I lived in Rugby between 1973 and 1983 until I was 18, and while most of my classmates were not Cov fans, hardly any of them actually ever went to games, whereas the Cov fans did. There were the odd Leicester fan here and there, but there were definitely more Cov fans. It was Cov fans who fronted up to the likes of Chelsea and West Ham when they stopped off in Rugby on their way back to London, while Leicester fans were nowhere to be seen.
There were also coaches organised to away games from the Saracens Head in the early 80's, with about 4 coaches used when we played at West Brom in the cup in 1982, so there were definitely a fair number of Cov fans in the town back then, even if that wasn't always represented in the classroom.