You've got a point IBS. It riles me every time. Quite a lot of games I sit in the family stand towards the away end and it's that chant that winds me up.
The thing is, most clubs who sing it will no doubt have the same problem (ie sheffield, Birmingham, teams around the north west, forest-when they inevitably sing it).