No. Don't stay quiet. But don't just dismiss someone's argument when you can't prove they're wrong. Especially when you haven't lived there experience.
As Shmmeee said, it's hard to know someone's intent, in this case that goes for both Akinfenwa and the fella who threw the insult.