Not just the victim. Anyone can claim it had a hate element and it will be flagged by the CPS. For sentencing to take it into account obviously it has to be proven in court like anything else.
https://www.cps.gov.uk/legal-guidance/racist-and-religious-hate-crime-prosecution-guidance