I think you're right but there is this culture around masculinity that if you kick up a fuss you're being a twat or can't take a joke etc.
That's what needs to change and it's something we can all do. Call out blokes being twats and support people calling out blokes for being twats.