I am quite sure that 'most' white western guys (I'll get to why) do not understand the receiving end of racism properly unless they have something similar to compare to. They want to, from good will, it is just not an easily understandable thing. Most brains that weren't exposed to it for a long time probably cannot take in the width of being notified/reminded umpteen times a day doing daily things like reading texts on pretty much anyhing from signs to instruction manuals, watching pictures of anything from products to commercials to children's drawings, seeing the subtle eye movements of cashiers, teenagers, old people, security guards, body language, not to mention reading the news where it is an absolute fuck-your-equality reminder fest -daily- no matter the intention of the article and it's been like that for decades.
It is far more subtle than you think I would guess, it is just that most people from ethnic minorities often don't make a fuss, out of experience. At some point most have worked through processes to not end up taking the road where you bitch about it, or become cynical, or slowly lose self-value or become a hater yourself. At an adult age plenty would say "doesn't bother me" but that's adaptation speaking, it says nothing about whether the case in point is fair.
It is also not always about obvious defining events where you went on a bus and someone yelled something and you were a sad panda. The every day wearing and tearing effect is also there. There probably isn't a simple way to deal with it, it'll be a long process thanks to people in the past setting norms that still persist.
It might seem unfair to a young white westerner (derp) who didn't create this problem and wants to do good, to feel like they're being blamed since it is obviously not their fault, most people know that. Most of the time you can tell when the intention is harmless. Yet at the same time damage is still caused daily and people who are feeling it every day often decide to not play the bitch-about-it card or even allow that perspective to enter their mind.
I now imagine women notice many of the same subtle things with sexism, things that I as a guy don't even notice, from seeing similarities with the ethnicity issue. I used to downplay the sexism thing when I was in my early teens before I one day was mature enough to draw that parallel in my head and realised there is definitely a case there.
edit: to clarify I'm just talking from my own experience, I imagine this issue takes different forms depending on where you live.