I'm finding your defensiveness irrational. In the UK Masons are mostly seen as a rather boring and mundane back-scratching old boys' club - possibly slightly sinister, perhaps a bit comical, but mostly quite ignorable.
Outside the fringe, no one here associates Masons with either pro- or anti-Semitic sentiments, pro- or anti-Muslim sentiments, or with racism.
Which is not to say that - e.g. - some of the Police and judiciary aren't racist and/or authoritarian. They clearly are.
But that's because some of them are vile and ignorant people, not because of what they get up to on Lodge nights - if they belong at all.
But you're ignoring everyone else's experience besides your own. And not a little British political history too.
Implying that any less than vehemently positive opinion about the Masons is equivalent to near-Godwin-ish anti-Semitism doesn't seem particularly rational in this context.