There are not enough plain racists arguing against scarves and for the downright exclusion of minorities to be successful...

Discriminators always look to expand their following by corrupting respectable views. Middle Age anti-Semites invented the blood libel. The slave-holders in the US South incorporated the traces of ancient Middle East slavery in the Bible into their ideology, making modern slavery divinely ordained. 19th century anti-Semites adapted religiously rooted anti-Semitism by adding Enlightement-era ethnic nationalism. Throughout the ages, the enslavement of other people for the enrichment of the few in the form of colonialism was justified to the home population with lofty goals (Roman Empire: bring civilisation and peace, Spain: bring Christianity to the pagans, British Empire: bring civilisation, US empire: bring freedom & democracy). The fact that Islamophobes used secularism to mainstream their racism/xenophobia is specific to the case, and what really matters is that people's secularism was subverted, rather than that secularism was subverted.

_{*Lunatic*, n.
One whose delusions are out of fashion.}

Think of conscientous objection

Good point, I haven't thought of that kind of law-breaking.

atheist privilege to eliminate religiously informed ethics from the public sphere

Hm, now I also arrived at the point where I don't understand your use of "privilege".

privilege - definition of privilege by the Free Online Dictionary, Thesaurus and Encyclopedia.

a. A special advantage, immunity, permission, right, or benefit granted to or enjoyed by an individual, class, or caste.

I don't see how any of that fits your usage above. Perhaps you meant another word? I would guess you meant "policy".

_{*Lunatic*, n.
One whose delusions are out of fashion.}

By the way, if atheist privilege to eliminate religiously informed ethics from the public sphere had prevailed, the movement for conscientous objection would probably have been too weak to be successful.

That's turning social reality on its head. It's evidence of religious privilege that conscientious objectors who claimed to be based on secular humanist principles would have been treated more harshly than those claiming religious principles. Possibly because the authoritarians administering military service were inclined to accord religious objections greater respect. That is religious privilege. Not a "failure of atheist privilege to prevail".

A society committed to the notion that government is always bad will have bad government. And it doesn't have to be that way. — Paul Krugman