Why are you "talking with workers" just go read the policy. I read all of the policies surrounding my business, and they stated nothing remotely close to that.
Just ignore it and make them fire you.
Do not ignore it, request information from HR on religious exemptions. Send it in email, send it by certified mail if you have to. They have to work with you to provide accommodations, it's the law.
If you ignore them, it makes it easier for them to get rid of you.
Your company is required by law to provide you any policy that applies to you. You have to be knowledgeable of that policy for it to be enforceable. Either someone is lying to you, or you are not asking the right questions loud enough.
I received my religious exemption today.