If we're the ones giving the inch it's not really possible for them to take a mile. That's just us giving them a mile.

The slippery slope only exists when you have a weak society that's incapable of agreeing to some things but not all. That can't say yes to stuff that's fine but also no to stuff that isn't. And while that may describe people now it's not a hard rule that always has to apply. If you give a large enough group of people something many will want you to go to the next step and give them more, but that doesn't mean you have to.