I've never had any ill effect beings around family or coworkers. Maybe you're just experiencing very strong psychosomatic effects, I got an itchy throat from nowhere from seeing a bunch of maskers back then that went away the moment they were out of sight. Or you did just catch something and blamed it mentally on them like how people always blame the last thing they ate for food poisoning despite it taking a week for the microbe to incubate in them so who knows where you really caught it.