The "M" in MAC stands for Medium meaning the physical data wire running around your office or home or whatever. The term originates in times before Wi-Fi but you can say your wireless LAN is also similarly a Medium. The "I" in TCP/IP stands for Internet, and the only way from your Medium to the Internet is via a router.

You are correct that your MAC address is visible all over your Medium in every single packet of data as you rightly point out. Do you care though? There is probably no Dr. Evil tracking you in your own apartment. As soon as that packet needs to go the Internet then it's all different - the router sends it to the Internet but the MAC address is not needed any more (once it leaves your Medium) so the router drops it before sending.

TL;DR your MAC address may be visible very locally but not on the Internet.

Oh yeh and a lot of modern devices (Android phones for sure) will deliberately randomise your MAC address if you configure that in the settings. Not a whole lot of use but prevents public Wi-Fi from recognising you as "that same guy who was here last week."