You are conflating the changing of defined borders of nation states with the internal structure of the society. From the perspective of the Jews themselves, they are well aware of which tribe they come from. This is especially true of the Levites, from whom the Aristocratic Class is established. It is predominantly from this group that the Cabal originates, so it is important to appreciate that borders do not, and have never defined the Tribes of Israel as you suggest.