I think that the term originates from a military conflict (I think in Spain) where it was known that a military commander had four columns of soldiers marching on a city.

The commander claimed he actually already had a fifth column which had already infiltrated the city and would activate from inside the city walls once the fighting began.

Therefore, this instance could refer to "infiltration, not invasion" or it could refer to the Q community already dispersed around the world ready to spring into action when the time is right.