Great point, here is some additional context:

Canada and the American Revolution

There were twenty British North American colonies or provinces in 1776, so why did only thirteen of those colonies declare independence that year or confirm it by war and treaty in 1783?

The rebelling colonies did not target for inclusion the isolated fishery that was Newfoundland, nor Rupert’s Land, the Hudson Bay Company’s far northern trading territory. Nor, looking to the south, were they interested in immediately including the East and West Floridas, which Britain had acquired from Spain at the end of the Seven Years’ War in 1763. No, the thirteen British colonies that became the founding dominions of the United States focused on another colony acquired in 1763: the province of Quebec. They also initially hoped that Nova Scotia and the island colony of St. John’s (later renamed Prince Edward Island) would join their cause as well.

Colonists resisting the British government’s policies did not come easily to the decision to reach out to Canada’s inhabitants, for they were suspicious of les Canadiens – the French Canadians – who vastly outnumbered Anglo settlers in the Quebec province. Those who did want the French Canadians to join them, in turn, recognized that they would have to address Canadien hostility against les Bostonnais (a term that covered not just Bostonians but other New Englanders and Americans). Most French Canadians distrusted those who had long campaigned to conquer their colony (accomplished in the French and Indian War in 1763), insulted the Catholic religion, and belittled their culture.

Sounds like "Le Bostonnias" ruined the opportunity for the merger to happen.