Here’s the full quote. They compressed it a bit when making the meme —

“Gentlemen! I too have been a close observer of the doings of the Bank of the United States. I have had men watching you for a long time, and am convinced that you have used the funds of the bank to speculate in the breadstuffs of the country. When you won, you divided the profits amongst you, and when you lost, you charged it to the bank. You tell me that if I take the deposits from the bank and annul its charter I shall ruin ten thousand families. That may be true, gentlemen, but that is your sin! Should I let you go on, you will ruin fifty thousand families, and that would be my sin! You are a den of vipers and thieves. I have determined to rout you out, and by the Eternal, (bringing his fist down on the table) I will rout you out!”