Old school kerosene lanterns and stoves would also be nice backup. I also have a ventless and no power gas heater that runs on natural gas or propane. I have it all plumbed up. I just need to drill a hole in the wall and make the final connection. You can buy them at Lowes or Home Depot.

Also if you are on a budget the best thing to have is a headlamp and a pile of batteries. That's what I mostly used when I lost power for 2 weeks a few years ago.