The cheapest heating system to install depends on the specific needs of the home and the region. Electric space heaters and wood stoves are usually the cheapest options, but their operating costs may be higher than other options in the long run. In some regions, forced air heating systems can also be relatively inexpensive to install.