Cost of housing soars: How high can it go?

The median price of a single-family home sold in March was up 8.7 percent, twice the gains housing was seeing a year ago. What's behind rising prices?

The housing recovery is coming at a steep price—for both homebuyers and renters. The reasons can be found in the unique supply and demand circumstances that were left in the wake of the worst housing crash in history.

For several years, would-be buyers stayed on the sidelines, unwilling or unable to buy a home. That pushed rents higher. Homeowners, who might have wanted to move up, also stayed put, either underwater on their mortgages or afraid to sink more money into a new home. And builders never returned even close to the pace of historical norms.

Now demand is finally surging again, thanks to higher rents and a slowly improving economy, but supplies of homes for sale are anemic. There were just 2 million existing homes for sale at the end of March, a 4.6 month supply at the current sales pace. A six-month supply is considered healthy. The lack of homes for sale pushed the median price of a single-family home sold in March up 8.7 percent, more than twice the gains housing was seeing a year ago.

"We are not seeing the inventory gains we had expected," said Lawrence Yun, chief economist of the National Association of Realtors. "It's not healthy."

Sales of existing homes did jump to the highest level in 18 months in March, but first-time buyers were still barely a third of the market. They should be more like 40 percent of it. Realtors say that if sales stay at this pace, 2015 will end up being the strongest year for housing in nine years, but that's a big "if" since in order for sales to happen there needs to be more homes for sale.

Read More Tiny Seattle house takes a final stand

"For sales to build upon their current pace, homeowners will increasingly need to be confident in their ability to sell their home while having enough time and choices to upgrade or downsize. More listings and new home construction are still needed to tame price growth and provide more opportunity for first-time buyers to enter the market," added Yun.

While sales are improving, they are nowhere near where they should be given demographics and market demand. Household formation is still all on the rental side, and homeownership is sitting at a 20-year low and expected to fall further. That is why rents continue to rise, up 3.7 percent from a year ago, according to a monthly rent index from Zillow.

Read More Philly housing lifted by grads who stick around

"By the end of the year, Zillow expects growth in rents to outpace growth in home values," noted the company's latest report.

With renters paying ever more for their housing, they are less likely to be able to save for a down payment on a home. The mortgage market, while loosening ever so slightly, is not helping much. Rates are still near historic lows, but the bar is high when it comes to credit score and personal debt levels. Younger Americans are saddled with more student loan debt than ever before, which only exacerbates the problem. Income growth is also nowhere near keeping pace with home value growth.

Read More Mortgage applications rise 2.3%, led by homebuyers

"Further 8 percent home price gains is not going to bring the first-time household into buying instead of renting," said Peter Boockvar, chief market analyst of The Lindsey Group.

It begs the question: Is there a breaking point? Soaring home prices at the end of 2013, fueled by investors at the low end of the market, brought an unexpected slide in sales throughout much of 2014. Prices then began to ease in the latter half of the year, only to turn sharply higher again at the start of this year.

The Realtors' Yun said his hope for the best sales year in nearly a decade could be quashed "if affordability gets out of hand."

Read More US foreclosures rise, repossessions mounted