A basic lack of homes is taking a terrible urban toll – affordability is social justice. Our only choice is to build, build, build

San Francisco's Chinatown - one of the most densely populated parts of the city. Bidding wars and multiple applications are now common in the city for both buys and lets. Photograph: Michael Layefsky/Flickr Vision

The first time I heard the story, I have to admit, I didn't believe it. A friend was apartment hunting and – despite being employed, personable and trustworthy – was having a difficult time finding a place. After a few weeks, she was invited to a "pre-showing", where landlords show the apartment to a select group of prospective tenants before having an open house.

The apartment was perfect, if expensive: think hardwood floors, lots of light, high ceilings. She was smitten. As she was strolling through what she thought might become her new home, however, a man in hip clothes walked up to the landlord and said, "I'll give you a year's rent, in advance". Needless to say, she did not get the unit.

As I say, I was sceptical. Then I heard basically the same story again, from a completely different source, about a completely different apartment. Then I heard a version of it about a flat in London, another about a competitor bribing a rental agent in New York, and another about the strategy of the pre-listing offer, where deep-pocketed tenants make generous bids to the owners of buildings where workers are making major repairs, to secure places that may soon be available before other tenants can even see them.

That's when I realised the truth of these stories is sort of irrelevant. The ruthless, rich renter stealing apartments from the professional middle class has clearly manifested an urban archetype for our day. Which makes perfect sense, because housing affordability is the urban crisis of our day. Here in the Bay Area, the median list price for a two-bedroom home in San Francisco is now more than $900,000 (£550,000). The median monthly rent for a similar home is over $3,000. If you can find either, that is: housing supply is so tight that bidding wars on real estate and multiple applicants for rentals are the norm.

Of course, San Francisco merely provides one outrageous example of a truly global trend. With the next billion poor people making their way to the world's cities, a revival in middle-class cultural preferences for urban living and the booming wealth of global economic hubs, cities are straining at the seams. The demand for urban housing has far outstripped supply in cities around the world. And when lots more people compete for the same homes, prices soar. Scrambles for real estate are reality now from Shanghai to Sao Paulo.

On a planet of cities, affordability is social justice. The hardships rising rents cause low-income people – the strained budgets, the overcrowding, the increased risk of homelessness and the pushing out of the urban poor into suburbs where their time and money is further taxed by longer commutes and more expensive transportation – undermine hardworking people's prospects and worsen income inequality, with serious consequences for all of us.

Cities have not been building enough new housing for generations. Photograph: Reuters

The debate about restoring affordability to our cities is often rancorous and out of date. Often we see the problem framed simply as a matter of regulating landlords or punishing developer greed. Renters' rights, subsidies for the poor, price controls and the like can play a role in buffering the poor from the market. All, though, have also shown themselves to be seriously inadequate given the scale of the gulf between available housing and demand for homes. Public housing can't keep pace. Nor do schemes aimed at keeping people from moving to cities succeed, even when they're repressive and unjust. Indeed, we know the approaches that won't work: all of them, other than building a lot more housing.

To make housing affordable again, we need to catch up to decades-worth of unmet demand, over the next few years. In many cities, this means goals measured in the tens of thousands of new homes; in the fastest-growing cities, it means hundreds of thousands. Build enough housing and (economists and experience both tell us) prices should at least stabilise. Want social justice? Build a lot more housing.

In order to build that kind of housing we need efforts that are new not only in scale, but in approach. We can build some housing incrementally, without changing the skyline or cityscape, but not anything like enough. To produce enough homes to matter, fast enough, we're going to have to fundamentally alter parts of our cities. That, of course, demands a local government willing and able to plan and permit such widespread change. It also takes an array of homebuilders doing the actual work, often in more innovative and low-cost ways, like more collaborative housing, manufactured buildings and flexible living spaces. Most of all, it takes broader public insight into how large-scale development can improve our cities.

That development can make cities better has not been a popular idea in recent decades. But there is a large new constituency for change, today: the young. Though the fact is often ignored in our debates, housing affordability is a generational issue. The people most disadvantaged by unaffordable housing are the young. In many cities, high costs and low-paying jobs mean that for young people renting a great apartment is a pipe dream, never mind owning a home. In that context, change doesn't look bad at all.

More, young people tend to understand that new housing, well-built, can be a tremendous force for positive transformation. Built in sufficient amounts, new housing can check the rise in housing costs; but with good design, it can also help grow new people-centered streets, increase use of bikes and transit, and promote healthier and more active lives.

Housing construction and more residents help the local economy and create far more jobs than subsidies for large corporations. More taxpayers help our cities balance their books though larger revenues and more efficient infrastructure. New housing is one of the few truly powerful tools citizens and local officials can actually make use of to improve our cities and the world.

More housing can even help us meet the global climate crisis. We know that in compact and walkable communities, each person uses less energy (and thus emits less carbon dioxide) than a similar person in more spread-out areas. A planet of small ultra-rich cities and sprawling low-density slums will be a planet warming faster than a planet of dense, mixed-income, transit-friendly cities.

San Francisco is an outrageous example of a truly global trend – that competiton for housing has hit a critical level. Photograph: Patrick Batchelder /Alamy

Combine compact development on a large scale with a push for more sustainable urbanism, and we can see the outlines of cities that will work in the 21st century take form. We can enable low-consumption lives through better public services and sharing systems. We can convert excess roads and parking lots into great streets, green spaces and parks, making denser neighborhoods nicer to live in then they were before and more resilient to weather extremes. We can promote green building and more sustainable water and energy infrastructure, making rising prosperity even more ecologically frugal. We're only just glimpsing how much hope for the planet rests on getting cities right.

Building cities defines the global economy already; building them better will define who succeeds and fails in the years ahead. It's not a zero-sum game. More people inventing ways to house everyone while improving our cities can mean better cities everywhere. We can create more widespread prosperity for everyone. To do that, we must have the courage to aim big, experiment, let people try new ideas, take creative risks; we must stop being so afraid that every change is a threat. Because in the real world, the biggest threat is no change at all.

> via The Guardian