Why do markets need designing? Lending Smith’s 'invisible hand' a hand

The most celebrated theory in all of economics, the so-called First Welfare Theorem, says that, provided there is perfect competition and perfect information, free markets allocate resources resources efficiently.

In other words, it says the father of economics Adam Smith was right, that the “invisible hand” of the market works in the best interest of society.

Often the conditions are not met, and much of modern economics is about how to improve the otherwise suboptimal outcomes delivered by unfettered markets.

That needn’t mean economists giving up on markets, it might just mean acting more like engineers, ensuring they work better.

In the early 1970s, petroleum engineers Capen, Clapp, and Campbell noticed that the market for US oil leases was performing badly.

Companies that had won auctions suffered unexpectedly low returns, a phenomenon that became known as the “winner’s curse”.

The reason was that the highest bid for the leases reflected the most optimistic estimate of the oil they contained, while the true value was probably closer to the average estimate.

The first to tackle the problem was Robert Wilson, one of the winners of this year’s Nobel Prize in Economics, coming up with the idea of a common value auction, in which information about the nature of what was for sale was shared.

When houses are sold, it can take the form of an independent inspection report handed to all bidders.

Perhaps surprisingly, the sharing of information can result in higher prices, because bidders don’t feel the need to underbid to avoid the winners curse.

Wilson’s PhD student with whom he shares this year’s Nobel Prize, Paul Milgrom, added further realism by incorporating private values.

In the case of oil, it might be that the value of the lease is determined not only by the amount of oil but also by the bidder’s unique method of exploitation. In the case of a house, it might be that a family of five really needs four bedrooms, but for a smaller family they are less important.

Until Milgrom, auctions were auctions

The prevailing auction wisdom at the time was based on William Vickrey’s “Revenue Equivalence Theorem,” which said that all types of auction yielded the same result for the seller.

Milgrom’s crucial discovery that in settings with both common and private values, the format matters.

In an ascending so-called English auction, bidders use the prices at which other bidders drop out to update their estimates of the product’s worth. The auction allows them to revise their estimates back to the mean and protects them from overbidding.

But no such learning is possible in a sealed-bid tender where bidders simultaneously submit their offers in a single round, exposing them to the winner’s curse.

The “greatest auction ever”

The 1994 United States broadcast spectrum auction was hailed by the New York Times as “the greatest auction ever”. It set off a chain of spectrum auctions worldwide that raised over US$200 billion.

Before then frequencies had mainly been given away to worthy recipients or allocated by lottery.

The problem Wilson and Milgrom faced in designing it was that if the value to an operator of a licence in one region depends on whether it has a licence in an adjoining region, a one-off set of simultaneous auctions will make it impossible for it to know how much to bid.

As well, a speculator can try to buy a blocking region, hoping to be bought out later so the overall winner can create a national network.

So they invented an entirely new auction format, the Simultaneous Multiple Round Auction which starts with low prices and allows repeated bids across many areas, so that geographic patterns of ownership can evolve in a single process.

In 2012 Milgrom designed a two-stage “incentive auction” for the Federal Communications Commission in which broadcasters were asked to nominate the price at which they would give up parts of the spectrum they owned which the Commission would then resell to phone companies and others at a huge profit.

Read more: Nobel economics prize: Wilson and Milgrom's insights into auctions could drive down carbon emissions

Eventually conducted in 2017, the twin auctions freed up 84 MHz at a cost of US$10.1 billion. The Commission sold 70 MHz for US$19.8 billion, leaving 14 MHz free and delivering taxpayers a profit of US$10 billion.

Economists as engineers

Market design has since matured into an interdisciplinary field where economists, computer scientists, and operations researchers collaboratively design, test, and build new solutions for problems of government, businesses, and society.

In Australia, the AGORA Centre for Market Design designed a novel exchange for trading rights to catch fish in the state of NSW.

Read more: Making auctions work: the winning ideas behind this year's Nobel Prize in economics

It ended two decades of political debate by providing a market-based response to a major problem faced by fisheries worldwide – the reallocation of rights in programs designed to prevent overfishing.

The “economist as engineer,” a vision first articulated by 2012 Nobel Prize winner Alvin Roth (another student of Wilson) has become reality in no small part due to Milgrom and Wilson’s pioneering work.