Why art market indexes are problematic at best – and misleading at worst

Since the 1960s, indexes have been used to persuade collectors to think of art as an investment. But beware the lines that always go up.

Illustration by Katherine Hardy

Art price indexes have long been used by anyone seeking to justify returns, prices, and portfolio diversification when buying into the art market. They regularly appear in the financial and art market press, as well as being published by banks and private companies. Through various methodologies, they aim to gauge the movement of markets – whether the art market as a whole, a sector, or a particular artist. But the people who know what they’re talking about tend to argue that indexes are at best meaningless and at worst completely misleading.

On the websites of new ‘art investment’ companies, indexes are used to send a clear message: this is where the smart money is. The New York-based online platform Masterworks is a clear example. An index of its own making sits on its homepage, showing that contemporary art has outperformed the S&P 500 stock market index for the past 25 years, achieving a 13.8% annual ‘appreciation’ versus 10.2% annualised returns for the S&P 500.

Of course, the platform is careful to comply with financial regulations. ‘Past performance is no guarantee of future returns,’ reads the disclaimer. But Masterworks and its competitors are part of a long tradition of businesses that have used indexes as a tool to persuade people more interested in profit than aesthetics to funnel their money into the art market. ‘Our investors aren’t art people,’ a Masterworks representative told Artnet in April. ‘They are just investors.

The lines always go up

The origin of art indexes can be traced to the 1960s. After two decades of post-war art market growth, Peter Wilson – chairman of Sotheby’s from 1958 to 1980 – set about revolutionising the auction house client base. He was aided by PR man Brigadier Stanley Clarke, who maintained that real revenue for Sotheby’s would come from the ‘common man’, not from the connoisseur.

Central to their efforts – alongside the introduction of black-tie events and other flourishes – was the creation of art indexes, published monthly in The Times of London and later syndicated to the New York Times, comparing the increased prices of selected artworks to stock market returns. Their reason was straightforward: actively bringing in a new group of clients – investors – would push up the price of art. All they had to do was encourage the perception of art as a viable investment. A young statistician and journalist, Geraldine Norman (née Keen), was hired to run the index.

As little more than advertising, it is unsurprising that the lines on the indexes always went up. But in 1971, Norman decided to report the fact that Sotheby’s and Christie’s were keeping unsold lots hidden, contributing to a false image of the market. Wilson, outraged with Norman, pulled the plug on the indexes.

By that time, Sotheby’s had accomplished its job of establishing the idea of art as an investable asset – even though the indexes received immediate backlash from dealers, journalists and, importantly, economists and academics. In 1986, during a boom in the art market, William J. Baumol of Princeton concluded that people ‘should not let themselves be lured […] by the illusion that they can beat the game financially’. But of course, in a market boom, these words of warning fell on deaf ears.

The biases within art indexes

‘There is no superior approach to making an art index,’ says William N. Goetzmann, of the Yale School of Management. ‘What you’re really looking at is a statistical result that’s got lots of noise and error and problems.’ Goetzmann is one of a small pool of academics who have studied the problematic nature of indexes as tools to measure the viability of art as an investment.

One problem is survivorship bias. In the same way that unsold lots were excluded from the Times-Sotheby’s Index, the majority of indexes today still ignore them: only the ‘winners’ are counted. This bias is compounded by the major auction houses providing a ‘floor’ to the market through guarantees or by selectively withdrawing lots. Without reserves and guarantees, a work might sell for half or quarter of its low estimate, depending on the day. What this means in effect is that an index systematically underestimates the real risk of investing in art.

Then there is selection bias. Works of art in high demand are more likely to be sold at auction than those in low demand, which collectors tend to either keep or sell privately – activity that will never be tracked on the index. Goetzmann previously created an art index using the repeat sales methodology, which focuses on repeated sales of the same works of art over time. This approach is particularly vulnerable to selection bias, as Goetzmann now acknowledges: ‘I wish it had dawned on me back then that if something has sold twice, that may not be that common, and it could be that people don’t want to sell unless [the work] exceeds their own reservation price.’

Selection bias appears in other fields such as venture capital investing. For start-ups, valuations come when money is raised, an event that tends to happen when a company is doing well. Tracking company valuations and returns over time, you would only see the successes.

There are also other factors to consider, relating to the particular nature of art collecting. ‘One reason these indexes go up and up is that they aren’t showing net prices,’ says Melanie Gerlis, art-market columnist for the Financial Times and author of Art as an Investment? A Survey of Comparative Assets (2014). ‘Nowhere in these returns is the holding cost of art, the insurance, the shipping, the conservation, the auction house fees.’

Perhaps unsurprisingly, indexes created by academics systematically generate far lower ‘returns’ on art at a higher risk than those created by art market businesses. In 2013, Christophe Spaenjers, Associate Professor of Finance at HEC Paris, and his co-author, Luc Renneboog of Tilburg University, examined more than one million paintings and works on paper that sold at auction between 1957 and 2007, finding an annualised return of 3.97% on works of art – similar to that of corporate bonds, but at a much higher risk and far below the 10%+ returns touted by advocates of art as investment. Spaenjers and Renneboog used the hedonic method, controlling for the differences in art (size, medium, artist) while maintaining a large data set, mitigating one of the failures of the repeat sales methodology.

Roman Kräussl of the University of Luxembourg and Stanford University, who is a frequent co-author of both Goeztmann and Spaenjers, puts it bluntly: ‘If you don’t collect for non-financial motivations, like aesthetic return, status, prestige, then don't get into art, it will just screw you.’ Kräussl co-authored a paper in 2016 that found that adjusting for selection bias in art indexes dropped returns to 6.3% per annum, and made investing in art ‘not attractive’.

A vicious cycle

If the range of artworks that comes to auction is random, selection bias is lower. Traditionally, consignments have been driven by certain random forces – known in the trade as the three Ds of death, debt and divorce. But if you are in a period when a certain type of profit-driven buyer is incentivised, the selection bias increases.

‘Today, only a very small portion of our sales are driven by the three Ds,’ said a senior contemporary art specialist at one of the leading global auction houses, who agreed to be interviewed on the condition of anonymity. ‘Of course, the best items still come from estates, but the market right now is driven by people wanting to make money. People have paid crazy prices for art, especially in Asia. So now sellers expect prices twice or three times the low estimate.’

‘And the indexes themselves fuel the bias,’ Gerlis points out. ‘If you’re a buyer, perhaps you see Masterworks and all its SEC stuff, and you think “OK, I’ve got $100,000. I think it’s safer to own part of a Warhol [through fractional ownership] than to help support four contemporary artists.” And you end up with a self-fulfilling prophecy.’

Time and again, academics have thrown cold water on the accuracy of art indexes. Yet they have survived – supported by an industry that is terrible at understanding data as well as financial and legal professionals who don’t care to understand the idiosyncrasies of the art market.

The new retail-investor-aimed platforms such as Masterworks and Yieldstreet in the US, 360X in Germany, or Mintus in the UK claim to be doing things differently from their failed antecedents. Their strategies for making art an attractive investment include only focusing on the top end of the market, avoiding middle men, and attempting to improve the liquidity of art through a share trading marketplace (although the extent to which this goal has been realised is not clear).

But what hasn’t changed is art indexes being used as marketing to lure people who don’t understand their flawed nature. If later down the line things don’t quite go to plan, those using these indexes could be accused of misrepresenting the truth, or even false advertising. Perhaps only then will non-academic audiences begin to examine them more closely – and realise that the emperor has been naked all along.