Imagine a world where our measuring devices were not constant.  A ruler would shrink at some times and grow at other times.  When you got an annual physical checkup, your recorded body temperature would increase by one degree per year (even though your real temperature would be unchanged).  A pound today would weigh less than a pound did five years ago.  After several years, the 12-gallon gasoline tank in your car would hold 13 gallons.  This would be a very strange world.  In the real world, we assume our measuring devices remain constant over time.  But this is an erroneous assumption if we are measuring the stock market.

It has been widely reported that the U.S. stock market, as measured by the Dow Jones Industrial Average (DJIA), recently hit a new high.  Pundits have pontificated on the significance of this milestone.  Some have even questioned the validity of this statistic.  But few have detailed the scope of the issue.  In order to truly figure out whether the stock market is at a new high, one must take into account sampling error, inflation, dividends, and taxes.

If one were going to sample the entire population of the U.S., choosing only 30 of the biggest cities would not be very representative.  It would be even worse if we weighted the influence of each of these cities by the number of letters in their names.  This is analogous to the situation with the DJIA, a price-weighted, quasi-representative slice of the U.S. stock market.  If a larger and more representative sample is available, why not use that instead?  A group of 500 stocks is better, but is still subject to a mismatch in representation if it primarily samples only the biggest companies (S&P 500).  3000 samples (the Russell 3000) are much closer to full representation, but why not use the full sample if available?  Several stock index providers furnish an index of the entire U.S. stock market, and numerous ETF providers include a total U.S. market ETF in their offerings.  Examples include the Vanguard Total Stock Market ETF (VTI), the SPDR Dow Jones Total Market (TMW), the Wilshire 5000 Total Market ETF (WFVK), and the Schwab U.S. Broad Market ETF (SCHB).

Another contributor to the inaccuracy of index measurement is inflation.  Stocks are priced in dollars, which is the measurement device we use when we compare historical prices with today’s prices. However, dollars shrink in value over time, more commonly known as the inflation rate.  So today’s 14,000 DJIA is not the same as 2007’s 14,000 DJIA because today’s dollars are worth less.  If you use the consumer price index as a proxy for the inflation rate (which is subject to endless debates about its accuracy), today’s 14,000 is about 10% lower than that of 2007.

However, inflation is not the biggest distorter of the measurement device.  Today, that characteristic belongs to dividends.  Every time a company pays a dividend, the stock price goes down by the dividend amount.  Currently, the DJIA loses about 360 points per year due to dividend payments.  Said another way, if an investor actually buys the 30 stocks in the DJIA (or its closest ETF equivalent: the SPDR Dow Jones Industrial Average, DIA), and the index level starts the year at one point and ends the year 360 points lower, the investor has actually broken even.  If the index ended the year unchanged, the investor would actually be 360 points (roughly 2%) ahead.  This same feature applies to the S&P 500 and many other indexes.  The S&P 500 currently loses over 30 points per year due to dividend payments.

The distortion created by dividend payments works in an opposite direction to that of inflation.  Consider a hypothetical investor who purchased DIA on the day it reached its 2007 high.  He would have about 15% more nominal dollars when the DJIA surpassed its old high earlier this month due to dividend payments.  After taking inflation into account, the investor would still be ahead, but only by about 5% or so.  In prior decades, inflation distorted the index value more than dividends. Therefore, the combined contribution of inflation and dividends to index measurement inaccuracies may be either positive or negative depending on the time period of interest.

If this is beginning to sound a bit complicated, I have bad news.  It’s going to get worse. Thus far I have glossed over the handling of the funds received from dividend payments.  Should we assume that that money stays in cash (to make the calculations simpler) or should it be reinvested?  If the point of the measurement device is to accurately measure how stock prices are behaving and not how a mixture of stocks and cash are doing, then dividends should be reinvested in the same stocks.  This could be done on a stock-by-stock basis as each stock in the index pays its dividend, or it could be done in more of a batch mode by reinvesting in a group of stocks after a chunk of dividends have been received.  In either case, the calculations are gnarly.  Determining the magnitude of the effects of dividend reinvestment is extremely difficult.  But it is easy to figure out whether this effect would add to or subtract from the real return, as opposed to a return calculated from a naïve comparison of a change in the index value.  In the example cited above, reinvestment would have occurred at prices lower than 14,000 throughout the period from 2007 to 2013, thus boosting the return.

Let the complications continue.  Taxes enter the picture.  Gains are taxed on nominal differences rather than inflation-adjusted differences.  Depending on the rate of inflation, this could wipe out our hypothetical investor’s entire gains.  Before inflation he has at least a 15% gain due to dividends.  (I am using the term “at least” because I cannot easily calculate the addition return due to dividend reinvestment.  All I can say is that it is a small positive number.)  Taxes will reduce this amount by 15% or 20% or 32.4% or some other number that is highly dependent on the investor’s income and location (state taxes).  And some portion of the gains (dividends) can be taxed at a different rate than other portions (capital gains).

But should tax complications affect the measurement of the index?  Not really, because the point of the measurement tool is to inform people about the performance of the stock market, not to measure each individual’s real post-tax, post-inflation return.  But tax considerations should not be ignored, because an investor could find himself in the situation of investing in an asset that has the appearance of appreciating nicely over time, but after inflation and taxes are taken into account, he is worse off than he was initially.  In this case, the reported measurement of the index would mislead a naïve investor into thinking he is doing better than he really is.

So if we use a broader sample than the DJIA or S&P 500, we find that since the last apparent high water mark, the stock market has actually done better than the popular averages would indicate, since small stocks not included in the DJIA or S&P 500 have outperformed large stocks.  Failing to take inflation into account will lead one to overestimate the real return.  Failing to take dividends and dividend reinvestment into account will lead one to underestimate the real return.  Taxes will muck things up further.  If you add all of this together, you will find that the stock market actually hit a new high months ago, when few were watching.  Most investors were paying attention to more popular, ever-changing measurement statistics that mislead more than inform.

What the financial world really needs is a widely-reported, precise measurement device that does not require expertise in forensic accounting to decipher.  One that would accurately answer a simple question: What would a dollar invested in the U.S. stock market on any given day in the past be worth today?