Thursday, October 09, 2008

Piracy Statistics and the Importance of Journalistic Skepticism

Timothy B. Lee has been explaining at Freedom to Tinker why journalistic skepticism is important when considering piracy statistics. It's worth reading in full:

" If you've paid attention to copyright debates in recent years, you've probably seen advocates for more restrictive copyright laws claim that "counterfeiting and piracy" cost the US economy as much as $250 billion. When pressed, those who make these kinds of claims are inevitably vague about exactly where these figures come from. For example, I contacted Thomas Sydnor, the author of the paper I linked above, and he was able to point me to a 2002 press release from the FBI, which claims that "losses to counterfeiting are estimated at $200-250 billion a year in U.S. business losses."

There are a couple of things that are notable about this. In the first place, notice that the press release says counterfeiting, which is an entirely different issue from copyright infringement. Passing stronger copyright legislation in order to stop counterfeiting is a non-sequitur.

But the more serious issue is that the FBI can't actually explain how it arrived at these figures. And indeed, it appears that nobody knows who came up with these figures and how they were computed. Julian Sanchez has done some sleuthing and found that these figures have literally been floating around inside the beltway for decades. Julian contacted the FBI, which wasn't able to point to any specific source. Further investigation led him to a 1993 Forbes article:

Ars eagerly hunted down that issue and found a short article on counterfeiting, in which the reader is informed that "counterfeit merchandise" is "a $200 billion enterprise worldwide and growing faster than many of the industries it's preying on." No further source is given.

Quite possibly, the authors of the article called up an industry group like the IACC and got a ballpark guess. At any rate, there is nothing to indicate that Forbes itself had produced the estimate, Mr. Conyers' assertion notwithstanding. What is very clear, however, is that even assuming the figure is accurate, it is not an estimate of the cost to the U.S. economy of IP piracy. It's an estimate of the size of the entire global market in counterfeit goods. Despite the efforts of several witnesses to equate them, it is plainly not on par with the earlier calculation by the ITC that many had also cited.

It's not surprising that no one is able to cite a credible source because the figure is plainly absurd. For example, the Institute for Policy Innovation, a group that pushes for more restrictive copyright law, has claimed that copyright infringement costs the economy $58.0 billion. As I've written before, these estimates vastly overstate losses because IPI used a dubious methodology that double- and triple-counts each lost sale. The actual figure—even accepting some of the dubious assumptions in the IPI estimate, is almost certainly less than $20 billion. But whether it's $10, $20, or $58 billion, it's certainly not $250 billion.

There are a couple of important lessons here. One concerns the importance of careful scholarship. Before citing any statistic, you should have a clear understanding of what that figure is measuring, who calculated it, and how. The fact that this figure has gotten repeated so many times inside the beltway suggests that the people using the figure have not been doing their homework. It's not surprising that lobbyists cite the largest figures they can find, but public servants have a duty to be more skeptical.

The more important lesson is for the journalistic profession. Far too many reporters at reputable media outlets credulously repeat these figures in news stories without paying enough attention to where they come from. If a statistic is provided by a party with a vested interest in the subject of a story—if, say, a content industry group provides a statistic on the costs of piracy—reporters should double-check that figure against more reputable sources. And, sadly, a government agency isn't always a reliable source. Agencies like the BLS and BEA who are in the business of collecting official statistics are generally reliable. But it's not safe to assume that other agencies have done their homework. The FBI, for example, has made little effort to correct the record on the $250 billion figure, despite the fact that it is regularly cited as the source of the figure and despite the fact that it has admitted that it can't explain where the figure comes from.

Julian gives all the gory details on the origins of the $250 billion figure. He also digs into the oft-repeated claim that piracy costs 750,000 jobs, which dates back even further (to 1986) and is no more credible. And he offers some interesting theoretical reasons to think that the costs of copyright infringement are much, much less than $250 billion."

No comments: