Solveig Singleton

The Downloading Studies Revisited

From Music Row Law, a review of two studies now supporting the view that P2P downloading actually increases sales of physical media; the downturn in CD sales through music stores is thus a result of other factors (such as the rise of Walmart).

I remain skeptical. In analyzing this data, assumptions are key. Many other studies show harm.

The earlier study, by Strumpf, professor of business economics at the University of Kansas Business School and Felix Oberholzer, seemed to operate on some peculiar assumptions (one being that downloads of popular tunes have the same impact on sales as downloads of more obscure ones). However, their data is not available for re-analysis.

Stan Leibowitz has a concise critique of the Canadian study as well as a paper in the Journal of Law & Econ. His use of data is extremely careful.

Among other things, he concludes:

All the papers that I have seen by other economists, except for one notable exception, find some degree of harm (to record producers) caused by file-sharing. These include papers by Blackburn, Hong, Michel, Peitz and Waelbroeck, Rob and Waldfogel and Zentner. The lone exception, but the most heavily publicized, is a paper by Oberholzer-Gee and Strumpf, which I believe is littered with errors and disingenuousness as discussed in greater detail below.

His critique of the Canadian study notes:

The result that has attracted the most attention comes from this quote from the A/F report: “Among Canadians who engage in P2P file-sharing, our results suggest that for every 12 P2P downloaded songs, music purchases increase by 0.44 CDs.” Since there are 14 songs on a typical CD, this means that for each CD equivalent of song downloaded, sales of CDs would increase by one half of a CD.

To arrive at this conclusion the authors limit their sample to only those who download music from peer-to-peer sites. Limiting the sample in this way seems nonsensical. When we test the efficacy of a drug we compare those who take the drug with those who do not. If we limited our observations to only those users who take the drug we would be giving up our most useful and important information. It is possible that dosage differences across users might still provide some information about the overall impact of the drug, but the most important information is whether the drug, at any reasonable dosage, causes a change compared to no drug at all…

Our interest is in the CD purchase behavior of consumers and the ‘treatment’ is peer-to-peer downloading. The best test for that is to compare the group that downloads with peer-to-peer against the group that does not download. …

A/F have since responded that they do not have a controlled experiment, such as that above, and imply that somehow that changes the logic of the above example. It does not. If we were to examine the impact of tobacco smoking we would compare the smokers to the non-smokers even though it is not a controlled experiment. In fact, this is how the studies were done. It would be illogical to examine only smokers. So A/F still need to provide a cogent explanation for their decision.

Although I am not sure whether A/F report this statistic, the average number of files downloaded from peer-to-peer networks in their sample of downloaders is 30 (24 — weighted values are in parentheses) files per month (the data are publicly available here). This is the equivalent (in terms of the number of tracks) of 26 (20) CDs per year. According to the A/F quote reported above, this would mean that the average downloader increases their purchase of CDs by 14 (10) units per year. Yet the same data indicate that downloaders only purchase an average of 9 (6) CDs per year. Thus, A/F’s reported result is impossible since downloaders cannot have increased their consumption by 14 (10) units and yet, after this increase, only consume 9 (6) CDs per year.

For more, see his home page.

December 4, 2007 | Comments |

discussion by DISQUS

Add New Comment

Viewing 6 Comments

    • ^
    • v
    Did you look at who funded those studies ?

    I know that the Liebowitz analysis of the Industry Canada funded paper was funded by the music industry.

    I'm not aware of any paper that wasn't funded by the music industry that has shown harm from peer-to-peer downloading.
    • ^
    • v
    Chris, I think you're barking up the wrong tree if you're arguing that Liebowitz's research is influenced by funding sources. If there were a betting market in it, I'd be at least 25:1 against. He's just not that kind of guy.
    • ^
    • v

    The experiment you propose isn't controlled, and I don't think you can suggest one that is. The problem is the groups are self selecting - I suspect those who use peer to peer are more likely to buy CDs anyway. Those who are not into peer to peer downloading are most likely not interested in new music anyway. They are happy with their Elvis CDs (or even vinyl), and only buy new ones if/when the old ones wear out.

    If you can get some who currently use peer to peer to stop for a few months you could have a controlled study. However I suspect most of your control group would drop out - those that remain would be a self selected group and no longer a valid control.

    • ^
    • v
    Henry, Liebowitz raises some very good points though. For example, verified sales figures for CDs in Canada are not consistent with the self-reported figures in the Canadian study, and the authors don't seem to have been concerned about this.

    Liebowitz also ran the regressions referred to in the Canadian paper and obtained different results. This raises questions as to WTF the authors were doing.





    • ^
    • v
    There are further complications. Because of the nature of the music biz today, with more independent artists selling CDs or downloads on their own websites and live shows that may not be tracked by official statistics, those figures might be lower than the actual total.

    In addition, there is a vibrant used CD market. A person may respond to hearing free downloaded music by buying a pre-owned album. Was this tracked in the study? Just because a P2P downloader wants to own the CD doesn't mean he/she will necessarily be willing to pay full price for it -- it might stimulate a perusal of eBay or the used section of amazon.com.

    Also, if the study were run for calendar year 2007, would new Radiohead album downloads, or the box sets they are selling from their own website, have been counted in the totals this study used?

    The market is fragmenting, and this makes accurate tracking difficult.
    • ^
    • v
    From Music Row Law, a review of two studies now supporting the view that P2P downloading actually increases sales of physical media; the downturn in CD sales through music stores is thus a result of other factors (such as the rise of Walmart).

    Well, please admit the difference between illegal P2P downloading and P2P downloading. P2P itself is not illegal, although it is possible to use P2P to break the law, as it is possible to use a car or a telephone to break the law.

    So, Solveig, if you mean illegal P2P downloading please say so.

    To arrive at this conclusion the authors limit their sample to only those who download music from peer-to-peer sites. Limiting the sample in this way seems nonsensical. When we test the efficacy of a drug we compare those who take the drug with those who do not. If we limited our observations to only those users who take the drug we would be giving up our most useful and important information.

    This is a conceptually flawed comparison. One of the key things any scientific research needs is a hypothesis, and that hypothesis is supported by an underlying mechanism.

    In the case of the pharmaceuticals the mechanism of their consumption and health result include an understanding of the cause and effect relationship that exists between consumption of an active substance, and it's effect on the physical body.

    However, in the case of P2P downloading the cause effect relationship, which necessarily include the complexities of human behavior, breakdown.

    Concrete example of this: I don't buy any more CD's because of the big four label's attitudes towards P2P technology. I suspect that many who fall into the same category as I do [that is, upset at the actions of the strong IP crowd] also are using P2P, perhaps [unlike me] for infringement. But if restrictions had never been placed on P2P, they never would have developed the antipathy they have for the big labels, and would have continued purchasing CD's.

    In this case, among this group, restrictions on P2P availability decreases CD sales, while non-resrictions on P2P availability would increase, even maximize sales sales.

    This kind of a conjecture has a very interesting characteristic in common with very large planning projects: untestability. The effects of the hypothetical course of action are so great that the 'experiment' cannot be undone, and therefore cannot ever be tested. But as with a planning project, the imperative to act can very greatly outweigh the limits on information that purely rational processes can provide. What is to be done in such cases? It's a very big question, too big for this post...
blog comments powered by Disqus