I continue to be surprised by the amount of weight given to downloads as a metric for OSS success. A topic Matt Asay also touched on recently over at The Open Road. Like Matt, I’m talking OSS at the product or company level (i.e., not OSS projects) and by success I mean sales.
For me, a high number of downloads only signals that people are willing to trial a product. Downloads do not equal actual use of the product, and trial is a far cry from success. If we take it as a given that consumers see value in the product, the company still has to convert those downloads into paying customers, and this challenge is where the problems of using downloads as an indicator of success become apparent.
The first problem is that not all downloads are created equal. If downloads are from commercial buyers who want to “try before they buy,” then increasing downloads is a good thing, whether open source or not (e.g., Microsoft’s Express Editions). However, if one’s downloads are largely by enthusiasts where no procurement channel exists, then the value of those downloads, beyond possibly generating demand for the product from the bottom up, is minimal when one is going after IT leaders who buy commercial products. It is the conversions that matter, and not the downloads.
Now, I realize that number of downloads will continue to receive attention as an indicator of success in open source. It is an easy metric to track, and while a less-is-more argument can be made in specific cases, I would rather see more downloads than not. That said, total downloads is still misleading.
Different products have different potential market sizes, so total downloads cannot be used to make comparisons across a number of products. Total downloads simply doesn’t provide a robust enough picture. It is the equivalent of saying that Company X is successful solely because it generates $250 million in revenue. Revenue doesn’t tell you anything about profitability, share, or rate of growth. Company X could be unprofitable with revenues declining at a rate of 20% a year. Revenue, like total downloads, is only part of the picture.
Sure, total downloads is worth knowing, but I am more interested the patterns and trends of those downloads. Are they steady/growing/declining? To what degree do they coincide with release dates? Etc. It is this second level of detail that starts to tell you whether a project is successful, not total downloads.
Hitting those total download plateaus, whether it be 25,000/month or 1,000,000 overall, is a great opportunity from a marketing standpoint to generate press and interest, but it doesn’t tell you whether or not an OSS product is successful or not. It’s just a small part of the equation.