In my never-ending quest to study the Popularity of Data Analysis Software, I recently read the 2013 Edition of the Wisdom of Crowds Business Intelligence Market Study by Dresner Advisory Services, LLC. In it, I found the table below which displays the “Wisdom of Crowds”, or what most people would call survey results.
As you can see, among high growth business intelligence vendors, Tableau comes out on top with a mean score of 4.40. Not long afterwards, I saw this press release that claimed, “TIBCO Spotfire Named the Leader in ‘High Growth Business Intelligence’ Market Segment. Spotfire Achieves ‘Best in Class’ in Wisdom of Crowds SME Business Intelligence Study.” That can’t be right, can it? I downloaded the report, expecting it to be for 2014. However, it was from the same year as the previous one I had read, 2013, and it contained the following familiar-looking table.
To my surprise, Spotfire was now in first place, with Tableau in 3rd. Then I belatedly noticed the “SME” in Tibco’s headline. It turns out that the second report was a based on a subset of the first, selecting responses only from Small and Mid-sized Enterprises (SMEs). While the reports looked identical to me at first, the subset report does include “Small and Mid-sized Enterprises” right in its title. So both company claims are correct once you figure out who is being surveyed.
But what does this mean for the value of a #1 ranking from Dresner? The reports break the 23 vendors down into 5 groups: Titans, Large Established Pure-Play, High Growth, Specialized, and Emerging (they mention Early Stage ones too, but don’t rank them). This approach results in four or five vendors per table. By doing the report twice, once for all respondents and again for SMEs, there are 10 opportunities for companies to be #1 in any given year. In the two Dresner reports from 2013, 7 of the 23 companies are #1. Companies are more likely to purchase distribution rights to reports when they come out looking good. Since Dresner makes a living selling reports, that gives a whole new meaning to the term Business Intelligence!
