David Menninger's Analyst Perspectives

I am happy to share some insights gleaned from our latest Value Index research, which provides our assessment of how well vendors’ offerings meet buyers’ requirements. The Ventana Research Value Index: Collaborative Analytics and Business Intelligence 2019 is the distillation of a year of market and product research efforts by Ventana Research. Drawing on our benchmark research and expertise, we apply a structured research methodology built on evaluation categories that are designed to reflect the real-world criteria incorporated in a request for proposal to vendors in analytics and business intelligence. Using this methodology, we evaluated vendor submissions in seven categories, five relevant to the product (adaptability, capability, manageability, reliability and usability) and two related to the vendor (TCO/ROI and vendor validation). This research-based index is the first such evaluation to assess the full business value of collaborative analytics and business intelligence software. You can learn more about our Value Index as an effective vendor selection and RFI/RFP tool at https://www.ventanaresearch.com/value-indexes.

This Value Index research evaluates the following vendors that offer products that deliver analytics and business intelligence as we define it: BOARD International, Domo, Infor, Information Builders, IBM, Looker, Microsoft Corporation, MicroStrategy, Oracle Corporation, Qlik, Salesforce.com, SAP, SAS, Tableau Software and Yellowfin. Eleven of these 15 suppliers responded positively to our requests for information and provided completed questionnaires and demonstrations to help in our analysis of their analytics and business intelligence products. Along with briefings and information where provided, online material that was publicly available was used for the analysis. This research includes products generally available as of August 2018. The following vendors declined to participate or did not respond to our invitation: Microsoft, Oracle, Qlik and Tableau. To organizations considering products from these vendors, we recommend extra scrutiny as part of the software assessment because they did not make their technology available for the Value Index evaluation process.

Unlike many IT analyst firms that rank vendors from an IT-only perspective or that focus on a specific type of analytics and BI, Ventana Research has designed the Value Index to provide a balanced perspective that is rooted in an understanding of business drivers and needs. This approach not only reduces cost and time but also minimizes the risk of making a decision that is bad for the business. Using the Value Index will enable your organization to achieve the levels of efficiency and effectiveness needed to optimize analytics and business intelligence. 

We urge organizations to do a thorough job of evaluating analytics and business intelligence systems and tools and offer this Value Index as both the results of our in-depth analysis of these vendors and as an evaluation methodology. The Value Index can be used to evaluate existing suppliers and also provides evaluation criteria for new projects; applying it can shorten the cycle time for an RFP.

The Value Index for Collaborative Analytics and Business Intelligence in 2019 finds IBM first on the list with Information Builders second and Yellowfin third. Companies that score in the top three in any category earn the designation Value Index Leader. IBM has done so in four of the seven categories; Infor and Information Builders are a Value Index Leader in three categories; and BOARD, Domo, MicroStrategy, SAP and SAS in two categories.

Our Value Index methodology determines the weighting for each of the seven categories to best represent the needs of buyers. Here is our specific analysis for each of the categories.

The Capability category makes up 65 percent of the Value Index rating. It is designed to assess how well the product supports communication and interaction among individuals involved in analytics business intelligence processes. The assessment includes capabilities for collaborating, informing, managing and implementing decision-making processes. Ventana Research evaluated 100 different function points in 14 different categories to assess capabilities. In this category IBM, Information Builders and Yellowfin are Value Index Leaders.

Usability is also necessary for meeting a wide range of the business needs of executives, management, managers, workers, analysts and those responsible for administration of analytics and BI. The weighting for this category is 10 percent of the Value Index score. Vendors’ products are evaluated in this category on the support they provide for the range of roles in the analytics process — executives, managers, analysts, users and IT administration. Domo, MicroStrategy and SAP are the top three vendors in this category with Yellowfin very close behind.

For analytics processes to operate efficiently, the software on which they run must be able to scale and perform reliably on existing architectures; thus reliability is weighted at 5 percent of the Value Index. MicroStrategy, Infor and Information Builders are the Value Index Leaders in this category. Adaptability is weighted at 5 percent; this category assesses support for configuration and development and integration across usage, processes, applications and data. Infor, IBM and SAS are the highest-rated vendors in this category.

Manageability, which includes administration, security and licensing, is weighted at 5 percent of this Value Index. Manageability proved to be a challenging category for some — vendors either performed well or struggled to meet business and IT needs for effective installation, deployment and administration. Information Builders, BOARD and SAP are the top three vendors in this category.

In TCO/ROI, a category that evaluates vendor support for buyer investment evaluation through information, tools and services to help with the business case, makes up 5 percent of the Value Index score. Leaders Domo, Infor and IBM are well prepared. However, many of the others struggled to provide the tools and information organizations need to make a sound buying decision that is based on the product’s benefits and costs.

The Validation category contributes 5 percent to the overall Value Index score. The leaders here are BOARD, IBM and SAS, but this assessment finds wide variation in the level of detail that vendors provide regarding many areas including services and support. A lack of information about a vendor’s product support is a challenge that an organization looking to evaluate and select an analytics and BI product simply should not have to face. After all, a vendor’s success is about not just technology but also its relationship with the customer and the support it provides.

As noted above, this Ventana Research Value Index evaluates the software in seven key categories. The chart below places the product-related and customer assurance scores on the X and Y axes respectively to provide a visual representation of our Value Index scores. Vendors whose products scored higher in aggregate in the five product categories place farther to the right; the combination of scores for the two customer assurance categories determines their placement on the vertical axis. In short, vendors that place closer to the upper-right on this chart scored higher than those closer to the lower-left.

We warn that close vendor scores should not be taken to imply that the packages evaluated are functionally identical or equally well suited for use by every organization or for a specific process. Although there is a high degree of commonality in how organizations handle analytics and business intelligence, there are many idiosyncrasies and differences in how they do these functions that can make one vendor’s offering a better fit than another’s with a particular organization’s needs.

After more than a decade of technology advances, all the products we evaluated are feature-rich, but not all the capabilities they offer are equally valuable to users. Moreover, the existence of too many capabilities may be a negative factor for an organization if it introduces unnecessary complexity. Nonetheless, one company may decide that a larger number of options is a plus, especially if some of them match its established practices or better support a new initiative that is driving the purchase of new software.

Other factors besides features and functions or assessments about the vendor can turn out to be a deciding factor. For example, a company may face budget constraints such that the TCO evaluation can tip the balance to one vendor or another. This is where the Value Index methodology and the appropriate weighting can be applied to determine the best fit of vendors and products to your specific needs.

Our firm has made every effort to encompass in this Value Index the functional requirements and capabilities of our Collaborative Analytics and Business Intelligence blueprint, which we believe reflects what a well-crafted RFP should contain. Even so, there may be additional areas that affect which vendor and products best fit your particular requirements. Therefore, while this research is complete as it stands, utilizing it in your own organizational context is critical to ensure that products deliver the highest level of support for your projects in this area. You can get more details on our site as well as the Value Index Market Report.

Regards,
David Menninger
SVP & Research Director