David Menninger's Analyst Perspectives

We recently published the results of our benchmark research on Big Data to complement the previously published benchmark research on Hadoop and Information Management. Ventana Research undertook this research to acquire real-world information about levels of maturity, trends and best practices in organizations’ use of large-scale data management systems now commonly called Big Data. The results are illuminating.

Rather than make predictions for 2012, which are everywhere right now, I want to look back at some of the surprising events of 2011. I think it’s worth considering what happened that wasn’t expected and what these things might tell us about the information technology market. Here, in no particular order, are the most important ones I see.

Talend recently announced version 5 of its information management platform, which emphasizes unifying its various components. Through a combination of development activities, acquisitions and partnerships, Talend has been steadily building its portfolio of information management capabilities. In addition to its core data integration capabilities, it has added data quality, master data management, application integration and with this release business process management (BPM).

Cloudera’s recent Hadoop World 2011 event confirmed that the world of big data is getting even bigger. As I wrote of last year’s event, Hadoop, the open source large-scale data processing technology, has gone mainstream. And while 75% of the audience attended this year for the first time and so may not have realized the breadth of Hadoop’s acceptance, statistics announced in the opening keynote show widespread use of it. Mike Olson, Cloudera CEO, reported that the event was sold out, with 1,400...

Informatica recently introduced HParser, an expansion of its capabilities for working with Hadoop data sources. Beginning with Version 9.1, introduced earlier this year, Informatica’s flagship product has been able to access data stored in HDFS as either a source or a target for information management processes. However, it could not manipulate or transform the data within the Hadoop environment. With this announcement, Informatica starts to bring its data transformation capabilities to Hadoop.

Teradata recently held its Partners User Group meeting (Twitter hashtag #TDPUG11) in San Diego. Analysts were briefed previously on some of the announcements, which I covered in an earlier post.

Oracle made several announcements at its recent Open World event demonstrating its strengths in the business computing market but also that it is standing on the shoulders of giants. The company has developed the expertise, processes and market share to scale out the ideas and innovations of others. Don’t get me wrong: That statement is not an indictment. Large organizations often have challenges with innovation. They are not as nimble as their smaller competitors. On the other hand, small...

Recently Karmasphere introduced version 1.5 of its Analyst product which helps organizations analyze “big data” stored in Hadoop, the open source large-scale data processing technology. An independent software vendor focused exclusively on the Hadoop market, Karmasphere made available a community edition of its developer product in September 2009 and launched the company in March 2010. Since then it has been active and visible in Hadoop-related events including Hadoop World, the IBM Big Data...

Splunk may be one of the biggest software companies you’ve never heard of. I’ve been following the seven-year-old company for over six months now and recently attended its second annual user conference. Splunk focuses on analyzing large volumes of machine-generated data in underlying applications and systems, which includes application and system logs, network traffic, sensor data, click streams and other loosely structured information sources. Many of these “big data” sources are the same...

For months the speculation was rampant, and now the rumors have proven to be true. Yahoo has officially announced that it will become a player in the emerging Hadoop market. Hadoop provides distributed computing capabilities that enable organizations to process very large amounts of data quickly. Backed by Yahoo and Benchmark Capital, a new entity called Hortonworks has formed around a team from Yahoo that consists of more than 20 key architects of and contributors to the Apache Hadoop project....