Monday, November 12, 2007

Polarization of BI Solutions

It almost seems like the big BI companies figured out they had to be a part of something even bigger to justify their already sky-high license costs. Just within last 12 months, we've seen Hyperion get bought Oracle, Business Objects get bought by SAP, and now today Cognos got bought by IBM.

And I can't help but feel that Enterprise BI software has become a dying breed.

If you are a fortune 1000 type of organization, chances are you already have some sort of elaborate licensing agreement with at least one of these acquirers (Oracle, SAP, or IBM). So now, they have one more thing to sell you so that you can have a check mark for your BI initiatives without having to drive too far to a different vendor.

I guess I shouldn't be so cynical. As BI software, Hyperion, Business Objects, and Cognos are pretty feature-rich, and have many compelling qualities. My problem is with their complex enterprise deployment model, proprietary nature, and staggering cost of ownership that continues standing in the way of making BI available to the masses. And their recent acquisitions further polarizes the world of BI haves and have-nots.

Perhaps this is how corporate world will come to demand a new breed of BI solutions. Back in 2002, when we started our marketing analytics company, we thought (and still do) open source and software-as-a-service were going to drive this new trend. And it's good to see companies like JasperSoft, Pentaho, Swivel, and LucidEra lead the way in this direction. What's more important than just looking for open source and/or SaaS solutions to BI, is to demand "openness" -- openness in terms of architecture, sharing of insights, and just as importantly, openness about cost of ownership. How many times you have seen a BI initiative where the entire budget got spent just on data integration and/or cleansing? How about the cost of those special consultants who seemed to be the only earthlings that understood the esoteric workings of a popular yet proprietary BI software package?

Above all, one needs to keep in mind that BI is about leveraging all available data to get a clearer picture of what's going on in the business, be able to focus on the most relevant issues, and make better decisions. Instead of taking a centralized approach of one system doing it all, a good BI system today needs to act more like a network that can connect to various data sources, systems, API's, web services, etc. What if your BI system acted more like a mashup that lets you combine compatible information sources and cross-reference them as you please? Swivel has an approach close to this except that it expects all information to be uploaded by its users. It could be interesting if Swivel could connect to some common public data repositories (like geographical locations, weather, stock prices).

The key here is to have BI systems that enable mashup of information resources and analyses, a la web 2.0, as opposed to being traditional "enterprise" solutions that need your business to bend over backwards to fit into their proprietary framework and terminologies. Which to me conjures up images of Oracle, SAP, and IBM.

Maybe I liked them better when they were Hyperion, Business Objects, and Cognos. At least they didn't say -- "BTW, we also do BI".


Adelino said...

Great post as usual, and good to see you blogging regularly again.
I think your observations are right on, I'd just add that BI providers have to start offering more complex analysis capabilities in addition to the simple dicing and slicing and basic analytics.
I've taken you your observations at:

Bhupendra said...

Great Post Sandeep. I have never used any open source BI platform till now (in exception to R for few months for running CART). Thanks a lot for providing this information.

Do you know any opensource product which is very good for Segmentation (with implementation of CART, Chaid, K-means Algorithm, Genetic Algorithm etc)? I will be grateful to know if someone provides that.


Sandeep Giri said...

Thanks for your feedback Adelino. A great example of leveraging public domain data is on -- which I covered in an older post. What I'd like to see is for this to be extended into a web-services like model, where a BI application can go through an index of "data services" on the web and see what data repositories are available, and furthermore, who certifies the accuracy/validity of that data. This can evolve parallely in 2 models -- first in the public domain where government and other public organizations start publishing their data (which in turn increases their visibility via data consumption), and second in the commercial domain where partnering entities expose their data.

Re your second comment on going beyond slicing and dicing, I couldn't agree more. My hope is to evolve our open source project OpenI into that direction (if I can muster up more resources :-) but the idea is to have a more wizard-like way of asking questions whose answers are handled via predictive models, etc. on the backend, but don't make statistical knowledge a requirement to use the BI tool

Sandeep Giri said...

Bhupendra -- with the exception of R and Weka project, not sure if there are many others out there in open source data mining / statistics. But as I mentioned in my response to Adelino's comment, that's the direction I'd like to take our open source project OpenI into, using R and/or Weka as the backend. The challenge is always going to be usability -- the application needs to focus on segmentation and the way people go on about it, while also applying stat models in the right context.