Commoditization of Business Intelligence

I received an email from Seth Grimes, a very familiar voice in the BI community, with an interesting question -- Do you now see BI as a commodity market?

He was referring to an earlier post about my open source BI project OpenI, where I'd mentioned:

The state of business intelligence software market has been very much controlled by a few big players. The situation is very similar to how the J2EE application server market was before JBoss, or how database server market used to be before MySQL and Postgres emerged as serious alternatives, or how OS market was before Linux. Pretty soon we will talk about the BI platform market in the same manner, because open source and open standards are driving the commoditization of BI as we speak. It is just a matter of time.

That was a few years ago. So, I asked myself -- well, how do I feel now? Have I learned anything?

The question is a tough one -- something I've always grappled with. Ultimately, it depends on what do we mean by "business intelligence". If we go by the current big commercial players' definition -- then BI is more about a software tool providing capabilities around data warehousing/ETL, OLAP, analytical modeling, and visualization. So by that account, I'd definitely stick to my original thoughts and say it's a commodity market.

However, a more relevant question might be -- does having these capabilities make a business intelligent? In reality, what I've seen is that it comes down to an analyst (or group of analysts) who (a) know how to work a "BI" tool, and (b) have some fundamental expertise in the business domain they are analyzing. So, the "BI" tool is more about facilitating the job of an analyst or a general business user. You could argue that by making performance metrics, etc. more easily accessible to a business user, the BI tool is helping them make more effective decisions, but it is making a big assumption that the user knows how relevant the performance metrics are for the business.

In the end, my take on this is that, the most effective BI tools are domain-centric, i.e. they embody some inherent knowledge about a particular business domain -- so, not only they are extremely efficient and accurate about compiling all the performance metrics and making them available, they also "understand" the applicability of those metrics and can almost act like expert systems in guiding crucial decisions. This, I don't think is a commodity market. It needs to be grounded into specific industry domains to be effective.

Would love to hear your thoughts.

Tweet

OpenI Listed in Top 10 Free BI Apps

Ask anyone who is involved in an open source project, getting recognized is one of the greatest kicks you get out of the whole deal.

So, when my Google alert picked up this news on OpenI listed in the Top 10 Free BI Apps list, it absolutely made my day. Of course, a lot of the credit goes to all the folks who have contributed to this project, and the open source community that has supported us all this time. And thanks to Tamina Vahidy for recognizing the project.

When we started OpenI back in July 2005, we just wanted to subsidize our R&D. We needed a BI platform to deploy our analytical models, and instead of opting for commercial BI platforms which would have never fit into our cost model, we decided to develop a BI platform using available open source components, and also as an open source project of its own. We figured if we get even a couple of people outside of our company to pitch -- whether it was design help, or just thinking through requirements, use cases we hadn't encountered -- that alone would pay for the efforts to make it open source.

Well, not only we got design help and advice from a great deal of smart folks in the space, we even have people contributing code. I remember someone (maybe Steve Weber) making a point about open source development model -- not all the smart people in the world work for you, so the only way to get them involved in your projects is via open source (ok, you may argue crowdsourcing ideas such as Netflix's contest, but I don't have a $ 1 million to give away in prize money :-)

So -- here we are -- working on version 1.3 of the product. We are using it internally as the web front-end of our commercial product. Of course, it has ways to go (see roadmap) -- but as contribution and recognition keep coming in, it just seems like a much more rewarding way to develop software.

Tweet

Is Open Source Java More Restrictive under GPL?

Today will most probably go down as an important day in the history of software engineering -- Sun open sourced Java under GPL license! (Sun's CEO Jonathan Schwarz's blogs on it as well)

While this is a very welcome news, the choice of GPL as the license has me a bit concerned. Two years ago, I was researching the various open source licenses like mad for our own open source project OpenI, and decided against GPL because of its "viral" nature (we ended up with Mozilla Public License, MPL). The "viral" aspect of GPL (section 2(b)) says:

You must cause any work that you distribute or publish, that in whole or in part contains or is derived from the Program or any part thereof, to be licensed as a whole at no charge to all third parties under the terms of this License.

and later further clarifies:

..it is not the intent of this section to claim rights or contest your rights to work written entirely by you; rather, the intent is to exercise the right to control the distribution of derivative or collective works based on the Program.

In addition, mere aggregation of another work not based on the Program with the Program (or with a work based on the Program) on a volume of a storage or distribution medium does not bring the other work under the scope of this License.

When someone uses programming language software like Java to build their own software, it is tricky to determine if their work is "derivative", "collective", or a "mere aggregation". In such a case, all the software code is written in Java. The software requires a Java Runtime Environment, which acts like a virtual operating system, to run. So, Java as a software construct becomes very tightly coupled with each new software program built on Java.

This is a contentious issue where there has been a lot of debate. See this blog entry at O'Reilly and its comments back in 2004

For example, with Java being GPL now, what happens when I build a software program in Java and would like to distribute it along with a Java runtime engine (JRE) for user convenience? Since I have built the software using Java and I am combining the JRE in my distribution, will my software be considered a "derivative work" or a "collective work"? If my software is considered "derivative" than by definition of GPL, I *MUST* distribute my software under GPL, which can severely affect the commercial viability of my software.

So, who decides what is derivative vs collective? Unfortunately, most likely it will be lawyers rather than software developers. And that may very well cause at least some amount of concern amongst those who build commercial (i.e. non open source) software using Java for a wider distribution (i.e. for use outside of one's organization, most probably selling commercially licensed copies of software to customers).

Now it is probably unlikely that Sun will come after commercial software developers who use Java claiming their software fits the category of "derivative". However, if any organization has to spend any legal resources to investigate this, it affects the use of Java in commercial space.

I think Sun should make it very clear to the community why they chose GPL, and how they want to assure the community (both open source and commercial) that their software development efforts in Java are not subject to the "viral" aspect of GPL.

UPDATE: CLASSPATH EXCEPTION TO THE RESCUE

After digging through more details, I found 2 things that address my concern:

1. Sun is distributing Java under GPL with the ClassPath exception, and describes in their FAQ the reasons behind this -- with an example almost the same as I posted earlier where the classpath exception allows me to distribute my software written in Java bundled with a JDK or JRE without requiring GPL:
It allows you to link an application available under any license to a library that is part of software licensed under GPL v2, without that application being subject to the GPL's requirement to be itself offered to the public under the GPL.

2. Following one anonymous commenter's suggestion, I read the original GNU Classpath license text , and it does clearly describe the exception:
As a special exception, the copyright holders of this library give you permission to link this library with independent modules to produce an executable, regardless of the license terms of these independent modules, and to copy and distribute the resulting executable under terms of your choice, provided that you also meet, for each linked independent module, the terms and conditions of the license of that module.

Still, my suggestion to Sun still remains the same: you can't assume everyone will go through the trouble (granted it wasn't much) of digging through the details of the classpath exception, and as such, they are likely to remain under false assumptions re GPL restrictions on Java. So, don't assume -- everytime you mention Java and GPL, make sure you talk about the classpath exception and how it enables you to distribute your software written in Java under any license.

Tweet