Another New Beginning

This week I completed my tenure as an employee at Responsys and started my new venture OpenI -- a company that provides open source business intelligence software and services to businesses that want to be data-driven in their operational strategy.

I guess you can call me a serial entrepreneur now, since OpenI will be my fourth startup -- last one being Loyalty Matrix, which was acquired by Responsys in 2007. I am happy to say that the marketing analytics technology we built at Loyalty Matrix found a way to express itself as Responsys's own analytics product Interact Insight. It was interesting to see the formal structures it requires in a more established company to release a product -- valuable lessons that I'll surely apply in future product releases. It is also great that Responsys will remain a client of OpenI, so that we can advance this technology in a mutually beneficial fashion (and also that OpenI has a few clients from the get go :-).

OpenI will partner with Codemandu, a software development company in Kathmandu, Nepal that has provided the engineering help for OpenI in the past. Codemandu will help us deliver support and integration work for our clients. So -- if you have software projects in business intelligence, reporting, and/or analytics (or know of someone who does) -- we are here for you :-) Basically, if you are an on-demand company that stores transactional data for your customers, we can help you build an on-demand analytics product based on OpenI -- something you can private-label and up-sell to your customers.

So, needless to say -- next couple of months are going to be crazy, and pretty exciting. Personally, I have a lot of pent-up ideas on making BI more accessible and actionable, and we will be toying around with these ideas in OpenI. And given the nature of open source, these experimentations will happen in public domain -- and so you'll see some fun stuff appear on this blog and OpenI site.

The BI landscape has definitely evolved since OpenI started back in 2005. Most of the big guys (Busienss Objects, Hyperion, Cognos, SPSS) have been acquired by even bigger guys (SAP, Oracle, IBM). On the open source BI side, Pentaho and JasperSoft have done a remarkable job in leading the sector. Plus there has been a great deal of movement in on-demand BI as well - with Swivel, GoodData, and PivotLink, and also at desktop level with Tableau. We will definitely give our best shot to stand on the shoulders of these giants and raise the bar a bit differently.

I recall Sting (lead singer of The Police, for the benefit of our younger readers) say this in a Rolling Stone interview once when asked about his unique singing voice -- something like "Nobody can sing like me -- I'm not saying that I have the best voice in Rock 'n Roll, it's more like someone can sing better or worse, but they can't sing exactly like me"

So, this I can say -- OpenI will be unique in its approach to BI. Stay tuned..

cheers,

Sandeep

Tweet

Data Visualization is about Telling a Story

First off -- Hans Rosling is an inspiration to us all in the business of analytics and data visualization. Not only this story is extremely relevant, but the way he shows the numbers -- there is a lot to learn. I will make an attempt here to deconstruct his latest TED talk in terms of what a good BI tool show do, and also how this is a great use case of how great BI users behave:



BI Features Used by Hans Rosling:

• The most prominent is the use of Time as a special type of "dimension". The tool knows that Time will support the concept of a Play button. This is still very novel -- most BI tool, OpenI included, treat Time as any other dimension -- you can drill up, drill down, set date filters, or date ranges -- but that's about it. Taking a lesson from here, what we should do instead is that the moment there is a Time dimension, user should have the option to "superimpose" Time in "Play" mode within a given analysis -- this should result in a Video Player like slider widget appear at the bottom of the analysis with a big old Play/Pause button next to it


• Notice how he first presents the data bubbles in dual-axis graph and then transitions it over to a map view. This makes the concept of "background canvas" a dynamic entity for presenting data. How many other choices a user can have (in addition to dual-axis and map overlay) to use as the context in which the data should be presented


• He keeps only 1 attribute per axis - country in X-axis, and % of population with HIV on Y-axis, and everything else (gender, per capital income, etc.) is treated as a filter (in OLAP speak). This keeps the visual very clear on its message. I have often struggled with OLAP based analyses, which have multiple dimensions on each axis, which makes sense sometimes in the table view, but the chart-view is completely horrid. Single data attribute per axis is a way to address that


• When it comes to drilling further into data, he basically clicks on a country bubble -- and it can either split by income groups, or only the specific country goes on a time play motion while others stay the some, etc. -- the key for me here is that drilling down is best done at the visual level -- somewhere on the chart/graph itself the user should be able to isolate a data group (in this case a country bubble), and have a choice on drilling down or move it back and forth in time

Hans Rosling as a BI User/Presenter

• Emotion, emotion, emotion... he is so far away from the stereotype of a statistician making a presentation. He cares about what he's presenting. The numbers are real people -- they get sick, and they can either get better or they can die.. you can feel that empathy as he presents.


• Al Gore did this first (that I can recall) in The Inconvenient Truth when he brought a crane ladder to hoist him up so he can point to the tallest bar in the chart that he is showing. Maybe a bit too melodramatic -- but it drives the point, and also makes a more visceral connection with the data. Hans Rosling stands on top of a table at the beginning of the presentation to explain the different numbers he is presenting, and the audience is at once connected and engaged


• His bringing of the long metal pole to point to the numbers instead of your generic laser pointer ("I have solidified the laser beam") is another way to get more personal and physical to show how involved he is


• Ultimately he has leverages the BI tool to make a presentation, to tell a compelling story. Earlier in my career, we worked on a feature with another BI tool that automatically generated powerpoints from its charts. Yes, it was pretty crude, and didn't really work that well usabilitywise -- but the point is, this was definitely a feature aimed at helping users build a story off the various charts and grahps and analyses. People want to tell a story -- the BI tool should help them do that.

Ultimately, watching Mr. Rosling is definitely inspirational -- I can only hope that OpenI will one day does the things he's shown us in this presentation. I'm sure we will get there in due time, but it is the spirit in which BI tools are used, and their ultimate message.. that's the important thing to keep in mind as we move the product forward.

Tweet

BI 2.0, Next Generation BI, and Everythig New and Improved

My fellow blogger Bhupendra Khanal has an interesting post that mentions the challenges associated with BI 2.0/Information 2.0 (he also plugs OpenI, my open source project, which is much appreciated -- Bhupendra, may OpenI karma come back to you thousand-fold :-) 

Software industry, not unlike any other, contains a lot of hype and probably sometimes even more so with all this 2.0 buzz, which probably seems cool to the industry insiders, but is definitely confusing to the market.

Take BI 2.0 (or Information 2.0) for example - what in the world does it mean? Well, turns out, at the end of the day, to most BI vendors, it means more fancy charts and graphs and dashboards, except this time they'll have rounded corners, larger fonts with brighter colors, and maybe a fit of Flash and/or Ajax thrown in for a good measure to demonstrate live interactivity.

All this is fine and well, but all this is also pure BS if you are not helping your user make better decisions, or informing them of something new.

If BI 2.0 or Information 2.0 is to be seen as the "next generation" (it seems you cant' escape these cliches), then it needs to go beyond charts/graphs/dashboard paradigm. BI applications and tools need to be rooted in the knowledge worker's workflow - and should be cognizant of the types of decisions that need supporting. BI needs to be aware of the domain context - i.e. which industry are you supporting? which area - marketing, finance, operations, research..? Because without this, the best BI can do is to provide nice visuals and hope and pray that the user knows how to translates them into intelligence and action.

But software can be better than that if is stops being lazy. And that's my hope with our work in OpenI. We certainly started in the charts/graphs/dashboards paradigm, so we are as guilty as anybody. But as they say in any 12-step plan, "acceptance" is the first step -- and now, we are moving towards a future of BI software that caters to the root need for intelligence -- i.e. not only that you see your data clearly, but you also see it in your specific business context, and get immediate option to act upon it.

For e.g. a marketing analytics BI application - once it incorporates the data about customers and marketing campaigns and resulting purchases -- should not wait for a user to define dashboards and reports, but rather already provide a suite of analyses that answer the most typical marketer's questions - i.e. how effective are my campaigns , who are my best customers/prospects, and what tactics work the best for individual customer segments? And don't stop there btw -- if you have identified some new and interesting customer segments - it should integrate with an online campaign manager tool to immediately launch new campaigns; or publish the list of your most valuable customers to your e-commerce engine or call center platform which know how to treat them in a special way; etc. etc.

Similar scenarios can be applied to other industries and domains too. We as BI software developers just need to imagine differently. And that's what 2.0, new version, or next generation is all about - next level of imagination.

Tweet

Taking BI Beyond Charts and Graphs

I attended a talk at the monthly BI SIG meeting at SDForum by Christian Marcazzo from Spotfire, now a part of Tibco. I have long admired Spotfire's innovations on data visualization front, so I was curious how they see BI from the whole Human-Computer Interaction (HCI) aspect, and couple of things stood out.

First - if we look at consumer-centric data applications (Zillow, Google Finance, etc.) and compare their interfaces to more traditional enterprise BI applications, it's amazing to see how the latter just doesn't even attempt to look good.

Why is that?

Because enterprise BI app developers aren't under the same pressure to seduce their users like consumer data apps. Zillow, Google Finance, et al live and die by the community they create, so for them, user experience in paramount, and it shows. Most BI apps, on the other hand, are almost developed under the assumption that users are under a “thou shalt always use this BI software” executive order, and as such don't have much leverage in rejecting a software based on poor or sub-optimal user experience.

So they begrudgingly use the BI software for its least interesting/effective use - churn out one report after another. The BI app basically becomes a report production factory.

That brings me to my second point - for BI to be more than charts/graphs/dashboards, it needs to be part of the user's workflow. Now the term "workflow" means a lot of different thing to differnt people, and has recently become a popular box in BI markitechture diagrams - but to me, it basically means that BI app needs to know the various contexts under which its users are using it, and provide a way to add intelligence/insight to the process. BI app by itself should almost be invisible.

Zillow users think of themselves as a home buyers/sellers, not a real estate data analysts. Typical Google Finance users are checking out their portfolio and evaluating stocks, but don't think of themselves as financial analysts. So, why in the world BI applications are hell bent to think of their users are data analyst first, rather than understanding the specific tasks they are trying to accomplish more intelligently?

That's where workflow comes in. BI app needs to understand the nuances of the business domains their users are in, make intelligence available in their task workflow where it's needed, and provide a clear way to act upon that intelligence. Too often we think of BI as a separate app where a user will do analysis, and then the users will jump to other apps where they can take actions -- that's now how users see the world. And without understanding the users, BI apps can't really provide intelligence.

It's time to turn this model around. BI apps should think more like mashups --pull data from any "public" repository with REST like API's, make anlayses available to share and tweak, and make the resulting insights be integrateable to other apps. The more lines get blurred between BI apps and the rest, more successful its adoption is going to be.

Tweet

What is the Cost for Different Phases of Outbound Marketing

A colleague recently sent me an email asking:

I'm trying to find out the cost/spend associated with different phases of outbound marketing campaigns. At a high level, I'm trying to understand the process as,

• Idea generation/ Message theme discussions (e.g. what is the campaign all about)
  
• Associated content generation (web site promotion, hard print material, email
  content generation...in summary, (creative + message) generation  )
  
• Outbound execution : actual delivery, publishing of hte message

Can you provide guidance as to,

1. If I missed any major step(s)
2. What % of total cost will be allocated to each of the above steps.....here if you can add the vertical (retail, hitech software, hi-tech mfg etc), it would help me more

Not that I'm an expert, but my response was as follows - see if you agree or better yet, can add in your 2 cents:

I think you have identified the key themes. I tend to think about outbound marketing in the following categories

Target

• Who will you contact? who is your audience? what is your access to that market? If you want to  go direct (email, direct mail, telemarketing), how are you going to obtain contact information
  -- homegrown lists, purchased lists?
• Is there a segmentation strategy applicable? If so, what are you costs/efforts to define/implement it?

Message

• Once you know who you will reach out to, you need to craft your message. This involves figuring out the creative for each media (email layout, direct mail layout, video or radio ad, etc.) and producing it
• Different elements of the message - creative content, offer, promotion, etc.

Execution

• How will the message get out? What are the different media channels? Are you going to work with an agency that can manage all channels, or do it yourself?
• How will you co-ordinate the different channels? e.g. someone who got an email offer ends up calling your telemarketing center, are they all in sync?
• How well are you able monitor your campaigns in progress and how quickly can you respond to feedback?

Optimization

This is more around anayltics, but a critical part (of course I'm biased :-) -- which is to look at the operational metrics of all campaigns and optimize mainly for 2 things - determine the most profitable/relevant segments and for each segment, figure out the optimal contact strategy

Cost-wise, execution will be the biggest chunk, probably 50-60% of overall cost, closely followed by "target" (acquisition of contact information or markets).  Rest is probably evenly divided.

Tweet

Polarization of BI Solutions

It almost seems like the big BI companies figured out they had to be a part of something even bigger to justify their already sky-high license costs. Just within last 12 months, we've seen Hyperion get bought Oracle, Business Objects get bought by SAP, and now today Cognos got bought by IBM.

And I can't help but feel that Enterprise BI software has become a dying breed.

If you are a fortune 1000 type of organization, chances are you already have some sort of elaborate licensing agreement with at least one of these acquirers (Oracle, SAP, or IBM). So now, they have one more thing to sell you so that you can have a check mark for your BI initiatives without having to drive too far to a different vendor.

I guess I shouldn't be so cynical. As BI software, Hyperion, Business Objects, and Cognos are pretty feature-rich, and have many compelling qualities. My problem is with their complex enterprise deployment model, proprietary nature, and staggering cost of ownership that continues standing in the way of making BI available to the masses. And their recent acquisitions further polarizes the world of BI haves and have-nots.

Perhaps this is how corporate world will come to demand a new breed of BI solutions. Back in 2002, when we started our marketing analytics company, we thought (and still do) open source and software-as-a-service were going to drive this new trend. And it's good to see companies like JasperSoft, Pentaho, Swivel, and LucidEra lead the way in this direction. What's more important than just looking for open source and/or SaaS solutions to BI, is to demand "openness" -- openness in terms of architecture, sharing of insights, and just as importantly, openness about cost of ownership. How many times you have seen a BI initiative where the entire budget got spent just on data integration and/or cleansing? How about the cost of those special consultants who seemed to be the only earthlings that understood the esoteric workings of a popular yet proprietary BI software package?

Above all, one needs to keep in mind that BI is about leveraging all available data to get a clearer picture of what's going on in the business, be able to focus on the most relevant issues, and make better decisions. Instead of taking a centralized approach of one system doing it all, a good BI system today needs to act more like a network that can connect to various data sources, systems, API's, web services, etc. What if your BI system acted more like a mashup that lets you combine compatible information sources and cross-reference them as you please? Swivel has an approach close to this except that it expects all information to be uploaded by its users. It could be interesting if Swivel could connect to some common public data repositories (like geographical locations, weather, stock prices).

The key here is to have BI systems that enable mashup of information resources and analyses, a la web 2.0, as opposed to being traditional "enterprise" solutions that need your business to bend over backwards to fit into their proprietary framework and terminologies. Which to me conjures up images of Oracle, SAP, and IBM.

Maybe I liked them better when they were Hyperion, Business Objects, and Cognos. At least they didn't say -- "BTW, we also do BI".

Tweet

Commoditization of Business Intelligence

I received an email from Seth Grimes, a very familiar voice in the BI community, with an interesting question -- Do you now see BI as a commodity market?

He was referring to an earlier post about my open source BI project OpenI, where I'd mentioned:

The state of business intelligence software market has been very much controlled by a few big players. The situation is very similar to how the J2EE application server market was before JBoss, or how database server market used to be before MySQL and Postgres emerged as serious alternatives, or how OS market was before Linux. Pretty soon we will talk about the BI platform market in the same manner, because open source and open standards are driving the commoditization of BI as we speak. It is just a matter of time.

That was a few years ago. So, I asked myself -- well, how do I feel now? Have I learned anything?

The question is a tough one -- something I've always grappled with. Ultimately, it depends on what do we mean by "business intelligence". If we go by the current big commercial players' definition -- then BI is more about a software tool providing capabilities around data warehousing/ETL, OLAP, analytical modeling, and visualization. So by that account, I'd definitely stick to my original thoughts and say it's a commodity market.

However, a more relevant question might be -- does having these capabilities make a business intelligent? In reality, what I've seen is that it comes down to an analyst (or group of analysts) who (a) know how to work a "BI" tool, and (b) have some fundamental expertise in the business domain they are analyzing. So, the "BI" tool is more about facilitating the job of an analyst or a general business user. You could argue that by making performance metrics, etc. more easily accessible to a business user, the BI tool is helping them make more effective decisions, but it is making a big assumption that the user knows how relevant the performance metrics are for the business.

In the end, my take on this is that, the most effective BI tools are domain-centric, i.e. they embody some inherent knowledge about a particular business domain -- so, not only they are extremely efficient and accurate about compiling all the performance metrics and making them available, they also "understand" the applicability of those metrics and can almost act like expert systems in guiding crucial decisions. This, I don't think is a commodity market. It needs to be grounded into specific industry domains to be effective.

Would love to hear your thoughts.

Tweet

OpenI Listed in Top 10 Free BI Apps

Ask anyone who is involved in an open source project, getting recognized is one of the greatest kicks you get out of the whole deal.

So, when my Google alert picked up this news on OpenI listed in the Top 10 Free BI Apps list, it absolutely made my day. Of course, a lot of the credit goes to all the folks who have contributed to this project, and the open source community that has supported us all this time. And thanks to Tamina Vahidy for recognizing the project.

When we started OpenI back in July 2005, we just wanted to subsidize our R&D. We needed a BI platform to deploy our analytical models, and instead of opting for commercial BI platforms which would have never fit into our cost model, we decided to develop a BI platform using available open source components, and also as an open source project of its own. We figured if we get even a couple of people outside of our company to pitch -- whether it was design help, or just thinking through requirements, use cases we hadn't encountered -- that alone would pay for the efforts to make it open source.

Well, not only we got design help and advice from a great deal of smart folks in the space, we even have people contributing code. I remember someone (maybe Steve Weber) making a point about open source development model -- not all the smart people in the world work for you, so the only way to get them involved in your projects is via open source (ok, you may argue crowdsourcing ideas such as Netflix's contest, but I don't have a $ 1 million to give away in prize money :-)

So -- here we are -- working on version 1.3 of the product. We are using it internally as the web front-end of our commercial product. Of course, it has ways to go (see roadmap) -- but as contribution and recognition keep coming in, it just seems like a much more rewarding way to develop software.

Tweet