OpenI Listed in Top 10 Free BI Apps

Ask anyone who is involved in an open source project, getting recognized is one of the greatest kicks you get out of the whole deal.

So, when my Google alert picked up this news on OpenI listed in the Top 10 Free BI Apps list, it absolutely made my day. Of course, a lot of the credit goes to all the folks who have contributed to this project, and the open source community that has supported us all this time. And thanks to Tamina Vahidy for recognizing the project.

When we started OpenI back in July 2005, we just wanted to subsidize our R&D. We needed a BI platform to deploy our analytical models, and instead of opting for commercial BI platforms which would have never fit into our cost model, we decided to develop a BI platform using available open source components, and also as an open source project of its own. We figured if we get even a couple of people outside of our company to pitch -- whether it was design help, or just thinking through requirements, use cases we hadn't encountered -- that alone would pay for the efforts to make it open source.

Well, not only we got design help and advice from a great deal of smart folks in the space, we even have people contributing code. I remember someone (maybe Steve Weber) making a point about open source development model -- not all the smart people in the world work for you, so the only way to get them involved in your projects is via open source (ok, you may argue crowdsourcing ideas such as Netflix's contest, but I don't have a $ 1 million to give away in prize money :-)

So -- here we are -- working on version 1.3 of the product. We are using it internally as the web front-end of our commercial product. Of course, it has ways to go (see roadmap) -- but as contribution and recognition keep coming in, it just seems like a much more rewarding way to develop software.

Tweet

Is Open Source Java More Restrictive under GPL?

Today will most probably go down as an important day in the history of software engineering -- Sun open sourced Java under GPL license! (Sun's CEO Jonathan Schwarz's blogs on it as well)

While this is a very welcome news, the choice of GPL as the license has me a bit concerned. Two years ago, I was researching the various open source licenses like mad for our own open source project OpenI, and decided against GPL because of its "viral" nature (we ended up with Mozilla Public License, MPL). The "viral" aspect of GPL (section 2(b)) says:

You must cause any work that you distribute or publish, that in whole or in part contains or is derived from the Program or any part thereof, to be licensed as a whole at no charge to all third parties under the terms of this License.

and later further clarifies:

..it is not the intent of this section to claim rights or contest your rights to work written entirely by you; rather, the intent is to exercise the right to control the distribution of derivative or collective works based on the Program.

In addition, mere aggregation of another work not based on the Program with the Program (or with a work based on the Program) on a volume of a storage or distribution medium does not bring the other work under the scope of this License.

When someone uses programming language software like Java to build their own software, it is tricky to determine if their work is "derivative", "collective", or a "mere aggregation". In such a case, all the software code is written in Java. The software requires a Java Runtime Environment, which acts like a virtual operating system, to run. So, Java as a software construct becomes very tightly coupled with each new software program built on Java.

This is a contentious issue where there has been a lot of debate. See this blog entry at O'Reilly and its comments back in 2004

For example, with Java being GPL now, what happens when I build a software program in Java and would like to distribute it along with a Java runtime engine (JRE) for user convenience? Since I have built the software using Java and I am combining the JRE in my distribution, will my software be considered a "derivative work" or a "collective work"? If my software is considered "derivative" than by definition of GPL, I *MUST* distribute my software under GPL, which can severely affect the commercial viability of my software.

So, who decides what is derivative vs collective? Unfortunately, most likely it will be lawyers rather than software developers. And that may very well cause at least some amount of concern amongst those who build commercial (i.e. non open source) software using Java for a wider distribution (i.e. for use outside of one's organization, most probably selling commercially licensed copies of software to customers).

Now it is probably unlikely that Sun will come after commercial software developers who use Java claiming their software fits the category of "derivative". However, if any organization has to spend any legal resources to investigate this, it affects the use of Java in commercial space.

I think Sun should make it very clear to the community why they chose GPL, and how they want to assure the community (both open source and commercial) that their software development efforts in Java are not subject to the "viral" aspect of GPL.

UPDATE: CLASSPATH EXCEPTION TO THE RESCUE

After digging through more details, I found 2 things that address my concern:

1. Sun is distributing Java under GPL with the ClassPath exception, and describes in their FAQ the reasons behind this -- with an example almost the same as I posted earlier where the classpath exception allows me to distribute my software written in Java bundled with a JDK or JRE without requiring GPL:
It allows you to link an application available under any license to a library that is part of software licensed under GPL v2, without that application being subject to the GPL's requirement to be itself offered to the public under the GPL.

2. Following one anonymous commenter's suggestion, I read the original GNU Classpath license text , and it does clearly describe the exception:
As a special exception, the copyright holders of this library give you permission to link this library with independent modules to produce an executable, regardless of the license terms of these independent modules, and to copy and distribute the resulting executable under terms of your choice, provided that you also meet, for each linked independent module, the terms and conditions of the license of that module.

Still, my suggestion to Sun still remains the same: you can't assume everyone will go through the trouble (granted it wasn't much) of digging through the details of the classpath exception, and as such, they are likely to remain under false assumptions re GPL restrictions on Java. So, don't assume -- everytime you mention Java and GPL, make sure you talk about the classpath exception and how it enables you to distribute your software written in Java under any license.

Tweet

"All marketers are liars". Seth Godin is a marketer. Hence?

Sorry for the cheeky title, but Seth sort-of ticked me off today with his post "Nobody Knows Anything".

First off, as someone who makes his living providing marketing analytics, I get a bit, let's say annoyed, when someone like Seth starts a blog post with "There are two kinds of marketing analysis, both pretty useless".

I calmed down a little bit when later in the post he conceded "Here’s the really good news: in addition to analysis, marketing today offers something that actually works: a process".

But his post in general has an attitude that says "Marketing is not science", and that "most marketing breakthroughs come down, sooner or later, to luck".

Well, I am not a guru like Seth (or as "lucky" in terms of having a couple of bestsellers under my belt), and he definitely knows a lot of things I don't -- but this is my blog :-) -- so, I will say that marketing is just as much of a science as social science is. It may not have equivalents to Newton's law of gravity or Einstein's theory of relativity, but marketing analytics does have some tried and tested ways to leverage data to make smart predictions about future behavior of customers and prospects.

Now, once equipped with the intelligence that marketing analytics provides, it is completely up to the marketers on how successfully they can change their strategy and tactics to yeild results (so, marketing is only partially scientific) -- but marketing analytics will almost always put a marketer somewhere above pure dumb luck.

Of course, you may agree with Seth, or not -- but I just had to get this off my chest.

Tweet

Marketing Analytics for Salesforce.com

This week and last, we've had 2 good conferences in San Francisco, very relevant for marketing analytics. Last week it was DreamForce'06 from salesforce.com, and earlier this week, we had DMA'06.

Let me talk about DreamForce first because it has become a fad these days (specially for anyone working in BI or SaaS) to integrate with salesforce.com using AppExchange. Since early this year, I have been eagerly trying to find an angle between our business and salesforce.com, so DreamForce'06 was obviously very relevant.

A good friend of mine from college, John Barnes, was at DreamForce'06. John is the VP of Technology at Model Metrics, a Chicago-based firm that specializes in customization and integration of salesforce.com with legacy systems. I pinged John to find out more about what salesforce.com is actually doing about marketing automation and analytics, which was very insightful (thanks John).

Salesforce.com website describes marketing autmation as a key component of their platform, which also includes marketing analytics. Turns out while it is possible to define and manage campaigns using salesforce.com, the marketing analytics bit only provides some very basic reports.

So my idea is fairly simple. If people are actually running high-volume direct marketing campaigns using salesforce.com, then we will write an AppExchange component to suck in all the campaign and customer data into our platform and then provide much more sophisticated predictive analytics and other fun stuff on our platform -- make a lot of money, retire early, speak at next DreamForce (sorry I get carried away with this vision thing).

Anyway, about 10-20% of SF customers are using the marketing functionality today. The SFA is the main use but marketing is growing more and more. But 10% of 22,000+ customers is still a good market to go after. And the marketing analytics (and reporting) on SF is lacking. The main obstacle to making it better is that their API does not have a join capability so the only way to do better reporting today is to have a copy of SFDC locally and use replication software to keep it up to date (by DBAmp or Relational Junction on the AppExchange).

Sounds like a good opportunity.

So, overall I am happy with DreamForce'06. I had a chance to shoot breeze with an old college buddy, and learn about a very feasible way for us to get into the AppExchange game. Now I need someone who will go in on this with me as a "design partner" :-)

Tweet

Better Blogging by Chemistry

Elliott Ng, a friend, ex-colleague and fellow blogger, has put an interesting commentary on Top 10 tips from (to) a novice blogger posted by Avinash Kaushik.

My take on it is that a blog's success depends on how effective it is in starting conversations -- which means you either get comments like this, or someone links your post on their blog expanding on the topic -- or simply email the link around with some comments.

Why would someone do that?

Well, only if they actually care about what you write about. And caring is more of an emotional response rather than intellectual one. Scoble, Doc Searls, et al really stress on "having a voice", which happens when you combine passion and get awawy from corporate-speak, IMHO.

Scoble's point on being easy to find is also important -- but instead of going the SEO route, it is more important to find interesting conversations in the blogosphere and participate. If you are an active participator with a unique and compelling voice, the search engines are bound to pick you up.

Personally, I found it helpful to write a post outlining my reasons for blogging -- and to the rest of the bloggers out there, novice and experts alike, I'd love to lob the question -- what have you found to be most effective at starting conversations? Was it different than what you'd initially expected?

Tweet

How to win $1 Million from Netflix?

Fellow blogger Michael Fassnacht noted rightly that Netflix has caused quite a stir for marketing data geeks with their recent $1 million prize offer for "substantially improving" their existing Cinematch algorithm to make more accurate predictions of "how much someone is going to love a movie based on their movie preferences".

Call it "crowdsourcing", or harnessing "group smart" -- the approach is intriguing, and one of a kind. Being a curious soul myself, I decided to register a team from our company to check this out (who knows what may happen? we can be smart sometimes with enough luck :-)

A few interesting facts:

• The contest is actually slated to go another 5 years until 2011, the bar being raised each year to improve over last year's winner
• A fine but important distinction: the algorithm needs to predict how someone will rent a movie, NOT what movie someone will rent
• At first glance, the data provided by Netflix seems pretty "skimpy" in terms of richness. Basically you get:
• List of movies
• List of ratings assigned for each movie by an extensive list of Netflix members
• My first reaction was that having extra information on the movies themselves might help. There's a bunch of stuff available from IMDB . However, apparently there are license restictions and also Netflix doesn't really consider extra data to be valuable in improving their algorithm (see the discussion thread )

The "enjoy the journey, not the destination" mantra may be apt for this contest. As you can see on the discussion forum on netflix , this process has invited all sorts of interesting conversation on the validity of approaches, whether Netflix has provided enough data, why should one even bother, etc. etc. -- a dream peer review IMHO, albeit a bit too noisy. So, Netflix should be getting a lot more than their money's worth via this process -- not just by getting better algorithms and the PR buzz, but also by leveraging an almost open-source-type process to involve external community for their internal R&D.

At the moment, I agree with Michael's assessment that trying to solve this with ratings data alone might not be the best way to go. There seem to be so many other interesting dimensions that should influence somone's movie rating: movie characteristics like the cast, director, etc., review from critics, local media review, geo/demographic information about the Netflix member, among others. None of these are being considered in the current algorithm. I can understand Netflix's hesitancy to interface with 3rd party resources, but perhaps they should make all the datapoints within Netflix's movie database available for this contest -- and second, encourage contestants to add their own qualitative datapoints. If the goal is to approach this as a pure improvement of a data mining problem -- then increasing the depth of data should help.

I'll keep you all posted how far we get on this. Being a small company, we will do this in the copious amount of spare time left over after working on existing client work that pays the bills. Still, it should be a lot of fun.

Tweet

An Asterisk that Cost 2 Million Dollars

This just in: a colleague of mine pointed out an analysis that showed a sudden spike in the number of new trial subscriber signups for one of our clients. In early 2005, they had just introduced a new product version and through mid 2006, they were averaging around 3,000 new trial subscribers each week -- which was less than half of what they were getting with the previous product version.

What had happened?

Turns out this client had several product feature descriptions listed during the trial sign-up process. One of these product descriptions happened to have an asterisk next to it, which was explained at the bottom of the page saying it required a credit card number upfront to use it. For a casual observer, it wasn't clear to which product feature the asterisk actually belonged. So, during a content review session, someone caught this and said -- wow, people are looking at our new product and they think they need a credit card to subscribe, which they really don't, and they get very hesitant.

So, in July, a quick content change was made. The asterisk was removed.

Since we are tracking all the transactions, within 7 days after this change, we started seeing a sudden spike in new trial subscriptions, which has levelled off above 7,000 new trial signups a week (for now). More than double of what was happening prior to the removal of the infamous asterisk.

My colleague did a quick calculation on revenue impact, and it basically translated roughly $2,000,000 in additional revenue in the coming 12-months because of the asterisk removal.

I hope you don't have a similarly expensive asterisk ANYWHERE on your website.

Tweet

Can Analytics Influence Direct Marketing Creative Process?

Direct Marketing Association (DMA's) Northern California chapter hosted its first independent meeting yesterday (Thursday Sep 21st) at Intuit's campus in Sunnyvale. This summer, the national DMA "abandoned" formal support for all its local chapters asking them all to go on their own. It's good to see that Northern California DMA has managed to make this independent start, hopefully they'll get adequate local support to thrive.

The main speaker was Bill Mirbach, VP of Direct marketing and direct sales at Intuit -- presenting a talk titled " Owner's Manual for the Creative Process". Now, I'll be the first admit that professionally direct marketing creative is the last thing we deal with -- at our work, we leverage direct marketing data for predictive analytics, so while we can measure and predict if creative version A will perform better than version B for a target audience, that is very different from the "creative process" itself. So, I was intrigued with the topic.

Bill has been around the silicon valley high tech industry. The highlight of the talk for me was a story he shared going back to 1984 when he helped the founder of a fledgling software company called Intuit with their direct ads. The talk was mostly about how companies should chose vendors and vice versa for the creative process, not the artistic aspect itself -- so, during Q&A, I asked Bill how does knowledge of your audience impact the direct marketing creative process? Can one leverage their direct marketing data, knowledge of their customer segments, etc. to make the creative process more effective? What's been his experience?

Here's what I heard:

Yes, a better knowledge of one's audience and their likes and dislikes about an organization's products and services certainly helps the creative person to craft their message more effectively? However, while this sounds logical, this is not what normally happens. The creative process is more the product of the discipline and idiosyncracies of the "creator", rather than driven by data-derived intelligence.

Bill mentioned a particular test where they wanted to measure the performance of a scare-tactic message ("if you don't use our product, you'll be sorry") versus a benefits-focused message ("and you can get X, Y, and Z at the click of a button") -- where the creative person just didn't believe in scare-tactic and came up with a very tepid "scary" message (which obviously didn't perform well). Whereas he used a different creative person who specialized in scare-tactic message (scary thought, pardon the pun) -- which turned out to be very effective.

What does this tell us?

I don't know how much of Bill's story is the norm or exception, but he certainly has been around direct marketing creative people a lot more than I have -- so I must respect his POV. Still, it seems like rather than asking creative people to use marketing data driven intelligence to fine tune their message -- it's probably better the other way around -- i.e. leverage the analytics to find out what type of messages you want to be send out to different segments -- THEN find the right creative team to craft those messages.

What do you think?

Tweet

Passion for Data Visualization

I found this on Christopher Ahlberg's blog, and I completely agree that this is a true display of "passion for information visualization".

http://www.ted.com/tedtalks/tedtalksplayer.cfm?key=hans_rosling&flashEnabled=1

The software used for the presentation is at http://www.gapminder.org and is also a google tool (http://tools.google.com/gapminder) It appears to be "bundled" with the global economic data, not sure if there is an open decoupled version that one can point to their data and play around. Although it seems flash-based and pulling data from static data sources (didn't seem like rdbms, but I could be wrong) -- but this would be a great way visualize OLAP data.

What's interesting is the concept of a "play" button for the time dimension, which makes a great use of animation to see how different quantities (measures) change over time. It also manages the screen real-estate well to put different dimensions on X or Y-axis. But most of all, this truly exemplifies what data visualization is all about -- it goes beyond the realm of charts and graphs that take a while to decipher, and rather tells a very clear, compelling, and visual story. Very impressive!

Tweet

We've got charts and graphs to back us up, so f@#$ off!

Recently I had a potential partnership discussion to evaluate whether our predictive analytics technology could provide key insights from this potential partner's (I'll call them company XYZ) marketing database. Here's how my conversation went with Mr. X, an exec at company XYZ, (which is a marketing technology company):

Me: "So, what are they key business pain points for your clients that we can analyze?"
Mr. X: "Well, you know, the usual stuff -- marketing ROI, cut costs, increase sales, etc."
Me: "ah.. yes, but can we delve a bit deeper? Where exactly clients' marketing programs need help?"
Mr. X: "what do you mean?"
Me: "Well, are they more worried about increasing acquisition volume, or is it more about predicting high-LTV customers, or is it more about retention? I'm trying to get a sense of what is their #1 issue?"
Mr. X: "they don't know.. it's probably all of that stuff"
Me: "It's important that we get a sense of priority, because otherwise we are talking about applying analytics without really knowing what we are trying to optimize"
Mr. X: "well, you are the analytics expert -- you need to tell them what to analyze. They don't think like you, worrying about success metrics, etc. They ask us to run marketing programs, and now we'd like to sell them some analytics. I can tell you what data we have on their marketing programs, now you tell me what kind of analytics you can provide me that I can sell."
Me: "But we do need to understand their business objectives before determining what analytics is relevant enough so that they'll pay for it"
Mr. X: "What I need from you is some screenshots --- some charts and graphs that show what kind of analytics you can do -- I'll be more than happy to review that and tell you if we can work together"

Needless to say, I didn't sense a true spirit of partnership here, but I did sense an attitude that I find more often than I'd like that analytics is all about producing charts and graphs that the user will somehow find useful.

Which, IMHO, is total BS!

But I can't blame Mr. X too much because this is a pretty common perception of analytics in the marketplace. Recently I talked to a marketing exec who said --"everytime I meet with the analytics guys from our agency, they basically have this big ream of a powerpoint deck filled with one chart after another -- and I don't want to see all that stuff; all I want them to do is to tell me what relevant insight(s) did they find, and what course of marketing action would they recommend, and it's like pulling teeth to get them to move beyond charts and graphs and talk about action."

Look at the websites of any business intelligence software provider, or analytical software provider -- and I will guarantee you that you will see a bunch of fancy charts and graphs, and dashboards with enough dials and speedometers to make you dizzy. Somewhere along the line, maybe we have forgotten that the purpose of analytics is to equip us with insights that enable better decision making.

So, first off, the type of analysis being done has to be aware of what type of decisions we are exactly expecting to improve; and second, the result of the analysis needs to be presented in a fashion that is "integrated" into the decision making. Maybe you list your recommended actions next to your charts and graphs, or maybe you somehow highlight the figures and trends that demand attention. My point -- don't leave it upto your user figure out the action based on the fancy charts and graphs, find out what decisions users are trying to make, and provide information that fills in that gap between analytics and actionable insight.

Tweet

Google blogsearch excludes blog entries made on Google's new blog tool

I like the new beta version of google's blogging tool. I was one of those who couldn't switch their blogs on the older product to the new one. So, I just started a new blogsite since I didn't have that many older posting.

But -- to my dismay, none of my new postings on the beta blogger were showing up when you do a blogsearch. So, I decided to do a quick test. I made the same postings from my new beta blogger account to the old bogger account and did a search immediately. And lo and behold, the entries from the old blogger turned up instantly, but not the new blogger.Try this for yourself:

1. Go to blogsearch.google.com
2. Search for "tie global poverty" -- entry of my last blog post
3. Refine search to within last hour (or day, depending upon).
4. You'll see that while my recent post on my old account shows up, my original post on the new blog doesn't.

Why is Google not indexing the beta blogger entries?

Tweet

TiE's session on The Pivotal Role of Entrepreneurship in Addressing Global Poverty

Part of my job is to leverage every networking opportunity to get the word out on the company. With that intent, and also to learn a bit more about social enterpeneurship, this afternoon I drove down to Santa Clara for TiE (The Indus Enterpreneurs) member networking session titled "The Pivotal Role of Entrepreneurship in Addressing Global Poverty".

If you are an enterpreneur living close to a TiE chapter, and haven't been to their events, it's definitely worth checking out (they always have great Indian food at each gathering). Not surprisingly, the crowd at the Silicon Valley chapter is a good mix of mostly Indian enterpreneurs and seasoned industry execs, so it's pretty decent networking. Today was a bit different, because instead of promoting our individual companies, the crowd was more interested in how enterpreneurs can play an active role in tackling global poverty.

Dr. Bill Musgrave from The Enterpreneurs Network(TEN)-Silicon Valley gave the keynote, with many anecdotes on enterpreneurs finding viable business models serving the "bottom of the pyramid". Particuarly striking was his quote of former president Clinton: "I have never seen uneven distribution of intelligence, but always seen uneven distributions of opportunity". Dr. Musgrave argued that to make the grand changes needed to address global poverty, governments and large corporations haven't proven to be much effective, but it is rather the enterpreneurs who are most effective at empowering the poor. His call of action to the roomfull of enterpreneurs was to seek out such opportunities and make a difference.

The session also gave 2-minute "open mike" pitch slots to any interested attendee to promote their company/cause. I was particualrly impressed with Vipin S (?) from Intel who's starting a program in India to enable local farmers to grow crops for biodiesel fuel. Also interesting was Navaneethan Sundaramoorthy's pitch on Association for India's Development's local chapter , which has a collection of development projects that harness the collective resources of local Indian diaspora.

I couldn't pass this "open mike" opporutnity to talk about my friend Mahabir Pun's work in Nepal. Mahabir and I went to the same college in Nebraska. Upon graduation, while I took the more common path of getting a job and later launching start-ups, Mahabir went back to Nepal, back to the same rural village where he grew up, and started a grade school for the local kids deprived of education, and also started several income-generating micro-projects. I should do a post describing his work in detail, but check out Himanchal Education Foundation's web site in the meanwhile, which is the foundation that tries to get financial help and recruit volunteers for the school. It is a very inspiring story of sheer will that is making a difference without much help from government and NGO's who are now finally warming up to this project.

Afterwards, I got a chance to chat with several other folks who are somehow involved part-time with similar social projects, big and small -- and as I write this, I feel a lot more hopeful about these enterpreneurs making a lasting social impact. I sensed a similar drive to make social changes as we have in launching new companies and seeking successful exits. Key is, how effective will we be in pooling our resources and provide helping hands (I should say, empowering hands) to organizations such as Mahabir's himanchal.org. Just like with any new venture, we don't have the answers when we start, but with the right will, we will find a way.

Tweet

Qualitative Data vs Behavioral Data: Who do You Pay Attention To?

Yesterday during a call with a potential client, this topic came up. This is a well-known desktop software company, and they have a unique challenge: Their primary measure of customer loyalty is the "Reichheld Score" aka the Net Promoter Score, which is based on customer responses to a single question -- "Would you recommend us to a friend?". Now, the interesting thing is that while this company is doing rather well in the market, their overall net promoter score is not that great. In fact, their score is lagging behind other comparable software manufacturers.

So, the obvious question is -- why doesn't their net promoter score correlate with company growth? Which metric should they rather measure as a driver of company growth?
As we talked, I also found out that this company hasn't done much in terms of evaluating the behavioral data on their customers -- you know, stuff like actual purchase transactions, new purchases versus repeat purchases, customer complaints, returns, etc. And I couldn't help but think that perhaps this is where they should start look first. A great deal of research work has shown us that past behavior is the best predicator of future behavior, and this is true when it comes to measuring customer loyalty as well. A 2002 HBR article "mismanagement of customer loyalty" describes this as:

Simply put: Not all loyal customers are profitable, and not all profitable customers are loyal. Traditional tools for segmenting customers do a poor job of identifying that latter group, causing companies to chase expensively after initially profitable customers who hold little promise of future profits. The authors suggest an alternative approach, based on well-established "event-history modeling" techniques, that more accurately predicts future buying probabilities. Armed with such a tool, marketers can correctly identify which customers belong in which category and market accordingly.

So why isn't this company looking at behavioral data on its coustomers? I didn't get a clear answer, but could it be that it is easier to conduct surveys rather than dig deep into data, specially when the data volumes are huge and the data is scattered around different corporate silos? Could this company be viewing the analysis of behavioral data as a long, complex exercise that involves getting down and dirty with data warehouses and analytical modeles, when all they needed was a nice simple metric that would measure customer loyalty and nicely correlate with company growth?

Yes, I am being a bit facetious -- but while I don't dispute the value of qualitative research, I think in this case, they are more applicable AFTER an initial study of behavioral data. This company needs to understand the "WHAT" first (i.e. what is going on with my customers? which customer attributes/behavior are best indicators of company growth?), and then they can apply qualitative reserach to understnd the "WHY", i.e. why things are happening the way they are.

Tweet

Would you consider a SaaS/on-demand solution for marketing analytics?

By normal conventions for my job, I'm supposed to answer a "hell, yes!" to this question because I work for a company that does on-demand marketing analytics. But all hype aside, let's explore this and see where it makes sense.

We all know, specially the direct marketers, that data can be leveraged to get a better understanding of customers/prospects which results to more targeted and effective marketing programs. This concept has been around for decades and generally well accepted. For a while, data used to be the challenge where an organization wouldn't have much data on its customers -- but in the age of web, RFID, CRM explosion, etc. -- people have way too much data to know what to do with it. Still then, why are we being bombarded with spam and other irrelevant marketing messages?

A lot of marketers, specially in the SMB sector would answer -- well, it's hard to do this. Analytical solutions to leverage your customer data are tough to implement. It's either some expensive software/hardware (think BI/CRM/Analytics solution providers), or some elaborate marketing service provider (agencies, mailhouses, etc.) with heavy-duty hourly rates, or else you left with your own devices to put together an analytics team who has:

1. Technology know-how to deal with large datasets and apply heavy-duty analytics
2. Business know-how to understand vital issues challenging your business and marketing
3. Strategic approach to analysis to avoid not seeing the forest for the trees

Sounds familiar?

What direct marketers, and a lot of other data-driven decision makers, seem to be lacking is an agile yet reliable set of tools that help them "see through" their data without having to own expensive hardware/software infrastructures, or having to pay for service provider hours throught the nose. And I think, a Sofware-as-a-Service (SaaS) or on-demand approach can be a very valid alternative approach to get there.

Because let's say you had an option to go with a SaaS solution provider for analytics, someone who had an easy way for you to upload all your marketing data securely and reliabley, and then let you define what exact answers you were looking for, and based on that apply the relevant analytical models and provide you the answers in a form you can actually understand. I am not talking about showing a bunch of fancy charts, graphs, and (yes) the dreaded dashboards -- but actually providing you with deliverables you can use.

Things like a downloadable list of all customers who are likely to bail out on you in the next 30 days. Plus the optimal set of retention tactics derived via a set of data mining models that analyze all past retention programs. Or, it could be a list of customers who are most likely to purchase a certain product -- or conversely, list of customers who will be really upset if you try to direct market them.

Now, I know these deliverables may sound simple, or "gee, everyone can do that" -- but at most of the companies I have worked with, I still don't see this. It baffles me as well, but most companies aren't even at this level of understanding.

Yes, I am biased because my job is to build and promote SaaS or on-demand analytics; but I also know that market will only buy a solution if it makes sense, only if it is truly a better alternative to other solutions. I don't know that. I have confidence in our approach, and with the different customers who we have been able serve via this model, but honestly, that's not enough data points. And as a solutions provider, I sure would like to make sure we are building/providing things to our customers that add a great deal of value.

So, what do you think? Do you feel SaaS/on-demand is a viable model for analytics? If you were in a situation to choose, would you go with an on-demand analytics solution (provided analytics is not the core competency of your businees)?

I think it's surely worth a shot to consider.

Tweet

Why am I doing this @#$%?

In the summer of 1990, the same day Saddam invaded Kuwait, I boarded a Northwest flight from my native Nepal and 20-odd hours later landed amongst the beautiful cornfields of Nebraska (state tag line is "the good life") to start by bachelor's degree in computer science, which later extended to a master's. After finishing my master's and a first taste of Internet startup, somehow I ended up in San Francsico, where I live today -- went through another string of startups, the latest having something to do with deriving "actionable insights" from customer databases, in other words -- business intelligence.

Why should you care? Well, I don't know, but..

I have come to believe that blogs are perhaps the best way to share your experience, both professionally and personally, with the widest possible audience who as a group are always smarter than you.

So that's why I'm here -- to share and explore conversations that haven't happened yet, but I know they will take me (and perhaps you as well) into fascinating aspects of "business intelligence" (and its repercussions on a practitioner) -- or at least get us all a good laugh.

So, come back, come often.. get my RSS feed, trackback, whatever works for you. But let me know if you find these postings useful, boring, or pointless, or what would you rather talk about.

Overall -- let's have some fun chats.

Tweet

Our baby OpenI is a Finalist on SourceForge: Please Vote!

One of my many professional roles is that of the project lead for OpenI, which is an open source BI web application that enables interactive analysis and reporting from OLAP, RDBMS, and data mining data sources. I am very proud to annouce that OpenI is now a finalist for the 2006 SourceForge Community Choice award in the Enterprise category. This is special because this puts OpenI is now in a select group of top 12 projects out of the 116,000-plus projects that exist today on SourceForge.

Now the time has come to cast the final deciding votes. Please go to the link below, select Enterprise category, and check OpenI -- it's that easy

http://sourceforge.net/awards/cca/

March 23rd is the last day of voting, so please vote now. And when you're done, please pass the word around -- we can use all the support/publicity :-)

A bit more on OpenI --at our company Loyalty Matrix, we developed OpenI out of necessity. Our main business is doing customer analytics and delivering actionable marketing insights. Microsoft SQL Server, with Analysis Services for OLAP, is our platform of choice. We needed a thin client web front end for client access. Nothing special you say.

But, the .MAC group at Apple is one of our clients. Guess what, Apple really doesn’t like to use Internet Explorer!

After spending significant time and money on comercial components, that claimed to be browser independent, we punted and decided to search out open source tools. Starting only a year ago last July, we are now about to release version 1.2 of OpenI which includes support for both SQL Server 2000 and 2005. Loyalty Matrix has been using OpenI in production for over six months to serve a number of clients including 24 Hour Fitness and Olivia in addition to Apple.

OpenI is a web-based application for business intelligence reporting, that enables users to analyze data and publish results over the web. Today, OpenI is consistently in the top 100 most active open source projects, with a growing and thriving community. More on OpenI at http://openi.sourceforge.net

Thanks for your support.

Tweet

When it comes to prediction, Machine Learning kicks CHAID and CART's Tails

Conventional wisdom for direct marketing analytics regards CHAID and CART as cutting edge analysis for discovering correlations. (see "Optimal Database Marketing" by Drozdenko/Drake, Chapter 8, 2002)

However, what's not commonly known is that these are more than 20-year old techniques, and recent academic work has shown that analyses like CHAID and CART are more applicable for hypotheses testing rather than predictive analysis. And it's the latter that's more key to direct marketers. So, what's the cutting edge technology for predictive analytics, you say? Machine Learning!

In our own experience for conducting predictive analysis for direct marketing optimization, we have found that machine learning techniques such as Random Forests(tm) are far superior to the single-tree based analyses like CHAID and CART. Machine Learning has far better accuracy for prediction, mainly because instead of trying to come up with a single best tree algorithm for prediction, it develops many different simple trees (many trees = forest!) and collectively combines them into a "forest model".

Tweet