A question for marketing analytics folks

In the last few years, marketing analytics has taken off in a big way in India. It is the relatively little known outsourcing success story. Firms from the US send their data to India for sophisticated analysis, specific to marketing questions, and people in India, typically MBAs, analyze the data and report findings. Marketics is a well known firm in the field, with HP Analytics and Dell Analytics also doing a fair bit of business that way.

A few of my friends work in those firms, and I have often been curious about the statistical packages they use for the analysis. My research also involves wrestling with marketing data, and I have used several packages over the last couple of years. In the rudimentary polling I have done, it seems like SPSS, SAS, Lisrel, MPlus, Minitab etc are the popular packages. A few of them, very few, use MATLAB. All of these are proprietary, and no doubt expensive, especially for companies.

Here is my question. I have yet to meet a person from the marketing analytics industry who uses R. They also did not seem to be aware of others in their firm using R. I wonder why that is. It is a very powerful package, which can do everything that the aforementioned packages can do, and more. It is a lot faster. Now ideally of course, you would want to use C or C++, but that involves programming which the analysts might not feel comfortable doing. R does not have that problem either. It is not as "black box" as other packages, but it is still fairly simple.

Plus, R is open source, hence free. The R community is also very active, so just one google search can help you find packages written by others which can allow you to do everything, from complex regressions to cluster analysis to structural equation modeling, to segmentation, to.... literally anything that a marketing analyst would want to do. And did I mention it is FREE? Yes, I did.

So why isn't R being used as extensively in the marketing analytics industry? Has it even been tried? Have the bosses or senior analysts even thought of switching to R? Is it because the salespeople for proprietary packages have been able to wield more influence than they should? Or is there some other reason for sticking to bulky and proprietary packages that I am missing?

Coolest Waste of Money Ever

I had gone to the building vending machine. Put in my dollar, pressed D4, but the candy bar, instead of falling, got stuck and dangled from its perch. And guess what? It was a Twix!!! That gave me such a major kick (if you don't understand why, never mind).

Fortunately, I had another dollar, so I put it in, pressed D4 again, and got both the Twix bars. There was no need to fashion a candy line-up.

Hillary is just like Monica

No, not that Monica, you gutterbrains! Monica from FRIENDS. Watch this clip and tell me if Monica's attitude towards losing does not remind you of Hillary Clinton. Wonder when Bill will interrupt her like Chandler to say "They left!"

Marathi Food and Marathi (Non)Migration

Through Amit Varma's blog I came across this interesting article by Vikram Doctor. He wonders why Maharashtrian food is not popular outside the state.

I should also qualify my initial statement about Maharashtrian food not being popular: it is, but only within Maharashtra. In Mumbai you’ll find several simple eateries in areas like Dadar or Girgaum serving vegetarian food (Dattatreya, Phansikar, Prakash, Tambhe Arogya Bhavan) or non-vegetarian, with an emphasis on seafood (many places calling themselves Malvani, Konkani or Gomantak restaurants).

Sindhudurg offers a more upmarket option, as does Viva Paschim (which used to feature in-meal entertainment with lavni artistes and two men dressed as a dancing cow) and now there’s Diva Maharashtracha where occasionally decent food is overshadowed by its phenomenally self-promoting owner.

On the street, you get snacks like vada-pav, kande-poha and missal (especially from the zhunka-bakhar stalls which ended up serving almost anything except that besan curry-roti mixture). And elsewhere in the state you’ll find restaurants serving local specialities with savour and style.

But only within the state. Outside, if you ask people about Maharashtrian food you’ll mostly get blank looks, or some vague knowledge of vada-pav, thanks to the Sena, which recently flew a vada-pav maker to Delhi to dish it up for national leaders, and chains like Jumbo Vada Pav which are taking it national (Dheeraj Gupta, Jumbo’s owner, tells me that local tweaking is needed, like buttering the pav to get Gujaratis to buy it: “We made this simple change and volumes jumped 250% in one month!”).

Beyond this though there’s not much knowledge, and I find this puzzling because many of the factors that leave a cuisine obscure do not apply in Maharashtra’s case.

I think Doctor has ignored a rather obvious explanation. My guess is the answer lies in the non-immigratory nature of the Marathi manoos.

Think about it. Who made idlis and dosas a national breakfast fare? People from places like Udupi who migrated from their states to other places in search of livelihood. Who made the generic Punjabi fare popular? The Punju migrants. Heck, even the street kulfis you see have the name "mevad" plastered on them because people from the Mevad province travelled to other states to sell kulfis and make a livelihood.

This is true worldwide too. Italian and Chinese food is more popular in the US than Indian food, because the purveyors of those foods got an immigration head-start. Meanwhile, Indian food is a lot more popular in Britain because Indians started migrating there in big numbers much earlier than to the US.

The Marathi manoos is not particularly known for leaving the state in search of livelihood. The causality here is ambiguous. One explanation could be cultural, and sociologists could explain why Marathis don't venture out too much. Another one could be just realistic and practical. Maharashtra in general and Mumbai in particular have been the original "sinks" for immigration because of employment opportunities. Everyone wants to come to Bombay. There was a big stream of immigration from the south in the 40s, 50s, and 60s, and since then North Indians have been filling the city. Gujaratis, Punjabis (pre-green-revolution reasons) and Marwadis have always been known to immigrate and set up businesses.

But the Marathi restaurateur didn't need to leave the state. He had no incentive to. If a Kolhapuri wanted to sell misal or a Malvani wanted to sell sea-food, they could just move to Bombay(or Pune) and prosper. Why go all the way to Delhi or Kolkata when you have massive markets just a few hours away? Also, relatively speaking, the infrastructure and law and order has been better in Maharashtra than most states. If everyone is coming to Bombay, why would you want to go in the opposite direction?

And maybe this lack of an incentive to immigrate..... or even stronger - the lack of a disincentive to stay where you are, perpetuates the phobia against going elsewhere. Talk to Marathis working in white-collar jobs in Delhi, Bangalore or Chennai, and most will have plans afoot to get a transfer to Pune or Bombay. I don't think this "take me home, to the place, I belong" sentiment is as strong with people from other places.

And who knows, maybe the immigration-phobia also dampens entrepreneurial spirit. If you are someone who has moved thousands of kilometres from your home to make a livelihood by setting up a food business, it means you probably have or develop (direction of causality very ambiguous) a greater drive to succeed and grow. So most successful marathi food business are happy to reach a certain level of financial success and then maintain the status quo. Whereas, udupis and punjabis and suchlike want to keep growing.

Let me end with a personal anecdote. When we moved into our house in Pune in the late 80s, the locality we lived in was on the outskirts of the city. Rickshaws charged huge premiums to go there. If we wanted to go to a proper sit-down restaurant, we had to travel a few kilometres on my dad's loyal Lambretta. So we often had to make do with the meagre options available in the vicinity.

There were 3 food stalls. One was a vadapav seller. The other was a bhelpuri wallah. And the third was an idli-dosa stand. All of us who grew up in the locality patronized these places regularly. Twenty years later, the locality is no longer on the outskirts. It is heavily populated with a lot of traffic.

The erstwhile idli-dosa stand is now a sit-down restaurant called Kinara. The vadapav and the bhelpuri guys.... you guessed it.... still have stalls. Not that they are doing badly. Whenever I go to Pune, I eat bhel and vadapav from those stalls regularly, and talk to them at length, because they have literally seen me growing up. Their sales volumes are quite high and they make more money than I did after my MBA :). They used to live in slums, but now live in apartments. Their kids went to college, did well, and are white-collar workers themselves. Both guys are moderate successes and happy with their lives.

But they were just content hiring one or two guys to wash the dishes, cut the pavs, cut the onions alongside them standing at the stalls. While the guy from Udupi invested his profits smartly, hired more people, bought a bigger shack, then an even bigger, then bought a small shop, then expanded, until he now owns a biggish restaurant called Kinara (Remember the scene from Kamal Hasan's silent movie Pushpak, where they show how the Pushpak grew from a tiny shack to a big hotel.... that was the transformation we saw happening with Kinara over 20 years).

And here's the real kicker. The last time I visited the bhelwallah, he told me about how he had been taking vacations all over the country, how he enjoyed visiting Goa, Bangalore, Agra, Delhi, Shimla, and so on. Then a guy who owns a shop next to him told me the bhelwallah could be in Australia now. Apparently one of his regular patrons who, like me, grew up in the neighborhood, had moved to Australia, owned a successful restaurant, and had all but offered the bhelwallah a blank cheque to move with him. He ensured he would get him a visa, get him a house, and set up a "marathi fast food restaurant" with him. The bhelwallah refused. Why?

"ikde sagla first class chalala aahe na... kashala ugaach dusrikadey jaaycha traas?" (Everything is going fine here... why should I take the trouble of moving elsewhere?)

And in that lies a big part of the answer to Vikram Doctor's question.

R.I.P. The Greatest Tendulkar Ever

Vijay Tendulkar is no more.

IPL as Liberalization?

Amit Varma and Neelakantan have made the point that the emergence of IPL is analogous to the liberalization of the Indian economy in the 1990s. Both have also said that a lot.. LOT more needs to be done.

I am not as optimistic as them (maybe I am overstating their optimism here). Yes, the IPL has brought money into the game for players. Yes, it has given chance for some players to shine on talent, and not be selected based on political issues. But to me, he IPL seems more like a previously corrupt and completey autocratic dictatorship relaxing its grip and throwing its citizenry some carrot slices. Not unlike Cuba allowing people to have cellphones (gasp!).

For all the apparent good that the IPL had done to the game, there is perhaps an equal if not heavier ill that it has inflicted. Think about how the BCCI is muscling out the ICL using unethical and even illegal (according to the English court judgment) means. Also think about the fact that underneath this apparent veneer of professionalization and corporatization, the fact remains that BCCI still calls the shots. Yes, we have people like Asnodkar and Gony benefitting, and it might seem like the monopsony in cricket is broken. On the other hand, you have the case of Shoaib Akhtar, who was banned by the PCB for political and Draconian reasons, and had to jump through a lot of hoops before the BCCI benevolently made his participation in the IPL possible.

The only real good that has come out of the IPL is that it has given cricketers a big share of the pie, has opened up newer avenues for them, and made it somewhat easier to be judged on merit. But I fail to see it having any real impact on what is wrong with Indian cricket. BCCI office bearers are still elected through shady horse-trading practices. The BCCI is still not accountable to anyone. And selection for the Indian team will still be driven by regional biases, because the IPL circumstances don't have any impact on the skewed incentive structures there. In fact, the IPL might take BCCI politicization to newer levels. What if team owners or their proxies run for BCCI office, and win? Just like past BCCI Presidents have pushed players from their own states or zones, a Mukesh Ambani proxy would push for players from the Mumbai Indians, and a Malya proxy would push for players from the Bangalore Royal Challengers.

If anything, the IPL has just enabled the BCCI to stmap out dissent and competition, further strengthen its monopoly as well as monopsony, but cloak it in a friendlier garb. Hardly liberalization.

The real liberalization in my opinion is the emergence of the ICL. ICL is a real competition to the BCCI. ICL is accountable to the shareholders of Essel Group and IL&FS. Everything that you want in terms of liberalization is on offer from ICL, as well as the competition between ICL and IPL.

This is what is wrong with the American media

After coming from a third world country to a first world country, I am often surprised at how some complete jackasses-without-a-clue actually have their own TV and radio shows. An excellent example is the right wing radio host Kevin James. Now, I have never heard James before, but there are many similar to him who infest the American airwaves. James' stupidity, ignorance and childishness is exposed here by, of all the people, Chris Matthews! Getting pwned by Chris Matthews is like being hit for 6 sixes in an over by Glenn McGrath.

A bit of a background first, before I present the clip to you. Barack Obama has said often on the campaign trail that he would like to talk to even enemies, including the Iranian president. Bush gave a speech to the Israeli parliament in which many believe he took an indirect shot at Obama by comparing him to those who appeased Hitler. In doing so, Bush unwittingly conformed to Godwin's law. There is a difference between talking to enemies and appeasing/negotiating with enemies. Nixon went to China. Reagan talked to Gorbachev. Appeasement is wrong, but talking is not. What Chamberlain did wrong was appeasing Hitler.

Now you would figure that anyone who paid attention during high school history classes would know what Chamberlain did wrong, what the Munich Pact was all about, and so on. Not Kevin James! Just look at this exchange. It is hilarious. All he seems to know is that Chamberlain was an "appeaser".



If such idiots get their own shows even in advanced countries like the US, I think the Indian media is doing OK.

Iron Man

This was a match made in heaven - Robert Downey Jr. playing Tony Stark. Downey's natural sass is one of the mot under-utilized resources in Hollywood, mainly due to his own problems. I have always wondered how he would have played Daniel Ocean, because no matter how suave Clooney is, he just doesn't measure up to Downey in terms of original ratpackishness. Tony Stark is as close an approximation as is possible to answer that question. Splendid performance.

From a natural casting to major cognitive dissonance. "Pure" villains are uncommon these days. Villains are now given some sort of a gray area with a weak but sellable justification for going bad. So Obidaya is a rare delight, almost reminiscent of Mogambo or Lotiya Pathan types, if 80s bollywood were a yardstick. Cognitive dissonance sets in because the role has been carried off in a sinisterly perfect way by the guy cannabically etched into my brain as "the dude". Just imagine.... the dude selling arms to terrorists... plotting to kill blondes named Pepper... spreading destruction in a massive suit... and no rugs in play. It is kinda cute when he speaks that one line in hindi/urdu though. Looks and sounds like Bob Cristo.

Making Use of Big Brother

If you want to make a music video, but can not afford any filming equipment, what do you do? A band from Manchester hit upon an ingenious idea.

Unable to afford a proper camera crew and equipment, The Get Out Clause, an unsigned band from the city, decided to make use of the cameras seen all over British streets.

With an estimated 13 million CCTV cameras in Britain, suitable locations were not hard to come by.

They set up their equipment, drum kit and all, in eighty locations around Manchester – including on a bus – and proceeded to play to the cameras.

Afterwards they wrote to the companies or organisations involved and asked for the footage under the Freedom of Information Act.

"We wanted to produce something that looked good and that wasn't too expensive to do," guitarist Tony Churnside told Sky News.

"We hit upon the idea of going into Manchester and setting up in front of cameras we knew would be filming and then requesting that footage under the Freedom Of Information act."

Only 25% of the organizations complied with their requests, but the footage was still enough to make a music video. Here is the finished product -



Link via email from Shantanu.

Hillary's Shifting Goalposts

Every time something goes against her plan, Hillary or her campaign comes up with a new metric, or parameter or definition of success. After losing North Carolina heavily and winning Indiana only narrowly, she and her supporters quoted Obama out of context saying "Indiana will be a tie-breaker" and proclaimed victory. But there now seems to be an air of inevitability around Obama. Even the media which unrealistically pretended this race was alive, seems to be giving up. But Hillary keeps shifting goalposts.

Here is an excellent compilation of her goalpost-shiftiness.

When you boil it all down, in determining the Democratic Party‘s presidential nominee, only one vote really matters. The 50-something conservative registered Democrat who‘s not independent but not a part of the base, and skipped college so they can go straight into teaching rather than become a casino worker, who votes on domestic issues but not in the primary or caucus and in a big state that doesn‘t border Illinois, that has elected female governors and members of Congress, but didn‘t vote Republican in 2004, won‘t vote Republican in 2008, and didn‘t vote for Jesse Jackson in 1984 and 1988 during an all day vote except between the hours of 7:00 and 7:15 p.m. Oh, they don‘t object their vote being overruled by superdelegates.

The Greatest Captain (N)ever

Ian Chappell has written a rare brilliant article about Warne's captaincy. It is not often that I find myself nodding in agreement as I read an article. That I read the article soon after Warne's unfancied Rajasthan Royals had beaten Dhoni's formidable Chennai Super Kings, made the nodding even more fervent.

There is this je ne sais quoi that bursts out from every molecule of Shane Warne. Call it his cockiness, his arrogance, his style, his unparalleled talent.... whatever it is, it makes it difficult to take your eyes off him when he is on field. Two of my surliest days have been when a) Steve Waugh was chosen to captain Australia after Taylor, and b) Ponting was chosen to captain Australia after Waugh. The first surly day, I shared with my friend Neeraj, another Warne fanatic, who would sit with me in circuit design classes with a red cricket ball in his hand, which he would "leg spin" from one hand to another, under the desk.

Finally we are able to witness Warne's captaincy in full glory. And it is delightful. Yet, in my opinion, the real spectacle is yet to come. That will happen a few months later, in Las Vegas. When Shane Warne takes part in the world poker series. Anyone who has seen how Warne operates, and who loves poker (especially Texas Hold 'Em) can share my excitement at seeing the master of guile sitting at a table, playing with chips as he ponders his next raise.

Idiots!

Indian politicians from all parties alike are making me do the unthinkable. They are making me defend Bush!

All the man said was -

"Worldwide there is increasing demand. There turns out to be prosperity in developing world, which is good. It's going to be good for you because you'll be selling products in the countries, you know, big countries perhaps, and it's hard to sell products into countries that aren't prosperous. In other words, the more prosperous the world is, the more opportunity there is," the US President said.

"It also, however, increases demand. So, for example, just as an interesting thought for you, there are 350 million people in India who are classified as middle class. That's bigger than America. Their middle class is larger than our entire population.

"And when you start getting wealth, you start demanding better nutrition and better food, and so demand is high, and that causes the price to go up," he said.

Whether he is factually right or wrong, and whether he is downplaying the role of ethanol or not, the question is - how exactly is he "blaming" India? Why are the Indian politicans acting as if he has insulted our national honour? The most idiotic statement, unsurprisingly, comes from... you guessed it -

"At a time when millions of people in India are unable to get enough food to eat and suffer from malnutrition, Bush's insensible remarks about India's prosperity affecting global food prices are adding insult to injury," CPI-M general secretary Prakash Karat told PTI.