Month: October 2012

Discovering intent for retargeting

Text-based Intention Analysis

At Aiaioo Labs, we have been developing text analytics tools for the discovery of intent for a number of years, as explained in the following articles:

  1. Intentions and Opinions
  2. Prior Work on Intentions

In recent months and years, we have seen other firms come up with similar technologies, studies and initiatives:

  1. SocialNuggets offers a study on the ROI of intent analysis
  2. Solariat has been trying to convince Facebook to use intention analysis for ad placement
  3. WisdomTap did a study on the power of intent analysis with us

These are all text-based studies, using the content of conversations on social media to discover intentions.

However, the most recent attempts at improving the targeting of ads have focused on methods for discovering intent without using any unstructured information.  Below, we look at two of the methods.

One of the methods is advertisement retargeting.


Retargeting involves showing ads to customers based on intent discovered from past visits to sales portals.  For instance, if you visited Amazon’s books section, you have demonstrated the intent to purchase books.  It’s clear as crystal, and there is no text analysis involved.

Now if Amazon tells Facebook that a particular user visited Amazon’s books section, Facebook can use the information to show suitable Amazon ads to that user, even after the user has left Amazon.  That is called retargeting.  It is in essence a kind of intention analysis, but without the use of text.

Here is an article about retargeting on Facebook:

It’s Become Tragically Clear That Facebook Chased The Wrong Business For Years

Another method is the use of special purpose social networks.

Special Purpose Social Networks

When most people go on LinkedIn, they have careers or business contacts on their minds.  So, LinkedIn might work well for job ads or B2B branding exercises.

This article on Forbes talks about this type of intent analysis:  Why Consumer Intent Drives The Value Of Social Networks

It’s very interesting to see wholly new ways to guess at social networking users’ intentions.


  • Advantages:  doesn’t need data from outside the website
  • Disadvantages:  might be perceived as an invasion of privacy, needs conversations, low volume, low accuracy


  • Advantages:  high volume, high accuracy, no privacy issues, needs no conversations
  • Disadvantages:  needs data from outside the website

Special purpose networks

  • Advantages:  high volume, no privacy issues, needs no conversations
  • Disadvantages:  low accuracy, the space of intentions that can be assumed is limited per network

Measuring the efficiency of retail and the possible implications

I read a beautiful BBC article today titled “How much will the technology boom change Kenya?

It is about how information delivered over mobile phones can improve people’s lives.  Here is an example:

“Ms Oguya, 25, is the creator of a mobile phone app called M-Farm, designed to help small-scale farmers maximise their potential.

Ms Oguya herself grew up on a farm. She realised people like her parents had two main problems.

Firstly, they did not always know the up-to-date market price for a particular crop.

Unscrupulous middlemen would take advantage of that and persuade farmers to part with their produce at lower prices.

Using M-Farm, a farmer can now find out the latest prices with a single text message.”

This reminded me first of all about the price of pomegranates in Bangalore.  A kilo of pomegranates costs Rs. 120 in Bangalore city.  The price at which traders buy pomegranates from farmers is Rs. 12 per kilo.  In other words, what the producer gets from the consumer is a mere 1/10 of the price a consumer pays for the product.

A year ago. I had written an article on an anti-corruption blog titled “Why Walmart’s measure of efficiency might be flawed“.

I quote from it:

“Walmart’s definition of efficiency is the cost of a product. They say they increase the efficiency of the entire market by lowering the cost of products (by getting an item manufactured in a low-cost geography).”

“However, in my opinion, there is a better way to measure efficiency, and I feel that retailers highlight lower prices as a measure of efficiency only because it is the only measure possible under the current conditions of lack of transparency in retail.”

“Take for example a $10 product that used to cost $5 to manufacture in the USA (potential profit margin of $5). Now, Walmart has a profit motive to move its manufacture to a geography where it costs less than $1 to manufacture if they can still charge $8 for it. Say, the transportation cost is $1. If the product sells for $8, they still have a profit margin of $6 which is higher than $5.”

“However, if I measure efficiency as the percentage of money paid by a customer that is being delivered to the manufacturer, there has actually been a drop in efficiency. The percentage of the price that went to the manufacturer dropped from 50% to only 12.5% (one dollar out of eight).”

“You would never donate your money to a charity without first asking what percentage of your donation was reaching the beneficiary, would you?”

Now let’s do the math for pomegranates.

By using the ratio of purchase price to sale price as a measure of efficiency, we find that the efficiency of the retail mechanism for pomegranates is a mere 10%.

If farmers became better informed, they might work to discover ways to improve the efficiency – either by bypassing middlemen – I remember the farmer’s market in Raleigh, NC, an amazingly simple idea, that did just that – or by negotiating better prices for themselves.

End consumers might also choose to patronize outlets that pay fairer prices to the producers if they could find out how much was really paid to the manufacturer.

That might also help bring back manufacturing to the USA.

Statistics for Linguistics

I recently wrote an article on how machine learning algorithms can be considered extensions of linguistic rules.

The article I am currently sharing with you is also related to linguistics.

It is in the form of a set of slides and describes statistical concepts and tools that might come in handy for linguists.

Here are the slides:

The goal of these slides was to teach a team of linguists about statistics and machine learning.

The main sections are:

  1. Motivation for learning new tools using examples from algorithms and their application to education and poverty elimination.
  2. What to annotate
  3. How to develop insights
  4. How to annotate
  5. How much data to annotate
  6. How to avoid mistakes in using the corpus

Gandhi’s internal conflicts

I recently gave a lecture on machine learning.  When I concluded the lecture, I was given a book as a gift. It was an abridged copy of Gandhi’s autobiography.  Today is the 143rd anniversary of Gandhi’s birth, so I will share a few excerpts from the book.

Excerpts from Gandhi’s Autobiography

Gandhi went to England as a young man to study.  He wrote:

I decided to take dancing lessons at a class and paid down 3 pounds as fees for the term. I must have taken about six lessons in three weeks. But it was beyond me to achieve anything like rhythmic motion. I could not follow the piano and hence found it impossible to keep time.

He participated in the Boer and Zulu wars (he set up an Indian Ambulance Corps that would assist the South African government):

In order to give me a status and to facilitate work, and also in accordance with the existing convention, the Chief Medical Officer appointed me to the temporary rank of Sergeant Major and three men selected by me to the rank of Sergeants and one to that of Corporal.

To hear every morning reports of the soldiers’ rifles exploding like crackers in innocent hamlets, and to live in the midst of them was a trial. But I swallowed the bitter draught especially as the work of my Corps consisted only in nursing the wounded Zulus. I could see that but for us the Zulus would have been uncared for. This work, therefore, eased my conscience.

He later also wrote:

I make no distinction, from the point of view of ahimsa, between combatants and non-combatants. He who volunteers to serve a band of dacoits, by working as their carrier, or their watchman while they are about their business, or their nurse when they are wounded, is as much guilty of dacoity as the dacoits themselves. In the same way those who confine themselves to attending to the wounded in battle cannot be absolved from the guilt of war.

About the support he extended to England during World War 1, he wrote:

It was quite clear to me that participation in war could never be consistent with ahimsa.  But it is not always given to us to be equally clear about one’s duty.

I used to issue leaflets asking people to enlist as recruits.  One of the arguments I had used was distasteful to the Commissioner:  ‘Among the misdeeds of the British rule in India, history will look upon the Act of depriving a whole nation of arms as the blackest.  If we want the Arms Act to be repealed, it we want to learn the use of arms, here is a golden opportunity.  If the middle classes render voluntary help to Government in the hour of its trial, distrust will disappear and the ban on possessing arms will be withdrawn.’

From Crude Rules for Classification to Machine Learning

I recently gave a lecture at a college in Bangalore where I explained to a roomful of undergrads how machine learning algorithms can be constructed incrementally starting from crude rule-based algorithms.  I explained how you could start with a list of words and make incremental improvements till you obtained a Naive Bayesian Classifier.

Here are the slides on the transition from crude rule based methods to machine learning that I presented at the college (showing how the incremental transition can be made).

What can Game Theory learn from Social CRM and vice versa?

Customer service can now be delivered through social media channels (Social CRM) in addition to traditional closed channels like feedback forms (closed in the sense that the issues described are not made public).  But there are benefits and costs to using social media as a channel for receiving complaints.

Complaints voiced on social media about a product might greatly affect a potential customer’s decision to buy the product.  So, brand owners have a clear incentives to respond to and resolve complaints on social media.

On closed channels like call centers or customer service email addresses since there is no danger of negative feedback being seen by and influencing another person’s buying decisions.

So, social media gives users a greater bargaining power compared to traditional complaint channels. This gives brand owners one more incentive to resolve complaints – it is no longer just a question of a complainor’s satisfaction, but also of an unknown potential buyer’s impressions of the brand.

So, there is a clear incentive for firms to respond to customer complaints on social media channels.

However, a recognition of the greater bargaining power on social media combined with the greater ease of voicing a complaint on a social media channels might give users an incentive to choose social media channels instead of closed channels to voice negative sentiments.

So, if a service provider does not listen and respond to complaints voiced on social media, there might just be fewer expressions of negative sentiment on social media in the first place.

So, you can argue both for and against using social media channels to handle customer complaints. In this whitepaper we attempt to study the opposing motivations using game theory.