Month: November 2012

Sentiment Analysis, Intention Analysis and the Direction of Fit

Direction of Fit

As you know, for some years now, all of us who form part of the NLP research team at Aiaioo Labs have been working on a technology for text analysis called ‘Intention Analysis‘.

It was something few had heard of when we started.

Today, a lot more people know the term.

But there has been not a great deal of research work published on Intention Analysis in the last 20 years.

So, we’re really happy to be one of the first research teams in ‘recent times’ to delve into the subject again.

We’re also really thrilled to be able to let you know that we’ve just been allowed to demonstrate our work on Intention Analysis at the COLING 2012 conference which will be held in Bombay (now officially known as Mumbai).

I hope I shall get to meet many of you in Bombay in a couple of weeks.

The theory that defines and shapes our work on Intention Analysis is known as ‘Speech Act Theory’.

One of the earlier philosophers to work on it was John Rogers Searle.

He augmented the theory with the concept of Intentional States.  (Incidentally, Intentional States are not even defined in the Wikipedia).

According to Searle, Intentional States could be either Beliefs or Desires.

He differentiated Beliefs from Desires by their direction of fit.

The direction of fit of an intentional state is said to be ‘mind-to-world’ if through the performance of the speech act, a mental state is established, revealed or altered.

The direction of fit of a speech act or intentional state is said to be ‘world-to-mind’ if the performance of the speech act alters the state of the world.

So, sentiment analysis is all about Beliefs.  The direction of fit is mind-to-world.  You see things in the world, and form opinions about them.

Intention analysis on the other hand is all about Desires.  The direction of fit is world-to-mind.  You try to fit the world to a model of how the world should be that resides in your mind.

If you would like to learn more, you can find our paper here:  www.aiaioo.com/publications/coling2012.pdf

Social responsibility and I am ranting!

This post is just a series of rants:

RANT 1

Yesterday, I went to see a friend who was in hospital with a head injury after having been hit by a speeding bus.

She had been driving a scooter near the Town Hall when a bus turning left towards the City Market had knocked her down.

A passing policeman tried to hail some transport to carry her to a hospital.

Three auto-rickshaws refused to take the lady.

Finally the cop threatened a fourth rickshaw driver, and forced him to take her to the hospital.

The part that I could not believe at all, was that while she was unconscious, someone had stolen her wallet and her cell phone.

You can’t fix problems in a country where people care so little about each other.

What is interesting is that so many problems have solutions, but people need to care enough about each other to act on them.

I’ve written about ways of moving ambulances fast through a city, and about a method for tracing terrorists who organize coordinated attacks.

Thinking up solutions is one thing.  Developing a society where people care about each other is quite another.

RANT 2

Prevention of Corruption

I remember the anti-corruption protests last year.  There were many people who turned out to support Anna Hazare.

But there’s so much you can do every day, near your homes.

I saw a traffic cop near my office on Diwali day, flagging people down for bribes.

I went up to him, said “Come to the side.  I saw you take the bribe”.

And I snapped a picture of him using my mobile and noted his name down.

He begged me to let him go:  “Please don’t send that picture to my police station!”

So I told him, “I work around here.  If I see you do this again, I’ll send it.”

This is something anyone can do.

I read that in Egypt before the Arab Spring, cops would harass traffic all the time for bribes.

But now they can’t.  People in Egypt don’t let the police get away with small corruption anymore.  The article says: “Most heartening of all, Cairo’s notorious traffic cops no longer dare shake down motorists for bribes. The people simply will not accept it.”

RANT 3

Recently, two girls were arrested by cops in Thane for posting negative opinions about politicians on Facebook.  One of them was arrested for merely liking the comment.  All that the girl who posted the comment had said was: “People like Thackeray are born and die daily and one should not observe a ‘bandh’ [shutdown] for that.”

That was not the only case.  “In October, Ravi Srinivasan, a 46-year-old businessman in the southern Indian city of Pondicherry, was arrested for a tweet criticising Karti Chidambaram, son of Indian Finance Minister P Chidambaram.”

The law that was used against them was Section 66-A of the Information Technology Act, 2008, which ‘deals with messages sent via computer or communication devices which may be “grossly offensive,” have “menacing character,” or even cause “annoyance or inconvenience.” For offences under the section, a person can be fined and jailed up to three years‘.  Fortunately, the law is now being challenged by a Human Rights activist.

When the news broke, people did not vandalize the police station.  They vandalized the clinic run by the father of the girl !!!

It’s really time we stopped and asked ourselves some pertinent questions about ourselves.  Are we really people who can be proud of and trustful of each other?

How to prevent death and injury in stampedes – Part 2

Image

We made an assumption in my previous post on a technique for preventing stampedes … that the force that can result from a crowd of people falling increases with the slope and with the number of people up above that point.

If this assumption is valid, there are simple techniques for preventing deaths from stampedes.

One of the simplest techniques, (courtesy of Saravanan – Microsoft Research in Bangalore), is a path designed as shown in the above image.

There are horizontal sections to the path at regular intervals.  These sections ensure that the weight of people falling on anyone would be limited to the length of the path between two horizontal sections.

It’s a simple mechanism to prevent death from crushing during stampedes on sloping ground.

It turns out that the stampede at the Ganga last week happened on flat ground.  People seem to have died from being trampled over rather than from the pressure of a mass of people falling or leaning together.

Well, I can’t think of a fool-proof solution for being trampled just yet, but I believe that some day someone will find a way.

What is interesting is that so many problems have solutions, but people need to care enough about each other to act on them.

Thinking up solutions is one thing.  Developing a society where people care is quite another.

Contradictions in some Thoughts on Thinking

Image

The cartoon in this post is shown here under a creative commons license, and is by Kevin Spear.  The cartoon’s story contains a contradiction similar to one that I had later noticed in the very attractive thought (of Osho’s) that I posted yesterday and another that Ramana Maharshi is credited with.

You are bound to be attracted to their very romantic ideas about the futility of thinking, until it hits you that they surely thought a lot about thinking in order to come up with those thoughts.

These people really must have thought a lot!

To bring about peace means to be free from thoughts and to abide as Pure Consciousness. ~ Sri Ramana Maharshi

Thoughts can create such a barrier that even if you are standing before a beautiful flower, you will not be able to see it. Your eyes are covered with layers of thought. To experience the beauty of the flower you have to be in a state of meditation, not in a state of mentation. You have to be silent, utterly silent, not even a flicker of thought – and the beauty explodes, reaches to you from all directions. You are drowned in the beauty of a sunrise, of a starry night, of beautiful trees.  ~ Osho

Cognitive Dissonance

Image

Bet you did not see this one coming … but it made me think all night …

Thoughts can create such a barrier that even if you are standing before a beautiful flower, you will not be able to see it. Your eyes are covered with layers of thought. To experience the beauty of the flower you have to be in a state of meditation, not in a state of mentation. You have to be silent, utterly silent, not even a flicker of thought – and the beauty explodes, reaches to you from all directions. You are drowned in the beauty of a sunrise, of a starry night, of beautiful trees.

~ Osho

How to prevent death and injury in stampedes – Part 1

Image

Only a few days ago, many people lost their lives in a stampede on the banks of the Ganga river.  There is a structural solution that can save lives on crowded sloped paths.  The mechanism is described in the image.

It’s actually quite simple.  The force that can result from a crowd of people falling increases with the slope and with the number of people up above that point.

The number of people above a point is proportional to the length of the path above the point.  By cutting up the path with barriers like the ones I have shown in the picture, you can control that length and keep it from becoming excessive.

When I described this solution to my colleague Saravanan (this was back when I was interning in Bangalore four years ago), he suggested a simpler solution (which doesn’t involve barriers).

Anybody want to take a guess how a simpler solution might look and work?

How far is 5 from 4? Does art take you from 4 to 5? Does MAGIC happen between 4 and 5?

I was intrigued, not long ago, to see a tweet by Dr. Prabhakar Raghavan that went “Berkeley Study: Half-Star Change In Yelp Rating Can Make Or Break A Restaurant http://tcrn.ch/RAy1iM  nonlinear outcomes from linear(?) inputs“.

Anderson and Magruder at Berkeley observed Yelp ratings and tried to see if they impacted the likelihood of a restaurant being fully booked at peak dinner time (between the hours of 6 and 8 pm).

What they found was that:

An extra half-star rating causes restaurants to sell out 19 percentage points (49%) more frequently, with larger impacts when alternate information is more scarce

Moving from 3 to 3.5 stars reduces the likelihood of availability from about 90% to 70%. A fourth star reduces the likelihood of  availability further to 45%, and that possibility drops to 20% at 4.5 stars

So, the answer to the first question seems to be “A lot farther than you’d think” or “About as far as 4 is from 1” !

Now I am going to show you something else that is equally amazing!

The above economics result is very similar to what is being talked about in this XKCD cartoon on online star ratings!

Take a close look at the cartoon and you will see that the cartoonist is saying much the same thing as the economics researchers.

How did the artist hit it right on the head?

Just how did he do it?  (maybe the cartoonist read the research paper – the guy’s a scientist so it could have happened).

But could it just be that good artists are also incredibly perceptive and intelligent people?

Does art take you from 4 to 5?

I don’t have the answer to that, but I am going to point you to an intriguing paragraph in the book “Transforming Education through the Arts”.

“We found that compared with typical scientists, Nobel laureates are at least 2 times more likely to be photographers; 4 times more likely to be musicians; 17 times more likely to be artists; 15 times more likely to be craftsmen; 25 times more likely to be writers of non-professional writing, such as poetry or fiction; and 22 times more likely to be performers, such as actors, dancers or magicians.”

(From Page 7 of “Transforming Education through the Arts” in the article “The social and educational costs of neglecting the arts”, citing as the source: “Root-Bernstein and Root-Bernstein 2010: 4”).

So, maybe the answer to the second question is ‘maybe‘.

I say ‘maybe’ because there is another explanation.

In software, we have a saying that goes:

80 percent of the features take 20 percent of the time.  The last 20% of the features take 80% of the time.

However, this 80/20 rule seems to apply to many things in life.

You can be a better singer than 80% of the world quite easily.  It might even be possible to become a better singer than 97% of the world with a bit more work.  But it takes infinitely more effort to get from 97% to 99%.  To reach those levels of perfection takes enormous dedication and persistence.

This is the case in every art, and in every craft, and in every area of science, and in every area of technology.

Products become magical when you put in that enormous effort that takes it from 97% to 99% of perfection.

Dancers become magical when they go from 97% to 99%.

We all heard the story of Steve Jobs obsessing over the details of the iPhone even during sessions of chemotherapy.

Cartoonists become magical when …

You get the idea.

So maybe it’s not about art …

Or about science …

Or about technology …

Maybe it’s about having the strength, resilience and stamina to get you to the very top of the mountain and the obsession to keep going …

Maybe that’s what makes all the difference in any area of human endeavour – the intense effort and drive that only very obsessed people are capable of putting in is the magic (or madness) that takes something from 4 to 5.

Maybe that’s why 5 is so far from 4.