Mental Models and Art Aesthetics

An Exploration of the Mental Model that a Spectator might Form when Watching a Dance.

This is a post about a painting that I completed only last week.  But it was a painting that I’d been trying to figure out how to paint for all of 7 years.  It all started with a motorbike ride.

One day in 2006, I was riding down a mountainous road in North Carolina,
when I saw a hill of striking beauty (I believe it was somewhere near a town called
Marion).

It was evening and the colours were beginning to fade, but I could clearly
make out on one side of the hill striking trees that were reddish-orange in hue and I
found them beautiful against the light green that surrounded them and the blue skies
above.

I continued to ride down the road.  I rounded the hill and came to the other
side. I saw again, waves of green and yellow and touches of red and I was overcome
with a sense of being in the presence of the highest beauty that the eye could
behold.

So, I stopped and removed the camera from the knapsack behind me. From
various points along the road, I tried to photograph the hill.

However, I could not frame a shot that could evoke the beauty of
the hill that stood before my eyes.

That evening, I was troubled by a sense of failure, of bafflement at having failed to record the soul of the scenery that I had left behind. The hills around Marion were all extremely beautiful, but nothing else on the road had the perfection of the hill that I had not been able to photograph.

So, that evening, I sat and pondered a very interesting question that had to do with
aesthetics. The question was this: Does the mind experience beauty according to
images that it has seen, or does the mind experience beauty by a mental model it
has built of an object and the associations it has made between that model and
models of other objects?

Let me clarify what I mean by a mental model with an example. When we approach a
person whom we have never met before, we don’t immediately see all of that person.
We perhaps at first only see the front of the person, at a distance. Later we see more
of the person, their face, their profile, and how they look from behind. As we see
more and more of a person, we fill in more and more of the missing information about
how the person looks.

However, from the very beginning, we know that we are
looking at a person, even though we haven’t seen the person’s teeth or sides or
back. So we must have decided that the frontal image that we saw fit best into a
mental model of a person.  The mental representation may not just be a 3
dimensional representation. It may be a set of associations and memories over time, all woven together in complex ways.

So, the question that came to my mind was:  Is a human mind’s evaluation of
the aesthetics of an object based on a) any of the individual images of the object
that we have seen or on b) the entire mental model that we have constructed in our
minds of the object.

The more I thought about it, the more I began to feel that my experience with the hill
would be best explained by the conclusion that our aesthetic experience of an
object is determined by the mental model we have formed of it, and not by any image
of it, and here were the reasons for my leaning towards the latter conjecture:

a) If it is just a single view or image of an object that drives us to experience it as
beautiful or attractive, then our feelings about its attractiveness should change all
the time, based on small things like the angle at which we are viewing it. That does
not seem to be how we experience things in real life.

b) The experience of the hill led me to think that the latter explanation was the more
likely because if it was one of the views of the hill that made me think it beautiful, I
should have been able to capture the view with my camera. Since I couldn’t,
perhaps it was something I saw on the other side of the hill, combined with the
fading light and the landscape along the way (perhaps combined with the beauty of
some of the trees I saw upon the hill) that came together to contribute to the sense
of beauty that I experienced when looking at the hill.

That evening, I thought of a new kind of painting technique that would be possible if
the above conjecture proved true. It seemed that it might be worthwhile to paint a
picture from the point of view of the mental model a viewer might form if they saw the actual subject of the painting in context, possibly from different angles, or at different distances, and over an extended period of time.

But it was only last week, a good 7 years after I passed through Marion, NC, that I
tried to create a painting of the sort. Since I have recently watched many Indian
classical dance performances and some contemporary dance performances, I
attempted to paint all the things that might be associated with a dance performance
in the mind of a spectator. I constructed the painting like a fractal, starting with a
dancer and then representing the dancer at different levels of granularity and detail.

One of the things on the mind of the spectator would be the dancer’s body
(the whole form, from head to toe). Emoting is a key aspect of dancing in Indian classical dance.  So, in order to appreciate the emotion, a spectator would have to pay close attention to (and so form a model of) the dancer’s face, and of his or her eyes. So, I wove an
image of a forehead, and of an eye, into the painting. Another part of the body that
Indian classical dancers use a lot is the hands and the fingers. They use them to
denote various objects, characters and actions. I associated the movements of the
arms and fingers with flowing water, and plants with branches. So, I painted in some
flowing washes and let some negative white spaces take the form of trees in the
painting. Indian dance costumes are colourful, so I made everything in the painting
as bright as a classical dance costume.

The process of painting was a lot of fun. It was very different from the process of
scientific experimentation of course. There was none of the same rigour and I
wasn’t really looking to prove anything.

I liked the painting that came about and I have inserted a photograph of the same into this blog entry (it’s at the top).

But we don’t know yet whether mental models lie at the root of aesthetic
experiences. And you know what? As an AI researcher, I am just dying to find out! It
seems possible to design an experiment to determine the validity of the conjecture.

We could take an object whose facade is beautiful but whose side view is not. The
subjects of the experiment would be shown the front and side views of the building
and asked to rate the appearance of the front of the building. The control group
would be shown just the front of the building and asked to rate its appearance.

A significant difference in ratings (poorer ratings by subjects shown the front and
side view) would support the conjecture that a mental model lay at the root of the
aesthetic experience.

I hope that one fine day, we might be able to conduct the experiment, and if we do, I
shall share the results here with you!

As a footnote, I’d like to share this blog post http://aimeeknight.com/2012/04/10/the-new-aestheticperhaps/ by a blogger from (it appears to be – hmm, what a coincidence) North Carolina.  The article is about art that attempts to portray the world from the point of view of a machine – using a machine’s aesthetic – the phrase they’ve used is ‘New Aesthetic’.  I had not heard of the concept before, and it isn’t the same thing that I am working on, but since we’re discussing aesthetics and perception, it appeared relevant and interesting.

As a second footnote, I’d like to put in a word of explanation about why a firm of computer scientists and AI researchers should be interesting in painting.  Well, to many early AI researchers, AI was as much a quest to understand the human mind as it was a quest to solve difficult problems.  There were many attempts to comprehend the human mind by constructing logical and mathematical models of various cognitive processes (things the mind did) and seeing if the models could mimic the real thing.  So, AI research had a lot of common ground with Psychology.  And there is still a cross-disciplinary research area known as Cognitive Science that blends Psychology and AI research.  So, I just want to defend myself by saying that talking about aesthetics is not off-topic for this blog.

LearnLog: 30 Million Word Gap and Why You Should Talk to Your Kids a Lot

Reblogged from Dorai's Learn Log:

From Support children’s learning by talking with them over a book

Almost two decades ago, in 1995, Hart & Risley published the results of a landmark study [1] that linked success in school at age nine to the amount of talk they hear from birth to age three.  The actual difference in the amount of words children hear is astonishing; successful children heard on average 8 million more words per year than their struggling peers, leading to what was termed the “30 million word gap” by the time they turn four.

Read more… 201 more words

Dorai writes about a curious connection between how well a child does in school at age 9 and the amount of language a child is exposed to by age three.

What traffic can reveal about society

The last post was about Socially Optimal Solutions and the Nash equilibrium.

In this post, we examine traffic patterns under the same lens.

It appears bad traffic situations might be avoided if road users a) play by the rules or, b) play fair or, c) are trusted by other participants.

Scenario 1

In Scenario 1, the North-South traffic is waiting for the light to turn green.  Cars 1 and 2 are going South.  Cars 3 and 4 are going North.

Now, if the drivers of Cars 2 and 4 try to cheat by overtaking on the wrong side of the road, you could end up with Arrangement 2 (everyone is blocked).

Si if the drivers all respect the law, they would end up with the socially optimal solution (Arrangement 1 rather than Arrangement 2).

Scenario 2

Here, the intersection does not have traffic lights.  The North-South traffic has stopped.  Cars are accumulating in the North-South road, and so Cars 1, 2, 3 and 4 are not going anywhere.  Now, 3 can choose to leave a gap for 5 to pass through.

The game here is between Cars 3 and 4.  If Car 3 feared that Car 4 might try to overtake by taking advantage of the gap, it would move forward and block the intersection.

So, in this case, an expectation of unfairness could lead to a solution that is not socially optimal.  If people considered each other to be fair, there could be a socially optimal solution.

Scenario 3

No traffic lights.  There are two lanes going in the same direction (South to North).  Traffic is stalled in the North-South direction (Cars 3, 4, 8 and 9 are going nowhere).  Cars 3 and 8 have the option of leaving a channel open for Car 5 to pass through.

What’s interesting here is that if Car 3 suspects that Car 8 might close the intersection, Car 3 would do well to close the intersection itself (preventing Arrangement 2).

So, the socially optimal solution is obtained only if there is strong mutual trust between the participants in this traffic pattern game.

Summary

I find these traffic patterns interesting because they could be indicators of local ethics. When you visit a new place, you might be able to get an idea of how fair, trustworthy and rule-abiding the local people are, just by observing the traffic?

What game theory says about why gas stations are built next to each other

This lovely video tells you about the concept of a Socially Optimal Solution – a solution that works very well for society – and why such solutions are often not stable.

It also explains the concept of a Nash Equilibrium – a solution that is stable, but not always optimal for society.

It also tells you how aggressive competition can lead to results that are not optimal for society.

Article on a possible migration: to Australia from India

When we started writing the Aiaioo Labs blog, I’d written an article “An echo of voices“ on how Tamil, a South Indian language, is phonetically similar to Dyirbal, a native Australian language.

Now, a genetic study seems to have found some evidence of migration from India to Australia some 4000 years ago.

I still haven’t had the opportunity to learn any Australian languages.  I can’t wait to do so.

How algorithms can ‘shape’ our world – Kevin Slavin

In our first blog post of the year, we’d like to share a talk by Kevin Slavin.

It is about how math (especially algorithms) has ’transitioned from being something that we extract and derive from the world to something that actually starts to shape it.’

One of the posts we most enjoyed writing last year was related to this:

It was about algorithms to combat poverty.

We also wrote about how math might someday shape our values.

Frameworks for evaluating values

I recently came across a very interesting 2001 paper by Daphne Koller that dealt with influence diagrams and how they could be applied to game theory.  I came across the paper while doing some background reading on a talk on decision making in accordance with our core values by a friend of mine, Somik Raha.

Influence diagrams are a formalism (very similar to probabilistic graphical models) that are used for making decisions.

What Somik Raha has attempted to do is come up with a framework for making decisions while also taking one’s values into account (either as constraints or as inputs into the decision model).  To do that he proposes extensions to influence diagrams.

What I found interesting when I thought about Daphne Koller’s work and Somik’s together, is that they could possibly give you a framework to evaluate your values.

Koller’s formalism reduces to a game theoretic model, which can be evaluated to determine the outcome of the decisions made by a group of people.

Plug in a formalism based on Somik’s ideas and you just might be able to create a way to quantity the benefits of values.

The Importance of Values?

I have been thinking a bit about values these days because there has been a horrific gang rape in Delhi, and there have recently been numerous incidents of bad driving where friends of mine have been injured in Bangalore.  Then there is corruption.  Our society seems to be quite happy with inequality and vast differences in the distribution of wealth.  It make me wonder if our values are to blame.

I have often wondered whether some of our problems originate in our value systems and whether the value systems that we consider sacrosanct in India are really very good ones.

Let me take just a couple of values that most Indians would consider to be very good values

1. Non-violence
2. Obedience

and let’s discuss them in more detail.

Non-violence

This value appeals not just to people in India.  You see variants of the value of non-violence appear in Tolstoy’s writings and in Semitic religions, as you can see from the Bible (“turn the other cheek”) and the Quran (“give alms to one who begs from you, even if he comes on the back of a horse”).

The issue with this sort of value is that it makes a person (and those around him/her) extremely vulnerable to injustice.

In India, we restrict the liberty of women – in their choice of clothing, company and lifestyle – for fear that they could be in danger if they violated societal norms.  This shows that none of us want to fight society or cross swords with someone who might make disparaging comments about personal choices.

Moreover, possibly as a result of the value of non-violence, very few Indians if any are taught fighting skills in school.  So, even if a person really wanted to act, say to protect a friend, he or she might not really have the skills to take down an aggressor.

So, instead of protecting and standing up for people who might be vulnerable, we become their tormentors and make their lives more miserable, just so we don’t have to get our hands dirty, or because we don’t have the skills and strength to do squat.

I’ve written about how bribes are openly collected by traffic policemen.  It should be very easy to put a stop to such behavior if you’re willing to fight.

If non-violence is not a core value, then how do we protect people from tearing each other to bits?

We could start with a question like:  non-violence for what purpose?  (turning it into an extrinsic value)

If the answer is something like, “so that the weak feel protected”, why not make protecting the weak our core value?

I’d prefer teaching kids values like “Don’t ever turn your back on a bully” rather than values like “Don’t fight anybody, and just come home safe, child!”

Obedience

“Our child is such an obedient kid!” is something that I never questioned as a child, but these days I begin to see why that might not be such a good thing.

In India, we have trained Everybody to be obedient to Everybody. Parents expect complete obedience from Children. The Police expect complete obedience from People.  The Politicians expect complete obedience from Police.  The Older expect complete obedience from the Younger.  Teachers expect complete obedience from Students.  Bosses expect complete obedience from Reports.  Car Drivers expect Bike Riders to get out of their way.  You get the idea.

What makes this dangerous is that obedience is a word that is very close to subservience.

At the end of the day, you have a hierarchy of people in India with relations of subservience to each other.

This can be very dangerous in many ways.

1.  It can stifle creativity and problem-solving – with ideas flowing down a hierarchy rather than leaping up, down and across.  A good example is how parliament will not accept ideas from people for a bill against corruption (members of the Indian parliament actually claimed that parliament is supreme in a democracy – even though the constitution of India starts with the words ‘We the people’ – and in spite of the fact that a really large number of MPs have criminal cases pending against them).  Another is the Prime Minister refusing to come to meet protesters in Delhi.

2.  It can leave young people ill-equipped to defend their personal boundaries (and defend it strongly – with violence if necessary).  I read in a paper on rape that many rapists approach victims by testing their boundaries.  They make comments and otherwise violate the intended victim’s personal boundaries.  If these are not strongly resisted, the probability of an assault becomes greater.  Another strategy used by rapists is to move their intended victim to a new location where they are more vulnerable.  It is very important for people to be conditioned so that they do not obey an order by an attacker to relocate under any circumstances.

3.  The hierarchies usually put stronger parties in a dominant position (anyone bigger, stronger, richer gets to be the boss).  I have had friends knocked down by buses and by drivers in bigger vehicles, at times deliberately.  I have heard men say that they feel like knocking down girls who overtake them just to put them in their place.  In their place? In their place in the hierarchy no doubt!

4.  The obedience hierarchy allows someone else to make decisions for you in domains that should be your own personal decision space.  There have to be decisions that you make alone and where no one else has the right to intervene if you do not choose to include them in the decision making process.  This personal space is not well-accepted in India.

5.  Obedience just means conforming to rules.  It means valuing rules above truth.  I see that in Confucian and Indian philosophy a great emphasis has been placed on conforming to the rules of society or of the powers that be.  Gandhi for example set very interesting rules for himself and made sure that at any cost he would not break the rules (rules about food, about dressing, about his relationship with his wife).  His whole life was about conforming to difficult rules that he set for himself.  Ancient Greek philosophers on the other hand focused on learning about what is natural (or on what is true – on describing how things are) and on quality.

For the above reasons, I think that obedience as an absolute value is not entirely harmless.

I’d love to replace it with something else – something that would not have a boss-over hierarchy as a side-effect.

With what can we replace it?

What do you get from being obedient, or maybe who gets what by you being obedient?

Well, I don’t really know.

But I’d rather hear someone say “be honest” and “be trustworthy” and “be proud” than “be obedient”.

Summary

I understand that we as Indians are very proud of our values but I’ve tried to argue that our values need to be re-examined.  Especially in the context of the horrid rape that took place in Delhi.  Especially in the context of the fear that corrupt politicians can put into people who oppose them (remember the activist who was hacked to death on the orders of a corporator from Bangalore, or the shutdown that the former Chief Minister of the State ordered when he was about to be investigated for corrupt dealings).  Especially in the context of our reluctance to and our inability to put up a fight on behalf of the good, or the daring or the vulnerable and against things and people who need to be fought.

Personally, I’d love to see the day when we replace all our values with just the value of personal  trustworthiness.

Trustworthiness as a value would mean we’d fight for each other.  It would mean we’d protect the weak.  It would mean we’d be on time.  It would mean we’d be honest.  It would mean we’d be capable and skilled and strong.  It would mean we’d be proud of each other.  It might mean we’d never lose another war.

Reading Koller’s and Somik’s work you get the feeling that one day you might be able to evaluate the comparative benefits of two sets of values, and pick the better one, using plain math.

And hopefully, by showing them mathematical proofs, you can convince people to change their values and pick better ones for themselves.