Month: December 2017

Mechanical Consciousness and Attention

I’d written an article a few weeks ago touching on the subject of what makes us conscious.

I wrote that consciousness was merely models of the world stored in memory.  In other words, “anything – even if it is a machine – that can actively create a model of something that it senses (is aware of) and store it in memory in such a way as to permit retrieval of the model, is conscious of it.”

It then came to my notice that very similar thoughts had been expressed in a 2013 book by Michael A. Graziano, a Princeton neuroscientist, titled “Consciousness and the Social Brain”.

So, I purchased the book and read it.  It set out a theory of consciousness called the “Attention Schema Theory” that Graziano had discovered through his work on social neuroscience (in particular this paper that he’d written in 2011).

Graziano describes consciousness (awareness) as “the brain’s simplified, schematic model of the complicated, data-handling process of attention”.

I liked Graziano’s arguments on attention being required for consciousness and felt they deserved to be discussed in some depth here.

So, in this article, I go over Graziano’s Attention Schema Theory of consciousness and then share some further thoughts that I had about it.

Graziano’s Attention Schema Theory

Graziano gives a very concise and pithy description of what he means by “attention” and by “schema” and links them both very simply to awareness.  He says, and I quote:

Attention is not data encoded in the brain; it is a data handling method.  It is an act.  It is something the brain does, a procedure, an emergent process.  Signals compete with each other and a winner emerges – like bubbles rising up out of the water.  As circumstances shift, a new winner emerges.  There is on reason for the brain to have any explicit knowledge about the process or dynamics of attention.  Water boils but has no knowledge of how it does it.

I am suggesting, however, that in addition to doing attention, the brain also constructs a description of attention, a quick sketch of it so to speak, and awareness is that description.

A schema is a coherent set of information that in a simplified by useful way, represents something more complex.  In the present theory, awareness is an attention schema.  It is not attention, but rather a simplified, useful description of attention.

Then, Graziano goes on to provide certain reasons for thinking that awareness (what I called consciousness in my own post on “Mechanical Consciousness”) is an attention schema.

He identifies eight similarities between attention and awareness and argues that consciousness is the process of creating a representation of attention in the brain.

Eight key similarities

If the hypothesis is correct, if awareness is a schema that describes attention, then we should be able to find similarities between awareness and attention.  These similarities have been noted before by many scientists.  Here I am suggesting a specific reason why awareness and attention are so similar to each other:  the one is the brain’s schematic description of the other.  Awareness is a sketch of attention.  Below I list eight key similarities.

  1. Both involve a target.  You attend to something.  You are aware of something.

  2. Both involve an agent.  Attention is performed by the brain.  Awareness is performed by the “I” who is aware.

  3. Both are selective.  Only a small fraction of available information is attended at any one time.  Awareness is selective in the same way.  You are aware of only a tiny amount of the information impinging of your senses at any one time.

  4. Both are graded.  Attention typically has a single focus but while attending mostly to A, the brain spares some attention for B.  Awareness also has a focus and is graded in the same manner.  One can be mostly intently aware of A and a little aware of B.

  5. Both operate on similar domains of information.  Although most studies of attention focus on vision, it is certainly not limited to vision.  The same signal enhancement can be applied to any of the five senses, to a thought, to an emotion, to a recalled memory, or to a plan to make a movement, for example.  Likewise one can be aware of the same range of items.  If you can attend to it, then you can be aware of it.

  6. Both imply an effect on behaviour.  When the brain attends to something, the neural signals are enhanced … and have a greater impact on behaviour.  Likewise, when you are aware of something, you can choose to act on it.  Both, therefore, imply an ability to drive behaviour.

  7. Both imply deep processing.  Attention is when an information processor devotes computing resources to an information set.  Awareness implies an intelligence seizing on, being occupied by, experiencing, or knowing something.

  8. Finally, and particularly tellingly, awareness almost always tracks attention.  Awareness is like a needle on a dial pointing more or less to the state of one’s attention.  At any moment in time, the information that is attended usually matches the information that reaches awareness … Awareness is undoubtedly a close but imperfect indicator of attention.

I believe that Graziano also linked attention to consciousness because of his earlier work on the neuroscience of social behaviour where he postulates that one being identifies another as conscious by watching it direct its attention to something.  So, he seems to suggest at one point in the book that since we identify those things as conscious that appear to pay attention to something, attention must be central to consciousness.

Graziano also says that attention enhances awareness and awareness enhances attention.  He says:

If the theory is correct, then awareness is a description, a representation, constructed in the brain.  The thing being represented is attention.  But attention is the process of enhancing representations in the brain.

I must say that I agree with all of the above, with all that Graziano has said, except for this:

The thing being represented is attention. 

What is being represented is the world that is being perceived.

I agree with everything else.

So, I would write the above as: “awareness is a description, a representation, constructed in the brain.  The thing being represented is the world perceived by the senses.  But attention is the process of enhancing representations in the brain.

So I agree that attention is important for choosing which parts of the information that the senses collect get enhanced into more abstract representations.

I agree that when a viewer looks at a street full of people, the viewer’s brain may pay attention to one person on that street and be more aware of that person than the rest.  I think that what happens is that when the brain pays attention to that one person, it constructs and stores in memory a more detailed model of that person and less detailed models of all the rest of the scene of the street and associates them more strongly with things in the present.

However, explaining points 1 through 8 in the “eight key similarities” list is a bit more difficult.

The question here is what makes the one person one pays attention to appear more salient.

Graziano takes the view that explaining that salience requires the assumption that attention is required for consciousness.

I wonder if that is really required.

Graziano described attention as the process of enhancing a model of the world (creating a more detailed model of a part of a less detailed model of the world and associating those details more strongly with other salient things at the present time in memory).

But Graziano doesn’t explain why that which we attend to is salient in our awareness while other things are not.

He assumes that there is something that makes that happen and that is part of consciousness.

I think we can explain the salience of what we attend to if we make two assumptions:

  1. That attending to something allows us to make more memories of it as time passes.
  2. More recent memories are more salient.

So, let’s say there is a sequence of moments in time t1, t2, t3 and so on.

Let’s say at t1, a conscious viewer has looked out at a street without attending to anything.

His brain constructs rudimentary models of everything in view, perhaps allowing the user to discern humans and objects in the view.

Let’s say one of the humans draws the viewer’s attention so at time t2, the viewer transfers his/her attention to that human.

Now, according to Graziano’s views on attention, the viewer starts to form a more detailed model of the human who is the subject of the viewer’s attention at time t2.

If the viewer’s attention remains on the same human, then the viewer continues to store in memory models of the human that (s)he is paying attention to at t3.

This makes that human more salient in the viewer’s mind.

So, if we assume that more recent memories are more salient, we can explain consciousness without treating attention as something special, but merely as a process of creating and storing models of a part of what the senses are bringing to the viewer’s brain in memory.

We can explain all 8 similarities by using the above explanation/assumption of salience.

1, 3, 4, 6 and 8 all follow from the idea that our newest models in memory are those of what we attended to and therefore what we attended to is what is most salient in our awareness.

2 happens because the brain that is attending is also the same that is consigning models to memory.

5 and 7 follow from our definition of attention as a process of enhancing information.

So, it seems that we can explain all the above similarities by just speaking of attention as a process of creating models.

Our consciousness can still be explained merely in terms of those models that are stored in memory in a retrievable manner, without involving attention in the explanation.

I think there are advantages to involving attention in an explanation of consciousness.

Advantages of Involving Attention in an Explanation of Consciousness

a)  It immediately excludes devices like cameras from any suggestion of consciousness.

b)  It intuitively explains why some things in our awareness are more salient than others?

Disadvantages of Involving Attention in an Explanation of Consciousness

a)  It excludes other living things like trees that are not known to have an attention mechanism from being counted as conscious.  We don’t see trees paying attention to things.  However, trees are aware of and respond to sunlight, the seasons and other conditions in their environment.  So, I would argue that trees are conscious.  A view of consciousness as emanating from the creation of models of the world (with or without the use of attention to enhance models) might be better able to accommodate the plant kingdom.

b)  If we say that we are conscious of only those parts of the retrievable models in our brain/memory that we pay attention to (that make it into the attention schema, in Graziano’s words), then we have to answer the question of what it is in our attention that makes it so special that it can give a part of a model of the world the kiss of consciousness.  We could avoid the difficulty by saying instead that some parts of the models in our brain are salient because they were stored/refreshed/enhanced in memory more recently and therefore are more easily retrievable and have a more ponderous effect on anything we think or do.

So, I would prefer it if Graziano were to define consciousness as: “a description, a representation, constructed in the brain.  The thing being represented is the world perceived by the senses.  But attention is the process of enhancing representations in the brain.

That would result in the explanatory power he seeks and would also avoid the problem of assigning to attention the magical ability to create consciousness.

Mechanical Chef

Mechanical Chef

On November 17th, 2017 at the Maker Faire in Bangalore, on stage before an audience of a few hundred people, a machine cooked a simple Indian side-dish – a potato fry.

The next day it prepared a pot of rice.

In so doing, it became the world’s first machine to cook an Indian meal autonomously.

23593357_1514971115237798_1402609992841796738_o (1)

The Mechanical Chef is the coming to fruition of an idea that germinated in the year 2012, when a colleague and I were walking down Millers Road, on the way to a cup of coffee at a roadside store, and I asked the colleague if there was any way in which we could really change the world and ‘make a dent in the universe’ as it were, in the present millennium.

The colleague was a woman, and she said that if Indian women didn’t have to do any cooking, it would save a large fraction of half a billion women in India 3 hours a day every day of their lives.

That day in 2012, my colleague, Princiya and I started dissecting Indian cooking.

We came up with simple operations that underlay Indian recipes that did not require shaping (we aren’t aiming to prepare rotis {there’s a machine for that already}, idlis or dosas {again, there’s a machine available for that}).

And they were:

  1. Adding ingredients
  2. Heating
  3. Manipulations such as stirring, sieving, grinding, whisking and powdering
  4. Transferring from one container to another

And we came up with a machine that could do all the above.

At the time, there was a Chinese cooking robot which could cook stir-fried (wok-friend) dishes:

and there were several experimental machines like Moley which used expensive robotic arms for cooking.

There is even another team in Bangalore that is also working at present on a cooking robot for Indian food!


Our First Design

In 2012 we came up with a much simpler architecture which could support all the above cooking functions (the ones that would be needed for Indian cooking).  It was a type of sensor network for cooking.

It was an improvement over the Chinese automated wok in that it could cook multiple dishes at the same time, and it was an improvement over the robotic arms because it was much simpler and therefore more cost-effective.

We took the drawings to Dr. Raju of Systemantics (a maker of industrial robots in Bangalore) one day.  He listened to us patiently and observed that our machine would need very powerful motors in order to lift heavy vessels up against the force of gravity.

This is a common mistake that firms developing cooking robots make.  Going against gravity is never a good idea when you have to carry kilograms of ingredients in a vessel.  The motors end up having to be big and powerful and friction becomes a very big hindrance to smooth movement.

So, we went back to the drawing board.


Our Second Design

Last year (in 2016), we developed a machine that looked a bit like this one at MIT:

… and this one …

In our second design, the cooking vessels moved on a linear rail and ingredients dropped into them from above much like they do in the machines above.

We did not have a grinder and could not transfer ingredients from one container to another at will in our design.

But as we analysed Indian cooking recipes further, we realized that the vast majority of Indian dishes did not need any transferring of ingredients across vessels, and even if they did, we could stop the cooking at a certain point and ask a human to do the transfer without greatly impacting the automatic cooking experience.

We could also get by without grinding for many dishes if we used specially made powders in our cooking machine.

It was with this design that I went to my old friend Prashanth G. B. N., a mechanical engineer and designer of hardware systems, for his advice.

He took a look at the drawings and felt we would have to make the machine even simpler if it had to be used in Indian homes.


Our Third Design

He took a sheet of paper and sketched on it a design that had only rotary movements.

“Rotary motion is far simpler to build out than linear motion” he explained.

After developing a detailed drawing of the machine, Prashanth and I took out a patent on the same.


From Design to Reality

The Mechanical Chef however only turned into reality when Arpit Sharma, an aerospace engineer from Rajasthan joined me and solved all the mechanical problems that remained.

We had to solve the problem of dispensing powders from a removable and easy-to-wash and easy-to-assemble container.

We had to find ways to completely dissociate the parts that came into contact with the food from the parts that came into contact with the electricals so that the former would be easy to clean.

We had to minimize the number of parts that needed to be cleaned after every cook.

We needed to finesse the electronics and electricals – a job which was often summarily dropped into the lap of a young engineer from Mysore – Avinash Bharadwaj.

To support ourselves as we worked on all these problems and to pay for the hardware, we applied for a grant from the Department of Science and Technology of the Government of India through IIT-Bombay.  In October of this year, we received the grant.

The first prototype that Arpit Sharma built looked like this.


And it worked!


The Proof of the Pudding is in the Eating

Here’s a video of the machine at work.  And you’re welcome to stop by and be one of the first humans on earth to eat Indian food cooked by a robot.

Here’s how it stirs.

Here’s our mini funding pitch!

Here’s our team at the office with the Mechanical Chef.


There’s a little website for it:

Write to us if you’d like to drop by and see us.