Month: June 2016

Fun with Text – Managing Text Analytics

The year is 2016.

I’m a year older than when I designed the text analytics lecture titled “Fun with Text – Hacking Text Analytics“.

Yesterday, I found myself giving a follow on lecture titled “Fun with Text – Managing Text Analytics”.

Here are the slides:

“Hacking Text Analytics” was meant to help students understand a range text analytics problems by reducing them into simpler problems.

But it was designed with the understanding that they would hack their own text analytics tools.

However, in project after project, I was seeing that engineers tended not to build their own text analytics tools, but instead rely on handy and widely available open source products, and that the main thing they needed to learn was how to use them.

So, when I was asked to lecture to an audience at the NASSCOM Big Data and Analytics Summit in Hyderabad, and was advised that a large part of the audience might be non-technical, and could I please base the talk on use-cases, I tried a different tack.

So I designed another lecture “Fun with Text – Managing Text Analytics” about:

  • 3 types of opportunities for text analytics that typically exist in every vertical
  • 3 use cases dealing with each of these types of opportunities
  • 3 mistakes to avoid and 3 things to embrace

And the take away from it is how to go about solving a typical business problem (involving text), using text analytics.

Enjoy the slides!

Visit Aiaioo Labs


Mathematics of Religious Violence

Pulse Vigil in Minneapolis (courtesy of Wikipedia)

In this article, I attempt to study the mechanisms used by religions for the propagation of their beliefs and to establish that these mechanisms foster violence. Moreover, I show that over time, the tendency to violence in religions can only increase.

For the purpose of this article, I define a religion as a collection of religious fictions (a religious fiction being defined as a belief concerning a supernatural force).  I call such statements ‘fictions’ not because I assert the truth or falsehood of those statements, but because they tend to form part of semi-coherent narratives.

First, it will be necessary to understand the mechanism by which religions propagate themselves.

Like anything subject to the laws of evolution, religious memes compete to establish better defensive barriers and better incentives for promoters.

The mechanisms for their propagation have therefore evolved with time as can be seen through a study of ancient and modern religions.

The earliest mechanisms for growth probably involved incentives for individuals.

  1. Incentives for Individuals

Early religions seem to have been explanatory in nature, and their main function appears to have been that of explaining physical phenomena and how the world came into existence.

The earliest religions of almost all ancient civilizations – be they the Greek, the Indian, the Chinese, the Anasazi or the Maya – seem to have dealt with the question of how the world come into existence.

The mythology involved could have provided incentives for art, providing some story-tellers or bards with a means to earn a living, and hence spread through art.

As humanity developed the means to travel over larger distances, these stories would have spread.

The religions that would have been most likely to survive would have been the ones with the most powerful stories and the lengthiest narratives (which would have been more profitable for their storytellers).

With the advent of agriculture, as humans began to live in settlements with a high population density, a new mechanism for propagation would have become more effective.

  1. Incentives for Organizations

With static populations in highly populated areas like cities and towns, religions would have begin to market themselves through religious organizations that maintained places of worship or organized religious events. These would have helped them reach more people within a town or city, just like a store in a busy street corner or a fair in a fair ground sells more goods than a push cart vendor pushing his wares through the streets of a town.

Individual story-tellers would never have been able to generate a sufficient surplus to build similar edifices or organize large scale events.  So religions that could create organizations that could benefit from them – professional priests – would have become more successful.

Fictions such as “If you contribute V to do W at place X or before deity Y at time T, you will have a better chance of succeeding at Z” would have been used to induce people to pay money for a service.  The earnings from such service offerings would then have been used to maintain the organizations, set up edifices or organized events which would have served to further propagate the religion.

What is important to observe is that once religions evolved the ability to generate sufficient income to sustain a priesthood, they could grow exponentially, because the more priests they had, the more people they could convince of the truth of their religious fictions.  And the more people they convinced, the more they would earn from the consumers, and the more priests they could pay for.  This cycle would repeat with increasing rates of growth and the largest religion in any location would grow into a monopoly.

The tendency for the larger organization to win (network effects) could provide such religions with a defensive barrier against new religions.

There is evidence of such defensive barriers in history.  Religions like Buddhism could only propagate themselves to places where no previous organized religions existed.  And in almost all places where  Buddhism took root, there were additions to make the Buddha a deity.  The deification would allow priests to say something like: “If you contribute V to do W before this image of the Buddha, you will gain merit and have a better chance of succeeding at Z”.

Buddhism couldn’t take permanent root in India because the Hindu religion with which it was in competition had at all times a larger organization and many more followers.

Challengers to existing organization-backed religions invariably required political support from the top namely the direct support of the king.  Examples include the support of the Egyptian Pharaoh Akhenaten to the monotheistic cult of the Aten in Egypt, the support of the Mauryan Emperor Ashoka in the case of Buddhism in India and the support of the Roman Emperor Constantine the First in the case of Christianity in Europe.

So, eventually, all religions in the world where people lived urban lives and were connected by trade routes came to rely on organizations of priests for their continued existence and for their growth.

And these religions, once they became dominant in any part of the world, became very difficult to displace.

But then there developed a class of religions which could displace religions that relied on priestly orders alone.

These were religions that in addition to wielding priestly orders of their own, propagated themselves through symbiotic relations with political powers.

  1. Propagation through symbiotic relations with political powers

The class of religions that could replace those that developed large organizations were those that could persuade political forces to assist in their propagation through military force.

Such religions would have had to be of service to political groups.

The symbiosis between a religion and a political force would have therefore depended on that religion’s ability to provide definite advantages to political leaders in furthering their political ends.

Political power is typically increased through war between political entities.

The number of soldiers that can be raised for war would be dependent on:

  1. the degree to which such soldiers are not dissuaded by fear of injury or death,
  2. the rewards that may be gained by the soldiers from war spoils, and
  3. the motivation to make the effort in the service of a cause.

Religions that rely on this strategy for propagating themselves increase all the above incentives by a) helping people overcome the fear of death and fear of defeat, by b) sanctioning immoral actions in war and by c) providing people with an excuse to go to war (providing them something to fight for).

To do that, they would have had to take recourse to one or more of the following fictions:

  1. Religious fictions promising an after-life with benefits
  2. Religious fictions promising divine help in war
  3. Religious fictions permitting unethical behaviour in/after war
  4. Religious fictions that lead to the exclusion of other religious/political groups

Let’s examine each of these in turn:

Feature 1:  Religious fiction of a desirable after-life

The Norse religion promised Norsemen that death in war gave a Viking a ticket to Valhalla – the hall of the heroic dead.

In the Christian religion, there is a place where souls are believed to go if they’ve been good in their lifetimes,and it’s called Heaven.   Souls of religious martyrs are pictured as ending up there: “Also I saw the souls of those who had been beheaded for the testimony of Jesus and for the word of God …  They came to life and reigned with Christ for a thousand years.” (Bible, Revelation 20:4).  Some people also seem to interpret the verse “Whoever finds his life will lose it, and whoever loses his life for my sake will find it” (Bible, Matthew 10:39) as a promise of good things in the afterlife for someone who gives up his/her life for religious reasons.

In the religion of Islam, there is a concept of a paradise or Jannah (the garden of paradise). It might be possible to interpret the following “Did ye think that ye would enter Heaven without Allah testing those of you who fought hard (In His Cause) and remained steadfast?(Qur’an, sura 3 (Al-i-Imran), ayah 142) as a promise that fighting in the cause of Islam shall lead to benefits in the afterlife.  The hadiths also apparently consider Jihad to be one of the 8 doors to entering Jannah.

Such beliefs can have a huge effect on the decision making process of a person considering going to war for a religious cause or a political entity claiming to represent a religious cause.

The person would have two alternatives:

  1. Staying away from war and facing social disapproval.
  2. Going to war, risking dying or returning rich from plunder.

The first alternative would entail no risk of death, but there would be the pressure of disapproval from social circles (especially in societies with a strong concept of honor).  The second alternative would be very risky because the worst-case outcome would be death.

The second choice would therefore be a logical choice only if the rewards-to-risks ration were more palatable than the disapproval involved in the first choice.  The risks in the second choice (dying in war) would feel more acceptable if there were an enticement attached to the worst-case scenario (the enticement of heaven after death).

As a result, a religion which sanctioned or rewarded dying for a cause would be able to raise larger armies of soldiers for a political leader than a religion that didn’t.

The benefit to the religion would be support for propagation of the religion by the political leaders and their soldiers (who often join the fight just to propagate the religion).

Feature 2:  Religious fiction promising divine help in war and divine sanction for war crimes

Just as religious fictions promising a felicitous afterlife serve to bring more soldiers to the battlefield, religious fictions promising victory in war help to keep more soldiers on the battlefield (prevent them from abandoning the fight and running away).

This is because the more a soldier believed that their side would win, the lower would be their expectation of desertion by their buddies, and lower the chances that they themselves would desert.  Consequently an army of soldiers confident of winning would have a higher chance of really winning (all else being equal) and a lower chance of defeat and death.

The above effect has been explained mathematically using game theory in an earlier post on this blog.

Essentially, you can build a game theoretic canonical form representation of an army of 2 soldiers as shown below.

In a war, the benefits to each soldier can be modeled as a bi-matrix (normal-form game) as follows:

soldier 2 fights soldier 2 flees
soldier 1 fights 5, 5
–5, 0
soldier 1 flees 0, -5
0, 0
Normal form or payoff matrix of a 2-player, 2-strategy game

The first of the two numbers in the matrix represents the payoff to soldier 1.

The second of the two numbers in the matrix represents the payoff to soldier 2.

(The soldiers win something (represented by 5 points) if their army wins; they win nothing if their army loses; and they lose their life (represented by -5 points) if they do not flee and their army loses; we assume the army wins if both soldiers do not flee and loses if one or both flee).

If soldier 1 trusts soldier 2 not to flee the battlefield, the best strategy for soldier 1 is to stay and fight as well (since he will then get more benefits than if he flees).

If soldier 1 does not trust soldier 2 to stay on the battlefield (if he suspects that soldier 2 will run away), then the best strategy for soldier 1 is to run away himself (so that he does not remain on the battlefield and get killed).

So, this model shows that if two equal 2 man armies meet on a battlefield, the one whose soldiers trust each other more will win.

So, religions of a certain kind can supply two of the most important needs of a political leader keen on fighting a war – finding soldiers and keeping soldiers.

To do these, they need to have a good story of the afterlife, and they need to credibly promise victory in war.

An early Jewish holy book (carried forward into the Bible) called the book of the Judges contained stories of wars that were won as promised by god.  In the stories, the Jews are shown as losing wars when not true to their religion, and winning them when true.

In the case of Islam, something similar can be seen after the defeat of the Muslim army in the Battle of Uhud in 625 A.D.  After the loss, the prophet Muhammad is said to have explained the loss as follows: “Allah did indeed fulfil His promise to you when ye with His permission Were about to annihilate your enemy,-until ye flinched and fell to disputing about the order, and disobeyed it after He brought you in sight (of the booty) which ye covet. Among you are some that hanker after this world and some that desire the Hereafter. Then did He divert you from your foes in order to test you but He forgave you: For Allah is full of grace to those who believe.

Feature 3:  Religious fictions permitting unethical behaviour in/after war

One other feature of certain religions that could have helped political leaders was religious sanction for crimes that would otherwise not receive social sanction.  There are examples of horrific massacres (justified through the premise of permission from the divine).

In the bible is a story of a Jewish prophet by the name of Moses requesting the Jews not to spare anyone from a community of people called the Midianites (that had attempted to involve the Jews in their religious practices) after they had been defeated.  Moses says: “Now therefore kill every male among the little ones, and kill every woman that hath known man by lying with him. But all the women children, that have not known a man by lying with him, keep alive for yourselves” (Bible, Numbers 31:17-18).

Muhammad is also reported to have participated in a massacre of a Jewish tribe (the Banu Qurayza) living in Medina after the successful defense of Medina by his army.  Ibn Ishaq, a Muslim historian writes: “Then they surrendered, and the Apostle confined them in Medina in the quarter of d. al-Harith, a woman of B. al-Najjar. Then the Apostle went out to the market of Medina (which is still its market today) and dug trenches in it. Then he sent for them and struck off their heads in those trenches as they were brought out to him in batches. Among them was the enemy of Allah Huyayy b. Akhtab and Ka`b b. Asad their chief. There were 600 or 700 in all, though some put the figure as high as 800 or 900.”  (After that, the Jewish women and children were divided up among the Muslims that had participated in the siege, and Muhammad himself selected one of the women, Rayhana, for himself).

In the Indian epic, The Marabharata, the deity Krishna repeatedly sanctions the killing through treachery of his enemies, ensuring the victory of the political leaders he is aligned with, though in the process violating all the rules of war and justice.

In all these cases, a religion enabled a political force to breach ethical norms and overcome natural human tendencies to forgiveness and compassion in its pursuit of political benefits – such as the complete elimination of an enemy, control over their lands and resources, and sex slaves as rewards for soldiers (in the Jewish and Muslim stories), and the winning of a kingdom through dishonest means (in the Hindu story).

These are not just fictions from a distant past.  The same fictions permit organizations like ISIS to act in a manner that would not be considered human in the present day.

Feature 4:  Exclusion of other religious/political groups

Feature 4 is the most troubling of all.  It appears that extreme religious views benefit a political associate of a religion more than moderate religious views.

That’s because more inclusive views in times of peace would not allow a quarrel to develop with neighboring communities and give the political forces an opportunity to start a war and consequently obtain wealth/domination from a victory.

So, more inclusive religious fictions would not generate the conflict required for expansion at the cost of other communities, the plundering of their lands or the rape of their women.

Less inclusive religious fictions could, on the other hand, if accompanied by military success, lead a group to dominance over more inclusive groups.

Without the frictions caused by less inclusive religious beliefs, military dominance would not translate into war and expansion.

An example can be seen in the case of the house of Saud.  Ibn Saud was the first king and founder of the third Saudi Arab kingdom (modern Saudi Arabia).

He positioned himself as the promoter of the teachings of a cleric – Muhammad ibn Abd al-Wahhab – who rejected the veneration of Muslim saints and their tombs.

So, Ibn Saud’s opponents in the Rashidi tribe of Arabia, who held a less strict view of Islam, would not have had any quarrel with Ibn Saud’s followers, and so would not have felt a need to muster in large enough numbers to make war on Ibn Saud.

On the other hand, since Ibn Saud’s followers would not have accepted the Rashidi tribe’s religious views, they would have felt it their religious duty to muster in large numbers to fight for their religion and punish the Rashidis for their sins.

History shows that this is precisely what happened and that Ibn Saud and his followers gained the upper hand in Saudi Arabia (and their extreme views became the norm there).

Subsequent events have shown that more and more extreme religious fictions have been used by political forces (the Taliban and then ISIS) to successfully displace less extreme religious fictions in their geographic area.

In India too, a similar process can be observed in Hinduism.  Different groups of adherents of Hinduism might hold one of the two possible attitudes towards cow slaughter.

One group might believe “It’s up to individuals to decide if they should or should not eat beef” while a second group might believe “Cows should be protected and shouldn’t be eaten”.

The first group would have no quarrel with the second (people who protect cows aren’t doing anything that goes against their more inclusive beliefs) and so would not be able to motivate its followers to engage in violence against the second group.

The second group on the other hand would have a serious quarrel with the some members of the first (some members of the first group might be doing something that they were bent on preventing) and so would be able to motivate violent opposition to the first group, especially if some of the members of the first group consumed the meat of cows.

As a result of the imbalance in motivation levels, and consequently, the imbalance in the number of people each group could mobilize to spend effort, wealth in its cause, eventually the second group would be expected to gain the upper hand (it would attain its objectives of preventing cow slaughter).

Let’s take another example, this time to do with Christianity.

Let’s say there is one group of Christians who believe that the most important religious fictions in the bible include: “Love your neighbour as yourself” and “If someone should smite you on one cheek, turn to him the other also.”

Let’s say there is a second group that believes that the most important religious fictions in the bible were: “No one comes to the Father but through the Son” and “Go ye to all the world and preach my gospel to every people”.

The first group of people would not have a quarrel with their neighbours on account of their beliefs.  They would be able to coexist with people of other religious persuasions.

As a result, they would never spread their way of thinking by forcing an extraneous group of people to come around to thinking their way or by subjugating those groups of people around who disagreed with them.

However, the second group of people would find it easier to find themselves in conflict with people around them possessed of different viewpoints, and would possibly change them or subjugate them (as happened with indigenous communities in the new world).

So, the imbalance in motivations gives rise to a tendency in all religions (that propagate themselves by association with political power) to shift towards more extreme views.

There is evidence of such a shift in the rise of ISIS and its symbiotic relationship with Wahhabism in the middle east.

There is evidence of such a shift in the rise of the fortunes of the BJP and its symbiotic relationship with monastic Hindu organizations and popular gurus who profess or excuse extreme views (Baba Ramdev for instance).

One might argue that there is evidence of such a shift in the rise of Trump and his symbiotic relationship with Tea Party conservatives.


Can we measure the degree of extremism of a dominant group’s religious fictions?  It appears that we might be able to do so.

We have no studies for this, but it is possible that one measure of the extremism of the religious fictions dominating in a geographical area is the number (as a fraction of the population) of members of minority religions who succeed in living in those areas.

As the favoured religious fictions of a dominant religion X become more and more extreme, they should lead (by the mechanisms listed above) to the elimination of more and more people of other religions from those areas.

So, in Gujarat, the reduction of the number of people professing Islam and other religions in Hindu-dominated localities of Ahmedabad might serve as a measure of the increase in extremeness of views of people of the majority faith in those localities.

Similarly, in parts of Kashmir where the majority religion is Islam, and in Muslim majority countries  such as Pakistan or Bangladesh, the very small populations of non-Muslim minorities remaining (and the expulsion or intimidation of the remnants) might provide a measure of the extremeness of the views of Islam (and their further reduction might possible be a signal of the slow demise of the more moderate and inclusive Sufi views that were once possibly more popular there).


I wanted to say that we have argued above that in order to displace older organizational religions, it becomes necessary for religions to propagate themselves violently through symbiotic relationships with amenable political players.

Moreover, we have argued that once a symbiotic pact with political entities is firmly in place, a pressure to move in the direction of more extreme (less inclusive) views results.

In other words, by the principles of natural selection, religions are forced to morph into more virulent / harmful / intolerant forms.

Similar to the way in which humans societies have developed in obedience to economic rules that force us to destroy competing animal species and our environment, religions too have developed in obedience to political and sociological rules that put pressure on them to morph into increasingly destructive entities that feed off the hatred and weaknesses of humanity.


About the author

The author is Cohan, a researcher at Aiaioo Labs with a keen interest in history, economics and of course, machine learning.