Deep Learning for NLP

Deep learning is usually associated with neural networks.

In this article, we show that generative classifiers are also capable of deep learning.

What is deep learning?

Deep learning is a method of machine learning involving the use of multiple processing layers to learn non-linear functions or boundaries.

What are generative classifiers?

Generative classifiers use the Bayes rule to invert probabilities of the features F given a class c into a prediction of the class c given the features F.

The class predicted by the classifier is the one yielding the highest P(c|F).

A commonly used generative classifier is the Naive Bayes classifier.  It has two layers (one for the features F and one for the classes C).

Deep learning using generative classifiers

The first thing you need for deep learning is a hidden layer.  So you add one more layer H between the C and F layers to get a Hierarchical Bayesian classifier (HBC).

Now, you can compute P(c|F) in a HBC in two ways:

Product of Sums
Computing P(c|F) using a Product of Sums
Sum of Products
Computing P(c|F) using a Sum of Products

The first equation computes P(c|F) using a product of sums (POS).  The second equation computes P(c|F) using a sum of products (SOP).

POS Equation

We discovered something very interesting about these two equations.

It turns out that if you use the first equation, the HBC reduces to a Naive Bayes classifier. Such an HBC can only learn linear (or quadratic) decision boundaries.

Consider the discrete XOR-like function shown in Figure 1.

hbc_figure_1

There is no way to separate the black dots from the white dots using one straight line.

Such a pattern can only be classified 100% correctly by a non-linear classifier.

If you train a multinomial Naive Bayes classifier on the data in Figure 1, you get the decision boundary seen in Figure 2a.

Note that the dotted area represents the class 1 and the clear area represents the class 0.

Multinomial NB Classifier Decision Boundary
Figure 2a: The decision boundary of a multinomial NB classifier (or a POS HBC).

It can be seen that no matter what the angle of the line is, at least one point of the four will be misclassified.

In this instance, it is the point at {5, 1} that is misclassified as 0 (since the clear area represents the class 0).

You get the same result if you use a POS HBC.

SOP Equation

Our research showed us that something amazing happens if you use the second equation.

With the “sum of products” equation, the HBC becomes capable of deep learning.

SOP + Multinomial Distribution

The decision boundary learnt by a multinomial non-linear HBC (one that computes the posterior using a sum of products of the hidden-node conditional feature probabilities) is shown in Figure 2b.

Decision boundary of a SOP HBC.
Figure 2b: Decision boundary learnt by a multinomial SOP HBC.

The boundary consists of two straight lines passing through the origin. They are angled in such a way that they separate the data points into the two required categories.

All four points are classified correctly since the points at {1, 1} and {5, 5} fall in the clear conical region which represents a classification of 0 whereas the other two points fall in the dotted region representing class 1.

Therefore, the multinomial non-linear hierarchical Bayes classifier can learn the non-linear function of Figure 1.

Gaussian Distribution

The decision boundary learnt by a Gaussian nonlinear HBC is shown in Figure 2c.

Decision Boundary of a Gaussian SOP HBC.
Figure 2c: Decision boundary learnt by a SOP HBC based on the Gaussian probability distribution.

The boundary consists of two quadratic curves separating the data points into the required categories.

Therefore, the Gaussian non-linear HBC can also learn the non-linear function depicted in Figure 1.

Conclusion

Since SOP HBCs are multilayered (with a layer of hidden nodes), and can learn non-linear decision boundaries, they can therefore be said to be capable of deep learning.

Applications to NLP

It turns out that the multinomial SOP HBC can outperform a number of linear classifiers at certain tasks.  For more information, read our paper.

Visit Aiaioo Labs

Decongesting Bangalore’s Roads: An Analysis of the BDA’s Detailed Project Report on the Proposed Flyover

In response to public demand, the Bangalore Development Authority (BDA) has finally released the Detailed Project Report (DPR) for the proposed steel flyover in Bangalore.

In this article we’ll attempt to address (using points from the DPR), the key question of whether the flyover will be of any use to commuters.

The DPR contains a study of traffic volumes at each of the intersections, and summarizes its findings in the following three diagrams (page 58) which are very easy to understand.

dpr_530dpr_531dpr_532

The Numbers

The numbers in the diagrams are peak hour traffic numbers at all junctions affected by the project in terms of PCUs/hr.  PCU stands for Passenger Car Unit.

So, the first diagram says for example, that at peak hour traffic, there is the equivalent of 6175 cars entering from Hebbal and 6949 cars exiting at Rajbhavan Road in one hour.

The second diagram shows the impact of the flyover on ground-level traffic.  It shows that the number of cars entering at ground level from Hebbal will drop to 3088 while the number of cars exiting at Rajbhavan Road will drop to 4122.

The excess traffic (3087 incoming at Hebbal and 1393 outgoing at Rajbhavan Road) will be carried on the flyover.

The incoming numbers add up (3087 + 3088 = 6175) as expected.

The outgoing numbers don’t add up (1393 + 4122 < 6949)!

The outgoing traffic numbers don't add up because there will no longer be a right turn at Basaveshwara Circle.  So, a part of the traffic volume decrease has nothing to do with the flyover!

Anyways, these calculations, if assumed correct, point to a reduction in traffic by 20% to 50% at the ground level.

Travel Time Calculations

Will the reduction in traffic lead to a corresponding decrease in travel time?

Not necessarily.

The DPR contains no estimate of reductions in travel time.

There are three reasons for doubting there will be huge reductions in overall travel time:

  1.  The impact of the constriction of the road leading to and from the flyover because of the ramps of the flyover needs to be taken into account.  The bottlenecks at the ramps could lead to traffic piling up at the entrances and the exists of the flyover.
  2. If the total capacity of the roads carrying traffic away from the flyover is too low, it could lead to traffic queueing up on the flyover itself.

Those who prepared the DPR should have run a simulation of the traffic on the flyover, below it and on the roads leading into and out of the flyover to determine if any savings in travel time would result or if serious backups on the flyover and around it could cancel any benefits.

A case in point is the flyover from the Electronic City software technology park (STP) to the Silk Board.  It might allow traffic to move fast on it, but it might be slowing down traffic inside the STP and on the road below it at its exit.

Flyover Effectiveness Conclusions

We don’t know if the travel time will decrease significantly unless the required simulations are done.

Public Transportation Conclusions

However, it is possible from the DPR to draw conclusions about the effectiveness of public transportation.

We see from the study that only 2% to 3.5% of the vehicles on the roads are city buses (pages 42-46).

So, if we doubled Bangalore’s bus fleet (which you can for the cost of the flyover) it would not increase the traffic on these roads by much but could replace almost all the private vehicle traffic not just on this stretch but all over the city (assuming people make the switch from private to public transport, and each bus carries 100 passengers).

Explanation & Calculations

Here’s how we can calculate that.

The proposed flyover will cost approximately 1800 crore rupees.

A TATA bus costs about 20 lakhs rupees.

So, you can buy about 7000 buses (and hire drivers/conductors for a year and build facilities for them) for 1800 crore rupees (or get 9000 buses without drivers/conductors or facilities).  I’m assuming that a quarter of the price of a bus will get you facilities and pay for the driver and conductor.

Now the BMTC runs around 7000 buses today.  So, for the price of the flyover, one could double the bus fleet.

We can show that doubling the fleet can drastically decrease the volume of traffic on these roads.

Let’s say that 10,000 vehicles used that stretch of the road.  We know that 2% of those vehicles were buses.  That’s 200 buses.  Let’s assume that all the other vehicles carry on average 2 passengers.  So, except for the 200 buses, the remaining 10,000 – 200 vehicles carry 20,000 – 400 passengers.  If we double the buses, we have 200 more  buses.  Now each bus can easily carry about 100 passengers (50 seated and 50 standing).  So, that means we can carry 20,000 passengers just by adding 200 buses (doubling the bus count).  That is the entire carrying capacity of all the other vehicles on the road!

So with a negligible increase in traffic (from 2% of current traffic to 4%), we can accommodate all the passengers of the remainder of the traffic using that stretch of Bellary Road today.

More Benefits of Public Transport

But it’s not just that!

For the cost of the flyover, we’d have doubled buses all over Bangalore!

So, we’d have added an equivalent carrying capacity to all the private vehicles on all the roads in Bangalore for the cost of this flyover!

That’s what this BDA DPR tells us!

Environment Benefits of Public Transport

But that’s not all!

There are still more benefits!  Think of the reduction in pollution.  Replacing all those private vehicles with equivalent buses would reduce pollution by 95%.

Assumption

In the above calculations, I’ve assumed that everyone will give up private transport for public transport.

That won’t happen in real life unless you get the same convenience from public transport.

It could happen if, like with the metro, bus passengers are:

  1. assured of getting a bus from a known key location to a desired key destination every ten minutes or at a known precise time (with bus tracking) and
  2. assured the buses are not overcrowded (the pleasantness of the travel is comparable to the pleasantness of private transport).

If you can get that sort of predictability, and comfort, then for those travelling on those routes to work, it would make more sense to use public transport than to use private transport.

So, it may need a lot more than doubling buses (mere capacity matching) to assure convenience and ensure that people prefer public to private transport.  It would also need route planning, bus tracking and highly predictable key routes.

Other Proposals

There are many other proposals for reducing congestion along the North South Bellary Road.

Here’s one:  http://www.deccanherald.com/content/561722/rail-link-kia-less-rs.html

This article says that there is an operational railway line between Yalahanka and Channasandra, and this can be extended easily till the airport up North and Baiyappanahalli in the East, taking airport traffic off the Bellary Road.

Estimated cost: Rs. 150 crores.  And it’s a public transport proposal, so it takes a lot of cars off the roads.

Visit Aiaioo Labs
 

Of Barriers and Free Trade

Many economists insist that free trade benefits everyone.  The argument goes like this:

Model 1

FreeMarketScenario2

Let’s say there are two countries, Country 1 and Country 2, each with a bread factory and two citizens who buy bread from their factory.

There is a trade barrier in place preventing the factories from selling to the other country.

Each factory makes a profit of $2 on each loaf of bread.

The factory in Country 1 earns $4 from its sales.  The people spend $40 on bread.

The factory in Country 2 also earns $4 from its sales of bread.  It’s citizens only spend $20.

So, the net expenditure of money in Country 1 is $36, and in Country 2 $16.

Now, let’s see what happens if the trade barrier is removed.

Everybody now buys from the factory in Country 2.

So the factory in Country 1 earns nothing (it sells nothing) but its citizens spend only $20.

The factory in Country 2 earns $8 from its sales of bread to four citizens now.  The citizens of country 2 spend the usual $20.

So, the net expenditure of money in Country 1 is now only $20 and in Country 2 only $12.

Economists argue that through this mechanism, the free market benefits all trading parties.

But it turns out that not everybody wins if you consider a different model – one that includes businesses shutting down and unemployment.

Model 2

When there are trade barriers, everyone works for the factories in their own countries.

FreeMarketScenario3

With the barriers in place, the factory in Country 1 pays a salary of $20 to its two employees.

The factory in Country 2, being more efficient, pays a salary of $40 to its employees.

Now the net earnings in Country 1 (earnings – expenditure) are $4 (because the earnings cancel out the expenditure of the citizens, and the factory earns $4).  The net earnings in Country 2 are now $64 (because the citizens spend only $10 each while earning $40, and the factory earns $4).

Now let’s say free trade is introduced.

The bread factory in Country 2 starts selling its bread at $10 in Country 1.

The bread factory in Country 1 shuts down (because no one will buy its bread at $20).

The citizens of Country 1 who worked for the bread factory no longer have an income.  They still have to buy bread to survive (using their savings) but their expenditure is lower not at $20.

So now Country 1 ends up with a net earnings of $ -20 (a drop of $24).

The net earnings in Country 2 go up to $68 (and increase of $4).

So in this model, not only does the earnings of Country 1 go down but also the earnings of the world as a whole (because there are fewer people gainfully employed).

However, things don’t stop there.

Model 3

The bread factory realizes that by hiring the citizens of Country 1 to work for it instead of citizens of Country 2, it can cut its cost of production in half (because salaries in Country 1 are half that of Country 2).  By doing so, it can start making profits of $6 per loaf of bread.

So now you have the citizens of Country 2 facing unemployment (no earnings).

So the net earnings of Country 2 drop to $4 (a drop of $64).

The earnings of Country 1 on the other hand go up to $20 (a gain of $40).

But there is again a drop in the earnings of the world, because though the same number of people are employed, it is higher-paid people who are now out of a job.

This is not compatible with a model of free trade that continually improves things for everybody.

Of course, I’ve used a very simple model, but it reflects the truth that when free trade is introduced, there are winners and losers and that the weaker parties are the losers.

There seems to be a way to predict who the winners and losers in any attempt to introduce free trade will be.

Force Model

In models of military engagements, the principle of force is used to predict the outcome.  Given roughly equal equipment and training, the stronger force almost always wins.

The reason for that is shown in the following diagram.  Let’s say there is a red force with 4 soldiers and a green force with 8 soldiers.  Each side fires a shot every minute.  Each soldier aims at an enemy soldier and fires, with a 50% probability of hitting his target.  After the first volley, the red force would have fired 4 shots and hit 2 green soldiers.  But the green force would have fired 8 shots and hit 4 red soldiers.

So the losses would be disproportionately higher for the weaker party, and the green force would win the battle with the loss of two of their own.

ForceModel1

It appears the same model can be used to predict the winners in economics too.

In economic models, the contending forces would be firms or employees.

Larger and better equipped firms (which can produce cheaper products) can be expected to win (push small firms or more expensive employees in direct competition out of business) in any market where free trade is introduced.

For example, before India traded freely with the USA, there were many local soft drink manufacturers (Torino, Thums Up and Limca) in India.  When  Coca Cola and Pepsi entered the market, all the Indian soft drink brands had to sell out.

Similarly cheaper workforces (which cost less to hire) can be expected to win.

An example would be the use of immigrant labour in many countries (especially the USA) for agriculture.

So, how can a weaker economic segment (be they workers or industry) be protected?

Through barriers (barriers allowed by trade agreements).  Barriers (something that a force can shelter behind) may be used to protect weaker forces of any kind.

Here are a few barriers used in economics.  As you can see, barriers to trade are already in heavy use though people talk as if free trade is widely practised.

Barrier 1

The US workforce in the computer industry being more expensive than an Indian workforce, has been protected by visa and fee barriers.

There are visa quotas and a fee  levied on Indian firms – to the tune of $5000 per visa for H-1B and L-1 visas).  The fee costs the Indian software industry $1 billion to $1.5 billion annually.

Not palatable to Indian industry of course, but effective in protecting high-end jobs in the USA.

Now what I don’t understand is why the USA uses barriers to protect jobs for skilled people who are highly qualified and quite capable of acquiring more skills and competing on quality.

I would suppose that if the USA used barriers, they should probably do so to protect jobs at the low end of the economy (helping out poorly skilled or unskilled workers who’d find it harder to acquire more skills or compete on quality) as India does (through a minimum salary requirement for work visas, which is aimed squarely at keeping out cheaper Chinese labour from infrastructure [say road building] projects).

Barrier 2

During the colonial era, barriers to the import of Indian products were used to protect a nascent textile industry and jobs in England.

I quote from an 1840 English parliamentary inquiry about India (taken from the above article):

Before a British Parliamentary Committee in 1840] Montgomery Martin stated that he . . . was convinced that an outrage had been committed ‘by reason of the outcry for free trade on the part of England without permitting India a free trade herself.’ After supplying statistical data of Indian textile exports to Great Britain, he pointed out that between 1815–1832 prohibitive duties ranging from 10 to 20, 30, 50, 100 and 1,000 per cent were levied on articles from India. … ‘Had this not been the case,’ wrote Horace Wilson in his 1826 History of British India, ‘the mills of Paisley and Manchester would have been stopped in their outset, and could scarcely have been again set in motion, even by the power of steam. They were created by the sacrifice of Indian manufacture. Had India been independent, she could have retaliated, would have imposed prohibitive duties on British goods and thus have preserved her own productive industry from annihilation. This act of self-defence was not permitted her’” (Clairmonte 1960: 86-87).

Barrier 3

Another example can be seen in the policy of subsidizing the renewable energy industry in the USA and buying renewable energy locally.

The use of models to determine winners, as described above, can empower governments to decide which sectors of the economy are most vulnerable to competition (so action may be taken to protect them).

Many protective actions can be taken without violating free trade obligations.

Fair Barrier 1

The best example of a fair protection policy is progressive liberalization.

For example, when the Chinese government introduced foreign direct investment (FDI) into the retail sector in China, it did so gradually, increasing the percentage of ownership permitted to foreign owners over 15 years.

In the course of the 15 years, local firms learnt the tricks of the retail trade and were able to compete effectively against the new entrants.

In contrast, the Indian government went from discouraging the participation of Indian private firms in the railways and defense industries, to allowing 100% FDI in one shot, thus failing to give Indian firms the time to develop the capabilities or technical know-how to compete in these markets, with the result that bullet train and metro rail equipment needs to be imported in entirety (or imported with a ‘made in India’ veneer – assembled in Indian factories entirely owned and operated by French firms).

Fair Barrier 2

Another policy that could protect and benefit businesses in certain sectors is grants for nascent industries with vast future potential.  In the USA, there are small business grants, research grants and subsidies for small firms.

In contrast, the government of India seems to have failed to create a level playing field for local startups.  I recall many years ago (at the height of the internet boom) when the government of India had a small business grants program, it had a clause specifically leaving out Indian software startups.  Somehow the government of India had decided that they would not make research grants available to local startups in the highest-growth industry in India at the time.

Here’s more on how the Indian government seems to not be as accessible to local startups as it is to larger global startups/firms competing with them.  This is a problem that needs to be fixed.

It’s easier for a foreign startup or MNC to get forgiveness for inadvertent violation than it is for a domestic startup (today foreign startups enjoy better access to political leaders and therefore easier forgiveness).

Uber got 1.5 years of forgiveness on payments that Ola did not get, and Amazon got forgiveness for having its own supply centres, and not following the marketplace model, which Flipkart did not get. Flipkart had to change its model to comply with the marketplace model. Both Ola and FK were hurt by this favoritism to Uber and Amazon.

Fair Barrier 3

Another method is to strengthen the weakest parts of the workforce in the face of competition from migrant/immigrant labour.

One policy that could help protect and improve the lot of weaker parts of the US workforce might be higher taxation, and using the tax money to:

  1. Sponsor education programs (financial-need-based free college/training) to facilitate reemployment in growing industry segments.
  2. Revive industries that could employ that part of the workforce in larger numbers (the manufacturing industry).  One way to revive manufacturing that might be to create more outlets for US made goods and fund automation of manufacturing.
  3. Visa barriers (Indian work visas require a minimum salary that effectively protects less skilled workers).

Fair Barrier 4

Using regulatory and tax barriers to FDI (as is done in China) to generate scalable revenue for local businesses.

Conclusion

In conclusion, I wanted to point out that if governments could start using models such as the one above to calculate who might win in various scenarios involving free trade (instead of assuming that everyone wins), they might be able to formulate better and more equitable economic policies for their citizens.

The Fine Balance of FDI

Algodones_sand-dune-fence
Courtesy of the Wikipedia

Some governments, such as those of New Zealand and China have opposed FDI (Foreign Direct Investment) in various forms.

We examine in this article what their reasons might be.

  1. Do some countries reject FDI?

It appears that some do.

The Case of the USA

In the USA, there are no serious legal barriers to FDI, but there can be disapproval of and concerted attempts to block FDI.

In this article on factories set up by foreign investors in the USA, there is a discussion of opposition to Japanese manufacturers in the automobile sector decades ago.  I quote:

After Honda Motors opened the first Japanese auto plant in the U.S., in Marysville, Ohio, in the early nineteen-eighties, followed by an engine factory in nearby Anna, Ohio, the company faced an onslaught of vicious anti-Japanese ads on TV and in print, often supported by American manufacturing trade and labor groups.

The comparably subdued response to Chinese manufacturers speaks, on one hand, to changing circumstances, especially the broad acceptance of globalization in the United States and the desire, on the part of some politicians and business leaders, to create manufacturing jobs by whatever means necessary. But it also follows from a conclusion that American companies have reached about their Chinese counterparts: namely, that they are, thus far, relatively inconsequential rivals.


The Case of New Zealand

In October 2015, the government of New Zealand blocked the Chinese firm Pengxin’s attempts to buy (through a local subsidiary) a farm called Lochinver.

The government explained their decision as follows:

Land information minister Louise Upston told the BBC that its decision in September to block Pengxin’s purchase of Lochinver farm does not mean the country is not interested in attracting foreign investment.

“It’s [foreign investment’s] an important part of our economic strategy, but equally when we do have an application for sensitive land and assets – we will put it across the 21 criteria that we need to assess and make a decision based on that,” she says.

“We weren’t convinced that this particular application met that threshold, which is substantial and identifiable benefits for New Zealanders.”

Dr William Rolleston, president of the Federated Farmers of New Zealand, a group that lobbies on behalf of its farmer members, says some farmers are concerned about the scale of the purchases.

“New Zealanders don’t have an issue on ownership at a low level. No one would be concerned if 5% of farmland was owned by overseas buyers,” he says.

“But if 95% of the land in New Zealand was owned by overseas buyers, I think we would have an issue – it would reduce our strategic options in the future.”

Dr Rolleston’s sentiment echoes public concern from 2012 when Pengxin bought 16 dairy farms and sparked a debate about national identity.

During the last elections in 2014, opposition politicians stoked those fears by saying that New Zealanders risked becoming “tenants in their own land”

So, it appears that in New Zealand, the government wishes to keep resources such as land in local hands.


The Case of China

China seems to have laws that make it difficult for foreign firms to compete with local firms, making it a logical choice for them to outsource the manufacture of their goods to Chinese firms.

This is explained on Chinese Law Blog in “Building and Operating a China Factory.  Why Even Bother?

For example, if you are a Wholly Foreign Owned Enterprise (WFOE), you need to pay the government 20% of the profits of your China operations.

I quote:

First off, in your first year, you are going to essentially waste around $50,000 in just forming your WFOE, securing various government approvals, paying someone to figure out your taxes, and making up for all the mistakes you will make because you will be in, what is for you, a very strange land.  On top of that will be your taxes, which you are going to need to pay on just about everything.  Figure 20% on profits and even if you do not make profits, figure on them being imputed to you. And figure on having to pay around 40% to various of the Chinese governments as taxes on the salaries you pay your employees.

And all of this is going to mean that your costs are going to be considerably higher than whatever Chinese factory you are currently using to make your product. In Buying A Chinese Company? Why China Deals DON’T Get Done, I wrote of the way this domestic-foreign price differential works in the context of a client looking to buy its Chinese manufacturer:

I said that there is a good chance the Chinese manufacturer is paying half of its employees completely under the table and reporting to the government only half of what it was paying the other half. I then talked of how there is also a good chance the Chinese manufacturer is underpaying its taxes and of how its rent also may be paid under the table. I then said that this sort of thing may be all well and good for Chinese companies, but that if the US manufacturer were to buy this Chinese manufacturer, it would need to do so as a WFOE and it would then immediately be on a “whole ‘nother level” with respect to China’s various tax authorities.

Joint ventures with Chinese firms on the other hand apparently are difficult to retain control of.

This explains why Apple, instead of running its own factories in China, outsources the manufacturing to a Chinese firm – Foxconn.

So it appears that China, through its taxation mechanisms and through selective enforcement of its laws, deliberately blocks FDI into China.

Numbers show that in the past few decades, China received considerable inflows of FDI.

However, much of that FDI is attributable to Hong Kong (Hong Kong’s share of China’s FDI was 55% – ten times that of the USA).

The imbalance in FDI numbers can be seen in the following table taken from an out-of-print volume from the National Bureau of Economic Research in China.

china_fdi_table

So, it is evident from the above that most of China’s 1990s FDI came from Hong Kong and therefore was not really Foreign DI.

So, what benefits does blocking FDI give to a local area?


  1. Might barriers to FDI help economies?

In the case of China, the advantage seems obvious.

If foreign investors could open their own factories, they would hire Chinese labour and be able to own the scalable income from the proceeds of manufacturing, passing on only the salaries (which are low in China) to their employees in China.

It is only the inability of foreign investors to purchase and operate Chinese factories as cheaply that forces them to  outsource to Chinese firms.

So, the barriers to FDI in China help drive business to Chinese firms.

These policies can also force the transfer of technology to local firms.

Contrast this with the case of India.

India had a similar boom driven by cheap labour in the form of software services.

There were hundreds of small, medium and large firms in Bangalore offering software services to the financial sector in the USA.

However, a trend I have seen in recent years is for US firms to buy up medium-sized software firms in India and get their software related work executed by these now captive software teams than to outsource work to Indian firms.

So, the inability to get work drives many local IT services firms to sell their operations to US-based firms.

This effectively reduces the total revenue earned locally to just the salaries paid instead of salaries+profits.

I am not certain of how the numbers compare with the boom years but I think the absence of a road up the IT services value chain results in the middle being cut out, leaving only the very large firms and a host of relatively tiny shops in the local IT services market.

So, had FDI barriers existed in the software sector, it would have forced foreign consumers of IT services to access Indian programmers through an Indian corporate entity, increasing the share of the pie that accrued to the local economy, and increasing the quantity of business opportunities for local IT firms.


Take the case of New Zealand again.

The argument put forth by the government there is that they need to keep control of land (a scarce resource in New Zealand).

In other words, the restrictions on FDI are meant to allow local firms the retain control of revenue generating resources.

Contrast this with the software industry in India again.

In software the key resources are human resources (the engineers).  And around the world, software engineers are a scarce resource.

There was no barrier to FDI that would allow local firms to compete with foreign firms (which could pay more because of their stronger currencies) for those resources.

The result seems to be a stratification of software engineering resources by capability.

The engineers with the best skills (who could effectively develop products) mostly ended up in American software firms and can be paid salaries in the range of Rs. 60 lakhs per annum.

Engineers with less valuable skill-sets (who can at most configure, install, test or maintain products) seem to gravitate to Indian IT services firms where salaries seem to stagnate at about a quarter that amount.

This flight of resources might have contributed to preventing forays into product development by software services firms in India.

So, in a sense, by providing outside firms direct access to scarce resources, the opportunity to make a lot more money might have been allowed to slip away.

 

Conclusion

In conclusion, it may be said that sometimes, a clever use of barriers to FDI seems to help extract more revenue (by blocking access to valuable resources) and thus create larger surpluses for local businesses.  Those surpluses in turn help build more infrastructure and capacity and improve the economy.  The case of China, contrasted with that of India, seems to illustrate the case where such barriers might have been beneficial to the economy.

Fun with Text – Managing Text Analytics

The year is 2016.

I’m a year older than when I designed the text analytics lecture titled “Fun with Text – Hacking Text Analytics“.

Yesterday, I found myself giving a follow on lecture titled “Fun with Text – Managing Text Analytics”.

Here are the slides:

“Hacking Text Analytics” was meant to help students understand a range text analytics problems by reducing them into simpler problems.

But it was designed with the understanding that they would hack their own text analytics tools.

However, in project after project, I was seeing that engineers tended not to build their own text analytics tools, but instead rely on handy and widely available open source products, and that the main thing they needed to learn was how to use them.

So, when I was asked to lecture to an audience at the NASSCOM Big Data and Analytics Summit in Hyderabad, and was advised that a large part of the audience might be non-technical, and could I please base the talk on use-cases, I tried a different tack.

So I designed another lecture “Fun with Text – Managing Text Analytics” about:

  • 3 types of opportunities for text analytics that typically exist in every vertical
  • 3 use cases dealing with each of these types of opportunities
  • 3 mistakes to avoid and 3 things to embrace

And the take away from it is how to go about solving a typical business problem (involving text), using text analytics.

Enjoy the slides!

Visit Aiaioo Labs

Mathematics of Religious Violence

Pulse_Vigil
Pulse Vigil in Minneapolis (courtesy of Wikipedia)

In this article, I attempt to study the mechanisms used by religions for the propagation of their beliefs and to establish that these mechanisms foster violence. Moreover, I show that over time, the tendency to violence in religions can only increase.

For the purpose of this article, I define a religion as a collection of religious fictions (a religious fiction being defined as a belief concerning a supernatural force).  I call such statements ‘fictions’ not because I assert the truth or falsehood of those statements, but because they tend to form part of semi-coherent narratives.

First, it will be necessary to understand the mechanism by which religions propagate themselves.

Like anything subject to the laws of evolution, religious memes compete to establish better defensive barriers and better incentives for promoters.

The mechanisms for their propagation have therefore evolved with time as can be seen through a study of ancient and modern religions.

The earliest mechanisms for growth probably involved incentives for individuals.

  1. Incentives for Individuals

Early religions seem to have been explanatory in nature, and their main function appears to have been that of explaining physical phenomena and how the world came into existence.

The earliest religions of almost all ancient civilizations – be they the Greek, the Indian, the Chinese, the Anasazi or the Maya – seem to have dealt with the question of how the world come into existence.

The mythology involved could have provided incentives for art, providing some story-tellers or bards with a means to earn a living, and hence spread through art.

As humanity developed the means to travel over larger distances, these stories would have spread.

The religions that would have been most likely to survive would have been the ones with the most powerful stories and the lengthiest narratives (which would have been more profitable for their storytellers).

With the advent of agriculture, as humans began to live in settlements with a high population density, a new mechanism for propagation would have become more effective.

  1. Incentives for Organizations

With static populations in highly populated areas like cities and towns, religions would have begin to market themselves through religious organizations that maintained places of worship or organized religious events. These would have helped them reach more people within a town or city, just like a store in a busy street corner or a fair in a fair ground sells more goods than a push cart vendor pushing his wares through the streets of a town.

Individual story-tellers would never have been able to generate a sufficient surplus to build similar edifices or organize large scale events.  So religions that could create organizations that could benefit from them – professional priests – would have become more successful.

Fictions such as “If you contribute V to do W at place X or before deity Y at time T, you will have a better chance of succeeding at Z” would have been used to induce people to pay money for a service.  The earnings from such service offerings would then have been used to maintain the organizations, set up edifices or organized events which would have served to further propagate the religion.

What is important to observe is that once religions evolved the ability to generate sufficient income to sustain a priesthood, they could grow exponentially, because the more priests they had, the more people they could convince of the truth of their religious fictions.  And the more people they convinced, the more they would earn from the consumers, and the more priests they could pay for.  This cycle would repeat with increasing rates of growth and the largest religion in any location would grow into a monopoly.

The tendency for the larger organization to win (network effects) could provide such religions with a defensive barrier against new religions.

There is evidence of such defensive barriers in history.  Religions like Buddhism could only propagate themselves to places where no previous organized religions existed.  And in almost all places where  Buddhism took root, there were additions to make the Buddha a deity.  The deification would allow priests to say something like: “If you contribute V to do W before this image of the Buddha, you will gain merit and have a better chance of succeeding at Z”.

Buddhism couldn’t take permanent root in India because the Hindu religion with which it was in competition had at all times a larger organization and many more followers.

Challengers to existing organization-backed religions invariably required political support from the top namely the direct support of the king.  Examples include the support of the Egyptian Pharaoh Akhenaten to the monotheistic cult of the Aten in Egypt, the support of the Mauryan Emperor Ashoka in the case of Buddhism in India and the support of the Roman Emperor Constantine the First in the case of Christianity in Europe.

So, eventually, all religions in the world where people lived urban lives and were connected by trade routes came to rely on organizations of priests for their continued existence and for their growth.

And these religions, once they became dominant in any part of the world, became very difficult to displace.

But then there developed a class of religions which could displace religions that relied on priestly orders alone.

These were religions that in addition to wielding priestly orders of their own, propagated themselves through symbiotic relations with political powers.

  1. Propagation through symbiotic relations with political powers

The class of religions that could replace those that developed large organizations were those that could persuade political forces to assist in their propagation through military force.

Such religions would have had to be of service to political groups.

The symbiosis between a religion and a political force would have therefore depended on that religion’s ability to provide definite advantages to political leaders in furthering their political ends.

Political power is typically increased through war between political entities.

The number of soldiers that can be raised for war would be dependent on:

  1. the degree to which such soldiers are not dissuaded by fear of injury or death,
  2. the rewards that may be gained by the soldiers from war spoils, and
  3. the motivation to make the effort in the service of a cause.

Religions that rely on this strategy for propagating themselves increase all the above incentives by a) helping people overcome the fear of death and fear of defeat, by b) sanctioning immoral actions in war and by c) providing people with an excuse to go to war (providing them something to fight for).

To do that, they would have had to take recourse to one or more of the following fictions:

  1. Religious fictions promising an after-life with benefits
  2. Religious fictions promising divine help in war
  3. Religious fictions permitting unethical behaviour in/after war
  4. Religious fictions that lead to the exclusion of other religious/political groups

Let’s examine each of these in turn:

Feature 1:  Religious fiction of a desirable after-life

The Norse religion promised Norsemen that death in war gave a Viking a ticket to Valhalla – the hall of the heroic dead.

In the Christian religion, there is a place where souls are believed to go if they’ve been good in their lifetimes,and it’s called Heaven.   Souls of religious martyrs are pictured as ending up there: “Also I saw the souls of those who had been beheaded for the testimony of Jesus and for the word of God …  They came to life and reigned with Christ for a thousand years.” (Bible, Revelation 20:4).  Some people also seem to interpret the verse “Whoever finds his life will lose it, and whoever loses his life for my sake will find it” (Bible, Matthew 10:39) as a promise of good things in the afterlife for someone who gives up his/her life for religious reasons.

In the religion of Islam, there is a concept of a paradise or Jannah (the garden of paradise). It might be possible to interpret the following “Did ye think that ye would enter Heaven without Allah testing those of you who fought hard (In His Cause) and remained steadfast?(Qur’an, sura 3 (Al-i-Imran), ayah 142) as a promise that fighting in the cause of Islam shall lead to benefits in the afterlife.  The hadiths also apparently consider Jihad to be one of the 8 doors to entering Jannah.

Such beliefs can have a huge effect on the decision making process of a person considering going to war for a religious cause or a political entity claiming to represent a religious cause.

The person would have two alternatives:

  1. Staying away from war and facing social disapproval.
  2. Going to war, risking dying or returning rich from plunder.

The first alternative would entail no risk of death, but there would be the pressure of disapproval from social circles (especially in societies with a strong concept of honor).  The second alternative would be very risky because the worst-case outcome would be death.

The second choice would therefore be a logical choice only if the rewards-to-risks ration were more palatable than the disapproval involved in the first choice.  The risks in the second choice (dying in war) would feel more acceptable if there were an enticement attached to the worst-case scenario (the enticement of heaven after death).

As a result, a religion which sanctioned or rewarded dying for a cause would be able to raise larger armies of soldiers for a political leader than a religion that didn’t.

The benefit to the religion would be support for propagation of the religion by the political leaders and their soldiers (who often join the fight just to propagate the religion).

Feature 2:  Religious fiction promising divine help in war and divine sanction for war crimes

Just as religious fictions promising a felicitous afterlife serve to bring more soldiers to the battlefield, religious fictions promising victory in war help to keep more soldiers on the battlefield (prevent them from abandoning the fight and running away).

This is because the more a soldier believed that their side would win, the lower would be their expectation of desertion by their buddies, and lower the chances that they themselves would desert.  Consequently an army of soldiers confident of winning would have a higher chance of really winning (all else being equal) and a lower chance of defeat and death.

The above effect has been explained mathematically using game theory in an earlier post on this blog.

Essentially, you can build a game theoretic canonical form representation of an army of 2 soldiers as shown below.

In a war, the benefits to each soldier can be modeled as a bi-matrix (normal-form game) as follows:

soldier 2 fights soldier 2 flees
soldier 1 fights 5, 5
–5, 0
soldier 1 flees 0, -5
0, 0
Normal form or payoff matrix of a 2-player, 2-strategy game

The first of the two numbers in the matrix represents the payoff to soldier 1.

The second of the two numbers in the matrix represents the payoff to soldier 2.

(The soldiers win something (represented by 5 points) if their army wins; they win nothing if their army loses; and they lose their life (represented by -5 points) if they do not flee and their army loses; we assume the army wins if both soldiers do not flee and loses if one or both flee).

If soldier 1 trusts soldier 2 not to flee the battlefield, the best strategy for soldier 1 is to stay and fight as well (since he will then get more benefits than if he flees).

If soldier 1 does not trust soldier 2 to stay on the battlefield (if he suspects that soldier 2 will run away), then the best strategy for soldier 1 is to run away himself (so that he does not remain on the battlefield and get killed).

So, this model shows that if two equal 2 man armies meet on a battlefield, the one whose soldiers trust each other more will win.

So, religions of a certain kind can supply two of the most important needs of a political leader keen on fighting a war – finding soldiers and keeping soldiers.

To do these, they need to have a good story of the afterlife, and they need to credibly promise victory in war.

An early Jewish holy book (carried forward into the Bible) called the book of the Judges contained stories of wars that were won as promised by god.  In the stories, the Jews are shown as losing wars when not true to their religion, and winning them when true.

In the case of Islam, something similar can be seen after the defeat of the Muslim army in the Battle of Uhud in 625 A.D.  After the loss, the prophet Muhammad is said to have explained the loss as follows: “Allah did indeed fulfil His promise to you when ye with His permission Were about to annihilate your enemy,-until ye flinched and fell to disputing about the order, and disobeyed it after He brought you in sight (of the booty) which ye covet. Among you are some that hanker after this world and some that desire the Hereafter. Then did He divert you from your foes in order to test you but He forgave you: For Allah is full of grace to those who believe.

Feature 3:  Religious fictions permitting unethical behaviour in/after war

One other feature of certain religions that could have helped political leaders was religious sanction for crimes that would otherwise not receive social sanction.  There are examples of horrific massacres (justified through the premise of permission from the divine).

In the bible is a story of a Jewish prophet by the name of Moses requesting the Jews not to spare anyone from a community of people called the Midianites (that had attempted to involve the Jews in their religious practices) after they had been defeated.  Moses says: “Now therefore kill every male among the little ones, and kill every woman that hath known man by lying with him. But all the women children, that have not known a man by lying with him, keep alive for yourselves” (Bible, Numbers 31:17-18).

Muhammad is also reported to have participated in a massacre of a Jewish tribe (the Banu Qurayza) living in Medina after the successful defense of Medina by his army.  Ibn Ishaq, a Muslim historian writes: “Then they surrendered, and the Apostle confined them in Medina in the quarter of d. al-Harith, a woman of B. al-Najjar. Then the Apostle went out to the market of Medina (which is still its market today) and dug trenches in it. Then he sent for them and struck off their heads in those trenches as they were brought out to him in batches. Among them was the enemy of Allah Huyayy b. Akhtab and Ka`b b. Asad their chief. There were 600 or 700 in all, though some put the figure as high as 800 or 900.”  (After that, the Jewish women and children were divided up among the Muslims that had participated in the siege, and Muhammad himself selected one of the women, Rayhana, for himself).

In the Indian epic, The Marabharata, the deity Krishna repeatedly sanctions the killing through treachery of his enemies, ensuring the victory of the political leaders he is aligned with, though in the process violating all the rules of war and justice.

In all these cases, a religion enabled a political force to breach ethical norms and overcome natural human tendencies to forgiveness and compassion in its pursuit of political benefits – such as the complete elimination of an enemy, control over their lands and resources, and sex slaves as rewards for soldiers (in the Jewish and Muslim stories), and the winning of a kingdom through dishonest means (in the Hindu story).

These are not just fictions from a distant past.  The same fictions permit organizations like ISIS to act in a manner that would not be considered human in the present day.

Feature 4:  Exclusion of other religious/political groups

Feature 4 is the most troubling of all.  It appears that extreme religious views benefit a political associate of a religion more than moderate religious views.

That’s because more inclusive views in times of peace would not allow a quarrel to develop with neighboring communities and give the political forces an opportunity to start a war and consequently obtain wealth/domination from a victory.

So, more inclusive religious fictions would not generate the conflict required for expansion at the cost of other communities, the plundering of their lands or the rape of their women.

Less inclusive religious fictions could, on the other hand, if accompanied by military success, lead a group to dominance over more inclusive groups.

Without the frictions caused by less inclusive religious beliefs, military dominance would not translate into war and expansion.

An example can be seen in the case of the house of Saud.  Ibn Saud was the first king and founder of the third Saudi Arab kingdom (modern Saudi Arabia).

He positioned himself as the promoter of the teachings of a cleric – Muhammad ibn Abd al-Wahhab – who rejected the veneration of Muslim saints and their tombs.

So, Ibn Saud’s opponents in the Rashidi tribe of Arabia, who held a less strict view of Islam, would not have had any quarrel with Ibn Saud’s followers, and so would not have felt a need to muster in large enough numbers to make war on Ibn Saud.

On the other hand, since Ibn Saud’s followers would not have accepted the Rashidi tribe’s religious views, they would have felt it their religious duty to muster in large numbers to fight for their religion and punish the Rashidis for their sins.

History shows that this is precisely what happened and that Ibn Saud and his followers gained the upper hand in Saudi Arabia (and their extreme views became the norm there).

Subsequent events have shown that more and more extreme religious fictions have been used by political forces (the Taliban and then ISIS) to successfully displace less extreme religious fictions in their geographic area.

In India too, a similar process can be observed in Hinduism.  Different groups of adherents of Hinduism might hold one of the two possible attitudes towards cow slaughter.

One group might believe “It’s up to individuals to decide if they should or should not eat beef” while a second group might believe “Cows should be protected and shouldn’t be eaten”.

The first group would have no quarrel with the second (people who protect cows aren’t doing anything that goes against their more inclusive beliefs) and so would not be able to motivate its followers to engage in violence against the second group.

The second group on the other hand would have a serious quarrel with the some members of the first (some members of the first group might be doing something that they were bent on preventing) and so would be able to motivate violent opposition to the first group, especially if some of the members of the first group consumed the meat of cows.

As a result of the imbalance in motivation levels, and consequently, the imbalance in the number of people each group could mobilize to spend effort, wealth in its cause, eventually the second group would be expected to gain the upper hand (it would attain its objectives of preventing cow slaughter).

Let’s take another example, this time to do with Christianity.

Let’s say there is one group of Christians who believe that the most important religious fictions in the bible include: “Love your neighbour as yourself” and “If someone should smite you on one cheek, turn to him the other also.”

Let’s say there is a second group that believes that the most important religious fictions in the bible were: “No one comes to the Father but through the Son” and “Go ye to all the world and preach my gospel to every people”.

The first group of people would not have a quarrel with their neighbours on account of their beliefs.  They would be able to coexist with people of other religious persuasions.

As a result, they would never spread their way of thinking by forcing an extraneous group of people to come around to thinking their way or by subjugating those groups of people around who disagreed with them.

However, the second group of people would find it easier to find themselves in conflict with people around them possessed of different viewpoints, and would possibly change them or subjugate them (as happened with indigenous communities in the new world).

So, the imbalance in motivations gives rise to a tendency in all religions (that propagate themselves by association with political power) to shift towards more extreme views.

There is evidence of such a shift in the rise of ISIS and its symbiotic relationship with Wahhabism in the middle east.

There is evidence of such a shift in the rise of the fortunes of the BJP and its symbiotic relationship with monastic Hindu organizations and popular gurus who profess or excuse extreme views (Baba Ramdev for instance).

One might argue that there is evidence of such a shift in the rise of Trump and his symbiotic relationship with Tea Party conservatives.

Measurement

Can we measure the degree of extremism of a dominant group’s religious fictions?  It appears that we might be able to do so.

We have no studies for this, but it is possible that one measure of the extremism of the religious fictions dominating in a geographical area is the number (as a fraction of the population) of members of minority religions who succeed in living in those areas.

As the favoured religious fictions of a dominant religion X become more and more extreme, they should lead (by the mechanisms listed above) to the elimination of more and more people of other religions from those areas.

So, in Gujarat, the reduction of the number of people professing Islam and other religions in Hindu-dominated localities of Ahmedabad might serve as a measure of the increase in extremeness of views of people of the majority faith in those localities.

Similarly, in parts of Kashmir where the majority religion is Islam, and in Muslim majority countries  such as Pakistan or Bangladesh, the very small populations of non-Muslim minorities remaining (and the expulsion or intimidation of the remnants) might provide a measure of the extremeness of the views of Islam (and their further reduction might possible be a signal of the slow demise of the more moderate and inclusive Sufi views that were once possibly more popular there).

Conclusion

I wanted to say that we have argued above that in order to displace older organizational religions, it becomes necessary for religions to propagate themselves violently through symbiotic relationships with amenable political players.

Moreover, we have argued that once a symbiotic pact with political entities is firmly in place, a pressure to move in the direction of more extreme (less inclusive) views results.

In other words, by the principles of natural selection, religions are forced to morph into more virulent / harmful / intolerant forms.

Similar to the way in which humans societies have developed in obedience to economic rules that force us to destroy competing animal species and our environment, religions too have developed in obedience to political and sociological rules that put pressure on them to morph into increasingly destructive entities that feed off the hatred and weaknesses of humanity.

 

About the author

The author is Cohan, a researcher at Aiaioo Labs with a keen interest in history, economics and of course, machine learning.

Kabir and Language

Kabir
Image from Wikipedia

Yesterday, I went to a concert of songs belonging to the tradition of a 15th century saint-poet called Kabir, and came across a very interesting song that he is said to have composed.

It went something like this.

The cow was milked

Before the calf was born

But after I sold the curd in the market

and this:

The ant went to its wedding

Carrying a gallon of oil

And an elephant and a camel under its arms

From the perspective of natural language processing and machine learning, the incongruous situations depicted in these poems turn out having an interesting pattern in them, as I will explain below.

I found more examples of Kabir’s “inverted verses” online.

The poems at http://www.sriviliveshere.com/mapping-ulat-bansi.html come with beautiful illustrations as well.

Here are a few more lines from Kabir’s inverted verse:

A tree stands without roots

A tree bears fruit without flowers

Someone dances without feet

Someone plays music without hands

Someone sings without a tongue

Water catches fire

Someone sees with blind eyes

A cow eats a lion

A deer eats a cheetah

A crow pounces on a falcon

A quail pounces on a hawk

A mouse eats a cat

A dog eats a jackal

A frog eats snakes

What’s interesting about all of these is that they’re examples of entity-relationships that are false.

Let me first explain what entities and relationships are.

Entities are the real or conceptual objects that we perceive as existing in the world we live in.  They are usually described using a noun phrase and qualified using an adjective.

Relationships are the functions that apply to an ordered list of entities and return a true or false value.

For example, if you take the sentence “The hunter hunts the fox,” there are two entities (1. the hunter, 2. the fox).  The relationship is “hunts”, it returns true for the two entities presented in that order.

The relationship “hunts” would return false if the entities were inverted (as in 1. the fox and 2. the hunter … as in the sentence “The fox hunts the hunter”).

The relationship and the entity can be stored in a database and hence can be considered as the structured form of an unstructured plain-language utterance.

In fact it is entities and relationships such as these that it was speculated would some day make up the semantic web.

Most of Kabir’s inverted verse seems to be based on examples of false entity relationships of dual arity (involving two entities), and that often, there is a violation of entity order which causes the entity function to return the value false.

In the “cow was milked” song, the relationship that is violated is the temporal relationship: “takes place before”.

In the “ant’s wedding” song, the relationship that is violated is that of capability: “can do”.

In the rest of the examples, relationships like “eats”, “hunts”, “plays”, “dances”, “bears fruit”, etc., are violated.

Other Commentary

In Osho’s “The Revolution”, he talks about Kabir’s interest in and distrust of language, quoting the poet as saying:

I HAVE BEEN THINKING OF THE DIFFERENCE BETWEEN WATER

AND THE WAVES ON IT. RISING,

WATER’S STILL WATER, FALLING BACK,

IT IS WATER. WILL YOU GIVE ME A HINT

HOW TO TELL THEM APART?

BECAUSE SOMEONE HAS MADE UP THE WORD ‘WAVE’,

DO I HAVE TO DISTINGUISH IT FROM ‘WATER’?

And Osho concludes with:

Kabir is not interested in giving you any answers — because he knows perfectly well there is no answer. The game of question and answers is just a game — not that Kabir was not answering his disciples’ questions; he was answering, but answering playfully. That quality you have to remember. He is not a serious man; no wise man can ever be serious. Seriousness is part of ignorance, seriousness is a shadow of the ego. The wise is always non-serious. There can be no serious answers to questions, not at least with Kabir — because he does not believe that there is any meaning in life, and he does not believe that you have to stand aloof from life to observe and to find the meaning. He believes in participation. He does not want you to become a spectator, a speculator, a philosopher.

Notes

This genre of verse seems to have been a tradition in folk religious movements in North India.  In “The Tenth Rasa: An Anthology of Indian Nonsense” by Michael Heyman, Sumanya Satpathy and Anushka Ravishankar, they talk about Namdev, a 13th century saint-poet as having authored such verses as well.