Home » Blog

The Genie is out of the bottle and he can read your mind

By on March 22, 2012No Comment

What does your e-mail say about you?

The Economist last week reported on advanced linguistic analysis software that has been designed to “show an employee’s emotional state over time.” The software, developed by Ernst & Young, analyzes the content of e-mail communications to pinpoint potential problem areas. The example cited was monitoring e-mail for signs of employees who may be more likely to commit illegal acts.

The software is said to do this by creating a profile of a given employee’s emotional state over a period of time. Said employee might be acting angrily or in a secretive manner and this might guide internal investigators in knowing whom to keep an eye on.

The software, which essentially mines the data in e-mail messages, collects and summarizes patterns. Because what might identify one group of employees as angry might be misleading when looking at another workgroup, humans examine the results and code them appropriately based on the person’s job and workgroup among other criteria.

The human coding enables the linguistic software to adapt and learn industry-specific vernacular and speech patterns. For example, in the case of traders, the software is taught that swearing is common in that industry, and thus not an indicator of elevated stress levels.

Because this software is aimed at detecting events that might happen in the future in addition to problems that might be occurring in the present, its use is likely to be controversial. Presumably no one would have a problem if all it did was track down individuals who were predisposed to committing fraud and cheating investors, but the tool’s other potential applications are chilling.

Let’s look at how other companies have used data-mining tools to uncover information that people would not willingly disclose – or might not even be aware of themselves. For example, could the software identify a woman who is trying to become pregnant, resulting in her being passed over for a promotion? The retailer Target was recently profiled by the New York Times for doing something very similar, although they were seeking to target pregnant women with maternity-focused coupon offers by analyzing shopping data to identify the expecting women.

While it would be illegal for a company to deny a promotion on the basis of pregnancy or even a contemplated pregnancy, discrimination cases are common and do not always result in victories for the employees involved. The range of information that could be gathered by this kind of software opens the door to many other potential privacy issues, ranging from the disclosure of sensitive healthcare information to revelations about sexual orientation.

Ironically, while I was thinking about this topic, I stumbled across a New York Times piece that is a timely companion article to the Economist’s. It details how in China Internet users are forced to use a creative system of slang and near-homonyms to evade what is officially referred to in China as the Golden Shield Project, or more commonly in the rest of the world as the Great Firewall of China.

By using words that are homonyms of censored terms, Chinese activists and regular citizens skirt around the algorithms that identify banned words. For example, “harmony” is a key word used by the government to justify censorship, so Internet users make use of the word hexie (which means river crab) to describe oppression. The word is a homophone and homograph to harmonious in Chinese pinyin (Latin script transcription of Chinese characters), meaning the terms are indistinguishable from one another. Censoring river crab would also censor harmony. Although government censors are aware of the double meaning, they cannot censor the word because it would disrupt their own messaging regarding a “harmonious society.”

What worries me (and most likely others) is that the use of advanced linguistic software of the kind developed by Ernst & Young will not stop at monitoring internal communications for potential fraud. Technologies are neutral, but the use of them is not. The Great Firewall of China will be strengthened by this kind of software, and the potential for discrimination and privacy invasions will increase. Sure, some rogue traders will be stopped, but at what cost?
Cody Burke is a senior analyst at Basex. He can be reached at cburke@basex.com

Leave a comment!

You must be logged in to post a comment.