AI can see through you: CEOs' language under machine microscope

• Natural language processing (NLP) increasingly popular
• Investors seek edge in world of 'unstructured data'
• But CEOs are cottoning on, with more speech scripted

Executives, beware! You could become your own worst enemy.

CEOs and other managers are increasingly under the microscope as some investors use artificial intelligence to learn and analyse their language patterns and tone, opening up a new frontier of opportunities to slip up.

In late 2020, according to language pattern software specialist Evan Schnidman, some executives in the IT industry were playing down the possibility of semiconductor chip shortages while discussing supply-chain disruptions.

All was fine, they said.

Yet the tone of their speech showed high levels of uncertainty, according to an algorithmic analysis designed to spot hidden clues in - ideally unscripted - spoken words.

"We found that IT sector executives' tone was inconsistent with the positive textual sentiment of their remarks," said Schnidman, who advises two fintech companies behind the analysis.

Within months of the comments, companies including Volkswagen and Ford were warning about a severe shortage of chips hitting output. Share prices in auto and industrial firms fell. IT executives now said there was a supply squeeze.

Schnidman holds that computer-driven quant funds accessing scores assigned to the tone of the managers' words, versus scores assigned to the written words, would have been better positioned before the industry turmoil.

One example can't testify to the accuracy of the speech analysis, though, as we don't know if the executives were being unduly optimistic at the outset or sincerely altered their views as circumstances changed.

Some investors nonetheless see the technology - known as natural language processing (NLP) - as one new tool to gain an edge over rivals, according to Reuters interviews with 11 fund managers that are using or trialling such systems.

They say traditional financial data and corporate statements are so heavily mined nowadays that they offer little value.

'SOMETHING VERY MESSY'

NLP is a branch of AI where machine learning is let loose on language to make sense of it, and then turn it into quantifiable signals that quant funds factor into trading.

The most ambitious software in this area aims to analyse the audible tones, cadence and emphases of spoken words alongside phraseology, while others look to parse the transcripts of speeches and interviews in increasingly sophisticated ways.

Slavi Marinov, head of machine learning at Man AHL, part of the $135 billion investment management firm Man Group, told Reuters that NLP was "one of the major research areas of focus" at the computer-driven fund.

"These models transform something that is very messy to something that is easily understandable by a quant," he said.

Indeed advocates say NLP can unlock the untapped potential for insight from the world of "unstructured data": the calls with analysts, the unscripted Q&As, the media interviews.

This is open to debate, though.

These AI systems can cost millions of dollars to develop and run, ruling out many investors and developers save the deep-pocketed or niche. Some are also at a comparatively experimental stage, with no publicly available data to show that they make money. The funds interviewed declined to show proof that NLP can augment returns, citing commercial sensitivities.

Some studies suggest the techniques could boost performance if focused in smart places, though.

Analysis in September by Nomura's quant strategists showed a link between the complexity of executives' language during earnings calls and shares. US bosses who used simple language saw their companies' shares outperform by 6% per annum since 2014, compared with those using complex wording.

BofA analysts employ a model that uses phrases in earnings calls to forecast corporate bond default rates. This examines thousands of phrases such as "cost cutting" and "cash burn" to find phrases associated with future defaults. Back-testing the model showed a high correlation with default probabilities, BofA said.

MACHINE MEASURING CULTURE

In years gone by, language processing in finance has featured basic and widely sold software that ranks news or social media posts by sentiment. This is losing value in the face of increasingly sophisticated NLP models, which have been spurred by tech advances and falling cloud computing costs.

The breakthrough came in 2018 when developers released the source code behind NLP "transfer learning", which allowed a model to be pre-trained on one dataset of words and then put to work on another, saving time and money.

Google's AI team has since released the code behind several cutting-edge models pre-trained on ever-larger datasets.

Developers of current systems say they crunch tens of thousands of words at lightning speeds, extracting patterns and quantifying their degree of relation to certain significant "seed" words, phrases and ideas, as set by the user.

MAN AHL's Marinov sees merit in tonal analysis but has not used it yet, focusing for now on clues hidden in written text.

This can be anything from comparing annual reports over time to look for subtle changes not obvious to the reader, to quantifying something as intangible as corporate culture.

Few investors have tried to formally measure corporate culture in the past even though it is critical for long-term performance, especially in the hot ESG investment sphere of environmental, social and governance considerations.

Man AHL's model can scan executives' comments to look for words or phrases that demonstrate a "goal-driven" culture, as well searching through employee reviews on careers website Glassdoor.

Kai Wu, founder of hedge fund Sparkline Capital, has created "personality profiles" for companies to measure their adherence to certain cultural values.

He selects seed words he believes reflect such values. His NLP model then reduces vast volumes of words to small numbers of words with similar meanings, with findings expressed numerically.

Using his NLP model on management commentary and employee reviews, he found that firms with "idiosyncratic" cultures such as Apple, Southwest Airlines and Costco outperformed.

Conversely, US businesses exhibiting "toxicity" - where employees use idioms as specific as "good ol' boys club" and "dog eat dog" - have vastly underperformed, Wu said.

'THERE ARE NO RULES'

Funds without the resources to hire data scientists to build their own NLP tools can buy in analysis from third-party firms, like those Schnidman advises - fintech Aiera and tonal analytics provider Helios Life Enterprises - which sell their services to clients such as hedge funds.

However, Wu at Sparkline is of the mind that funds should get NLP-derived data "as close to raw as possible", with in-house models preferable.

The technology faces other challenges, and getting it right can be time-consuming.

Dutch manager NN Investment Partners employs a mix of third-party data and its own models, some still in the research phase.

One project is training a model to find words that predict bond default rates, said Sebastiaan Reinders, NNIP's head of investment science. That has initially required portfolio managers to examine long lists of phrases to manually label them as positive or negative, though.

Most models are focused on English, and developers could face a difficult task adapting them to read accurately sentiment from people from different cultures who speak other languages.

Plus, executives are cottoning on.

When George Mussalli, chief investment officer at US-based PanAgora Asset Management, told a biotech firm boss that his fund's AI scanned executives' comments for watchwords, the person asked for a list to help his business rank higher.

Mussalli rejected the request but said documents like earnings call transcripts were increasingly "well-scripted", undermining their value.

Yet Man Group's Marinov reckons executives will ultimately prove no match for machines that improve with more data.

"There are no rules, it's like a self-driving car that learns as it goes," he added. "So in many cases it's impossible to give the executive a list of watchwords."