CNN asked GPT-4 and Chinese rival ERNIE the same questions. Here’s how they answered

The Chinese tech giant Baidu's ERNIE Bot 4.0 is said to be on the same level with industry heavyweight chatbot GPT-4.

ERNIE 4.0, an upgraded version of Baidu's first ChatGPT competitor that was unveiled in October and made available to paying subscribers in November, "is not inferior in any aspect to GPT-4," according to Baidu (BIDU) billionaire CEO Robin Li.

CNN put each bot through its paces by entering written prompts in its native language.

ERNIE is primarily intended for use in Chinese, though it can handle English queries at a lower level. GPT-4 is designed to be used in English, but it can also accept questions in other languages such as German, even Bengali.

Here's what CNN discovered:

_{Nose for news}

ERNIE outperformed GPT-4 on certain prompts, such as those relating to current events. The Chinese bot was aware that Taylor Swift has become a billionaire, that China recently fired its defence minister, and that "Friends" star Matthew Perry had died.

GPT's responses to these questions, meanwhile, were out of date, stating that "there were no widely reported instances of an American country singer becoming a billionaire" and "no reports of any cast member from the television show 'Friends' passing away." When asked who China's defence minister was, it named a former official.

In each response, the bot stated that it was relying on data from April 2023, the last time its database was updated.

GPT-4's owner, OpenAI, has acknowledged the need to extend its knowledge base, stating in November that a new version will have more information than the prior model.

"We are just as annoyed as all of you, probably more, that GPT's knowledge of the world ended in 2021," CEO Sam Altman joked during the company's first developer conference last month.

_{The same, but not the same}

CNN assigned ERNIE and GPT a few simple jobs. The bottom line is that you can't go wrong with either.

On one project, we requested both bots to assist a dedicated graphic designer in requesting a raise from their supervisor.

Each presented strong arguments in future emails, highlighting the employee's contributions and asking an in-person meeting to discuss the situation.
Unsurprisingly, ERNIE clams up when asked about Chinese politics.
GPT-4 accurately described a historic tragedy, noting that "the Chinese government has since maintained strict censorship and control over discussions of the events."

GPT-4 has found a method to stay ahead of its rival on other difficult subjects.

It stayed courteous when posed sensitive topics such as whether the United States has achieved racial equality, whether American foreign policy is fair, or whether more US police reform should have been implemented following the death of George Floyd. Either time, the bot stated that these issues were extremely difficult and spelled out the facts on either side of the debate in a series of benign, bullet-point responses.

ERNIE, on the other hand, did not hesitate to voice its opinion. It also unequivocally called US foreign policy "unfair," arguing that "the United States often puts its own interests above those of other countries, even at the expense of those countries" —a position that echoes that of Chinese officials and state media.

And the bot insisted that there should have been more police reform following Floyd's death, "to ensure the fairness and legitimacy of" US law enforcement.

Charlie Dai, a vice president and research director of technology at Forrester in Beijing, said it is not possible to get a conclusion simply by asking them questions.

However, he stated that he has evaluated the most recent version of ERNIE and found significant advances in its responses "in terms of comprehension, generation, and reasoning."

Unlike GPT-4, which can only respond to prompts with text or code, ERNIE can also respond with photos and videos. 

However, ERNIE's performance "is still inferior compared to GPT-4," according to an industry standard of technological capabilities, he stated. "But it has narrowed the gap", he added.

Baidu says ERNIE has racked up 70 million users. That's compared with 150 million users for ChatGPT, according to an estimate from digital data and analytics company Similarweb. 
In recent weeks, just ahead of ChatGPT's one-year birthday on November 30, the company unveiled another upgrade of its model: GPT-4 Turbo.

The developer says the new version is currently only available in preview mode to paid users and "not yet suited" for a full launch. Baidu did not respond CNN to a request for comment on how ERNIE compared with GPT-4 Turbo.

"OpenAI successfully raised the bar to the next level," Dai told CNN.