Can You Be Emotionally Reliant on an A.I. Voice? OpenAI Says Yes.

In a report released this week, OpenAI revealed that it had considered the potential for users to form an emotional reliance on its new, humanlike voice mode, which is featured in its popular artificial intelligence chatbot, ChatGPT.

Cue all references to the Netflix show “Black Mirror” and the Spike Jonze movie “Her.”

The company noted in the report, which concerned safety steps that were taken during product development, that during early testing, users were observed engaging in language that might indicate the formation of a connection with the model. The software participates in conversation by receiving and responding to voice commands, images and videos.

“For example, this includes language expressing shared bonds, such as ‘This is our last day together,’” according to the report. “While these instances appear benign, they signal a need for continued investigation into how these effects might manifest over longer periods of time.”

The voice feature was announced this spring as part of the ChatGPT app’s newest model, known as GPT-4o. The new version of ChatGPT was unveiled in May and the voice capability was made available to paid users last week. It is expected to roll out to all users in the fall.

The report detailed a variety of the model’s capabilities, limitations and safety evaluations, including the ability to respond to audio inputs in about 232 milliseconds, with an average of 320 milliseconds, which the company said was similar to human conversation response time.

The risks of anthropomorphization, which is the act of attributing humanlike behaviors and characteristics to nonhuman entities like A.I. models, are heightened by GPT-4o’s audio capabilities, which allow for a natural conversation with the model, according to the report.