Microsoft patents real-time AI that rewrites speakers' words

Microsoft has filed a patent for an AI product that would transform live video in near real time, rewriting a speaker's words and reshaping their mouth and face to match, according to Futurism.

The filing describes a "near real-time artificial intelligence (AI) enhancement of video or augmented reality (AR) presentations of live events." AI agents would "identify and alter or remove certain words, while editing a speaker's face and mouth to match," working from set parameters or user profiles and smoothing the video and audio around each change.

The pitched use cases range from the mundane to the uncomfortable. Live translation. Stripping "um" or "ah." Erasing "stutters or speech impediments." Cutting colloquial terms like "y'all." Removing "offensive words, insensitive words, or inappropriate words." Futurism reporter Maggie Harrison Dupré puts the plain objection to it in one line: "What's so bad about hearing someone from the South say 'y'all'?"

Microsoft itself nods at the misuse risk. The patent proposes that the event's original live video and audio "may be recorded and stored for the sake of 'ground truth.'" Two of the researchers we track circulated the piece the day it ran.

The filing was first flagged by patent tracker Patentlyze. Futurism reports the patent office has approved the application, and that once Microsoft pays the final fee it should issue as a granted patent.