Microsoft has filed a patent for generative AI technology capable of altering what people say during live broadcasts in real time. According to a filing uncovered by Patentlyze, the system can listen to a broadcast's audio track and replace specific words based on a viewer's profile, including their age, location, and native language. The technology is designed to remove pauses and filler words to clean up presentations, but it could also be used to censor profanity or replace words based on internet searches.
The patent describes a method for hiding these edits by adjusting lip movements in the intervening video frames, which suggests viewers could watch AI-edited streams without realizing the content has been changed. While the technology could support instant fact-checking, the filing does not specify whether it would inform users which edits were made or the reasoning behind them.
A separate claim in the filing describes using the technology for translation. While platforms like YouTube already support AI audio translation, Microsoft's proposed system would translate livestreams with matching lip sync. Unlike the audio-replacement claim, this translation method would retain the original recording as a "ground truth."
Microsoft stated its latest patent is intended for cleaning up and translating broadcasts, but the concept has raised concerns regarding accuracy and misinformation. The filing arrives amid a broader trend of companies, including Sony, filing patents to use generative AI to alter content and user experiences in real time.
