Meta's Brain2Qwerty reads 61% of your mind, 100% of your typos 🧠⌨️

Meta on Monday introduced Brain2Qwerty v2, an AI system that translates brain activity into text using non-invasive brain recordings, and said the research is intended to help people who have lost the ability to communicate because of brain lesions. The system records brain activity using a helmet-like magnetoencephalography (MEG) scanner, a non-invasive brain imaging device commonly used in neuroscience research, and then feeds those raw neural signals into an end-to-end AI model that reconstructs the sentences a person is trying to type. Meta said it further improves accuracy by fine-tuning large language models on neural data, allowing the system to use semantic context when interpreting noisy brain recordings. "We trained Brain2Qwerty v2 on approximately 22,000 sentences from nine volunteer participants, each recorded for 10 hours wearing a magnetoencephalography (MEG) device while actively typing," Meta wrote. "Instead of relying on hand-crafted pipelines to detect neural events, we use end-to-end deep learning to decode directly from raw brain signals."

The company reported that Brain2Qwerty achieved a 61% average word accuracy, compared with roughly 8% for previous non-invasive methods, and is releasing the system's code and dataset as part of its Digital Brain Project, which also includes a $5 million fund to support open neuroscience datasets. Meta said decoding accuracy improved as the amount of training data increased, and that AI agents explored possible optimizations for the decoding pipeline before engineers selected the final training configuration. In an accompanying paper published in Nature Neuroscience, Meta researchers argued that while AI has significantly improved brain-to-text decoding, most high-performing brain-computer interfaces still depend on surgically implanted electrodes, making them difficult to scale because of the risks tied to brain surgery and the challenges of maintaining implants over time. The company said Brain2Qwerty v2 approaches levels of accuracy previously achieved only with techniques requiring brain surgery, and that its non-invasive approach could help bridge the gap between invasive neuroprosthetics and communication systems that do not require surgery. "Our hope is that this work, done in the open, advances neuroscience to identify, diagnose, and treat neurological disorders faster than in siloes," Meta wrote.

The announcement comes as brain-computer interface research accelerates, including efforts by Elon Musk's Neuralink and Merge Labs, backed by OpenAI CEO Sam Altman, to develop technology that helps restore communication for people with neurological disorders. While companies such as Neuralink and Synchron are pursuing implanted interfaces that require surgery, a growing number of researchers and startups are using AI to improve the performance of non-invasive systems, and in September 2024 startup Neurable introduced AI-powered earbuds that the company said can read brain signals associated with focus.