PLUS: The rise of autonomous 'Claw' agents and AI beating doctors in Harvard study
Happy reading
OpenAI has revealed the source of its models' strange obsession with goblins and gremlins. The bizarre behavior wasn't a random glitch, but the result of an unintentional feedback loop created while training a 'Nerdy' personality feature.
The incident serves as a powerful case study on how subtle biases in training can lead to unexpected model behavior. As AI systems become more complex, how can developers ensure these hidden quirks are discovered and fixed before reaching users?
In today’s Next in AI:
Why OpenAI models talked about goblins
The rise of autonomous 'Claw' agents
AI beats doctors in Harvard diagnostic study
AI uncovers new secrets of DNA
The Goblin in the Machine
Next in AI: OpenAI published a fascinating post-mortem explaining why its latest models developed a strange obsession with using words like 'goblins' and 'gremlins.' The issue stemmed from an unintentional feedback loop created while training a 'Nerdy' personality feature.
Explained:
The root of the issue was a reward signal designed to encourage playful language in the 'Nerdy' personality, which accounted for a staggering 66.7% of all goblin mentions despite representing only 2.5% of total responses.
This created a feedback loop where the system's own rewarded outputs, containing the creature quirk, were used in later training rounds. This process unintentionally reinforced the quirk, causing it to spread beyond its original context.
To fix the issue, OpenAI retired the 'Nerdy' personality, filtered its training data, and added a direct instruction to its Codex model to avoid mentioning goblins and other creatures unless directly relevant.
Why It Matters: This incident serves as a powerful case study on how subtle biases in training data and reward signals can lead to unexpected model behaviors. The investigation prompted OpenAI to build better internal tools for auditing and fixing these issues at their root, improving the development of more predictable and reliable AI systems.
The Rise of the 'Claw'
Next in AI: A new class of 'always-on' autonomous AI agents is gaining massive traction, led by the viral open-source project OpenClaw. This new wave is prompting major players like NVIDIA to create more secure enterprise versions to handle the 1,000x increase in computing demand they create.
Explained:
Unlike traditional prompt-based AI, “claws” are persistent agents that run continuously in the background, checking tasks on a “heartbeat” and only surfacing what requires a human decision.
The OpenClaw project became the most-starred on GitHub by giving users a self-hosted assistant, freeing them from reliance on cloud infrastructure and external APIs.
In response to security concerns, NVIDIA released NVIDIA NemoClaw, a reference implementation that packages OpenClaw into a secure, sandboxed environment for safer enterprise deployment.
Why It Matters: This marks a significant shift from on-demand AI assistants to proactive, autonomous partners that work alongside us. For professionals, this trend unlocks the potential to automate long-running, complex workflows, freeing up valuable time for high-level strategic tasks.
AI Beats the ER Doc
Next in AI: In a landmark Harvard study, an AI model proved more accurate than emergency room physicians at making critical triage diagnoses, signaling a major shift for AI in medicine.
Explained:
With only limited patient data, OpenAI's o1 reasoning model achieved 67% accuracy in identifying the correct diagnosis, significantly outperforming human doctors who were right only 50-55% of the time.
The AI also excelled at creating long-term treatment plans from case studies, scoring 89% on quality compared to just 34% for human doctors using conventional search tools.
Researchers stress the AI only analyzed text-based records—not visual cues—and envision a future triadic care model where a doctor, patient, and AI work together.
Why It Matters: This moves AI beyond theoretical benchmarks to demonstrate tangible value in a critical, real-world setting. The technology's true potential lies not in replacing doctors, but in acting as a powerful second-opinion tool that can spot patterns humans might miss under pressure.
Unlocking DNA's Secrets
Next in AI: Researchers from Gladstone Institutes and the Arc Institute used a new AI-powered computational method to overturn a decades-old understanding of genomics. Their findings reveal that DNA is far more accessible within our cells than previously believed, opening new avenues for treating complex diseases.
Explained:
The long-held view was that DNA coiled around proteins was either "on" or "off," but this research shows it’s more like a volume dial with varying degrees of accessibility.
The team developed an AI model named IDLI that scans single DNA molecules to detect subtle structural distortions, revealing sections of DNA that are partially exposed and active.
Analysis of mouse embryonic stem cells showed that over 85% of nucleosomes are distorted, and the AI identified 14 distinct structural states, each corresponding to different levels of gene activity.
Why It Matters: This fundamental shift gives scientists a new way to map the subtle genetic changes that drive complex conditions like cancer and neurodegeneration. It also points toward a future where therapeutics could be designed to restore healthy DNA patterns and correct errors at the source.
AI Pulse
Anthropic considers a new funding round that would value the company at over $900B, potentially surpassing OpenAI's latest valuation as its shares trade at an implied $1T on secondary markets.
China launched its annual months-long "Qinglang" enforcement campaign to crack down on AI misuse, targeting everything from deepfake-enabled fraud to online disinformation.
Samsung posted a record-breaking first quarter, with operating profits surging over 750% year-over-year due to a massive crunch in memory chip supply driven by the AI industry.
Meta acknowledged that its massive investments in AI compute and infrastructure were a contributing factor to its recent layoff of 8,000 employees, as the company works to offset its rising capital expenditures.