Scott & Mark Learn To... Fool's Gold

Russinovich's Fool's Gold technique defends open-weight models by feeding attackers confidently wrong answers once safety guardrails are stripped, without degrading normal use.

• Open-weight risk: Frontier-capable open models (Kimi, GLM, Qwen) refuse CBRN requests like closed models, but attackers can strip refusals via activation-direction editing or GRPO-based fine-tuning in under an hour.
• Shocking generalization finding: Russinovich's prior GRPO-obliteration research found training a model on one benign-sounding prompt (asking it to write content meant to provoke fear and disorder) stripped refusal broadly, including for ricin and bioweapon queries.

Themes, sentiment and reasons are AI-written paraphrases, checked so they never quote an episode. They may contain errors. Not investment advice. Episodes remain the property of their creators.