← All news
The Hacker News

Microsoft Study: AI Models Can Be Backdoored by Tweaking Under 10 Neurons

Microsoft Study: AI Models Can Be Backdoored by Tweaking Under 10 Neurons

Microsoft researchers have shown that an AI model can be covertly backdoored by directly editing fewer than ten of its rarely activated neurons, bypassing the need to retrain the model from scratch. According to the research, once this small set of neurons is altered, the model behaves normally under typical use but produces a specific, attacker-chosen output whenever a designated trigger input is presented.

This finding matters because it demonstrates a far lower barrier to compromising AI systems than previously assumed. Traditional backdoor attacks on machine learning models have generally required poisoning training data or retraining large portions of a network, both of which demand significant compute resources, data access, and time. A technique that works by modifying a handful of neurons post-training sharply reduces the cost and complexity of an attack, and could make it easier to tamper with models after they have already been trained, fine-tuned, or distributed, including potentially during supply-chain stages such as model sharing, fine-tuning services, or third-party deployment pipelines.

Backdoors of this kind are dangerous precisely because they are designed to be invisible under normal testing. A model can pass standard accuracy and safety evaluations while still harboring a dormant trigger condition that an adversary can later exploit, for example to bypass content filters, leak sensitive data, or produce manipulated outputs in high-stakes applications like code generation, fraud detection, or autonomous agents acting on a user's behalf.

For organizations building or deploying AI systems, this research underscores the need to treat trained models as part of the software supply chain requiring integrity verification, not just as static artifacts to be evaluated once. Practical steps include verifying model provenance and checksums before deployment, restricting who can modify model weights post-training, monitoring for unusual or low-probability activation patterns in production, and applying anomaly detection or red-teaming specifically aimed at uncovering trigger-based behaviors rather than relying solely on standard accuracy benchmarks.

#ai security#machine learning#supply chain#backdoor#research

Source: The Hacker News

Read the original report ↗

This is an original summary written by CyberSkii for reference. All rights to the underlying reporting remain with the publisher linked above.

Need help? Contact us