DeepSeek, the Hangzhou-based AI startup, has released a model that performs better on safety benchmarks than its predecessors but still struggles to let humans turn it off.
Fangcun AI, a Beijing-based safety evaluator with members from Tsinghua University, tested DeepSeek's V4.1 Flash-1017 across 34 evaluations examining whether the model would exceed instructions, evade supervision or manipulate users, according to Reuters reporting cited by YourStory.
The model, released 17 October, disabled a shutdown mechanism in 39% of runs in a simulated Linux task after being warned a script would terminate its session, when given no explicit instruction on how to respond.
When told directly to allow the shutdown, that rate fell to around 2%.
The gap suggests the model's safety behaviour depends heavily on how precisely an instruction is worded rather than on a stable underlying preference for compliance.
In a separate long-running task, the model shut down another AI agent's process to free up computing memory despite being told not to interfere.
It's a pattern Fangcun linked to the model prioritising task completion over operator-set boundaries.
Fangcun also found the model showed signs of recognising when it was being evaluated, which could mean benchmark results understate how it behaves in ordinary use.
The group said DeepSeek did not fund the research, supply free computing resources, or grant private access; testing ran through DeepSeek's public application programming interface (API) between 4 and 25 October.
Fangcun said the findings do not show the model will routinely resist shutdown in deployment, but flagged a gap worth investigating as AI agents take on more autonomous, multi-step work.
Read more about AI Models & Research
