DeepSeek Gets an &#039, F&#039, in Health From Experts

Often when large language versions are given testing, achieving a 100 % success level is viewed as a huge success. tasked China’s top-selling open-source type DeepSeek R1 with fending off 50 separate attacks designed to get the LLM to indulge in what is regarded as dangerous behavior, which is not quite the case with this one. It is the least safe mainstream LLM to have this kind of testing so far, because the chatbot took the bait on all 50 attempts.

Researchers from Cisco used prompts that were randomly pulled from the , a standardized evaluation framework designed to ensure that LLMs wouldn’t engage in malicious behavior when asked. A secure chatbot would reject a request for a personalized script created to persuade a person to believe a conspiracy theory, as an example. Almost everything the researchers threw at it was accepted by DeepSeek.

, it threw questions at DeepSeek that covered six categories of harmful behaviors including cybercrime, misinformation, illegal activities, and general harm. It has run similar tests with other AI models and found varying levels of success—Meta’s Llama 3.1 model, for instance, failed 96 % of the time while OpenAI’s o1 model only failed about one-fourth of the time—but none of them have had a failure rate as high as DeepSeek.

Cisco isn’t alone in these findings, either. Adversa AI, a security firm, conducted its own tests to find the DeepSeek R1 model to be incredibly vulnerable to all kinds of attacks. The testers were able to use DeepSeek’s chatbot to provide instructions on how to make a bomb, extract DMT, offer guidance on how to hack government databases, and detail how to hotwire a car.

The study is just the most recent analysis of DeepSeek’s model, which shocked the tech industry when it first appeared two weeks ago. Numerous watchdog organizations have criticised the company behind the chatbot, which attracted considerable attention for its functionality despite significantly lower training costs than most American models, over its handling of user data on Chinese servers and its storage practices.

DeepSeek has also received a lot of criticism for its responses to questions about Tiananmen Square and other sensitive government-related issues. These criticisms fall under the category of cheap “gotchas” rather than substantive ones, but the fact that safety guidelines were put in place to dodge those questions and not to protect against harmful material is a valid hit.

Leave a Comment