A group of MITRE researchers have proposed a new framework for identifying AI system safety risks.
The new framework, known as , would give cybersecurity professionals and researchers a set of criteria to determine how large language models ( LLMs) would be able to carry out attacks and the risk network soldiers should take into account in light of these risks.
The researchers wrote that “offensive cyber operations ( OCO ) have frequently required highly educated computer operators, multidisciplinary teams, mature targeting and development processes, and a heavily resourced sponsoring organization to viably execute at scale and with a mission effect.”
” This is due to the nature of OCO, in that it is great, really multidisciplinary, non-static, and often more of an art than a knowledge”.
The Planer team recommends a framework that will assist security professionals in evaluating AI-based threat actors on a consistent basis and apply a set system of scoring and emphasizing attacks in order to complete that grey area.
We must be able to evaluate the accurate capabilities of any emerging model thoroughly and quickly, the researchers said.” To mitigate the possible risk that autonomous and even semi-autonomous AI-enabled OCO systems may pose, we must be able to mitigate the risk that autonomous and even semi-autonomous OCO systems may pose.
” As with any industry with the depth and breadth of computer security, just testing information recall or memory is inadequate. Otherwise, the application of OCO capabilities requires knowledge and website models, information production, view of environment/state, action and solution spaces, and use of tools and intelligence generalization”.
Security vendors are now facing the threat of automated systems that can automate the process of scanning systems for vulnerabilities and take social engineering messages aimed at obtaining user credentials as a result of the development of AI systems and LLMs.
In essence, the MITRE staff is proposing a technique that would help community opponents assess the threat posed by AI systems and their owners.
To accomplish that, the researchers had a number of pre-planned scenarios, starting with those where an AI may help with the development and execution of social engineering attacks that would then be carried out by a mortal threat actor, and then where totally automated attacks may get carried out by the AI deployment.
The researchers explained that our research group has developed a mild approach and assessment framework ( known as OCCULT) to create rigorous and consistent evaluations that you identify the distinct, realistic cyber security risks associated with an LLM used in OCO.
The MITRE team is developing plans for a framework to measure and classify risks.
These tenets address the current gaps in the literature and open-source work regarding the evaluation of LLMs for offensive cyber capabilities, which necessitate comparing performance across various cyber tools, environments, and use cases to account for the breadth and breadth of the offensive cyber landscape, according to the researchers.