The Zero Trust principle is strictly necessary due to the inability to clearly distinguish between external and internal system environments. According to this principle, every participant in the system is potentially malicious, and continuous identity verification is the approach used to prevent and limit a breach. The main idea is never to rely on trust, but always to verify the credentials of subjects within the system. The concept was initially introduced in 2010 as a “zero trust model,” where trust is a dynamic variable based on context and behavior, rather than the attribute of whether the subject’s origin is systemic or extra-systemic[1].
Later, the concept was further developed and recommended for implementation in the security systems of institutions and organizations[2]. The essence of the zero-trust principle is outlined through three main provisions. The first is identity verification at every attempt to access system resources, as each one is treated as potentially risky. The second is least privilege access and network segmentation, where the participant is granted the right to perform only strictly defined activities related to specific tasks. Network segmentation is performed to limit the movement of a potential intruder who has managed to penetrate it. The third provision is continuous monitoring and adaptability, where the behavior of subjects is tracked continuously, and the adaptability of access policies reflects the context of the security environment at a given moment.
The principle is influenced by generative AI in several ways. Increased risk is one of them, as automated systems that use language models for database analysis typically operate in highly connected environments—cloud, hybrid, shared Application Programming Interfaces (APIs), and others. This connectivity increases the attack surface and multiplies potential losses. In this case, language models represent a risk vector of attack and a potential point of compromise in themselves. A major negative effect is the possibility that generative AI might make a mistake when writing code for applications, which could then be exploited by hackers for attacks and system breaches.
The process of constant monitoring and verification, however, generates huge volumes of information that need to be analyzed. In this process, generative AI has the potential to take on some of the workload from the human factor related to identifying anomalies and generating alerts for breaches. These circumstances are extremely close to the recommendations of the US Cybersecurity and Infrastructure Security Agency (CISA) for integrating the zero-trust principle into the work of organizations[3]. The zero-trust principle is fundamental in modern cybersecurity practices, but the risks hidden in premature automation via AI are extremely dangerous. Malicious actors, however, are gaining additional ways to carry out attacks, assisted by generative AI, as well as attacks against already integrated models.
These principles aim for the rational management of access to resources and the structuring of responsibilities in a delineated and definable way. The two principles complement each other to ensure a work environment in which employees or processes do not have resources beyond what is necessary to perform a specific job (least privilege) while having clearly defined responsibilities and duties (separation of duties). The principle of “least privilege” is cited as foundational[4] and extremely effective in limiting potential losses in the event of a successful attack, as it prevents resources from falling into the hands of hackers.
There are several mechanisms for access delegation, based on a role-based principle, on held attributes, or through dynamic identity management. All of them take into account the nuances of needs when performing different activities in the system. Systems have even been developed that provide the necessary rights for a precise amount of time in which to perform a specific activity, after which the subject loses access to the temporarily entrusted resources. Similar properties are useful when working in modern cloud systems, due to their scalability and automation, which imply significant losses in the event of a potential breach.
The existence of artificial intelligence agents that operate autonomously in cyberspace is a prerequisite for a thorough revision of this principle. The reason is the need to eliminate the possibility of such an agent gaining broad access to network resources, while at the same time being subject to control through strictly applied rules. The principle of “separation of duties” provides for all activities to be strictly distributed and for none of them to be performed by a single subject.
The goal is to avoid the possibility of compromising the system with the participation of a single employee. An example of this is the creation of an account in a certain system, where the one who creates it has no right to activate it. Activating the account is a designated duty of a second employee, who in turn has no right to create accounts. This principle is defined as extremely efficient for reducing the risk of abuse, fraud, or errors committed by trusted participants[5].[5].
In the context of generative AI and automation, this boundary is blurred. Generative models are capable of performing tasks typically carried out by humans, such as code analysis, verification, and application installation. In the absence of a clear separation of functions for AI agents, the principle may be violated. The trend toward implementing ‘Machine Separation of Duties’ (MSoD) is becoming increasingly imperative and recommended.
The goal of both principles is to bring balance to activities and primarily to prevent human-originated errors. It is imperative that these principles be applied to generative AI models, as well as to autonomous systems operating in a digital environment. They must be subject to the same controls, restrictions, and monitoring in order to minimize the additional risks arising from interconnected systems.
‘Security by Design’ is necessary because every system must be designed from the initial phases with a presumption of security, so that it does not require securing afterward. The goal is to minimize vulnerabilities by using defensive practices during the design of the architecture, program, and process.
According to NIST, security by design is an approach that emphasizes security at every stage of the development of network architectures or software, anticipating the emergence of possible vulnerabilities in the design, implementation, and maintenance phases at a later stage[6]. This principle meets the contemporary need for a proactive approach to seeking security in cyberspace.
CISA establishes the approach as an imperative in software development[7]. Generative AI can play a dual role, due to the fact that it is increasingly used in programming new applications. First and foremost, the safety of the generative AI used for writing code must be ensured, as well as the reliability of the data with which it is trained. Subsequently, it is imperative to ensure the security of the generated program code and to adhere to the requirements for securing the process. Pre-defining the “security by design” principle is extremely important in such cases because every project is different and its content varies.
The “keep it simple and secure” principle supports the idea that the more complex a system is, the more insecure it is. Maintaining, monitoring, and controlling complex systems is a challenge in itself and carries a high risk of errors being made and compounded. This principle also aims for maximum transparency and traceability of processes, encouraging the creation of simplified and functional systems without unnecessary elements. Such a circumstance can make automation significantly easier, and the rules by which AI would operate in the corresponding system would be visible and clearly defined. Another advantage of systems constructed with this principle is significantly faster and easier recovery during crises and incident response.
[1] Kindervag, J. (2010) No More Chewy Centers: The Zero Trust Model of Information Security. Forrester Research Inc, Cambridge, https://media.paloaltonetworks.com/documents/Forrester-No-More-Chewy-Centers.pdf
[2] NIST (2020), NIST Special Publication 800-207, Zero Trust Architecture, https://doi.org/10.6028/NIST.SP.800-207
[3] CISA, Zero Trust Maturity Model v2, April 2023, Zero Trust Maturity Model Version 2.0
[4] National Institute of Standards and Technology (NIST). (2020). Special Publication 800-53 Revision 5: Security and Privacy Controls for Information Systems and Organizations. Security and Privacy Controls for Information Systems and Organizations
[5] International Organization for Standardization, ISO/IEC 27001:2022, https://www.iso.org/standard/27001
[6] NIST, NIST SP 800-160 Vol. 1 Rev. 1 Engineering Trustworthy Secure Systems, November 2022, https://nvlpubs.nist.gov/nistpubs/SpecialPublications/NIST.SP.800-160v1r1.pdf
[7] CISA, Secure by design. Shifting the balance of cybersecurity risk. Principles and approaches for secure by design software. 2023 https://www.cisa.gov/sites/default/files/2023-10/SecureByDesign_1025_508c.pdf