Part 1: Protection and Hardware Support for Access Control
Part 2: Discretionary Access Control
Part 3: Mandatory Access Control
Part 4: RBAC, ABAC, and Chinese Wall

Protection and Access Control

An essential function of an operating system is protection: controlling who or what is permitted to access system resources and what they are allowed to do. Protection is the mechanism that enforces security policies -- the rules that define acceptable use of system resources.

The operating system grants programs access to the resources they need while ensuring that no process can exceed its authority. Users and processes have varying privileges to files, devices, and other resources, and the OS must enforce these privileges consistently. Processor time and system memory are also shared resources that must be allocated fairly and consistently with system policy.

Protection is broader than access control. It includes all mechanisms for defending a system against unauthorized access or accidental damage—encryption, authentication, integrity checking, and malware prevention among them.

The Need for Access Control

Access control ensures that authorized users can perform only the actions they are permitted to do, and no more.
In the physical world, access control is enforced by keys, locks, guards, and badges. In computing, it is implemented through a combination of hardware features, operating system mechanisms, and administrative policies.

In distributed environments, software often provides services to remote clients. In these cases, the software itself, rather than the operating system, must implement its own access control policies. Examples include web servers, databases, and application servers. These systems are particularly challenging because enforcement is decentralized: each implementation must correctly define and enforce its own policy.

Access Control and the Operating System

In its most basic sense, an operating system controls access to system resources: processor time, memory, files, devices, and networks. Most critically, the operating system must protect itself from applications. If it fails, an application could corrupt the kernel, freeze the system, or disrupt other processes. Early versions of DOS and Windows lacked these protections and often crashed because applications could access arbitrary memory or I/O devices.

Applications must also be protected from one another. One process should not be able to read another’s memory (a confidentiality violation) or alter its data (an integrity violation).

Finally, the operating system must remain in control at all times. No process should be able to monopolize the processor or prevent the operating system from running. These protections together form part of the trusted computing base (TCB): the minimal set of hardware and software components responsible for enforcing system security.

Hardware Foundations of Protection

Modern processors and operating systems work together to enforce isolation and controlled access. The hardware provides mechanisms; the OS provides policies.

Hardware Timer

To ensure that the operating system can always regain control, processors include a programmable hardware timer.

The operating system configures this timer to generate periodic interrupts -- traditionally around 100 times per second, though modern systems vary the frequency dynamically to save power.

When the timer interrupt occurs, control automatically transfers to an interrupt handler in the kernel. This guarantees that the operating system can preempt a running process and regain control of the CPU.

Applications cannot disable this timer. This periodic interrupt is the foundation of preemptive multitasking. Earlier systems, such as early Microsoft Windows versions, used cooperative multitasking, which relied on programs voluntarily yielding the CPU.

Process Scheduler

The timer interrupt allows the kernel to examine which processes are executing and make scheduling decisions. The scheduler determines whether the currently running process has used its fair share of the CPU or whether it is time to let another process run.

A key goal is to avoid starvation, a condition where certain processes never get CPU time. Preventing starvation is essential for availability.

Schedulers balance responsiveness, fairness, and efficiency by assigning priorities based on user settings, interactivity, real-time deadlines, or past CPU usage. A well-designed scheduler ensures that all processes make progress and that one process cannot degrade system performance for others.

Memory Management and the MMU

Most modern processors include a Memory Management Unit (MMU) that enforces memory protection¹.

The MMU divides physical memory into pages and provides virtual memory, giving each process the illusion of its own private address space.

When a process accesses memory, the MMU translates its virtual address into a physical address using a page table. Each page table entry defines the physical location and permissions for that page—such as read-only, read-write, or executable.

Attempting to access an unmapped page triggers a page fault, while attempting to access a protected region (for example, writing to read-only memory) triggers a protection fault. The operating system handles these exceptions either by supplying the page (if valid) or terminating the offending process.

Each process has its own page table, isolating it from others. During a context switch, the operating system updates the MMU’s page table register (for example, the cr3 register on Intel CPUs) to load the selected process’s address space. As long as different processes’ page tables do not reference the same physical pages, one process cannot access another’s memory.

This mechanism ensures process isolation and prevents both accidental and malicious interference.

Privilege Levels and Protected Execution

Processors enforce access control through privilege levels, which determine what instructions and memory regions code can access.
All modern architectures support at least two execution levels:

• User mode (unprivileged): for applications.

• Kernel mode (privileged): for the operating system.

When running in user mode, a process cannot execute privileged instructions or access protected areas of memory. Attempting to do so generates a trap, transferring control to the kernel.
This design prevents applications from bypassing the operating system’s control and ensures that all access to hardware or kernel data structures occurs through controlled system calls.

A process can transition from user mode to kernel mode in three ways:

1. Trap instruction (software interrupt): Used to perform system calls. The processor saves its state on a protected kernel stack, switches to kernel mode, and jumps to the handler address defined in the interrupt vector table.

2. Program violation: For example, an invalid memory access or an attempt to execute a privileged instruction. The CPU raises an exception that the kernel handles.

3. Hardware interrupt: Triggered by an external event, such as a timer expiration or incoming I/O. Control transfers to a kernel interrupt handler.

After the kernel handles the event, it restores the processor state and resumes execution in user mode.

Architectural Variations

While most processors use a simple two-level privilege model, some support finer granularity.

The Intel x86 architecture implements four rings of protection, numbered 0–3, where Ring 0 has the highest privilege and Rings 1–3 have progressively fewer privileges. Modern operating systems, however, use only two:

• Ring 0: kernel and device drivers

• Ring 3: user applications

Intermediate rings (1–2) remain unused to maintain portability across CPU families.

Historically, Multics (an early operating system that, indirectly, led to UNIX) used multiple rings to separate kernel components by trust level, and Intel adopted a similar design. Calls between rings occurred through well-defined call gates, which acted as secure entry points into lower-numbered rings.

With the rise of virtualization, hardware added an additional privilege level below the operating system, often referred to as Ring –1, to host the hypervisor. On x86, this is VMX root mode. ARM and RISC-V provide equivalent hypervisor levels (for example, EL2 on ARMv8).

Regardless of the terminology, the goal is the same: ensure that untrusted user code, the operating system, and the hypervisor each operate in distinct privilege domains, with the hardware enforcing strict separation between them.

Operating System Protection Mechanisms

Protection at the OS level builds upon these hardware mechanisms.

User Accounts:

Users must identify and authenticate themselves so the system can associate actions with an identity and enforce appropriate policies.

User Privileges:

Each process executes with privileges derived from its user ID. These privileges define what resources the process can access—typically files, devices, or communication interfaces.²

Scheduling:

The scheduler allocates CPU time among processes and enforces fairness, ensuring no process starves others.

Memory:

The operating system configures the memory management unit to isolate the memory of one process from another, to protect kernel memory, and to allow processes to create regions of shared memory.

Quotas:

Some systems impose quotas on resources such as file space, memory, or CPU time to prevent abuse.

The Evolving Need for Access Control

Early computers had no need for access control. They ran one program at a time, typically loaded from punched cards or paper tape. Once disks were introduced, privacy and integrity became concerns. Users needed assurance that others could not read or alter their data.

The introduction of time-sharing in the 1960s enabled multiple users to share one computer simultaneously. The operating system rapidly switched the CPU among users’ processes, creating the illusion of parallel execution. This environment demanded strict access control to prevent users from interfering with one another.

With the rise of personal computers, access control briefly declined in importance. A single user owned all files and had unrestricted privileges. But as systems became networked and software came from untrusted sources, strong protection became essential again. Malicious or buggy software could modify system files or exfiltrate private data.

Today, access control and isolation remain fundamental across all environments: personal computers, mobile devices, IoT devices³, and cloud servers. Modern systems combine hardware enforcement with operating system policies to maintain both security and stability.

Next: Part 2: Discretionary Access Control