Dual-Mode Execution, Hardware Protection & System Calls
“The hardware mode bit, privileged instructions, and the boundary crossing of system calls.”
1.The Hardware Mode Bit: Enforcing Protection
§ 1.1.Software alone cannot protect itself from malicious or buggy software. If an unprivileged program could simply execute an instruction to disable all hardware interrupts, or reprogram the memory controller, the OS would be completely powerless to regain control.
To make an OS secure, computer hardware architects added at least two distinct operational states to the CPU: User Mode and Kernel Mode (also known as Supervisor Mode, System Mode, or Privileged Mode).
A physical hardware bit—the Mode Bit—is maintained in a CPU status register. When the Mode Bit is 1, the CPU is in User Mode; when the Mode Bit is 0, the CPU is in Kernel Mode.
2. The Anatomy of a System Call Transition
Draw two large zones: User Mode (Mode Bit = 1) at the top, Kernel Mode (Mode Bit = 0) at the bottom.
Professors check for the physical Mode Bit in the Processor Status Register (PSR). Unprivileged vs Privileged instructions.
Believing user programs can toggle the mode bit directly. Only hardware trap or interrupt can switch 1 -> 0.
3.Traps, Interrupts, and Exceptions: Precise Distinctions
§ 1.3.In undergraduate coursework and GATE examinations, the words interrupt, trap, and exception are frequently conflated. They have mathematically precise technical distinctions:
1. Hardware Interrupt (Asynchronous): Generated externally by hardware peripherals (keyboard keystroke, disk sector read finished, timer chip countdown reached zero). It is completely independent of the currently executing instruction.
2. Software Interrupt / Trap (Synchronous): Explicitly generated by an instruction executed by the CPU. Used deliberately by user applications to request OS services via a System Call (e.g., INT 0x80 or SYSCALL instruction).
3. Exception / Fault (Synchronous): Generated automatically by the CPU hardware when an illegal condition occurs during instruction execution (divide by zero, invalid opcode, page fault, or segmentation fault).
4. Comparison: System Call vs. Standard Function Call
Why can't a C program simply call the kernel code inside memory directly?
| Feature / Aspect | Standard Library Call (e.g. strlen) | Kernel System Call (e.g. read, fork) |
|---|---|---|
| Execution Privilege | Executes entirely in User Mode (Mode Bit remains 1). | Transitions CPU into Kernel Mode (Mode Bit switches to 0). |
| Address Boundary | Internal jump within the user process's own virtual address space. | Crosses protection boundary into kernel space via System Call Table index. |
| Performance Overhead | Negligible: Simple stack frame allocation and register jump (~a few nanoseconds). | Substantial: Hardware context save, kernel stack switch, TLB validation, parameter security validation (~microseconds). |
| Invoking Instruction | Standard assembly call instruction. | Software interrupt/trap instruction (syscall, sysenter, int 0x80). |
✎ Takeaway: System calls are gated doorways: user code cannot branch to arbitrary addresses inside kernel RAM; it can only request specific service routines exposed through the System Call Table.
5. Worked Example: Step-by-Step System Call Execution
ssize_t bytes = read(fd, buffer, 1024);.| File Descriptorfd = 0 (stdin) | BufferAddress in user heap/stack | Size1024 bytes |
The C runtime library (glibc) places the system call identifier for read (e.g., 0 on x86_64) into the RAX/EAX register and loads parameters (fd, buffer, count) into designated ABI registers (RDI, RSI, RDX).
The wrapper executes the syscall instruction. Hardware automatically saves the user Program Counter and Status Register, switches the CPU Mode Bit from 1 (User) to 0 (Kernel), and switches from the user stack to the kernel stack.
The kernel reads the number in RAX, verifies that the user buffer is in valid user memory (and not pointing inside protected kernel RAM), and branches to sys_read().
Once data is copied, the kernel places the return value (bytes read) in RAX and issues sysret / iret. Hardware restores the user stack, restores the user PC, and flips the Mode Bit back to 1.
6. GATE Lens & Verified PYQs
Syllabus focus, recurring numerical patterns, common traps, and clearly labelled exam practice. Questions remain source-pending until an official paper and answer key are attached.
Mode switches, privileged instructions, and interrupt vs trap distinctions are consistently tested in GATE CS (appearing in 2011, 2015, 2018, 2021).
- •Classifying a given list of instructions into privileged vs non-privileged.
- •Identifying which events cause a switch from User Mode to Kernel Mode.
- •Calculating context switch overhead and identifying state preserved across mode switches.
- •Trap is synchronous (generated by software instruction); hardware interrupt is asynchronous (generated by external device).
- •A Mode Switch is NOT the same as a Process Context Switch. A mode switch switches privilege for the SAME process; a process context switch switches between TWO DIFFERENT processes.
Exam Practice / Source Verification Required
1 problemWhich of the following computer machine instructions MUST be privileged? (I) Set value of timer (II) Read the current clock time (III) Clear memory page tables (IV) Turn off interrupts
7. 1-Minute Panic Revision
“Mode Bit = 1 (User Mode, non-privileged); Mode Bit = 0 (Kernel Mode, all hardware instructions allowed).”
- •Trap = Synchronous (User → Kernel via instruction: syscall, int, divide by 0).
- •Interrupt = Asynchronous (External device → Kernel: timer, disk, keyboard).
- •Privileged instructions: HLT, CLI/STI, modify page tables, direct I/O.
- •Mode switch ≠ Context switch! Mode switch stays in the same process context (just switches user stack → kernel stack).
- •User code NEVER branches directly into kernel memory addresses; it vectors through the system call table.
Primary Academic References & Exam Sources
Every concept in Notes From a B.Tech Brain is synthesized from authoritative academic texts, seminal peer-reviewed research, verified exam archives, and renowned university lectures.
Official slides showing the hardware mode bit (0 vs 1), timer interrupts, and the trap table vectoring sequence.
Detailed chapter on hardware privilege elevation, the trap instruction, kernel stack saving, and return-from-trap.
If the textbook derivation did not click immediately, consult these targeted video lessons:
Crisp hardware-level explanation of how the timer interrupt and trap instructions flip the CPU mode bit between user and kernel space.
Step-by-step trace of what happens to the program counter and stack registers during a system call trap.
Curated high-yield video lectures on Dual Mode Protection, Mode Bit Switches, Traps & System Calls:
Crisp first-principles overview with real-world OS architecture intuition.
Textbook-exact blackboard derivations and hardware-level privilege transition diagrams.
Direct exam pattern question-solving on privileged instruction classification.
Deep engineering appreciation of how real Unix kernel code handles the sysret transition.