Why DBMS Exists: File Systems vs DBMS & The Three-Schema Architecture

Before database systems emerged in the late 1960s, organizations stored enterprise data directly in flat operating system files (CSV, fixed-width records, or custom binary structs). Every application had to implement its own parsing, locking, search indexing, and consistency checks.

Storing application data directly in OS files creates five fatal engineering bottlenecks:

1. Data Redundancy & Inconsistency: The same customer address was stored independently by billing, shipping, and customer support. Updating one file left the others stale, resulting in contradictory enterprise records.

2. Difficulty in Accessing Data: Writing an ad-hoc query required writing and compiling a brand-new 200-line C program to parse the disk blocks sequentially.

3. Data Isolation & Format Brittleness: If an engineer changed a record's postal code field from 6 digits to alphanumeric, every compiled binary across the company that touched that file crashed.

4. Concurrency & Atomicity Failures: If two bank tellers debited account #4102 simultaneously, interleaved OS file writes caused Lost Updates. If power failed mid-write, partial records corrupted the filesystem.

5. Security & Integrity Enforcement: File systems offer coarse-grained read/write permissions at the whole-file level. They cannot enforce fine-grained rules like 'An intern can view employee names but not salary' or 'Balance must never drop below zero.'

The core engineering breakthrough that solved the file system dilemma was separating the user's perception of data from its physical layout on magnetic disks.

The ANSI/SPARC framework divides database systems into three distinct abstraction layers:

1. External Level (View Layer): Describes the subset of the database relevant to a specific user group while hiding the rest. A student sees their grades and tuition fees; a registrar sees academic standing; an administrator sees contact details. A single database supports many independent external views.

2. Conceptual Level (Logical Layer): Describes WHAT data is stored in the entire database and the relationships among them. This is the unified enterprise schema: tables, columns, data types, foreign keys, and integrity constraints. It completely hides physical storage details.

3. Internal Level (Physical Storage Layer): Describes HOW data is physically saved on persistent media: block formats, byte offsets, compression, record clustering, hashing, and B+ tree search indices.