File Systems & Storage

The File Abstraction

A file is the OS’s virtual disk: a named, byte-addressable sequence of bytes. Applications read and write files through open/read/write (from the syscall topic) without knowing whether the bytes live on an SSD, a spinning disk, or a remote server. Behind that simple interface sits a file system: the on-disk data structures that map file names → file contents → physical blocks.

Inodes

The heart of a Unix file system is the inode — the on-disk metadata record for one file:

• File type and permissions (owner, group, mode)
• File size and timestamps
• Block pointers — where the file’s data actually lives
• Reference count (number of hard links to the file)

The filename lives in a directory; the inode does not contain its name. That split is why hard links work (two names pointing to the same inode) and why “file has no name, only an inode number” is true at the disk level (ls -i shows inode numbers). Directories are themselves files whose content is a table of name → inode entries.

Allocation Strategies

How does an inode point to a file’s blocks? Three classic schemes, still relevant today:

The indexed scheme uses a few direct pointers plus single/double/triple indirect blocks, so tiny files need only direct pointers while huge files chain through indirect blocks. This is why ext4 handles both a 4 KB file and a 16 TB file with the same inode.

Directories and Paths

A path (/var/log/app.log) is resolved by walking the directory tree: read /’s inode, look up var, read its inode, look up log, and so on. Each lookup is a disk access unless cached — which is why deep directory trees are slower than shallow ones, and why the kernel’s dcache (directory entry cache) is so important. Symlinks (ln -s) are files whose content is another path, resolved as a string; hard links are multiple directory entries pointing to the same inode.

The Page Cache and Buffering

File writes don’t hit disk immediately. The kernel keeps recently used file data in page cache (RAM). A write typically lands in cache and is flushed to disk lazily; a read may be served entirely from cache (memory-hierarchy shortcut from the Computer Architecture topic). Implications:

• Repeated reads are fast — but a crash can lose data still sitting in cache.
• fsync/O_SYNC forces data to stable storage — the expensive durability call that databases issue constantly.
• “Disk is 100% full” and “write says OK but power loss loses data” are both consequences of this layer.

This is also why measuring “RAM usage” on a Linux box shows lots of “cache” — it’s not wasted; it’s the OS buying back the memory-hierarchy latency gap.

Journaling and Crash Recovery

A crash mid-write can leave the file system half-updated: metadata says a block belongs to a file, but the allocation bitmap says it’s free. Rebuilding the whole tree from scratch (fsck) on every boot is too slow, so modern file systems journal: before modifying the on-disk structures, they write the intended change to a journal (a log), then apply it, then mark the transaction complete.

• Ordered journaling (ext4 default): data + metadata journaled; metadata applied before data is allowed to be overwritten.
• On crash: replay the journal to a consistent state — fast, bounded recovery.

Copy-on-write (CoW) file systems (btrfs, ZFS, APFS) take a stronger approach: never overwrite live blocks — write new copies and atomically repoint. They get snapshots and self-healing essentially for free at the cost of more metadata work.

The Virtual File System (VFS)

Linux supports dozens of file systems behind one API because of the VFS layer — a kernel abstraction defining the operations every file system must implement (open, read, iterate_dir, …). Applications and syscalls talk to VFS; VFS dispatches to the concrete driver (ext4, XFS, NFS, tmpfs, procfs). This is the same pattern as the syscall interface: one contract, many implementations, all checkable. It’s also why /proc and /sys appear as “files” — they’re file systems whose reads generate kernel data on the fly.

Practice Trajectory

1. Run ls -li and stat a file; identify its inode number, links, and block allocation.
2. Create a hard link and a symlink to the same file; stat both and explain the difference in inode numbers.
3. df -h and mount — identify the file system type of your root, home, and a tmpfs mount.
4. Write a program (or use dd + sync) that writes a file, then measure the difference with and without fsync.
5. Read /proc/meminfo’s Cached line and explain why “free” RAM with lots of cache is healthy, not alarming.

When It’s the Right Tool