Operating System Concepts
Classic OS interview material on one page: processes, threads, CPU scheduling, virtual memory, synchronization, deadlock, filesystems, I/O and IPC. Linux/Android kernel specifics live on linux-kernel-bsp.html — link there, do not rewrite GKI/device-tree/drivers.
- An OS multiplexes hardware, isolates programs, and exposes a stable syscall ABI. User code cannot touch devices or other processes' memory except through that ABI.
- A process is an address space plus OS metadata (the PCB). A thread is a schedulable execution context inside a process. Context switches and page faults are the two costs you must be able to narrate.
- Scheduling trades waiting, turnaround and response time. Know FCFS, SJF/SRTF, RR, priority and MLFQ; mention Linux CFS only as one implementation.
- Shared memory plus concurrency needs mutual exclusion. Deadlock needs all four Coffman conditions; prevention, Banker's avoidance, and detect-and-recover are three different policies.
- Virtual memory is demand-paged physical RAM plus a page table and a TLB. Replacement, working sets and thrashing are the interview cluster; COW and
mmapare the "how Linux actually uses this" cluster. - Kernel internals, drivers and Android bring-up are on Linux kernel & BSP. Language-level threads and memory live on C, Java and C++.
What an OS is: kernel vs user, privilege, syscalls
An operating system is the program that owns the machine. It multiplexes CPU, memory and devices among programs, isolates those programs from each other and from the hardware, and abstracts messy devices behind a small, stable interface: files, processes, sockets, virtual memory. Everything interesting in an OS interview is a consequence of that triangle: share, isolate, abstract.
Think of a city government. Residents (user processes) may not dig up the street or walk into someone else's house. They file permits (system calls). Civil servants in a restricted office (the kernel, running in privileged mode) update the land registry (page tables), dispatch police and ambulances (interrupt handlers), and assign road time (the CPU scheduler). Privilege rings are the locked doors of that office: ring 0 / exception level 1 can change the maps; ring 3 / EL0 cannot. A syscall is a resident walking up to the counter; an interrupt is a fire alarm that stops the counter mid-sentence.
Kernel vs user, dual mode
Dual-mode operation is the hardware rule that makes isolation possible. The CPU has at least two privilege levels. In user mode the instruction set is restricted: no direct device I/O, no write to page tables, no halt, no change of the privilege bit. In kernel mode those instructions are legal. The mode bit is switched by hardware on a well-defined entry (syscall, interrupt, exception) and switched back on a well-defined exit (return-from-exception). A program that could flip the mode bit itself would be the kernel.
The kernel is the privileged program that stays resident and handles those entries. User is everything else: shells, browsers, Android apps, even init and system daemons. "Userspace" is not "unimportant"; it is "not trusted with the MMU and the devices".
privilege (high) kernel mode page tables, devices, scheduler, syscall impl. ------------ controlled crossings only: syscall / IRQ / fault user mode apps, libc, runtimes, most of Android privilege (low) x86-64: ring 0 = kernel, ring 3 = user (rings 1–2 unused on mainstream OS) AArch64: EL1 = kernel, EL0 = user (EL2 hypervisor, EL3 firmware)
Privilege rings
Older textbooks draw four rings (0 most privileged through 3). Mainstream kernels use two: kernel and user. Extra levels exist for a hypervisor (type-1: the guest kernel is deprivileged) and for firmware. Interviewers want the idea — a hardware-enforced lattice of privilege — not a trivia list of every ARM exception level. If they ask "why not one mode?", the answer is: then a wild pointer in any program can rewrite page tables or program a DMA engine into another process's memory.
System calls
A system call is a controlled, numbered request: "open this path", "read this fd", "mmap this range", "clone a thread". The userspace stub (usually in libc; see C) loads a syscall number and arguments into agreed registers, then executes a trap instruction (syscall on x86-64, svc on ARM). Hardware saves user registers, switches to kernel mode, and vectors to a dispatcher. The kernel validates arguments (especially user pointers), does the work or sleeps, then returns a result and restores user mode.
- Stub libc puts the number and args in the ABI registers and traps.
- Entry hardware saves the user program counter and status, raises privilege.
- Dispatch kernel looks up the number, copies/checks user pointers.
- Work may complete now or block (I/O, lock, child
wait). - Return result in a register (or
errno-style error); privilege drops; user resumes.
open, read, write, mmap, clone/fork, futex. Linux implements them with a syscall table and a kernel entry path; Android apps usually go Java/Kotlin → ART → bionic → the same syscalls. The kernel walk lives on Linux kernel & BSP.Interrupt vs trap vs exception
These words get used loosely. In interview English, sort them by who caused it and whether it is restartable.
| Term | Who | Synchronous? | Typical example |
|---|---|---|---|
| Interrupt (IRQ) | Device / timer | No — asynchronous to the instruction stream | NIC packet, timer tick, disk completion |
| Trap / syscall | The running program, on purpose | Yes | svc / syscall instruction |
| Exception / fault | The running program, by accident or by design of VM | Yes | Page fault, divide-by-zero, illegal opcode |
| Fault | Subset of exceptions | Yes, often restartable | Page fault: kernel fills the page, retries the instruction |
| Abort | Subset of exceptions | Yes, usually not restartable | Hardware machine check, some double faults |
A trap is a deliberate synchronous transfer (syscall, breakpoint). An interrupt is asynchronous. An exception is synchronous but not a request for a service in the POSIX sense — the instruction itself cannot complete without kernel help (or should kill the process). Page faults are the important "exception that is not a bug" case: virtual memory is built on them.
probe(); that is the kernel page.Processes: PCB, states, context switch, fork/exec
A process is a program in execution: a private virtual address space, one or more threads, and a bundle of OS-owned resources (open files, credentials, signal mask, resource limits). The kernel's record of that bundle is the process control block (PCB) — a structure, not a POSIX type. Interviews treat "process" as the isolation boundary and "thread" as the schedulable entity inside it.
A process is a rented workshop. The lease file at the front desk is the PCB: tenant name (PID), key list (page tables / mm), tool lockers (file descriptors), emergency contacts (parent PID, signals). The workers inside the shop are threads; they share the floor plan and the lockers. A context switch is the front desk swapping which shop is on the main workbench: save the current tools' positions, load the next shop's. fork photocopies the lease and the shop; exec guts the shop and installs a different business under the same lease number.
What the PCB holds
Exact fields differ by kernel, but every interview PCB contains:
- Identity: PID, parent PID, credentials (UID/GID and derivatives), process group / session.
- State: new / ready / running / waiting / terminated (names vary).
- CPU context: where to resume (registers, stack pointer, program counter) — often per-thread, not per-process.
- Memory: pointer to page tables / address-space descriptor, heap/stack accounting.
- I/O and files: file-descriptor table, current working directory, umask.
- Scheduling: priority, class, accounting (CPU time used).
- IPC / signals: pending signals, handlers, wait queues.
On Linux the living PCB is task_struct (plus mm_struct, files_struct, …). That split — and how Android's LMK reads it — is kernel material. Here you only need: the PCB is the kernel object you context-switch and the object ps is describing.
Process states
admit
NEW --------------------> READY
| ^
dispatch | | preempt (quantum / higher priority)
v |
RUNNING
/ | \
block / | \ exit
(I/O, / | \
wait, / | v
sleep)/ | TERMINATED ---- reaped by wait() ----> gone
v |
WAITING --+
(I/O done / event / child exit wakes it back to READY)
Five-state model. Some books add "suspended ready / suspended blocked" for swapped-out jobs.
| State | Meaning | Who moves you |
|---|---|---|
| New | Being created; not yet schedulable | OS admits you to ready |
| Ready | Runnable, waiting for a CPU | Scheduler dispatches |
| Running | On a CPU right now | Preempt, block, or exit |
| Waiting / blocked | Cannot run until an event | I/O completion, signal, child exit, lock |
| Terminated | Finished; PCB may still exist (zombie) | Parent wait reaps the PCB |
Context switch cost
A context switch is the kernel saving one thread's CPU state and loading another's. Direct cost: save/restore registers, switch kernel stacks, switch address space (reload page-table root, which flushes or tags the TLB), jump to the new program counter. Indirect cost, often larger: cache and TLB pollution. The next process does not share the previous one's hot cache lines. Switching threads in the same process is cheaper: no address-space switch, TLB can stay (on a tagged TLB it stays anyway; on an untagged one a process switch must flush).
Do not claim "context switch = 1 µs" as a constant. Say: it is thousands of cycles plus a cold cache, which is why spinning a few dozen cycles can beat sleeping, and why huge thread counts with tiny quanta destroy throughput.
POSIX process model: fork, exec, wait, exit
UNIX-family interviews use POSIX as the example, even if the shop also runs Windows. The four calls are a complete life cycle:
forkCreates a child that is a copy of the parent. Both return fromfork: child gets 0, parent gets the child's PID. Address space is logically copied; modern kernels implement this with copy-on-write pages (see virtual memory).execfamily Replaces the current address space with a new program. Same PID, same open fds (unless marked close-on-exec), new text/data/stack.fork+execis "create a different program".exitTerminates: close files, release memory, notify parent, leave a zombie status behind until reaped.wait/waitpidParent blocks (or polls) until a child stops or exits, then collects the exit status and frees the zombie PCB.
pid = fork();
if (pid == 0) {
/* child */
execlp("ls", "ls", "-l", (char *)0);
_exit(127); /* exec failed */
} else if (pid > 0) {
/* parent */
waitpid(pid, &status, 0);
} else {
/* fork failed */
}
Zombie
- Child has exited
- Parent has not
waited - Almost no memory, but the PID and exit status remain
- A leak of PCBs/PIDs if the parent never reaps
Orphan
- Parent has exited while the child still runs
- The OS reparents the child (classically to PID 1 /
init) - The new parent will
wait, so the child will not stay a zombie forever
init. Android's zygote forks app processes from a pre-warmed runtime so exec is not on the cold-start path; that policy lives on Android frameworks. The portable interview answer is still fork/exec/wait/zombie/orphan.exit the address space is gone. What remains is a slot in the process table so the parent can read the exit code. The failure mode of ignoring children is PID exhaustion and ps full of defunct, not RAM exhaustion.Threads: user vs kernel, N:1 / 1:1 / M:N, TLS
A thread is the unit the scheduler actually runs: a register set, a stack, a thread-local storage area, and a scheduling state, all inside a process's address space. Threads in one process share code, heap, globals and file descriptors. They do not share stacks or (by default) the values of thread-local variables.
If a process is the workshop, threads are workers on the same floor sharing the same tool lockers and the same floor plan. They can hand each other a part without leaving the building (shared memory — cheap). They can also trip over the same cable (data races). A second process is a different building: to share a part you need a courier (IPC). That is the whole "why threads vs processes" question.
Why threads instead of processes
| Threads in one process | Separate processes | |
|---|---|---|
| Create / switch | Cheaper (shared mm, no fd-table copy) | Heavier; COW helps fork but not enough for fine-grain parallelism |
| Share data | Direct loads/stores + a lock | Need IPC (pipe, shm, socket) |
| Fault isolation | One wild pointer can corrupt all threads | A crash is contained (browsers use processes for this) |
| Credentials / fds | Shared by default | Independent |
Pick threads when the work is parallel or concurrent and shares a large working set (a server handling connections, a UI thread plus workers). Pick processes when you need a crash or security domain (a renderer, a plugin, a sandbox). Browsers and Android's isolated processes are the standard interview examples; see frameworks for app process policy and system design for multi-process services.
User threads, kernel threads, and the mapping
User-level threads are scheduled by a library in the process (a runtime, a green-thread package). The kernel sees one or a few tasks. Kernel-level threads are first-class schedulable entities; blocking in one does not freeze the others. The mapping is the interview taxonomy:
N:1 (many user : one kernel)
A library multiplexes N user threads onto one kernel thread. Switch is a function call (fast). One blocking syscall or one page fault blocks all of them. Cannot use more than one core. Historic user-thread packages.
1:1 (one user : one kernel)
Each application thread is a kernel thread. Blocking and parallelism work as you expect. Create/switch go through the kernel (still cheap enough). This is POSIX pthreads on modern Linux (NPTL) and the usual Android/ART native thread.
M:N (many user : some kernel)
A runtime parks M green threads on N kernel workers. Can hide I/O and use several cores. Hard to get right (scheduler on scheduler, signal delivery, stop-the-world). Seen in some language runtimes; interviewers care that you know the trade-off, not a vendor name.
clone with shared address space creates a kernel thread; pthreads is a library on top. Java threads on Android are native 1:1 threads managed by ART; the JVM-level memory model and synchronized live on the Java page. Do not describe "the kernel thread" as a Linux kthread unless asked; that is a kernel-internal worker, covered on Linux kernel & BSP.Thread-local storage (TLS)
TLS is storage that is addressed like a global but has a distinct instance per thread: errno, a thread ID, a scratch buffer, a PRNG state. Implementation sketch: the ABI reserves a register (FS/GS on x86-64, TPIDR on ARM) that points at a per-thread control block; TLS variables are offsets from that pointer. A "global" that is actually TLS is why errno is thread-safe without a lock.
fork in a multithreaded process, POSIX says the child has one thread (the one that called fork). Other threads vanish mid-critical-section; locks they held stay locked. The safe pattern is fork-then-exec, or to exec immediately. This is a favourite follow-up.CPU scheduling: bursts, algorithms, metrics
The CPU scheduler chooses which ready thread runs on a core. Workload is not a smooth stream of arithmetic: it is a sequence of CPU bursts separated by I/O bursts (or lock waits, or sleeps). Interactive jobs have short CPU bursts and frequent I/O; batch jobs have long CPU bursts. A good policy is fast for short bursts (so the UI feels alive) and fair enough that a long job still finishes.
A single bathroom on an airplane is the CPU. FCFS is the aisle queue: a passenger who reads a novel in there (long CPU burst) creates a convoy of people who only needed ten seconds. SJF is "shortest bathroom time first" — optimal average wait if you magically knew durations, rude if you guess wrong. Round robin is a kitchen timer on the door: everyone gets a quantum, then goes to the back. Priority is first class vs economy; without aging, economy never gets in. MLFQ is a set of timers: if you finish fast you stay in the short line; if you keep using your full quantum you get demoted to the "long haul" line.
Preemptive vs non-preemptive
Non-preemptive (cooperative): a thread runs until it blocks or yields or exits. Simple, but a runaway loop starves everyone. Preemptive: the timer interrupt or a wake-up of a higher-priority thread can seize the CPU. All general-purpose OS schedulers you will discuss are preemptive. "Non-preemptive SJF" still appears in textbook numericals; say so when you work a Gantt chart.
The classic algorithms
| Policy | Rule | Preempt? | Interview notes |
|---|---|---|---|
| FCFS (FIFO) | Run in arrival order to completion of the burst | No | Convoy effect; fair in a weak sense; terrible average wait if a long job is first |
| SJF | Shortest next CPU burst first | No | Minimum average waiting time among non-preemptive policies if bursts are known; you do not know the future |
| SRTF | Shortest remaining time first | Yes | Preemptive SJF; still needs a burst estimate; can starve long jobs |
| Round robin (RR) | Each ready job gets a time quantum q, then the tail of the queue | Yes | Good response time; q too small → thrash on context switches; q too large → FCFS |
| Priority | Highest priority ready job wins | Either | Starvation; fix with aging (priority rises with wait) |
| Multilevel queue | Separate queues (e.g. interactive vs batch), maybe different policies | Yes | Jobs do not move; misclassification is permanent |
| MLFQ | Several priority queues; demote jobs that burn their quantum, promote idle/short | Yes | Learns interactivity; needs aging or boosts so batch jobs do not freeze |
Convoy effect and aging
The convoy effect is FCFS's failure mode: one long CPU-bound job is at the head; a fleet of short, I/O-bound jobs wait behind it, then all run briefly and block, then all wake and wait behind the long job again. Average waiting time explodes. RR and SJF/MLFQ exist in part to break convoys.
Aging is the anti-starvation patch for priority (and for SRTF/MLFQ in spirit): the longer you wait, the more your priority rises, so a low-priority job cannot be deferred forever. Interviewers listen for the word after you say "priority can starve".
Linux example: CFS (mention only)
Textbook policies are what you work on paper. Real Linux desktop/server scheduling for normal threads is CFS (Completely Fair Scheduler; newer kernels discuss EEVDF as the fair-class evolution): each task has a virtual runtime; the leftmost task in a tree of vruntime runs next, so over a long window each job gets a fair share of CPU. Real-time classes (FIFO/RR POSIX) sit above that. Energy-aware and Android-specific policy (EAS, cgroup cpu sets, binder threads) belong on Linux kernel & BSP and power & thermal. In an OS-concepts interview, one sentence on CFS is enough: "Linux fair scheduling approximates ideal fair sharing via virtual runtime; it is not FCFS and not a textbook RR."
Synchronization: races, locks, classic problems
When two threads share memory, "interleaved loads and stores" is a first-class behaviour, not an accident. A race condition is when the correctness of a result depends on that interleaving. The fragment that must appear atomic is the critical section. The lock (or a wait-free algorithm) is how you make it atomic from the other threads' point of view.
Two clerks updating the same paper ledger. If both read "balance = 100", both add 20, both write 120, a deposit vanished. The ledger room's rule "only one clerk at the book" is mutual exclusion. A mutex is a single bathroom key on a hook. A counting semaphore is a club with N wristbands: N people may be inside. A condition variable is the buzzer that says "a table is free" — you still need the host (the mutex) to seat you, or two parties grab the same table.
Critical-section requirements
Peterson's algorithm and the textbook checklist still get asked. A solution must provide:
- Mutual exclusion At most one thread is in the critical section at a time.
- Progress If the critical section is free and threads want in, selection cannot be postponed indefinitely by threads that are not interested (no "the doorman went to lunch").
- Bounded waiting There is a bound on how many times other threads may enter after you have requested entry and before you are granted it (no silent starvation).
Hardware helps with atomic instructions (test-and-set, compare-and-swap, fetch-and-add). Software-only solutions exist on sequentially consistent memory but are not what you ship; you ship a mutex built on atomics plus an OS sleep queue (Linux: futex — details on the kernel page).
Mutex, semaphore, monitor, condition variable
| Primitive | Shape | Use |
|---|---|---|
| Mutex | Owned lock; lock/unlock; typically the owner must unlock | Protect a critical section; default choice |
| Binary semaphore | Count 0 or 1; P/wait decrements, V/signal increments; no owner | Can mimic a mutex, but also signal across threads ("it happened") |
| Counting semaphore | Non-negative integer; P sleeps at 0 | N identical resources, producer-consumer empty/full counts |
| Monitor | Language/runtime object: mutex + encapsulated data + condition variables | Java synchronized methods; Mesa/Hoare semantics in textbooks |
| Condition variable | Wait (atomically drop mutex and sleep); signal / broadcast | Wait for a predicate ("queue not empty"), not just exclusion |
Java monitors and wait/notify are on the Java page. C++ std::mutex / std::condition_variable are on C++. Here the OS point is: the kernel provides sleep-and-wake; the language library paints mutex/CV on top.
Producer-consumer and readers-writers
Bounded buffer (producer-consumer): producers write slots, consumers read them. You need (1) mutual exclusion on the buffer structure, (2) "do not overwrite a full buffer", (3) "do not read an empty buffer". Classic solution: mutex + two counting semaphores empty and full, or one mutex + two condition variables.
Readers-writers: many readers may share; a writer needs exclusive access. Variants: readers-preference (writers may starve), writers-preference, or a fair queue. Interviewers want you to state which starvation you accepted.
/* bounded buffer sketch — counts are the idea, not a compilable module */
sem_t empty = N, full = 0;
mutex_t m;
void produce(item_t x) {
sem_wait(&empty); /* sleep if no free slot */
mutex_lock(&m);
put(x);
mutex_unlock(&m);
sem_post(&full); /* a slot is now filled */
}
void consume(item_t *out) {
sem_wait(&full);
mutex_lock(&m);
*out = get();
mutex_unlock(&m);
sem_post(&empty);
}
Spin vs sleep
A spinlock waits in a loop on an atomic flag. It is correct for very short critical sections on a multiprocessor, and it is the only option in contexts that must not sleep (hard IRQ on Linux — kernel page). A sleeping lock (mutex) deschedules the waiter: useful when the hold time may include I/O or a long computation. Spinning wastes CPU and can invert priorities; sleeping costs a context switch. Hybrid locks spin briefly, then sleep (adaptive mutex).
Memory visibility (high level)
A mutex is not only exclusion. On a real machine, cores have store buffers and caches. Without a happens-before edge, thread A storing data = 42; ready = 1 can be observed by thread B as ready == 1 while data is still stale. Acquiring a mutex (or a correctly paired atomic load-acquire) is a barrier: you see all writes that happened-before the matching unlock. volatile in C is not that barrier (see C). Interview sentence: "locks give exclusion and publication; atomics need an explicit memory order."
Deadlock: Coffman, Banker's, livelock, inversion
Deadlock is a set of processes where each is waiting for a resource that another in the set holds, and none will release what they hold. No CPU progress, forever, unless an external agent kills or rewinds someone.
Four cars meet at a four-way stop and each blocks the next car's only path forward. Nobody can reverse because the textbook cars do not reverse. That cycle is deadlock. Livelock is four overly polite drivers who all inch forward and all back up, forever: they keep changing state but never cross. Starvation is a side road that never gets a gap because the main road is always busy — progress exists in the system, just not for you.
Coffman conditions
All four must hold for deadlock (with exclusive, non-sharable resources):
- Mutual exclusion The resource is not usable by two holders at once.
- Hold and wait A process holds at least one resource and waits for another.
- No preemption You cannot forcibly yank a held resource (or doing so is not in the policy).
- Circular wait A cycle: P1 waits for P2 waits for … waits for P1.
Break any one condition and deadlock is impossible. That is prevention: e.g. acquire locks in a global order (breaks circular wait); acquire everything up front (breaks hold-and-wait); make a resource preemptive (rare for mutexes; possible for the CPU).
Wait-for graph (single-instance resources):
P1 ----wants R2----> P2 ----wants R1----> P1
^ |
+-------------- cycle ---------------+
Cycle => deadlock when each resource has one instance.
Multi-instance: a cycle in the resource-allocation graph is necessary
but not sufficient — Banker's / reduction needed to be sure.
Prevention vs avoidance vs detection
| Policy | When | Idea | Cost |
|---|---|---|---|
| Prevention | Design time | Outlaw a Coffman condition | Can hurt concurrency (lock ordering is the usual real fix) |
| Avoidance | Before each grant | Stay in a safe state (Banker's) | Needs max-claim a priori; rarely used for locks, used as a numerical question |
| Detection + recovery | After the fact | Build a wait-for graph or run a reduction; then kill or roll back | Needs a victim policy; databases do this more than kernels |
| Ostrich | Never | Ignore; reboot if it happens | Honest description of many desktop lock bugs |
Banker's algorithm (avoidance)
Each process declares a maximum claim per resource type. The allocator tracks Allocation, Need = Max − Allocation, and Available. A request is granted only if, after a hypothetical grant, there still exists an order in which every process could run to completion with the remaining resources (a safe sequence). If not, the request waits even though enough units exist right now — those units are reserved to keep the state safe.
Interviewers want the intuition and a tiny matrix walk, not a coding of the nested loops. Mention that real lock libraries do not run Banker's: they use lock ordering, try-lock with backoff, or detection in debug builds.
Livelock, starvation, priority inversion
Livelock: processes keep responding to each other and change state, but make no useful progress (two people stepping aside in the same direction). Starvation: a process waits unbounded while others proceed (priority without aging; writers in a readers-preference lock).
Priority inversion: a low-priority thread holds a lock that a high-priority thread needs; a medium-priority thread runs and preempts the low-priority holder; the high-priority thread is stuck behind the medium one. Classic real-time bug. Priority inheritance: the holder temporarily runs at the waiter's priority until it unlocks. Priority ceiling is the other textbook fix: a lock has a ceiling priority; anyone who takes it boosts to that ceiling.
Memory management: addresses, fragmentation, paging, TLB
Programs speak logical (virtual) addresses. The MMU, using tables the kernel installs, translates them to physical addresses in RAM (or decides there is no mapping and raises a fault). That translation is the whole of memory protection and of relocation: the same binary can run at different places in physical memory without being rewritten.
A library user asks for "shelf B, slot 14" (a virtual address). That is not a GPS coordinate of a box in the warehouse. The card catalog (page table) says which warehouse pallet (physical frame) currently holds that slot. The librarian's sticky notes on the desk (the TLB) remember recent lookups so she does not walk the catalog for every book. If the catalog says "in the annex" or "not in the building", that is a page fault: fetch it, or deny access.
Relocation and binding
Compile-time binding assumes a fixed physical address (embedded without MMU). Load-time binding patches the binary as it is loaded. Execution-time binding (virtual memory) translates every access; the program never sees physical addresses. Interviews expect "relocation is why two processes can both think they own address 0x400000".
Fragmentation
| What it is | Where you see it | Mitigation | |
|---|---|---|---|
| External | Free memory exists but is split into holes too small for the next request | Variable-size partitions, classic segment allocation | Compaction; paging (allocate in fixed frames) |
| Internal | You were given a block larger than you needed; the slack is wasted inside the block | Paging (last page of a region); buddy/slab slack | Smaller pages / better sized bins; tolerate it |
Compaction slides allocated regions together to make one big hole. It requires execution-time relocation (or a stop-the-world move of physical pages and a table rewrite). Paging made compaction of user heaps unnecessary for the "cannot fit a process" problem; heaps still fragment in userspace (see C malloc).
Paging vs segmentation
Paging
- Fixed-size pages / frames (4 KiB classic; also 2 MiB/1 GiB huge pages)
- Process sees a flat virtual array
- No external fragmentation of frames
- Internal fragmentation on the last page of a mapping
- Table can be huge unless hierarchical
Segmentation
- Variable-size segments (code, stack, heap, modules)
- Addresses are (segment id, offset)
- Matches how programmers think
- External fragmentation returns
- Modern OS: paging is the mechanism; "segments" are VMAs / mappings on top
Multi-level page tables and inverted tables
A flat table for a 64-bit space is absurd (252 4 KiB pages). A radix tree of tables (3 or 4 levels on 64-bit Linux) stores only the populated branches. Each virtual address is split into a series of indices plus a page offset. The walk is several dependent memory loads — which is why the TLB exists.
virtual address (example: 3-level, 4 KiB pages)
[ unused | idx_L2 | idx_L1 | idx_L0 | offset ]
| | |
v v v
L2 table --> L1 table --> PTE --> physical frame
+ offset = PA
TLB hit: skip the walk, or skip the upper levels (page-walk cache).
TLB miss: hardware or software walker fills a TLB entry from the PTE.
An inverted page table is one entry per physical frame: "this frame belongs to (pid, vpn)". Lookup is a search (hash) rather than an index. Saves table space on huge address spaces; makes "translate this VA" more work. Interview fact, not a Linux default (Linux uses multi-level + hashed/specialised structures for some arches; do not invent a design).
TLB and effective access time
The translation lookaside buffer is a small associative cache of recent VPN → PFN translations (and permissions). Hit: almost the cost of a memory reference plus a tiny TLB lookup. Miss: walk the page table (L memory references for L levels), then access the data.
Speedup intuition: without a TLB, every load pays L+1 memory references. With hit rate h close to 1, you pay about 1. A drop from 99% to 90% hit rate is a large EAT jump because misses are so expensive. Context switches that flush an untagged TLB destroy h.
Page-fault path (portable)
- Hardware the instruction's translation misses in the TLB; the walker finds an invalid PTE, or there is no PTE; CPU raises a fault, saves the faulting VA and the program counter.
- Kernel classify access vs not-present vs permission vs bad address. Bad pointer → signal / kill (UNIX:
SIGSEGV). - Not present, legal mapping find or allocate a frame; if file-backed, schedule disk I/O and sleep; if anonymous, zero a frame; if COW, copy.
- Install PTE set valid, dirty/access bits as required; flush or fill TLB.
- Return the instruction is restarted (fault, not abort).
Linux mm details (VMA, anon vs file, major vs minor fault) are on Linux kernel & BSP.
Virtual memory: demand paging, replacement, COW, mmap
Virtual memory means a process's logical space can be larger than RAM, and only the pages it is using need frames. The backing store (swap, the filesystem) holds the rest. The mechanism is demand paging: a page is brought in when it is first touched (or when a policy prefetches it), not when the program starts.
Your desk (RAM) holds the papers you are using (the working set). The filing cabinet (disk/swap) holds the rest. You do not photocopy the whole cabinet onto the desk at 9:00. You pull a folder when you reach for it (demand paging). If you keep six projects "active" and the desk only holds two, you spend the day filing and unfiling (thrashing) and no project finishes. LRU is "put away the folder you have not touched in the longest time". The impossible oracle who knows the future is OPT.
Working set and thrashing
The working set W(t, Δ) is the set of pages referenced in the last Δ time units. If the sum of working sets exceeds RAM, the system pages continuously: CPU utilisation drops while the disk is busy. That is thrashing. Fixes: run fewer processes (admission control), give the working set more frames, reduce the working set (fix a memory leak, use a smaller cache), or add RAM. A common interview trap: "CPU is idle, load average is high, disk is saturated" — think thrashing or I/O wait, not "need a faster CPU".
Replacement algorithms
| Algorithm | Victim | Notes |
|---|---|---|
| FIFO | Oldest loaded page | Simple; can evict a hot page; suffers Belady's anomaly |
| OPT / MIN | Page whose next use is farthest in the future | Unrealisable (needs the future); lower bound for comparisons |
| LRU | Least recently used | Good approximation to OPT in theory; exact LRU is expensive (stack or timestamps on every ref) |
| Clock / second chance | Walk a circular list; skip pages with the referenced bit set (and clear it) | Hardware referenced bit + a hand; the LRU you actually ship |
| Second chance (explicit) | FIFO, but if the referenced bit is set, give another chance | Clock is the efficient picture of the same idea |
Belady's anomaly: for some reference strings, FIFO faults more with more frames. OPT and stack algorithms such as LRU do not. Mention it when you mention FIFO.
Copy-on-write and mmap
Copy-on-write (COW): after fork, parent and child share physical pages marked read-only. The first store faults; the kernel copies the page, assigns a private frame, and resumes. Cheap fork if you exec soon. Also used for snapshots and some mmap MAP_PRIVATE mappings.
mmap intuition: you ask the OS to map a file (or anonymous memory) into the address space. Loads page-fault in file pages (the page cache). Stores may stay in cache until msync or eviction. Versus read: you avoid an extra userspace copy if you can consume data in place; you also expose page-fault latency on first touch. Shared mappings are IPC (see below). Implementation and VMAs are kernel territory.
I/O and interrupts: PIO, IRQ, DMA, drivers as a concept
The CPU is a terrible photocopier. Moving megabytes between a device and RAM by executing load/store in a tight loop wastes a core and still may be slower than hardware that exists for that job. OS I/O interviews are about who copies and who waits.
Programmed I/O is you walking every box from the loading dock to the storeroom yourself, checking the dock after each box ("is the next one ready?"). Interrupt-driven I/O is you going back to real work; the dock bell (IRQ) rings when a box arrives; you still carry each box. DMA is hiring a mover: you set up a list (addresses, length), the mover fills the storeroom, and the bell rings once at the end. You must still keep the storeroom map consistent (caches, bounce buffers) — that bookkeeping is why "DMA is free" is false.
| Style | CPU during transfer | When it is used |
|---|---|---|
| Programmed I/O (polling) | Busy-waits or repeatedly checks a status register; CPU moves each word | Tiny, rare, or boot-time transfers; some very low-latency devices |
| Interrupt-driven | CPU does other work; IRQ handler moves data or schedules a bottom half | Low-rate character devices, completion notifications |
| DMA | CPU programs a controller; controller masters the bus | Disk, NIC, GPU, audio — anything bulky |
Blocking, non-blocking, asynchronous
- Blocking
readsleeps until the request can complete (or a signal hits). Simple; a thread is occupied for the wait. - Non-blocking
readreturns immediately with data orEAGAIN. You poll or useselect/poll/epoll. - Asynchronous I/O you start a request and are notified later (completion queue, signal, callback) without the thread sitting in the syscall. Overlaps compute and I/O in one thread.
These are orthogonal to PIO/DMA: DMA can back a blocking read. Interviewers mix the words; separate "how the bytes move" from "how the thread waits".
Device drivers as a concept
A device driver is the kernel (or userspace I/O server) module that speaks a device's register language and exposes a uniform interface: block read/write, character stream, network packet. The OS is designed so the filesystem does not know your SSD brand and the scheduler does not know your NIC. That is the idea. Binding to a device tree, probe, clocks and interrupts as Linux objects are Linux kernel & BSP — do not recite them here.
Filesystems: inodes, allocation, journaling, links
A file is a named, persistent stream of bytes (plus metadata). A directory is a listing that maps names to file identities. The identity on a UNIX-style disk is the inode (index node): owner, mode, timestamps, size, and the map from file offsets to disk blocks. The name is not in the inode; it is in a directory entry. That split is why hard links work.
The inode is a library catalog card: who owns the book, how long it is, which warehouse bins hold the pages. The directory is a shelf label that says "Moby-Dick → card #4281". A hard link is a second shelf label pointing at the same card. A symbolic link is a sticky note that says "see the label Moby-Dick" — it stores a path, not the card number. Delete the last hard link and the card (and bins) can be recycled; delete a name that a symlink used and the sticky note dangles.
Allocation: contiguous, linked, indexed
| Method | How blocks are found | Good | Bad |
|---|---|---|---|
| Contiguous | Start block + length | Sequential I/O, simple | External fragmentation; growth is painful |
| Linked | Each block points to the next | Easy grow; no external frag | Random access is a chain walk; one bad pointer loses the tail |
| Indexed | An index block (or tree of them) lists data blocks | Random access; grow with extra index blocks | Index overhead; small files waste an index block if you are naive |
FAT is a linked scheme with the links stored in a central table (the File Allocation Table): faster than chasing pointers in the data blocks, still a table walk for random access, and the table is a single point of corruption — hence FAT32's conservative reputation. A UNIX inode is indexed: a few direct block pointers, then single/double/triple indirect blocks. Extents (start+length runs) are the modern variant of "indexed but for long sequential runs".
Journaling and crash consistency
A crash mid-update can leave a directory that points at an unallocated inode, or a bitmap that disagrees with the tree. Crash consistency is the guarantee that after replay or fsck, metadata (and maybe data) is a state that could have existed. A journal (write-ahead log) writes intended metadata changes to a log, commits, then writes the home locations. After a crash, replay the committed log. Policies: journal metadata only (common), or data+metadata (safer, slower). Ordered mode (data to its place before metadata commit) is the usual middle path.
VFS (virtual filesystem) is the kernel's uniform inode/dentry/file-ops layer so open/read do not care whether the backing store is ext4, F2FS, a pipe, or procfs. Linux VFS objects are on the kernel page; the interview idea is "one syscall surface, many implementations".
Hard link
- Another directory entry for the same inode
- Same file, same data blocks
- Cannot span filesystems; usually not allowed for directories
- Link count in the inode; data freed at 0 (and no open fds)
Symbolic link
- A small file holding a path string
- May dangle; may cross filesystems
- Permissions on the target matter for most operations
- Can create cycles if you are careless
IPC: pipes, queues, shared memory, signals, sockets
Inter-process communication is how address spaces that do not share memory on purpose still exchange bytes or events. Threads in one process do not need IPC for data (they need locks). Processes do. The interview skill is picking the mechanism that matches lifetime, topology and performance — not listing every POSIX call.
Two offices. A pipe is a pneumatic tube between a parent and a child who set it up before they were born (anonymous, one-way byte stream). A named pipe (FIFO) is a tube with a lobby label so strangers can find it. A message queue is a mailroom with stamped envelopes (record boundaries, optional priority). Shared memory is a conference table both offices can write on — fastest, and you must chair the meeting (synchronise). A signal is a fire alarm: little payload, nasty if you try to have a conversation with it. A socket is the phone: works across machines, and also works across processes on this one (UNIX domain).
| Mechanism | Shape | Pick it when |
|---|---|---|
| Anonymous pipe | Byte stream, typically parent-child, unidirectional | Shell pipelines; fork a helper and feed it bytes |
| Named pipe (FIFO) | Same stream, lives in the filesystem namespace | Unrelated processes on one machine, simple stream |
| Message queue | Discrete messages, often with types/priorities | You need records, not a stream; multiple readers with selection |
| Shared memory + sync | Pages mapped in both processes | Bulk data, low latency; you already have a mutex/semaphore protocol |
| Signal | Async notification, tiny payload | Job control, "please die", timers — not a data path |
| Socket | Bidirectional, optional network, optional datagram | Client-server, privilege separation, anything that might go remote later |
Shared memory is the fastest path and the easiest to get wrong: you still need a mutex, futex, or lock-free protocol, plus a memory-visibility story. Signals interrupt a thread at an awkward moment; async-signal-safe functions are a short list (see C).
Protection and security basics: identity, ACL, isolation, TOCTOU
Protection is the OS mechanism: who may touch which object. Security is the policy plus the assumption that some programs are hostile. Interviews at this level want identity, access control, isolation boundaries, and one race: TOCTOU.
A building. UID/GID is your employee badge number and team. An ACL is a paper taped to each door listing names. A capability is a physical key: if you hold it, you open the door; the door does not look up your name. Isolation is not sharing a desk (separate address spaces, separate mount views). TOCTOU is checking that the hallway is empty and then walking — someone opened a trapdoor between the check and the step.
UID, GID, and the objects they guard
UNIX-style credentials: a real/effective/saved UID and GID (plus supplementary groups). The kernel stamps a process with them and, on each privileged operation, compares them to the object's owner and mode bits (or to an ACL). Root (UID 0) is the historical escape hatch; modern systems add capabilities (below) so "full root" is not the only privileged state. Android maps apps to distinct UIDs so two apps are two users at the kernel boundary — see frameworks.
ACL vs capability
ACL (access-control list)
- Object-centric: each file/socket lists who may do what
- Easy to answer "who can read this file?"
- Hard to answer "what can this process still do?" without scanning the world
- Revocation: edit the list
Capability
- Subject-centric: a token (fd, handle, key) is the right
- Easy to delegate: pass the fd
- UNIX fds are a capability-like design for open files
- POSIX process capabilities split root into bits (CAP_NET_ADMIN, …)
Do not confuse POSIX file mode bits (a tiny ACL) with Linux process capabilities (privilege bits) or with true object-capability systems. Say which one you mean.
Isolation
The default isolation primitive is the process address space: another process cannot load your memory without a mapping you both share or a kernel bug. Stronger isolation: separate UID, seccomp-style syscall filters, namespaces (mount, PID, network), virtual machines. Each step costs convenience and performance; system design interviews reuse this ladder for sandboxing a plugin or a multi-tenant service.
TOCTOU
Time-of-check to time-of-use: you stat a path, see it is safe, then open the same path. Between the two calls an attacker can replace the file with a symlink. Fix: operate on file descriptors (open then fstat, openat with a directory fd), or use atomic create flags (O_CREAT|O_EXCL). The pattern is general: any check that is not bound to the same kernel object as the use is a race.
Virtualization and containers (light)
A virtual machine is a full machine abstraction: virtual CPU, virtual memory, virtual devices, usually a guest OS. A container is a process (or process tree) with a filtered view of the host kernel: isolated namespaces and limited resources, same kernel, no second scheduler. Interviews mix them; keep the boundary clean.
A type-1 hypervisor is a landlord who owns the building and rents out finished apartments that each think they have their own electrical panel (guests). A type-2 hypervisor is an app on your laptop that runs another house in software (hosted VMs). A container is a lockable room in a shared house: same plumbing (the host kernel), different cupboard labels (namespaces) and a fuse box limit (cgroups). A plain process is just a person in the house with a normal lease (address space) and no extra room door.
Type-1 hypervisor
Runs on the hardware; guests are deprivileged. Datacenter and many phone hypervisors. Lower overhead, needs hardware virt extensions for a full guest kernel.
Type-2 hypervisor
A process on a host OS that implements a VM. Convenient for developers; an extra hop to real devices.
Trap-and-emulate
Guest runs; privileged ops trap to the hypervisor, which emulates the effect. Classic paper model. Fails if a sensitive instruction does not trap (historic x86). Modern CPUs add a guest mode so the hardware traps what the hypervisor needs.
| Process | Container | VM | |
|---|---|---|---|
| Kernel | Shared | Shared | Guest kernel (usually) |
| Isolation | Address space + UID | + namespaces + cgroups | Hardware + hypervisor |
| Density / start time | Highest / fastest | High / fast | Lower / slower |
| Attack surface | Kernel + your code | Kernel (larger ABI) | Hypervisor + virtual hardware |
Namespaces (Linux examples): PID, mount, network, UTS, IPC, user — each is a view. cgroups limit CPU, memory, I/O, freezer. Together they are the usual container recipe. They are not a second OS. Implementation knobs belong on Linux kernel & BSP; the concept belongs here.
Putting it together: a syscall walk and where to go next
Every topic on this page appears in one boring, complete story: read(fd, buf, n) from a userspace C program. Narrate it once in an interview and you have shown dual mode, processes, scheduling, VM, I/O, VFS and IPC-adjacent buffering.
You ask the city desk for a copy of a permit (the file). The clerk checks your badge and the permit number (credentials, fd → file object). If the paper is already on the counter (page cache), you get it immediately. If it is in the warehouse, a runner is sent (block I/O, DMA); you sit in the waiting room (sleep, state = waiting); the scheduler seats someone else. The dock bell rings (IRQ); the runner returns; you are called back to the counter (ready → running); the clerk photocopies into your notebook (copy to the user buffer), not theirs. You leave the building (return from syscall).
- Userspace The program calls
read. libc (bionic on Android) is C; calling convention anderrnoare C material. - Trap
syscall/svc: privilege up, kernel entry, syscall number forread. - Process context The kernel uses the current PCB/thread to find the fd table. The fd is a capability-like index to an open file description (offset, flags, vnode/inode).
- VFS The file's
readoperation: socket, pipe, or regular file. Pipes and sockets may block on emptiness — IPC plus scheduler. - Page cache / VM For a regular file, the offset maps to pages. Hit: copy to
buf(the user pointer was validated). Miss: allocate a frame, submit disk I/O, sleep. That sleep is a state transition to waiting; another thread runs. - I/O path The block layer programs DMA; the disk interrupts on completion. The waiter is marked ready. This is not a driver-writing question; the portable story is enough. Hardware specifics: Linux kernel & BSP.
- Copy out and return Bytes land in the user buffer. The syscall returns a count. The userspace thread continues. If this was a Java
FileInputStream, ART and JNI sit above libc — Java.
app / ART / JVM java.html (threads, GC — not the kernel)
|
libc read() c.html
| syscall
=============== dual mode ===============
kernel entry
fd -> file -> VFS
| linux-kernel-bsp.html (VFS, block, IRQ)
page cache / mm
| miss
block I/O + DMA + IRQ
| wake
copy_to_user(buf)
=============== return ==================
user continues
Data-structure questions about wait-for graphs or Banker's matrices can lean on DSA (graphs, BFS for a safe-sequence search). End-to-end product design that happens to include caches and queues is system design. Do not turn this page into a driver or GKI lecture.
read", spend most of the time on validation, blocking, page cache, and the scheduler wake-up. Name one Linux file (/proc as a VFS example) only if asked. Stop before device tree.Quick revision
- An OS multiplexes hardware, isolates programs, and abstracts devices behind syscalls.
- Dual mode: user code cannot change page tables or talk to devices; the mode bit flips only on trap, IRQ, or exception.
- Privilege rings: kernel vs user in practice; extra levels exist for a hypervisor and firmware.
- A syscall is a numbered, validated request issued by a trap instruction, not a normal function call.
- IRQ is asynchronous (devices, timer). Trap/syscall is synchronous and deliberate. Faults are synchronous and often restartable (page fault).
- A process is an address space plus kernel metadata; the PCB is that metadata.
- States: new, ready, running, waiting, terminated. Preempt: running to ready. Block: running to waiting.
- Context switch cost = register save/restore + optional address-space switch (TLB) + cold caches.
- POSIX life cycle: fork copies, exec replaces, exit leaves a zombie, wait reaps it.
- Zombie = exited, not waited; tiny, but it occupies a PID. Orphan = parent died; reparented and later reaped.
- Threads share the address space and fds; they have their own stacks and TLS.
- N:1 is fast and cannot parallelise or block independently; 1:1 is the modern POSIX default; M:N is a runtime compromise.
- TLS is per-thread globals, usually a register pointing at a control block plus offsets.
- CPU bursts and I/O bursts describe the workload; schedulers care about the next CPU burst.
- FCFS has the convoy effect. SJF/SRTF minimise average wait if bursts are known. RR needs a sane quantum.
- Turnaround = completion − arrival; waiting = turnaround − burst; response = first run − arrival.
- Priority without aging starves. MLFQ demotes CPU hogs and keeps interactive jobs responsive.
- Linux CFS is a fair-share example using virtual runtime; it is not a textbook RR. Details stay on the kernel page.
- A race means the outcome depends on interleaving. The critical section must be mutually exclusive.
- Critical-section trio: mutual exclusion, progress, bounded waiting.
- Mutex has an owner. Semaphores do not; counting semaphores track N resources.
- Condition variables wait for a predicate; always re-check the predicate after wake-up (Mesa semantics).
- Producer-consumer needs exclusion plus empty/full. Readers-writers needs a stated starvation policy.
- Spin if the hold is tiny and you cannot sleep; otherwise use a sleeping mutex. Hybrids spin then sleep.
- Locks also publish memory: unlock/lock (or acquire/release atomics) create happens-before.
- Deadlock needs all four Coffman conditions. Break one to prevent it.
- Banker's grants a request only if a safe sequence still exists. It needs declared maxima.
- Livelock: activity without progress. Starvation: you never run while others do.
- Priority inversion: a medium job preempts a low holder that a high job needs. Inheritance boosts the holder.
- Logical addresses are what the program issues; physical addresses are RAM. The MMU translates.
- External fragmentation = holes between allocations. Internal = slack inside a block or page.
- Paging uses fixed frames (no external frag of RAM). Segmentation uses variable segments (external frag returns).
- Multi-level page tables store only used branches of a huge virtual space. The walk costs L memory refs.
- EAT ≈ h(t_TLB + t_mem) + (1−h)(t_TLB + (L+1)t_mem). Hit rate dominates.
- Page fault: classify, get a frame, fill from disk or zero/COW, install PTE, restart the instruction.
- Demand paging loads a page on first touch. The working set is the recently referenced pages.
- Thrashing: working sets do not fit; the disk is busy and the CPU is idle. Run fewer jobs or add RAM.
- FIFO can Belady-anomaly. OPT is the unrealisable lower bound. Clock approximates LRU with a referenced bit.
- COW:
forkshares physical pages until a write;MAP_PRIVATEmmap is the same idea for files. mmapmaps a file or anonymous memory into the address space; first touch is a page fault, not areadcopy.- Programmed I/O: CPU copies every word. Interrupt-driven: CPU still copies, but after an IRQ. DMA: the device copies; IRQ means done or error.
- Blocking I/O sleeps. Non-blocking returns immediately (
EAGAIN). Asynchronous I/O starts now and completes later on a queue or callback. - A device driver is the translator from a device's registers to a uniform OS interface. Binding and
probeare kernel-page material. - A file is bytes plus metadata. An inode is the identity and block map. A directory maps names to inodes.
- Disk allocation: contiguous, linked, or indexed. FAT is a central link table. A UNIX inode is indexed (direct plus indirect blocks).
- A journal is a write-ahead log so a crash can replay metadata (and optionally data) to a consistent state.
- VFS is one syscall surface over many filesystem implementations (disk, pipe, proc-like).
- Hard link: another directory entry for the same inode. Symbolic link: a path string that may dangle or cross filesystems.
- Anonymous pipes fit parent-child byte streams. Sockets are bidirectional and can go remote. Shared memory is fastest and still needs a lock protocol.
- Signals notify; they are not a data path. Use async-signal-safe calls only in a handler.
- UID/GID label the process. An ACL lives on the object. A capability is a token you hold; an open fd is the everyday example.
- TOCTOU: a check on a path is not a check on the object you later use. Prefer fds,
openat, and exclusive-create flags. - Type-1 hypervisor sits on hardware; type-2 is hosted. Trap-and-emulate needs sensitive guest operations to actually trap.
- A container shares the host kernel and isolates with namespaces plus cgroups. A VM runs a guest kernel on virtual hardware.
read()is libc → trap → fd/VFS → page cache or block I/O sleep → DMA/IRQ wake → copy to user → return.
Glossary
- ACL
- Access-control list: per-object list of who may perform which operations. File mode bits are a tiny ACL.
- Address space
- The range of logical addresses a process may issue, translated by page tables to physical memory or to a fault.
- Aging
- Raising a waiting job's priority over time so a low-priority job cannot starve forever.
- Banker's algorithm
- Deadlock-avoidance check: grant a request only if a safe sequence still exists, given each process's declared maximum claim.
- Belady's anomaly
- For some reference strings, FIFO page replacement faults more when given more frames. Stack algorithms such as LRU do not.
- Binary semaphore
- A semaphore whose count is only 0 or 1. Can signal an event or mimic a mutex, but has no owner.
- Bounded waiting
- A bound on how many times other threads may enter the critical section after you have requested entry.
- Capability
- A token that is a right (an open file descriptor, a handle). Holding it authorises the operation; the object does not look up your name.
- CFS
- Completely Fair Scheduler: Linux's fair class. Tasks are scheduled by virtual runtime so each gets a fair CPU share. Mention only; internals are on the kernel page.
- cgroup
- Linux control group: a resource limit and accounting on a set of tasks (CPU, memory, I/O). A container building block.
- Circular wait
- Coffman condition: a cycle of processes, each waiting for a resource the next one holds.
- Clock algorithm
- Page replacement that walks a circular list and gives a second chance to pages with the referenced bit set. Practical LRU.
- Coffman conditions
- The four necessary conditions for deadlock: mutual exclusion, hold and wait, no preemption, circular wait.
- Compaction
- Sliding allocated memory together to coalesce free holes. Needs relocatable addresses; paging mostly makes it unnecessary for process placement.
- Condition variable
- A wait queue used with a mutex: wait atomically drops the mutex and sleeps; signal wakes a waiter who must re-check the predicate.
- Context switch
- Saving one thread's CPU state and loading another's. Process switches also switch address spaces and disturb the TLB and caches.
- Convoy effect
- FCFS failure: a long CPU job at the head forces many short jobs to wait, then they all block and re-form the convoy.
- Copy-on-write
- Share a physical page among mappings until a write; the write faults and the kernel copies the page.
- Critical section
- The fragment of code that must appear atomic with respect to other threads sharing the same data.
- Deadlock
- A set of processes where each waits for a resource another in the set holds, and nobody can proceed.
- Demand paging
- Bring a page into RAM only when it is first referenced (or when a prefetch policy chooses to).
- DMA
- Direct memory access: a device copies to or from RAM as bus master; the CPU is interrupted on completion or error.
- Dual mode
- Hardware privilege split: user mode cannot execute privileged instructions; kernel mode can. Crossings are traps, IRQs and exceptions.
- Exception
- A synchronous unusual condition caused by the current instruction (fault, trap, abort), as opposed to an asynchronous IRQ.
- External fragmentation
- Free memory exists but is split into holes too small for the next variable-size request.
- Fault
- A restartable exception. The textbook example is a page fault: the kernel repairs the mapping and retries the instruction.
- File descriptor
- Small integer index in a process's open-file table. Capability-like: the right to operate on that open file description.
- FIFO
- First-in first-out. As a scheduler it is FCFS. As a page-replacement policy it evicts the oldest loaded page and can Belady-anomaly.
- GID
- Group identifier on a UNIX-style process, used with UID for access checks and for group ownership of objects.
- Hard link
- A directory entry that names an existing inode. Same data, same inode number; last unlink (and last close) frees the file.
- Hold and wait
- Coffman condition: a process holds at least one resource while waiting for another.
- Hypervisor
- Software that virtualises a machine for guest OS instances. Type 1 runs on hardware; type 2 runs as a host process.
- Inode
- On-disk (and in-core) index node: owner, mode, timestamps, size, and the map from file offsets to blocks. Names live in directories.
- Internal fragmentation
- Wasted slack inside an allocated block or page because the allocator handed out more than the request needed.
- Interrupt
- Asynchronous signal from a device or timer that vectors into the kernel independently of the current instruction.
- Inverted page table
- One entry per physical frame, recording which (process, virtual page) owns it. Saves table space; lookup is a search, not an index.
- IPC
- Inter-process communication: pipes, message queues, shared memory, signals, sockets, and platform RPCs such as Binder on Android.
- Journaling
- Write-ahead logging of filesystem changes so a crash can replay committed metadata (and optionally data) to a consistent state.
- Kernel mode
- Privileged CPU mode in which the OS may program devices, change page tables and halt. Entered only through controlled vectors.
- Livelock
- Processes keep changing state in response to each other but never make useful progress (the polite-doorway problem).
- Logical address
- The address the program issues (virtual address). The MMU translates it to a physical address or a fault.
- LRU
- Least recently used page replacement. A good stand-in for OPT; exact LRU on every reference is expensive, so hardware uses a referenced bit and Clock.
- MLFQ
- Multilevel feedback queue: several priority queues; jobs that burn their quantum are demoted so interactive work stays responsive.
- mmap
- Syscall that maps a file or anonymous pages into the address space. Accesses fault pages in; shared maps are also IPC.
- Monitor
- Language-level construct: encapsulated data plus an implicit mutex and condition variables (Java
synchronizedis the usual example). - Mutex
- Mutual-exclusion lock with ownership. Lock/unlock a critical section. Contrast a semaphore, which has no owner.
- Mutual exclusion
- At most one thread is in the critical section (or holds the exclusive resource) at a time. Also a Coffman condition.
- Namespace
- Linux isolation of a name table (PID, mount, network, …) so a process tree sees its own view. A container building block.
- Orphan
- A live process whose parent has exited. The OS reparents it (classically to
init) so someone can wait and reap it. - Page fault
- The CPU trapped on a translation: not present, permission, or bad address. Legal not-present faults implement demand paging and COW.
- Page table
- Kernel-owned map from virtual page numbers to physical frames and permissions, usually a multi-level radix tree.
- PCB
- Process control block: the kernel's record of a process (or task) — identity, state, memory, fds, credentials, scheduling data.
- Physical address
- An address on the memory bus, in a RAM frame (or device memory). Programs do not use these under virtual memory.
- Pipe
- Unidirectional kernel byte-stream buffer. Anonymous pipes are typically parent-child; a FIFO is a named pipe in the filesystem.
- Priority inheritance
- A lock holder temporarily runs at the priority of its highest waiter so a medium-priority job cannot invert the high waiter.
- Priority inversion
- A high-priority thread waits on a lock held by a low-priority thread that is itself preempted by a medium-priority thread.
- Process
- A program in execution: address space, credentials, file table, and one or more threads. The usual isolation boundary.
- Progress
- Critical-section rule: if the section is free and someone wants in, the choice of who enters cannot be delayed by uninterested threads.
- Quantum
- Time slice in round robin (or a level of MLFQ). Too small wastes time on switches; too large behaves like FCFS.
- Race condition
- Correctness depends on the interleaving of concurrent operations. A lost update on a shared counter is the canonical example.
- Ready queue
- The scheduler's collection of runnable threads waiting for a CPU. Waiting time is time spent here, not time spent blocked on I/O.
- Round robin
- Preemptive scheduling: each ready job runs for a quantum, then is placed at the tail of the ready queue.
- Safe state
- A resource-allocation state for which there exists an order of processes that can all finish with the remaining resources (Banker's).
- Segmentation
- Variable-size logical regions (code, stack, heap). Matches program structure; reintroduces external fragmentation if used as the allocator.
- Semaphore
- Integer plus a wait queue: P/wait decrements and may sleep; V/signal increments and may wake. Binary or counting; no owner.
- Signal
- Asynchronous notification delivered to a process or thread. Tiny payload; not a data plane. Handlers may call only async-signal-safe functions.
- Socket
- Bidirectional endpoint. Internet sockets span hosts; UNIX-domain sockets span processes on one host with a filesystem or abstract name.
- Starvation
- A thread waits unbounded while others proceed. Priority without aging and some readers-writers policies are typical causes.
- Symbolic link
- A small file storing a path. It may dangle, may cross filesystems, and is resolved at use time.
- System call
- The stable, numbered kernel API. Userspace traps; the kernel validates arguments and performs the privileged work.
- Thrashing
- The system spends its time paging rather than running. Working sets do not fit in RAM; CPU utilisation falls while the disk is busy.
- Thread
- A schedulable execution context: registers, stack, TLS, inside a process's address space.
- TLB
- Translation lookaside buffer: a small cache of virtual-to-physical translations. Hit rate dominates effective memory access time.
- TOCTOU
- Time-of-check to time-of-use: a race between validating a path (or other name) and operating on it. Bind check and use to one kernel object (an fd).
- Trap
- A deliberate synchronous transfer to the kernel (syscall, breakpoint), as opposed to an unexpected fault or an asynchronous IRQ.
- UID
- User identifier on a UNIX-style process. Compared with object ownership and mode bits (or ACLs) on each privileged operation.
- User mode
- Unprivileged CPU mode in which applications run. Privileged instructions and direct device access are forbidden.
- VFS
- Virtual filesystem: the kernel's uniform layer so
open/readwork the same for disk filesystems, pipes, sockets and pseudo-filesystems. - Virtual memory
- Demand-paged, isolated address spaces larger than RAM, backed by frames plus a backing store, translated by the MMU.
- Wait-for graph
- Graph of processes waiting on processes. A cycle implies deadlock when each contended resource has a single instance.
- Working set
- The pages a process referenced in the recent window Δ. If working sets do not fit, the system thrashes.
- Zombie
- A process that has exited but whose parent has not yet waited. The address space is gone; the PID and exit status remain until reap.
Interview questions
Fundamentals
What is an operating system, and what three jobs does it actually do?
An OS is the privileged program that owns the machine. It multiplexes CPU, memory and devices among programs, isolates those programs from each other and from raw hardware, and abstracts devices behind a stable interface (files, processes, sockets, virtual memory). Interviews that ask "is it just a library?" want this: a library cannot enforce isolation because it does not control the mode bit or the page tables.
What is the difference between kernel mode and user mode?
The CPU has a privilege bit (or exception level). In user mode the running code cannot execute privileged instructions: no device I/O, no page-table writes, no halt, no flip of the mode bit. In kernel mode those instructions are legal. The hardware switches mode on a syscall, interrupt or exception and switches back on return-from-exception. User programs include shells, apps and most daemons; only the kernel (and a hypervisor at a higher level) is trusted with the MMU and devices.
What are privilege rings?
A hardware lattice of privilege. The textbook x86 picture has rings 0 (most privileged) through 3. Mainstream kernels use two: ring 0 / EL1 for the kernel and ring 3 / EL0 for user. Extra levels exist for a hypervisor (EL2) and firmware (EL3). The interview point is not the trivia of every ARM level; it is that isolation is hardware-enforced. If any program could raise its own privilege, a wild pointer could rewrite page tables or program DMA into another process.
What is a system call?
A numbered, documented request into the kernel: open, read, mmap, fork, clone, and so on. Userspace (usually libc) loads a number and arguments into ABI registers and executes a trap instruction (syscall, svc). Hardware raises privilege and vectors to a dispatcher. The kernel validates arguments — especially user pointers — does the work or sleeps, and returns a result. It is not a normal C function call: the stack, privilege and trust boundary all change. See C for the libc stub and Linux kernel & BSP for the entry path.
What is dual-mode operation, and why do we need it?
Dual mode is the hardware rule that user code cannot perform privileged operations. Without it, isolation is a social convention: any process could disable interrupts, remap memory or write another process's pages. With it, the only crossings are well-defined vectors the kernel handles. That is why "run the whole OS in user mode on paper" is a research idea that still needs some trusted TCB, not a reason to drop the mode bit on a phone.
Interrupt vs trap vs exception — what is the difference?
An interrupt (IRQ) is asynchronous: a device or timer fires independently of the current instruction. A trap is synchronous and deliberate (syscall, breakpoint). An exception is synchronous but not "please do this POSIX service" — the instruction cannot complete as issued (page fault, divide-by-zero, illegal opcode). In interview English, sort by who caused it and whether you will retry the instruction.
What is a fault versus an abort?
Both are exceptions. A fault is typically restartable: the kernel repairs the condition and retries the same instruction. A page fault is the important case — virtual memory is built on it. An abort is not restartable (serious hardware error, some double faults); the process or the machine is done. If you say "page fault kills the process", you have mixed a protection violation (bad address) with a not-present fault on a legal mapping.
What is a process?
A program in execution: a private virtual address space, credentials, a file-descriptor table, signal state, resource limits, and one or more threads. It is the usual isolation boundary. The kernel's record of that bundle is the PCB. Two processes do not share memory unless they both map the same pages (shared memory, shared file maps) or they are still sharing COW pages after fork.
What is a PCB, and what does it contain?
The process control block is the kernel object describing a process or task. Interview contents: PID and parent, state, CPU context (often per-thread), pointer to page tables / mm, file table, credentials, scheduling parameters, accounting, and pending signals. On Linux the living form is task_struct plus satellite structs — that split is kernel material. Here: the PCB is what you context-switch and what ps is displaying.
What are the process states and the main transitions?
Classic five: new (being created), ready (runnable, waiting for a CPU), running (on a CPU), waiting/blocked (cannot run until an event), terminated (exited; may still be a zombie). Dispatch: ready → running. Preempt: running → ready. Block (I/O, lock, wait): running → waiting. Wake: waiting → ready. Exit: running → terminated. Some books add swapped-out suspended states; mention them only if asked.
What is a context switch, and why is it expensive?
The kernel saves one thread's registers and kernel stack pointer and loads another's. If the next thread is in a different address space, it also switches the page-table root, which flushes or retags the TLB. Direct cost is thousands of cycles. Indirect cost is often larger: the next job does not share the previous job's hot cache lines. Same-process thread switches skip the mm switch and are cheaper. That is why a huge number of threads with a tiny RR quantum destroys throughput.
Explain the POSIX fork / exec / wait / exit model.
fork creates a child that is a copy of the parent (modern kernels: COW pages). Child sees return value 0; parent sees the child's PID. exec replaces the current address space with a new program; the PID and most fds remain. exit releases memory and files and leaves an exit status. wait/waitpid lets the parent collect that status and free the zombie PCB. The usual "run another program" path is fork then exec; Android's zygote is fork without exec for a pre-warmed runtime — see Android frameworks.
What is the difference between a zombie and an orphan?
A zombie has already exited; the parent has not waited. Almost no memory remains — only a process-table slot so the exit code can be read. Ignore children and you leak PIDs, not RAM. An orphan is still running after its parent died. The OS reparents it (classically to PID 1) so someone will wait. An orphan is not a zombie; it may become one later, and the new parent will reap it.
Process vs thread — what is shared, and what is not?
Threads in one process share the address space (code, heap, globals), file descriptors, and credentials. They do not share stacks or, by default, TLS values. Separate processes share nothing unless they set up IPC or inherited COW/mmap mappings. Threads are cheaper to create and switch and pass data by pointer; processes isolate faults and security domains. Browsers and sandboxed renderers pick processes on purpose.
What is a thread?
The unit the scheduler runs: a register set, a stack, TLS, and a scheduling state, living inside a process. "The process runs" is loose language — a thread runs on a core. A single-threaded process is just the special case of one thread. Java threads and the JVM memory model are on the Java page; here the OS fact is that a thread is a kernel-schedulable (1:1) or library-schedulable (N:1) execution context.
User-level threads vs kernel-level threads?
User-level threads are scheduled by a library; the kernel sees one task. Switch is a function call (fast), but one blocking syscall or page fault blocks all of them, and you cannot use more than one core. Kernel-level threads are first-class tasks; blocking and SMP work. Modern POSIX pthreads on Linux are 1:1 kernel threads. Language runtimes that multiplex green threads are an M:N or N:1 story on top.
What is thread-local storage (TLS)?
Storage that looks like a global but has one instance per thread: errno, a thread id, a scratch buffer. The ABI keeps a register (FS/GS, TPIDR) pointing at a per-thread block; TLS symbols are offsets from that pointer. That is why errno is thread-safe without a lock. It is not a substitute for a mutex on heap data you actually share.
What is a CPU burst vs an I/O burst?
A CPU burst is a stretch of execution that needs the processor. An I/O burst is a stretch waiting for a device (or a lock, or a sleep). Interactive jobs have short CPU bursts and frequent I/O; batch jobs have long CPU bursts. Schedulers estimate or observe the next CPU burst. Mixing them under FCFS is how you get the convoy effect.
Preemptive vs non-preemptive scheduling?
Non-preemptive (cooperative): a thread runs until it blocks, yields or exits. A runaway loop starves the machine. Preemptive: a timer tick or a higher-priority wake-up can seize the CPU. General-purpose OS schedulers are preemptive. Textbook numericals still use non-preemptive SJF or FCFS; say which you assumed when you draw the Gantt chart.
What is FCFS, and what is the convoy effect?
FCFS (first-come first-served) runs ready jobs in arrival order until each CPU burst finishes. The convoy effect is a long CPU-bound job at the head; short I/O-bound jobs wait, run for a moment, block, then queue behind the long job again. Average waiting time explodes. RR, SJF and MLFQ exist in part to break convoys.
How does round-robin scheduling work, and how do you choose the quantum?
Each ready job gets a time quantum q, then goes to the tail of the ready queue. Response time is good for interactive work. If q is tiny, context-switch overhead dominates (CPU spends its life saving registers). If q is huge, RR collapses to FCFS. Pick q large compared to a switch (milliseconds, not microseconds) but small compared to a human-noticeable delay. There is no universal number; state the trade-off.
Define waiting time, turnaround time and response time.
Turnaround = completion time − arrival time (how long the job was in the system). Waiting time = turnaround − CPU burst time (time in the ready queue, not I/O wait). Response time = first time on the CPU − arrival (how long until it first "twitches"). Quote averages over a set of jobs. Throughput is completed jobs per unit time; utilisation is the fraction of time the CPU is not idle.
What is a race condition?
A situation where correctness depends on the interleaving of concurrent operations. The lost-update: two threads read the same counter, both add one, both write, and one increment vanishes. Shared memory plus no synchronisation is sufficient. Data races are undefined behaviour in C and C++ (C, C++); in OS language, they are the reason critical sections exist.
What is a critical section?
The region of code that must appear atomic with respect to other threads that share the same data. Entry and exit protocols (a mutex, an atomic algorithm) implement that atomicity. The section should be as short as you can make it: hold time is contention and, if you spin, wasted CPU. I/O inside a critical section is a common design smell.
Mutex vs semaphore?
A mutex is an owned lock: the thread that locks it must unlock it; it is for protecting a critical section. A semaphore is a counter plus a wait queue: P/wait decrements and may sleep, V/signal increments and may wake. No owner — any thread may V. A binary semaphore can mimic a mutex but can also be used as a pure signal ("this event happened"). A counting semaphore tracks N identical resources. Prefer a mutex for exclusion; use a semaphore when the count or the cross-thread signal is the point.
Binary semaphore vs counting semaphore?
Binary: the count is 0 or 1. Counting: the count is a non-negative integer. Use counting for N buffers, N devices, or the empty/full counts in producer-consumer. Use binary for a single-event signal or a crude mutex. Neither has ownership; if you need "only the locker may unlock" and priority inheritance, you want a mutex.
What is deadlock, and what are the four Coffman conditions?
Deadlock is a set of processes such that each is waiting for a resource another in the set holds, and none will release what they hold. The four conditions, all required: mutual exclusion (the resource is not shared), hold and wait (hold one, wait for another), no preemption (you cannot yank the held resource), circular wait (a cycle). Break any one and deadlock cannot occur. That is prevention. Avoidance (Banker's) and detection-plus-recovery are different policies.
Logical address vs physical address?
A logical (virtual) address is what the program issues. A physical address is a location in RAM (or device memory) on the bus. The MMU, using kernel-installed page tables, translates one to the other or raises a fault. Relocation and isolation both come from this: two processes can both use address 0x400000 and not collide. Without an MMU you bind at compile or load time and you do not get this isolation.
Internal vs external fragmentation?
External: free memory exists but is carved into holes too small for the next variable-size request (classic segments, a naive heap). Fix by compaction or by allocating only fixed frames (paging). Internal: you received a block larger than you needed; slack inside the page or slab is wasted. Paging has internal fragmentation on the last page of a mapping. You tolerate it; you do not compact it away.
Paging vs segmentation?
Paging splits memory into fixed-size pages and frames. The process sees a flat virtual array. No external fragmentation of RAM; some internal slack; the table can be huge unless it is hierarchical. Segmentation uses variable-size logical regions that match how programmers think (code, stack, heap). External fragmentation returns. Modern OS: paging is the mechanism; "segments" are just named mappings (VMAs) on top of pages.
What is a page fault?
The CPU failed a translation: no valid PTE, or a permission violation, or a reserved bit. Hardware saves the faulting address and traps to the kernel. If the address is illegal, the process is signalled (UNIX: SIGSEGV). If the mapping is legal but not present, the kernel allocates a frame, fills it (zero, file, swap, COW copy), installs the PTE, and restarts the instruction. That restartable path is demand paging.
What is virtual memory?
Each process has a large, isolated logical address space. Only the pages it uses need RAM frames; the rest live on a backing store (the file, swap) or do not exist yet. Demand paging, the TLB, replacement, COW and mmap are the machinery. It is not "infinite RAM": thrashing and out-of-memory exist. On Android, the LMK may kill a process under pressure rather than swap forever — frameworks.
What is a file, an inode, and a directory?
A file is a persistent byte stream plus metadata. An inode is the identity: owner, mode, timestamps, size, and the map from offsets to disk blocks. A directory is a listing that maps names to inode numbers (or equivalent). The name is not stored in the inode. That split is why two names can refer to one file (hard links) and why renaming is a directory operation, not a rewrite of the data.
Hard link vs symbolic link?
A hard link is another directory entry for the same inode: same data, same inode number, cannot cross filesystems, usually forbidden for directories. The inode's link count drops on unlink; data is freed when the count is 0 and no fd remains. A symbolic link is a small file holding a path. It may dangle, may cross filesystems, and is resolved when you use it. Permissions on the target usually decide access.
What is IPC, and when would you use a pipe versus a socket?
IPC is how separate address spaces exchange bytes or events. An anonymous pipe is a unidirectional kernel buffer, typically set up by a parent and inherited across fork — shell pipelines. A socket is bidirectional, has a richer API (connect, listen, datagram vs stream), and can leave the machine. Use a UNIX-domain socket when two local processes need a bidirectional channel or credentials passing; use a pipe for a simple one-way inherited stream. Shared memory wins for bulk data if you will write the synchronisation yourself.
What do UID and GID do, and how is a process different from a VM or a container?
UID/GID are the credentials the kernel stamps on a process and compares to object ownership and mode bits (or ACLs). A process is an address space plus those credentials. A container is still processes on the same kernel, with extra namespace views and cgroup limits. A VM virtualises hardware and usually runs a guest kernel. Isolation strength and density go in opposite directions: process < container < VM for isolation; the reverse for density and start time.
Going deeper
Walk a system call from userspace into the kernel and back.
Libc places the syscall number and arguments in the architecture's ABI registers and executes a trap instruction. Hardware saves the user program counter and status, raises privilege, and vectors to the kernel entry. The kernel saves user registers, looks up the number, copies and checks user pointers (never trust a raw address), performs the work or sleeps, writes the return value, then return-from-exception drops privilege and resumes userspace. If the thread slept, another thread ran in between — that is the scheduler, not part of the stub. Kernel entry details: Linux kernel & BSP.
Why not run everything in kernel mode and skip syscalls?
Then every bug is a kernel bug. A wild pointer can rewrite page tables, program a DMA engine into another process, or disable the timer that preempts runaways. Dual mode plus a narrow syscall ABI is how we keep a browser tab from owning the machine. Microkernels move more policy to user servers, but they still have a privileged core that owns the MMU. The interview answer is isolation and a stable ABI, not "syscalls are fast".
Why is a process context switch more expensive than a thread switch in the same process?
A same-process thread switch saves/restores registers and kernel stacks but keeps the same page-table root. A process switch also switches the address space: load a new CR3/TTBR, flush or retag the TLB, and lose cache warmth for the previous working set. File tables and credentials may be switched too. That is why "just use more processes" is the wrong default for fine-grain parallelism, and why browsers accept the cost only when they want a crash domain.
What happens if a parent never calls wait() on its children?
Each child that exits becomes a zombie: the address space is gone, but the PCB slot and exit status remain. Enough of them exhaust the PID space; fork starts failing. Memory does not leak in the "child heap still allocated" sense. The fix is wait/waitpid (or a SIGCHLD handler that reaps), or to have the parent die so the children are reparented and reaped by init / a subreaper.
Who reaps an orphan, and why does that matter?
When the parent exits first, the OS reparents the live child. Classically the new parent is PID 1 (init), which loops on wait and collects zombies. Linux can reparent to a designated subreaper instead. The guarantee is: someone will wait, so orphans do not become permanent zombies. Service managers that spawn workers rely on this if a worker outlives its launcher.
Compare N:1, 1:1 and M:N threading.
N:1 multiplexes many user threads onto one kernel thread: cheapest switch, no parallelism, one blocking syscall freezes all. 1:1 maps each user thread to a kernel thread: blocking and SMP work; create/switch go through the kernel (still cheap). This is POSIX on modern Linux and the usual Android/ART native thread. M:N parks M green threads on N kernel workers: can hide I/O and use several cores, but you now have two schedulers, messy signals, and hard stop-the-world cases. Interviewers want the trade-off, not a brand name.
When do you choose threads over processes, and when the reverse?
Threads: shared working set, cheap hand-off of pointers, need many concurrent I/O or parallel CPU workers inside one program (a server, a UI plus workers). Processes: fault isolation (a renderer crash must not take the browser chrome), different credentials, a sandbox, or a blast radius you can kill. The cost of IPC and of a heavier create/switch is the price of that isolation. See system design for multi-process services and frameworks for app process policy.
What goes wrong in an N:1 threading package when one thread makes a blocking syscall?
The kernel deschedules the one task it sees. Every user thread in that process stops, including ones that were ready to run. The library cannot context-switch them because it is not running. Page faults have the same effect. That is the historic reason N:1 died for general-purpose servers. Workarounds (non-blocking I/O plus a user scheduler) reinvent M:N.
SJF vs SRTF?
Both pick the job with the shortest next CPU burst. SJF is non-preemptive: once a burst starts, it finishes. SRTF is preemptive: a newly arrived shorter remaining burst seizes the CPU. Both minimise average waiting time if you know the future. You do not; you estimate (exponential average of past bursts, or "interactive vs batch" feedback). Long jobs can starve; aging or a lower bound on share is the patch. These are textbook policies, not Linux CFS.
How does priority scheduling starve, and what does aging do?
If a high-priority stream is always ready, a low-priority job never runs. That is starvation, not deadlock (the system is making progress). Aging increases priority as wait time grows, so the low job eventually outranks the chatter and gets a turn. Mention aging whenever you mention priority, SRTF, or an MLFQ without boosts.
Multilevel queue vs multilevel feedback queue (MLFQ)?
A multilevel queue has fixed classes (interactive, batch, RT). A job is assigned a queue and stays there; each queue may have its own policy and a priority between queues. Misclassification is permanent. MLFQ lets jobs move: burn your quantum and you are demoted (you look CPU-bound); block early and you stay high (you look interactive). You need aging or periodic boosts so a long job is not buried forever. MLFQ is the "learn the workload" textbook design behind a lot of interactive OS folklore.
How do you compute waiting and turnaround time from a Gantt chart?
Write arrival times and burst lengths. Draw the chart under the policy (watch preemption points). For each job: completion is the time its last slice ends; turnaround = completion − arrival; waiting = turnaround − burst (only CPU burst; do not subtract I/O unless the problem included I/O in the "burst" column). Average the columns. Response is the start of the first slice minus arrival. State whether the policy is preemptive. A common arithmetic bug is using finish − burst and forgetting a late arrival.
What three properties must a critical-section solution satisfy?
Mutual exclusion: at most one thread in the section. Progress: if the section is free and someone wants in, selection cannot be postponed by threads that are not interested. Bounded waiting: a bound on how many times others may enter after you have requested entry. Peterson's algorithm is the software classic; hardware atomics plus an OS sleep queue are what you ship. A spin-forever lock can fail bounded waiting if a thread is never scheduled.
Solve producer-consumer (bounded buffer) with semaphores.
You need exclusion on the buffer and "do not write when full" / "do not read when empty". One mutex (or a binary semaphore used as a mutex) protects put/get. A counting semaphore empty starts at N; a producer P(empty) before filling and V(full) after. A consumer P(full) before taking and V(empty) after. Invert the P order (mutex first, then empty/full) and you can deadlock: you hold the mutex while sleeping for a slot. Always take the "resource count" semaphores outside the mutex, or use a mutex plus two condition variables and wait in a loop on the predicate.
What is the readers-writers problem, and what can starve?
Many readers may hold the data at once; a writer needs exclusive access. Readers-preference: a waiting writer can starve if readers keep arriving. Writers-preference: readers can starve. A fair queue (one waiter line, or a ticket) avoids both at the cost of extra state. Always state which policy you chose. Implementation is usually a mutex, a reader count, and a write lock (or two condition variables).
What is a monitor, and how does a condition variable work?
A monitor is a language construct: data plus operations that run with implicit mutual exclusion, plus condition variables. Java synchronized methods are the interview example (Java). A condition variable is not a lock. wait atomically releases the mutex and sleeps; signal/broadcast wakes waiter(s). Under Mesa semantics (the one you will ship), a wake is a hint: re-test the predicate in a loop. Hoare semantics hands the mutex to the waiter immediately; textbooks mention it, kernels and pthreads do not implement it that way.
When do you spin, and when do you sleep on a lock?
Spin when the critical section is a handful of instructions and the holder is running on another core (so the flag will clear soon) and you are not allowed to sleep (hard IRQ context on Linux — kernel). Sleep (mutex) when the hold may include I/O or a long computation, or you are on a uniprocessor (spinning waits for a holder that cannot run until you deschedule). Adaptive mutexes spin a little, then sleep. Priority inversion is worse with a spinlock held while a low-priority holder is preempted.
Deadlock prevention vs avoidance vs detection and recovery?
Prevention designs out a Coffman condition: global lock order (breaks circular wait), allocate everything up front (breaks hold-and-wait), or make the resource preemptive. Avoidance (Banker's) checks each grant against a safe-state test; needs declared maxima; rare for mutexes, common as a numerical. Detection builds a wait-for graph or runs a reduction, then kills or rolls back a victim. Many desktops are "ostrich": ignore and hope. Databases detect; lock-heavy servers prevent with ordering.
Explain Banker's algorithm in one interview paragraph.
Each process declares a maximum claim per resource type. The allocator tracks Allocation, Need = Max − Allocation, and Available. When a process requests units, pretend you granted them. Then search for a safe sequence: an order of processes that can each finish with Available + what they would free. If no such order exists, refuse (the waiter sleeps) even if the units are free right now. You stay in a safe state. Real lock libraries do not run this; they use lock ranking. Be ready to walk a 3×3 matrix if they put one on the board. Searching for a sequence is a graph/greedy exercise; see DSA if you want the loop structure.
Deadlock vs livelock vs starvation?
Deadlock: a cycle of waiting; no one in the set changes state in a useful way. Livelock: everyone keeps reacting (retry, backoff together, step aside the same way) but nobody makes progress. Starvation: the system progresses, but one participant never gets a turn. Fixes differ: break a Coffman condition or recover; add randomness/exponential backoff; age priorities or fair-queue. Do not use the three words as synonyms.
What is address binding, and when does relocation happen?
Compile-time binding assumes a known physical address (no MMU, or a fixed embedded map). Load-time binding patches the binary when it is loaded. Execution-time binding (virtual memory) translates every access; the program never sees physical addresses. Relocation is why the same ELF can run in many processes at once. PIC/PIE is the userspace cousin (see C); the OS story is the MMU.
What is compaction, and why did paging reduce the need for it?
Compaction slides allocated regions together so free holes become one region. It requires the ability to move a live object and fix every pointer or to rewrite translations. With variable-size physical partitions, you compact or you fail to place the next job. Paging allocates fixed frames: any free frame will do, so external fragmentation of RAM disappears (internal slack remains). Userspace heaps still fragment; that is a malloc problem, not an OS placement problem.
Why do we use multi-level page tables instead of one giant flat table?
A 64-bit space with 4 KiB pages has an absurd number of virtual pages. A flat array of PTEs would be huge even if the process uses a few megabytes. A radix tree (3–4 levels) stores only the branches you populated: unused quadrants cost a null upper pointer, not a forest of empty leaves. The cost is L dependent memory loads on a TLB miss. That cost is why the TLB — and huge pages that shorten the walk — exist.
What is the TLB, and how do you write the effective access time formula?
The TLB caches recent VPN → PFN translations (and permissions). Hit: about tTLB + tmem for the data. Miss: tTLB + L memory references for the walk + tmem for the data. EAT = h(tTLB + tmem) + (1−h)(tTLB + (L+1)tmem). A hit-rate drop from 99% to 90% is painful because misses are so expensive. A process switch that flushes an untagged TLB resets h. If they give numbers, plug them in and keep L explicit.
Walk a page-fault path at interview level.
Hardware: TLB miss, walker finds invalid PTE or no PTE, fault with the VA and the PC. Kernel: is this a bad address, a permission error, or a legal not-present mapping? Bad → signal/kill. Legal: find or allocate a frame. File-backed: schedule disk I/O and sleep (major fault). Anonymous: zero a frame. COW write: copy the page. Install a valid PTE, update TLB, return so the instruction retries. Minor faults (page already in cache, just not mapped here) skip the disk. Linux VMA details: kernel.
What is demand paging?
Do not load a program's pages at exec time. Map the address space as not-present (or file-backed holes) and fill a frame on the first touch. Startup is faster and unused code/data never occupies RAM. The downsides are a fault storm at first use (working-set warm-up) and the need for a replacement policy when frames run out. Prefetch / readahead is a policy on top, not a different mechanism.
Compare FIFO, OPT, LRU and Clock page replacement.
FIFO evicts the oldest loaded page: simple, can evict a hot page, Belady-anomaly. OPT evicts the page whose next use is farthest in the future: unrealisable, the lower bound you compare against. LRU evicts the least recently used: good approximation to OPT; exact LRU needs a stack or timestamps on every reference. Clock / second chance walks a ring; if the referenced bit is set, clear it and skip (a second chance). Hardware gives you the bit; Clock is the LRU you actually implement.
What is Belady's anomaly?
For some reference strings, FIFO produces more page faults with more frames. More memory is not always fewer faults under FIFO. OPT and stack algorithms (LRU) do not have this anomaly: a stack algorithm's resident set of size n is nested, so growing n only adds pages. If they give a string, simulate FIFO at two frame counts and show the fault counts. Mention it whenever you mention FIFO replacement.
What is copy-on-write, and where do you meet it?
Share a physical page among several mappings, marked read-only. The first store faults; the kernel copies the page, assigns a private writable frame, and resumes. fork uses this so the child is logically a full copy without a full copy. exec soon after means almost no copies happen. MAP_PRIVATE mmap of a file is the same idea: reads share the page cache; writes break off a private page. Snapshots and some VM balloon tricks use the same primitive.
When is mmap a better I/O API than read/write?
mmap maps the file into the address space. You can parse in place (no extra userspace copy) and let the page cache be the buffer. Random access is pointer arithmetic. Costs: first-touch faults (latency jitter), harder error reporting (a fault vs a return code), and you must still msync if you need durability. read is simpler, copies into a buffer you control, and reports errors on the call. Large read-mostly files and shared mappings as IPC are the usual mmap wins. Implementation: kernel.
Programmed I/O vs interrupt-driven I/O vs DMA?
PIO: the CPU moves every word and often polls a status register — simple, burns a core, fine for tiny or boot-time transfers. Interrupt-driven: the CPU does other work; an IRQ says "a byte/packet is ready"; the handler or a deferred path still copies. DMA: the CPU programs a controller (addresses, length); the device is bus master; an IRQ means done or error. DMA still needs cache maintenance and an interrupt. Do not start a Linux probe lecture.
Blocking vs non-blocking vs asynchronous I/O?
These describe how the thread waits, not how bytes move (DMA can back any of them). Blocking: the syscall sleeps until it can complete (or a signal hits). Non-blocking: return data or EAGAIN now; you poll or use select/epoll. Asynchronous: submit a request and be notified later (completion queue, callback) without sitting in the syscall. One thread can overlap compute and I/O with async; with blocking you use more threads. Do not call epoll "async I/O" unless you are precise: it is readiness, then you still read.
Contiguous vs linked vs indexed file allocation?
Contiguous: start + length. Great sequential I/O; external fragmentation; growth is painful. Linked: each block points to the next (or a FAT does). Easy growth; random access is a chain walk; one bad pointer loses the tail. Indexed: an index block (or a tree, or extents) lists data blocks. Random access is good; small files pay index overhead if you are naive. Real filesystems mix ideas: extents are "indexed, but runs of contiguous blocks".
Sketch FAT versus a UNIX inode.
FAT keeps a table with one entry per cluster: 0 means free, EOF ends a file, a number is the next cluster. Directories store the first cluster. Random access walks the table. The table is a single point of corruption. A UNIX inode stores a few direct block pointers and then single/double/triple indirect blocks, plus metadata (owner, mode, times, link count). Names live in directory entries that point at the inode. Extents replace many pointers with (start, length) runs on modern layouts.
What problem does journaling solve, and what does it not solve?
A crash mid-update can leave bitmaps, trees and directories disagreeing. A journal writes intended changes to a log, commits, then writes home locations. After a crash, replay committed records. That restores crash consistency of metadata (and of data if you journal data too). It does not make every write durable the instant it returns — that is fsync / barriers. It does not fix a disk that lies about flush. Soft updates and copy-on-write filesystems are other consistency designs; mention them as alternatives, not as Linux driver trivia.
What is the VFS idea?
The kernel presents one inode / open-file / directory API so read does not care whether the backing store is a disk filesystem, a pipe, a socket, or a pseudo-fs. Each implementation fills in operations (read, write, lookup). Userspace sees fds. That is why you can cat a file and also read from a pipe with the same syscall. Linux dentry/inode objects are kernel material; the interview word is "one surface, many implementations".
How do you choose among pipe, message queue, shared memory, signal and socket?
Pipe: inherited one-way stream (parent-child, shell). Named pipe: same stream, unrelated processes, still local. Message queue: record boundaries and maybe priorities; multiple consumers selecting by type. Shared memory: bulk and latency; you must add a mutex/futex protocol and think about visibility. Signal: "please die", job control, a timer — not a payload path. Socket: bidirectional, credentials, and a path to the network if you ever need it. On Android, app-to-system RPC is Binder, not POSIX queues — frameworks.
ACL vs capability — what is the difference?
An ACL lives on the object: "these principals may do these ops". Easy to audit one file; hard to list everything a process can still do. Revocation is "edit the list". A capability is a token the subject holds; possession is authorisation. UNIX fds are capability-like for open files (passing an fd delegates). POSIX process capabilities are a different thing: they split root into bits. Say which you mean. Isolation still starts with page tables; ACLs and capabilities decide authorised crossings.
What is a TOCTOU bug, and how do you close it?
Time-of-check to time-of-use: you stat a path, decide it is safe, then open the same path. An attacker swaps the name for a symlink in the gap. The check did not bind to the same kernel object as the use. Fixes: open then fstat on the fd; openat with a directory fd; O_CREAT|O_EXCL for exclusive create; never follow user-controlled paths from a privileged process if you can take a handle instead. The pattern is general, not only files.
Advanced
A mutex is more than exclusion. What is memory visibility, at OS-interview level?
Cores have store buffers and private caches. Thread A may execute data = 42; ready = 1 and thread B may observe ready == 1 while data is still stale, unless a happens-before edge orders those writes. Unlock/lock on a mutex (or a matching release/acquire atomic pair) is that edge: the locker publishes, the acquirer observes. volatile in C is not a lock and not a portable barrier (C). The OS contributes the sleep/wake side (a futex wait sees the same write that the waker published). Interview sentence: "locks give exclusion and publication."
Why do kernels spin in some paths and userspace usually sleep?
Userspace hold times are unpredictable (you might call into a library that pages or does I/O). Sleeping is the default. The kernel has paths that must not sleep (interrupt handlers, code that already holds a spinlock on Linux) and paths whose critical sections are a few instructions with the holder running on another CPU. There, spinning is correct and cheaper than a full deschedule. Mixing the two — spinning in userspace on a lock whose holder is preempted — wastes a core and can invert priorities. Hybrid locks exist because the first dozen cycles of a short hold look like a spin win.
Explain priority inversion and priority inheritance. Where does this show up on a phone?
Low-priority L holds a lock. High-priority H blocks on that lock. Medium-priority M runs and preempts L. H is now stuck behind M even though H outranks M. That is inversion. Inheritance lets L run at H's priority until it unlocks, so M cannot sneak in. Ceiling boosts anyone who takes the lock to a predeclared ceiling. On phones this is an audio/camera/RT story: a SCHED_NORMAL thread holding a lock that an RT or audio thread needs produces glitches. Android and real-time Linux use inheritance (including PI futexes) so the UI or audio path does not wait on a background job. Keep it light unless they are hiring for RT; power interaction is power & thermal.
What is an inverted page table, and why is it not the default story?
Instead of one tree per process indexed by VPN, keep one entry per physical frame: "this frame is (pid, vpn)" plus a hash to find a VPN quickly. Table size tracks RAM, not virtual space — attractive on huge address spaces. The downsides are: translate-this-VA becomes a search, shared pages need extra care, and the radix-tree plus TLB design is already good enough on the machines you will discuss. Mention it as an alternative organisation, not as "what Linux uses on ARM".
Define the working set and explain thrashing.
W(t, Δ) is the set of pages referenced in the last Δ time. If the sum of working sets exceeds RAM, almost every reference misses, the disk is saturated, and CPU utilisation falls (the CPU is waiting on paging). That is thrashing. Fixes: admit fewer processes, enlarge RAM, shrink the working set (leak, oversized cache), or a working-set / page-fault-frequency policy that suspends a job until there are frames. A trap: "the box is slow, add CPU" when the iowait and fault rates say otherwise.
How does the Clock (second-chance) algorithm use the referenced bit?
Frames sit in a ring with a clock hand. On a need for a victim, look at the page under the hand. If the referenced (access) bit is 0, evict. If it is 1, clear it and advance — that page got a second chance. A hot page keeps getting its bit set by hardware and is skipped. This approximates LRU without a timestamp on every load. A dirty bit can add a third chance (prefer clean victims to skip a writeback). If the hand spins too fast, you are already thrashing.
If OPT is unrealisable, why teach it?
OPT (MIN) is the offline lower bound: evict the page whose next use is farthest in the future. You cannot know the future in a general-purpose OS, but you can measure how close LRU or Clock came on a trace. If OPT still faults a lot, the working set does not fit — no clever replacement will save you. If OPT is fine and FIFO is terrible, the policy is the problem. That is why it appears in every numerical and in every "which algorithm is best" discussion.
What is a device driver, as an OS concept (not a Linux probe story)?
A translator: device registers, DMA descriptors and IRQs on one side; a uniform kernel interface on the other (block, character, network). The filesystem should not know the SSD brand; the scheduler should not know the NIC. Drivers are trusted because they run with kernel privilege (or in a userspace I/O server with a narrow DMA API). How Linux binds a compatible string and runs probe is Linux kernel & BSP. Here the point is the abstraction boundary and why a bug in a driver is a kernel bug.
How can a filesystem stay crash-consistent without a journal?
Several designs: fsck walks the tree after a crash and repairs (slow, last-century default). Soft updates order metadata writes so a crash leaves a reachable, slightly leaked state rather than a broken pointer. Copy-on-write / shadow paging (new tree, then an atomic root swap) never overwrites the old consistent tree. Journaling is the common middle path: a log plus home-location writes. Interviewers want you to name the problem (metadata graph torn by a crash) and at least two remedies, not a vendor's on-disk format.
Why are signals a poor IPC data path?
A signal is an asynchronous control event with almost no payload. It can interrupt a thread between any two instructions (except where masked). The handler may call only async-signal-safe functions; malloc, printf and most locks are unsafe. Re-entrancy and interrupted syscalls (EINTR) are the tax. Use signals for "terminate", "job control", "a timer fired". Use a pipe, socket or a dedicated thread waiting on a signalfd-style descriptor if you need to turn the event into ordinary control flow. See C for the safe-function rule.
Shared memory is the fastest IPC. Why do you still need synchronisation?
Two processes mapping the same pages are just threads with a more expensive create path: they can tear structs, lose updates, and see stale stores. You need a mutex, a process-shared futex, a semaphore, or a carefully ordered lock-free protocol — plus the same visibility story. You also need a way to agree that the mapping exists (a POSIX shm name, an inherited fd, a mapped file). Fast path is loads/stores; the hard part is the protocol. Binder exists on Android in part so most app code never writes that protocol — frameworks.
What is trap-and-emulate virtualization?
Run the guest in a deprivileged mode. Ordinary instructions execute natively. Privileged operations (load a page-table root, mask IRQs, issue I/O) trap to the hypervisor, which emulates the effect on virtual hardware and resumes the guest. This is the classic model. It requires that every sensitive instruction actually traps. Historic x86 had sensitive instructions that ran silently in user mode; that broke the model until hardware added a guest mode. Modern CPUs provide that mode; software VMMs still emulate devices.
When does trap-and-emulate fail, and what do hypervisors do about it?
It fails when a guest instruction is sensitive (it would reveal or change privileged state) but is not privileged (it does not trap). The hypervisor never gets control, so the guest sees the host's truth or changes state it must not. Fixes: rewrite those instructions (binary translation), or use hardware virtualization extensions that make them trap or that give the guest its own view. Interviewers want the definition of "sensitive but unprivileged", not a product name.
What do Linux namespaces and cgroups actually isolate?
Namespaces virtualise names: PID (you can be PID 1 in a box), mount (your own filesystem view), network (your own interfaces and addresses), UTS, IPC, user. cgroups limit and account resources: CPU, memory, I/O, freezer. Together they are the usual container recipe. They do not give you a second kernel or a second scheduler. A kernel bug is still a host bug. Implementation knobs: Linux kernel & BSP. The concept: names vs quotas vs hardware virtualization.
Walk read() from libc to disk and back. What OS topics fire?
Userspace: C read in libc (C); Java would sit above this via ART/JNI (Java). Trap into the kernel (dual mode). Current thread's PCB → fd table → open file (offset, vnode). VFS read: pipe/socket may block (IPC + scheduler). Regular file: page cache lookup. Hit: copy to the validated user buffer. Miss: allocate a frame, submit block I/O, state = waiting, another thread runs. DMA + IRQ complete; waiter goes ready; scheduler eventually runs it; copy out; return. Kernel path: Linux kernel & BSP. Do not continue into device tree.
What should you say about Linux CFS in an OS-concepts interview?
One paragraph, then stop. CFS (and its fair-class successors) approximate ideal fair sharing: each task has a virtual runtime; the task that has been cheated the most runs next, so over a window everyone gets a weight-adjusted share. It is not FCFS and not textbook RR. POSIX FIFO/RR real-time classes sit above the fair class. Energy-aware scheduling, cgroup cpu controllers and Android binder-thread pools are kernel / power material. If they want a Gantt chart, they want a textbook policy, not CFS.
Why is the address space the root isolation primitive?
Dual mode keeps user code from installing its own translations. Page tables keep process A from issuing loads that resolve to process B's frames (unless a shared mapping exists). Credentials and ACLs decide authorised kernel crossings (files, IPC). Namespaces and VMs add more views or another kernel. If the MMU story is broken, none of the higher policy matters: a write goes wherever the attacker maps it. That is why "isolation" answers should start with translations, not with a firewall.
DMA is off-CPU. What can still go wrong with caches and addresses?
The device writes RAM. The CPU may still hold a stale line in cache, or the CPU may write back over a just-arrived packet. You need coherent interconnects or explicit cache flush/invalidate around the buffer. The device also needs a physical (or IOMMU-translated) address, not the process's virtual address. Bounce buffers exist when the device cannot reach the memory. This is still conceptual: IOMMU and Linux DMA APIs live on Linux kernel & BSP.
Show how a virtual address becomes a physical address in a 3-level table.
Split the VA into unused high bits, three VPN indices, and a page offset. The page-table base register points at the L2 table. Index 1 selects an L1 table; index 2 selects a PTE; the PTE's PFN concatenated with the offset is the PA. Permissions live in the PTE (and sometimes in upper levels for entire regions). A TLB hit skips this. Draw the boxes; name "dependent loads". If they want Linux's 4-level/5-level names, send them to the kernel page.
What does POSIX say about fork() in a multithreaded process?
The child contains one thread: the one that called fork. Every other thread vanishes. If one of those threads held a mutex, the mutex stays locked in the child with no owner running to unlock it. The safe patterns are fork-then-exec (the new image has one thread and fresh locks) or to use pthread_atfork handlers that you actually understand. This is a favourite follow-up after "fork is just COW".
Wait-for graph vs resource-allocation graph: when does a cycle mean deadlock?
A wait-for graph has processes as nodes and an edge Pi → Pj if Pi waits for a resource Pj holds. With single-instance resources, a cycle is deadlock. A full resource-allocation graph also has resource nodes and assignment/request edges. With multiple instances, a cycle is necessary but not sufficient — you may still reduce the graph (Banker's-style) and finish. Detection is cycle-find plus, for multi-instance, a safe-sequence search. Graph algorithms: DSA.
mmap a file, write, then crash. What might be on disk?
Stores go to the page cache. Durability happens on writeback, msync, fsync, or eviction — not on the store instruction. A crash can lose dirty pages. Journaling of data (or a COW filesystem with a committed root) changes the story; metadata-only journals can leave a file whose size updated but whose new data did not, or the reverse, depending on mode. Interviewers want "mmap is not fsync" and a sentence on ordered vs data=journal, not an ext4 mount-option recitation.
Journal modes: metadata-only vs ordered vs data=journal. What is the trade-off?
Metadata-only: fast; after a crash the tree is consistent but file contents may be stale or contain stale junk in freshly allocated blocks if you are sloppy. Ordered: write data blocks to their home locations before committing the metadata that points at them — the usual middle path. Data=journal: log data too; safest and heaviest. Pick by whether "file contains zeros or old bytes after a crash" is acceptable. Databases often do their own WAL and want the FS out of the way (fsync semantics), which is a system design conversation.
Why is a container not "as isolated as a VM", even with namespaces?
A container calls the host kernel. The syscall ABI is large. A kernel memory-safety bug is often a host compromise. Namespaces hide names; they do not give you a second MMU for the kernel itself. A VM fails closed at the hypervisor and (usually) a guest kernel: the attack surface is virtual devices plus hypercalls, which is smaller if you keep the VMM tight. Density and start time go the other way. Use both: VMs as tenant boundaries, containers as deploy units, if the threat model says so.
Syscall overhead is high. What do programs do about it?
Batch: readv/writev, one mmap instead of many reads, io_uring-style submission queues (Linux-specific; name the idea: amortise the crossing). Avoid: stay in userspace with a cache, use huge pages to cut TLB misses, use shared memory for a hot IPC path. The OS topic is the crossing cost (mode switch + validation + possible schedule). Micro-optimising a single getpid is not the point; the point is that a chatty ABI kills throughput. See also system design for batching at service level.
How does Android's low-memory killer relate to thrashing, without turning this into a frameworks lecture?
Classic VM thrashes: it keeps everyone alive and pages. A phone would feel frozen and burn flash. Android prefers to kill a cached process (oom_adj / LMK policy) and keep the foreground working set in RAM. That is admission control by death, not a different page-replacement algorithm. The theory is still working sets; the product policy is on Android frameworks.
What is a process-shared mutex, and when does a futex enter the story?
A mutex in shared memory must live in memory both processes can see and must wake a waiter in another address space. Fast path is still an atomic in userspace. Slow path (contention) needs the kernel to sleep and wake — Linux's futex is that wait queue keyed by an address. You do not implement a futex in an OS-concepts interview; you say "userspace atomic plus a kernel sleep queue" and send details to the kernel page. Language wrappers: C++, Java (monitors are in-process unless you add IPC).
How do huge pages change the TLB and the page-table walk?
A 2 MiB or 1 GiB page occupies one TLB entry instead of 512 or more 4 KiB entries, so the TLB covers more of the working set and hit rate rises. The walk is shorter (you stop at a leaf that describes a large page). Costs: internal fragmentation, harder to move/evict, and allocation can fail if physical memory is fragmented. Mention as a TLB-capacity tool, not as a Linux THP tuning session.
Scenario & debugging
The machine feels frozen, but CPU utilisation is low. What OS stories do you check first?
If the CPU is idle, it is not "compute bound". Look at I/O wait and paging: thrashing (disk busy, high fault rate, load average high because tasks are stuck in D / uninterruptible I/O), a lock convoy, or a filesystem journal stuck on a dying disk. Memory pressure with swap or a storm of major faults matches the working-set story. On Android, LMK killing and restarting can feel like freeze-and-jank — frameworks. Do not start with a faster CPU.
ps shows hundreds of defunct processes. What happened, and how do you fix it?
Those are zombies: children exited, the parent never waited. RAM is probably fine; PID space is not. The parent is buggy (no SIGCHLD reap, or it ignored children). Fix the parent, or kill the parent so the zombies are reparented and reaped. If the parent is stuck in an infinite loop that never waits, that is a process-lifecycle bug, not a memory leak.
A student process calls fork() in a loop with no exec or exit. What happens?
A fork bomb: exponential process create until the PID table, memory (even with COW, each task_struct and kernel stack costs something), or a cgroup/nproc limit stops you. The machine becomes unresponsive because the scheduler and memory allocator are drowning. Defence is a process limit (ulimit/cgroup pids), not a clever scheduler. This is a resource-isolation question as much as a fork question.
Two threads increment a shared counter and the total is too small. Diagnose and fix.
Lost update: both read, both add, both write. It is a race on a critical section that is one increment. Fix: a mutex around the increment, or an atomic fetch-add. Do not use volatile as the fix in C. Mention visibility: after the lock, other threads must see the new value — a correct mutex already publishes. If they ask you to write the code, keep the critical section to the increment; do not hold the lock across I/O.
Dining philosophers: how does deadlock appear, and how do you prevent it?
Each philosopher picks up the left fork then waits for the right: hold-and-wait plus circular wait. Prevention: (1) a global lock order (even-numbered philosopher takes right first), (2) pick up both forks atomically (or via a waiter/arbitrator), (3) a counting semaphore of N−1 philosophers so one seat is always empty and the cycle cannot close. Detection is a wait-for cycle among five processes. This is the Coffman checklist in costume.
Audio glitches when a background thread holds a lock. What inversion story is this?
The audio or RT thread is high priority and needs the lock. A low-priority worker holds it. A medium-priority batch job runs and preempts the worker. The audio thread waits — inversion. Fix: priority inheritance on that lock, or do not take a sleep-lock on the audio path (lock-free ring buffer, try-lock and drop a frame). This is the light Android/RT note; do not dive into scheduler classes unless they ask — then kernel.
A process is stuck and will not die on SIGKILL. How is that possible in OS terms?
SIGKILL is not delivered until the thread returns to a killable context. If it is in uninterruptible sleep waiting on I/O (classic UNIX "D state"), it will not handle signals until the I/O completes or fails. The portable story: some waits are uninterruptible so a filesystem is not left half-updated. The Linux specifics (D state, hung task detector) are kernel. User takeaway: the disk or a driver is the suspect, not "kill −9 is broken".
Power was pulled during a copy. The directory listing looks wrong. What would journaling have changed?
Without a consistency scheme, bitmaps, directory entries and inodes can disagree: a name that points at a free inode, or allocated blocks that no file owns. A metadata journal would replay committed operations and restore a tree that could have existed. You might still lose the last un-fsynced data. fsck is the slow alternative. This is crash consistency, not "the journal makes every write immortal".
You set a 10-microsecond RR quantum to make the UI "snappier". Throughput tanks. Why?
A context switch is thousands of cycles plus cache/TLB loss. If the quantum is on the order of the switch, the CPU spends its life in the scheduler. Response time may even get worse because of the overhead. RR needs a quantum large versus a switch and small versus a human delay. Snappy interactive behaviour is more MLFQ/fair+priority than "tiny q".
A browser vendor asks: threads or processes for tabs?
Threads: cheaper, easy to share a cache, one wild pointer or one infinite loop (without preemption of that thread's siblings... actually 1:1 threads preempt) — more importantly, one memory-corruption bug in a renderer can take the chrome if they share an address space. Processes: crash and security isolation, separate UID/sandbox, higher RAM (duplicated heaps, unless you share via IPC or a zygote-style fork). Real browsers pick processes for renderers. That is isolation over IPC cost — system design plus this page's process vs thread table.
Two processes communicate through shared memory. One sees a flag set but the payload is stale. What did they forget?
They used a plain store to a flag without a release/acquire pair or a process-shared mutex. The payload write can stay in a store buffer or be reordered. Fix: put the payload write before a release-store on the flag, and load-acquire the flag before reading the payload; or just use a mutex around the publish. This is the visibility question in IPC clothing.
A service deadlocks in production. How do you talk about detect vs prevent vs recover?
First gather a wait-for graph: who holds which lock, who waits (thread dumps, lockdep-style traces). A cycle is the smoking gun for single-instance locks. Prevention going forward: lock ranking, smaller critical sections, try-lock with backoff. Recovery now: kill a victim thread or restart the process (databases roll back a transaction). Banker's is almost never how you fix a live mutex deadlock — you do not have declared maxima. Tools are platform-specific; the vocabulary is this page.
A container cannot see host process IDs. Which mechanism is that, and what can it still see?
A PID namespace: the box has its own PID 1 and its own numbering. It cannot name host tasks by their host PIDs. It still shares the kernel, so it can still issue the same syscalls (unless a filter is added) and a kernel bug still matters. Memory and CPU limits are cgroups, not namespaces. Network isolation is a network namespace. Keep the "names vs quotas vs kernel" triangle straight.
An app mmap()s a file, writes a header, crashes. Users report a torn file. Walk the durability story.
The store dirtied a cache page. No msync/fsync means the disk may have old data, new data, or a mix if writeback raced the crash. The directory may already show the new size if metadata was journaled first (or the reverse). Tell them mmap is a cache, not a durability API. For a header that must be atomic, write to a temp file and rename (atomic dir update) after fsync, which is also a system design pattern.
Right after launch, the app is slow and the fault rate is huge, then it settles. What is that?
Demand paging: the working set is being faulted in (code, data, mapped libraries). Major faults hit disk; minor faults map pages already in the page cache (shared libs). Then the working set fits and the fault rate drops. Fixes: demand-paging is working as designed; if the storm is too long, reduce the working set, prefetch the hot path, or keep a process warm (Android zygote is this idea — frameworks). Do not call it thrashing unless the fault rate stays high and the CPU is idle.
Pick IPC: a local log daemon versus a service that might move off-box later.
Log daemon on the same machine: UNIX-domain socket or a named pipe (records vs stream: prefer a socket or a message-oriented protocol so lines do not tear). Shared memory only if the volume is huge and you will write a ring buffer with a futex. Off-box later: start with a stream socket and a versioned protocol so the transport can become TCP. Signals are wrong (no payload). Android apps talking to system_server: Binder — frameworks.
A privileged helper stats a user path, then chmod it. Why is this a TOCTOU interview favourite?
Between stat and chmod the user can replace the path with a symlink to /etc/passwd (or any sensitive file). The helper applied a user-controlled name twice. Fix: open the path once (with O_NOFOLLOW if needed), fstat the fd, then fchmod the same fd. Bind check and use to one object. This is the exam version of "never operate twice on a raw path from an untrusted client".
Adding RAM made a "CPU-bound" batch job much faster. What was it really?
Thrashing or a working set that did not fit: the job looked busy but much of the "CPU time" was wait-on-page-fault, or the disk was the bottleneck. Extra frames let the working set stay resident, faults dropped, useful CPU rose. Check major fault counts and iowait before and after. A true CPU-bound job would have been at 100% compute with few faults and would not have doubled in speed from RAM alone.
One long compile sits at the head of the queue; short interactive jobs feel dead. Name the effect and a policy change.
Convoy effect under FCFS (or RR with a huge quantum). The long CPU burst blocks the short ones; they then all I/O-wait and rejoin behind it. Change to RR with a sane quantum, MLFQ (demote the compile), or a priority/fair class that reserves share for interactive work. CFS's fair share is the Linux example of "the compile does not own the machine" — one sentence, then stop.
Walk a blocking read() of a cold file from a C program to the disk IRQ and back.
C program calls read (C). libc traps. Kernel: fd → file → VFS. Page cache miss: allocate a frame, attach a bio, submit to the block layer, put the thread in waiting. Scheduler runs someone else. Disk DMA fills the frame; IRQ signals completion; the waiter is marked ready. Later it runs, copies bytes to the user buffer, returns the count. Privilege dropped. If this were Java, ART and JNI sit above libc (Java). Hardware/driver objects: Linux kernel & BSP.
Two processes mmap the same file. When do they share physical pages, and when do they get COW copies?
MAP_SHARED: they share page-cache pages; a store is visible to the other and is a candidate for writeback to the file. MAP_PRIVATE: reads share until a write; the write COWs a private page that will not go back to the file (and the sibling still sees the old page). After fork, the child's private maps are COW against the parent as well. Ask which flags they used before you debug "why doesn't the other process see my write?".
A profiler shows a huge context-switch rate. What causes that, and what do you change?
Causes: tiny RR quantum, too many runnable threads (1:1 thread-per-connection without a pool), lock ping-pong (threads wake each other for tiny critical sections), or blocking I/O with no batching. Fixes: fewer threads (event loop or a pool), longer quantum / fair scheduler, coarsen the locking, batch syscalls, use shared memory instead of a chatty pipe. Switching is not free; the indirect cache cost is why. DSA-level "too many workers" is the same idea as too many threads here.