Revise

Quick revision

The must-remember points from every topic. Skim this the night before or the hour before an interview.

1515 key points
CS Foundations / Programming Languages

C Language

Read the full topic →

  • C remains the language of kernels, bootloaders, libc/bionic, HALs and firmware because the ABI is stable and nothing is implicit.
  • Preprocess → compile → assemble → link; the compiler sees one translation unit at a time.
  • Headers declare; exactly one .c defines each external object or function (unless static or a careful inline).
  • Static .a copies used object files into the image; shared .so is loaded at runtime and needs PIC.
  • Integer promotions lift narrow types to int (or unsigned int) before most arithmetic.
  • Usual arithmetic conversions pick a common type; signed vs unsigned of the same rank becomes unsigned.
  • Signed overflow is UB; unsigned wrap is defined modulo 2n.
  • size_t is unsigned; never write if (len < 0) on a size_t.
  • intptr_t / uintptr_t hold a pointer as an integer; they are not a free pass for provenance games.
  • Plain char signedness is implementation-defined; use unsigned char for bytes.
  • Text = code; rodata = literals; data = initialised globals; bss = zero globals; stack = frames; heap = malloc.
  • realloc failure leaves the old pointer valid; do not overwrite it with NULL.
  • Alignment drives struct padding; sizeof includes tail padding.
  • Endian is a property of how integers are stored; the wire format is not automatically the host format.
  • Arrays decay to pointers except for sizeof, &arr and a few other cases; function parameters are already pointers.
  • Pointer arithmetic is in elements, only within the same array (or one-past-end, not dereferenced).
  • void * is a generic object pointer; do not dereference it; function pointers are a separate story.
  • const T * vs T * const: pointee vs pointer.
  • restrict means "these buffers do not alias"; overlapping memcpy is UB.
  • A C string is bytes plus a NUL; capacity must include that NUL if you store a string.
  • snprintf bounds the write, NUL-terminates if size > 0, and returns the untruncated length.
  • strncpy is not safe strcpy: it may omit the NUL and it zero-pads.
  • memcpy is restrict and forbids overlap; memmove allows overlap.
  • Do not memcmp structs for equality when they contain padding.
  • Union type punning is a documented compiler dialect; portable punning is memcpy.
  • Bitfield allocation order and straddling are implementation-defined; do not use them as a wire format.
  • A flexible array member is T data[] at the end; sizeof excludes it; allocate offsetof + n * sizeof(T).
  • Packed structs drop padding and can cause unaligned accesses on ARM; serialise with bytes or memcpy.
  • Include guards (or #pragma once) are per translation unit; they do not give you one definition across the program.
  • Prefer static inline over function-like macros: types, single evaluation, debuggable.
  • # stringizes, ## pastes tokens; both happen in the preprocessor, not the type system.
  • #include "…" searches the includer's directory first; <…> uses the include path; first -I hit wins.
  • static at file scope = internal linkage; on a local = static duration; the word is not one concept.
  • Tentative definition: file-scope int g; with no initialiser. Modern -fno-common forbids one per unit.
  • C99 inline is not C++ inline; you still need one external definition if the function is used out of line.
  • Undefined = no requirements; unspecified = one of several; implementation-defined = documented choice.
  • Unsequenced modify + modify (or modify + extra read) of the same scalar is UB: i = i++.
  • volatile forces accesses to that lvalue; it is not atomic, not a mutex, not a CPU barrier.
  • A data race is UB in C11 even if the store "looks atomic" on the CPU.
  • File descriptors are kernel ints; FILE* is a buffered libc stream; fileno / fdopen bridge them.
  • Read errno only after a documented failure; it is thread-local; save it before you log.
  • Make rebuilds a target when a dependency is newer; .PHONY is a name that is not a file.
  • After a crash: backtrace first. gdb/lldb need -g; optimised code will not match source perfectly.
  • ASan = spatial/temporal memory; UBSan = language UB; LSan = leaks; TSan = races; Valgrind = interpreter, slow, host-friendly.
  • Mutex unlock happens-before the next lock; condvar waits must loop on the predicate.
  • Publish with payload stores then a release store; consume with an acquire load then payload loads.
  • memory_order_relaxed is for independent counters, not for publishing a pointer to new data.
  • Android native = NDK/Clang + bionic, not glibc. Kernel C is a different world (no libc).
  • JNIEnv is per-thread; attach native threads; never cache another thread's env.
  • FindClass on a native thread uses the system class loader — cache jclass from the right context.
  • Local JNI refs die when the native method returns; delete in tight loops; global refs must be deleted.
  • JNI "UTF" is modified UTF-8; Java strings are UTF-16; binary data belongs in byte arrays.
  • fdsan aborts on close of an fd the runtime thinks someone else owns.
  • HALs are C vtables historically and AIDL/Binder now; do not invent a new ioctl protocol if AIDL exists.
  • C has no unique_ptr: ownership is a comment plus one free on every path, often via goto cleanup.
  • Whiteboard C is strlen/memcpy/bits/buffers, not graph algorithms (those live on the DSA page).
  • Check overflow before malloc(count * size); a wrapped size is a security bug.
  • Writing a string literal is UB; use an array if you need to mutate.
  • One-past-the-end pointers may be computed, not dereferenced.
  • LTO and -O2 make latent UB visible; "it works in debug" is not a proof of definedness.
CS Foundations / Programming Languages

Java Language

Read the full topic →

  • Java is the language; HotSpot is a JVM; ART is Android's runtime. Kotlin compiles to the same bytecode family; this page is Java.
  • javac emits class files (bytecode + constant pool), not machine code. ART's on-disk code is dex, then oat/vdex via dex2oat.
  • A class is (binary name, defining loader). Parent delegation asks the parent first so core types cannot be spoofed.
  • Initialization order: superclass statics, this statics, superclass instance + ctor, this instance + ctor. Compile-time constants do not trigger init.
  • Stack holds frames (primitives and references). Heap holds objects. Class metadata is Metaspace / ART native, not PermGen.
  • Object header: mark word + class pointer (+ array length). Compressed oops and compressed class pointers are optional 32-bit packs.
  • HotSpot: young/old (G1 regions). ART: Concurrent Copying, moving, Zygote image space kept stable for copy-on-write.
  • Escape analysis may scalar-replace a non-escaping new; do not claim every allocation hits the heap.
  • GC roots: locals, statics, JNI globals, thread/monitor internals. Weak / soft / phantom are not roots in the strong sense.
  • STW pauses mutators; concurrent GC still has short safepoint pauses. finalize is obsolete; use Cleaner.
  • equals is reflexive, symmetric, transitive, consistent, false for null. Equal objects must share a hashCode.
  • Never mutate a key while it sits in a hash map. Prefer immutable keys.
  • String is immutable; intern is a heap pool; do not intern unbounded input. StringBuilder for loops; skip StringBuffer.
  • Integer cache is typically −128..127. == on boxed integers outside that range is a bug.
  • clone is shallow; prefer a copy constructor. compareTo must agree with equals in sorted collections.
  • Composition over inheritance unless Liskov holds. Default methods evolve interfaces; they do not add instance fields.
  • Inner and anonymous classes hold this$0. Static nested + WeakReference for Android Handlers.
  • Records and sealed types are newer Java; mention them, do not assume every Android API level has them.
  • Generics erase. PECS: producer extends, consumer super. Heap pollution shows up later as ClassCastException.
  • Arrays are reified and covariant; generic lists are invariant and erased.
  • ArrayList grows about 1.5×; first add uses capacity 10. Prefer ArrayDeque over Stack/LinkedList for queues.
  • HashMap treeifies a bin at chain 8 only if table capacity is at least 64; otherwise it resizes.
  • HashMap is not thread-safe. Hashtable is coarse synchronized. ConcurrentHashMap is the concurrent map; no nulls.
  • Fail-fast iterators use modCount. CHM iterators are weakly consistent.
  • Checked exceptions must be declared or caught. Error is for the VM. Do not swallow InterruptedException.
  • try-with-resources closes in reverse and records suppressed exceptions. A return in finally discards the pending throw or return.
  • Happens-before: program order, unlock/lock, volatile write/read, start, join. A data race has almost no guarantees.
  • volatile is visibility and ordering of that field, not atomic i++. DCL requires volatile on the instance field.
  • final fields are safely published after construction if you do not leak this.
  • wait only inside the monitor, always in a while that re-checks the condition.
  • Latch = one-shot N-to-zero. Barrier = reusable "all arrive". Semaphore = permits. Lock + Condition = multiple wait-sets.
  • ThreadPoolExecutor: core, then queue, then max, then reject. An unbounded queue means max is never used.
  • Deadlock: circular lock wait. Livelock: busy yielding. Always ThreadLocal.remove on pooled threads.
  • CompletableFuture: compose with thenCompose; do not block the common pool with I/O.
  • Streams hurt on tiny data, boxed primitives, parallel() on the common pool, and side-effecting map/filter.
  • Optional is a return type, not a field or parameter. orElse(x) is eager; orElseGet is lazy.
  • Java serialization skips constructors and is a security hazard. On Android IPC use Parcelable, never Parcel-on-disk.
  • Defensive-copy mutable args and returns. Unmodifiable wrappers still share the backing list unless you copy.
  • native + System.loadLibrary on the Java side; C-side JNI is C language.
  • ART: dex in the APK, vdex verified, oat/odex compiled. Main thread is a Looper; block it and you ANR.
  • Binder calls run on the Binder pool. Boxed Boolean/Integer on Binder allocate and can be null.
  • Java is pass-by-value of the reference. Rebinding a parameter does not change the caller; mutating the object does.
  • Overload = compile-time, static types. Override = runtime, receiver type. Thread.start creates a thread; run does not.
  • String == is identity. Use equals unless both sides are interned by you on purpose.
CS Foundations / Programming Languages

C++ Language

Read the full topic →

  • C++ adds RAII, a stricter type system and templates on top of the C machine model; the Linux kernel stays C.
  • A translation unit is one .cpp after preprocessing. The ODR forbids two different definitions of the same entity.
  • inline means "this definition may appear in many TUs", not "please inline this call".
  • Templates instantiate per used specialization; the linker keeps one copy (COMDAT).
  • Name mangling encodes types; extern "C" disables it for C, JNI and dlsym.
  • ABI is layout + calling convention + vtable + standard-library object layout. libc++ and libstdc++ are incompatible.
  • Bases construct first, then members in declaration order, then the body; destruction is the reverse.
  • Virtual call: load vptr, index the slot, optionally adjust this, indirect call.
  • Virtual calls in constructors and destructors do not reach the most-derived override.
  • Deleting through Base* without a virtual destructor is undefined behavior.
  • Slicing: passing a derived object by base value drops derived data and virtual behavior.
  • Multiple inheritance may require a this-adjustment; virtual inheritance shares one base in a diamond.
  • Empty class sizeof is 1; empty bases can be 0 bytes (EBO). That is why a default unique_ptr is pointer-sized.
  • lvalue = identity, still alive; xvalue = identity, expiring; prvalue = temporary / initializer.
  • std::move is a cast to T&&. It does not move; the move constructor might.
  • Moving a const object usually copies. Do not return std::move(local).
  • C++17 guarantees elision for prvalues of the same type; NRVO is optional.
  • Rule of Zero: manage resources with members. Rule of Five: if you write one special member, write or delete all five.
  • Copy-and-swap gives the strong exception guarantee for assignment if swap is non-throwing.
  • RAII ties release to destruction so every exit path cleans up, including exceptions.
  • unique_ptr exclusive; shared_ptr shared + atomic control block; weak_ptr observes and breaks cycles.
  • make_shared is one allocation (object + control block). The object storage may outlive the object while weak refs remain.
  • Never shared_ptr<T>(this); use enable_shared_from_this only when a shared_ptr already owns the object.
  • auto_ptr transferred on copy and was removed; use unique_ptr.
  • new constructs; malloc does not. Pair new[] with delete[]. Placement new needs an explicit destructor.
  • alignas / alignof matter for SIMD and over-aligned types. std::launder after reusing storage (light).
  • A template T&& parameter is a forwarding reference; use std::forward.
  • Prefer overloading functions to specializing them. Specialize class templates; use partial specialization for families of types.
  • SFINAE drops bad candidates; C++20 concepts name the same constraints with better errors.
  • CRTP is static polymorphism (no vtable). Variadic packs expand with ...; C++17 adds folds.
  • vector is the default sequence: contiguous, amortized O(1) append, geometric growth (~2× on libc++).
  • vector reallocation invalidates all iterators, pointers and references.
  • map is O(log n) ordered; unordered_map is average O(1), worst O(n), rehash invalidates iterators.
  • SSO keeps short std::string inline (often 15 bytes on 64-bit libc++).
  • C++20 ranges are lazy non-owning views plus range algorithms; a view does not own data.
  • const is a type qualifier; constexpr may run at compile time; consteval must; constinit is static init (C++20, light).
  • Writing through const_cast to an object that was born const is UB.
  • Exception safety: no-throw, strong, basic, none. Destructors must not throw.
  • vector moves on reallocation only if the move is noexcept; otherwise it copies.
  • Android system C++ often uses -fno-exceptions and -fno-rtti for size and JNI/Binder safety.
  • A data race (conflicting non-atomic access, at least one write, no sync) is UB.
  • lock_guard is simple RAII; unique_lock works with condition_variable. Always wait with a predicate loop.
  • memory_order: relaxed = atomic only; acquire/release = publish; seq_cst = default total order.
  • Signed overflow, strict aliasing violations, dangling refs and iterator invalidation are UB.
  • Use-after-move is a valid but unspecified state for STL types; treating it as fully intact is a bug.
  • Prefer explicit lambda captures. [&] stored past the scope dangles. std::function type-erases and may heap-allocate.
  • Android NDK uses libc++. Do not pass STL types across mixed-STL or mixed-ABI .so boundaries.
  • HIDL C++ is legacy hwbinder; new vendor HALs are AIDL NDK (libbinder_ndk, ScopedAStatus).
  • SurfaceFlinger and netd are native C++ daemons; JNI is extern "C" and must not leak C++ exceptions into ART.
  • Default coding answers: RAII wrappers, unique_ptr, vector, no naked new, no returned locals, virtual dtor on bases.
CS Foundations / Core Computer Science

Data Structures & Algorithms

Read the full topic →

  • Big-O ignores constants and keeps the dominant term; always state both time and space, and name every input variable.
  • Amortized O(1): dynamic-array append and hash-map insert are occasionally O(n) but average O(1) because capacity doubles.
  • Rough budget: about 10^8 simple operations per second; n = 10^5 to 10^6 means O(n) or O(n log n).
  • Recursion depth counts as space: O(h) for trees, O(n) for skewed trees and deep DFS.
  • Hash maps give O(1) average lookup via hashing and buckets; worst case O(n) when keys collide. Java 8+ HashMap treeifies a bin only if it has 8+ entries and the table capacity is at least 64 (otherwise it resizes); a treeified bin is O(log n).
  • Replacing an inner search loop with a hash-map lookup is the most common optimisation.
  • Linked lists: use a dummy head, save next before rewiring, check fast and fast.next.
  • Floyd's cycle detection: after the pointers meet, reset one to head and step both by one to reach the entry.
  • Use collections.deque for queues in Python; list.pop(0) is O(n).
  • Heaps: O(1) peek, O(log n) push and pop, O(n) heapify; a size-k min-heap keeps the k largest.
  • Two heaps (max-heap low half, min-heap high half) give a running median.
  • A BST in-order traversal is sorted; validate with min/max bounds, not just children.
  • Diameter of a tree is the max of left-height + right-height over every node, not only the path through the root.
  • Fenwick tree: point update and prefix sum in O(log n) via i & -i; segment tree if you need min/max or range updates.
  • equals and hashCode must agree; mutable keys in a hash map get lost.
  • Tries give O(L) insert and search and natural prefix queries at the cost of memory.
  • BFS finds shortest paths in unweighted graphs; mark nodes visited when enqueuing.
  • DFS finds components, paths and cycles; directed cycles are detected with grey (on-stack) nodes.
  • Topological sort (Kahn): repeatedly remove in-degree-zero nodes; fewer than n output means a cycle.
  • Dijkstra needs non-negative weights, O((V + E) log V); use Bellman-Ford for negative edges.
  • Union-Find with path compression and union by size is effectively O(1) per operation.
  • Two pointers need a monotonic property (usually sortedness) to know which pointer to move.
  • Sliding window: expand right, shrink left while invalid; fails with negative numbers for sum constraints.
  • Binary search on the answer works whenever feasibility is monotonic; use a half-open "first true" template.
  • Prefix sums plus a hash map count subarrays with sum k in O(n), even with negatives.
  • Monotonic stack solves next-greater and histogram problems in O(n) because each index is pushed and popped once.
  • Interval problems start by sorting; meeting rooms uses a min-heap of end times.
  • Backtracking is choose, explore, un-choose; copy the path when recording and prune early.
  • Greedy needs a proof (exchange argument); coin change with {1, 3, 4} breaks greedy.
  • Comparison sorts are Omega(n log n); counting and radix sort beat it for bounded integer keys.
  • Stable sorts (merge, insertion, TimSort, counting) keep equal keys in original order.
  • Quicksort is fastest in practice but O(n^2) worst case; random pivots or introsort fix it.
  • DP needs optimal substructure and overlapping subproblems; define state, transition, base case, order, answer.
  • DP complexity = number of states x work per state.
  • 0/1 knapsack iterates capacity downward; unbounded iterates upward.
  • LIS is O(n^2) with DP or O(n log n) with patience sorting and binary search.
  • Interview flow: clarify, brute force, optimise, agree, code, test edge cases, discuss complexity.
CS Foundations / Core Computer Science

Operating System Concepts

Read the full topic →

  • An OS multiplexes hardware, isolates programs, and abstracts devices behind syscalls.
  • Dual mode: user code cannot change page tables or talk to devices; the mode bit flips only on trap, IRQ, or exception.
  • Privilege rings: kernel vs user in practice; extra levels exist for a hypervisor and firmware.
  • A syscall is a numbered, validated request issued by a trap instruction, not a normal function call.
  • IRQ is asynchronous (devices, timer). Trap/syscall is synchronous and deliberate. Faults are synchronous and often restartable (page fault).
  • A process is an address space plus kernel metadata; the PCB is that metadata.
  • States: new, ready, running, waiting, terminated. Preempt: running to ready. Block: running to waiting.
  • Context switch cost = register save/restore + optional address-space switch (TLB) + cold caches.
  • POSIX life cycle: fork copies, exec replaces, exit leaves a zombie, wait reaps it.
  • Zombie = exited, not waited; tiny, but it occupies a PID. Orphan = parent died; reparented and later reaped.
  • Threads share the address space and fds; they have their own stacks and TLS.
  • N:1 is fast and cannot parallelise or block independently; 1:1 is the modern POSIX default; M:N is a runtime compromise.
  • TLS is per-thread globals, usually a register pointing at a control block plus offsets.
  • CPU bursts and I/O bursts describe the workload; schedulers care about the next CPU burst.
  • FCFS has the convoy effect. SJF/SRTF minimise average wait if bursts are known. RR needs a sane quantum.
  • Turnaround = completion − arrival; waiting = turnaround − burst; response = first run − arrival.
  • Priority without aging starves. MLFQ demotes CPU hogs and keeps interactive jobs responsive.
  • Linux CFS is a fair-share example using virtual runtime; it is not a textbook RR. Details stay on the kernel page.
  • A race means the outcome depends on interleaving. The critical section must be mutually exclusive.
  • Critical-section trio: mutual exclusion, progress, bounded waiting.
  • Mutex has an owner. Semaphores do not; counting semaphores track N resources.
  • Condition variables wait for a predicate; always re-check the predicate after wake-up (Mesa semantics).
  • Producer-consumer needs exclusion plus empty/full. Readers-writers needs a stated starvation policy.
  • Spin if the hold is tiny and you cannot sleep; otherwise use a sleeping mutex. Hybrids spin then sleep.
  • Locks also publish memory: unlock/lock (or acquire/release atomics) create happens-before.
  • Deadlock needs all four Coffman conditions. Break one to prevent it.
  • Banker's grants a request only if a safe sequence still exists. It needs declared maxima.
  • Livelock: activity without progress. Starvation: you never run while others do.
  • Priority inversion: a medium job preempts a low holder that a high job needs. Inheritance boosts the holder.
  • Logical addresses are what the program issues; physical addresses are RAM. The MMU translates.
  • External fragmentation = holes between allocations. Internal = slack inside a block or page.
  • Paging uses fixed frames (no external frag of RAM). Segmentation uses variable segments (external frag returns).
  • Multi-level page tables store only used branches of a huge virtual space. The walk costs L memory refs.
  • EAT ≈ h(t_TLB + t_mem) + (1−h)(t_TLB + (L+1)t_mem). Hit rate dominates.
  • Page fault: classify, get a frame, fill from disk or zero/COW, install PTE, restart the instruction.
  • Demand paging loads a page on first touch. The working set is the recently referenced pages.
  • Thrashing: working sets do not fit; the disk is busy and the CPU is idle. Run fewer jobs or add RAM.
  • FIFO can Belady-anomaly. OPT is the unrealisable lower bound. Clock approximates LRU with a referenced bit.
  • COW: fork shares physical pages until a write; MAP_PRIVATE mmap is the same idea for files.
  • mmap maps a file or anonymous memory into the address space; first touch is a page fault, not a read copy.
  • Programmed I/O: CPU copies every word. Interrupt-driven: CPU still copies, but after an IRQ. DMA: the device copies; IRQ means done or error.
  • Blocking I/O sleeps. Non-blocking returns immediately (EAGAIN). Asynchronous I/O starts now and completes later on a queue or callback.
  • A device driver is the translator from a device's registers to a uniform OS interface. Binding and probe are kernel-page material.
  • A file is bytes plus metadata. An inode is the identity and block map. A directory maps names to inodes.
  • Disk allocation: contiguous, linked, or indexed. FAT is a central link table. A UNIX inode is indexed (direct plus indirect blocks).
  • A journal is a write-ahead log so a crash can replay metadata (and optionally data) to a consistent state.
  • VFS is one syscall surface over many filesystem implementations (disk, pipe, proc-like).
  • Hard link: another directory entry for the same inode. Symbolic link: a path string that may dangle or cross filesystems.
  • Anonymous pipes fit parent-child byte streams. Sockets are bidirectional and can go remote. Shared memory is fastest and still needs a lock protocol.
  • Signals notify; they are not a data path. Use async-signal-safe calls only in a handler.
  • UID/GID label the process. An ACL lives on the object. A capability is a token you hold; an open fd is the everyday example.
  • TOCTOU: a check on a path is not a check on the object you later use. Prefer fds, openat, and exclusive-create flags.
  • Type-1 hypervisor sits on hardware; type-2 is hosted. Trap-and-emulate needs sensitive guest operations to actually trap.
  • A container shares the host kernel and isolates with namespaces plus cgroups. A VM runs a guest kernel on virtual hardware.
  • read() is libc → trap → fd/VFS → page cache or block I/O sleep → DMA/IRQ wake → copy to user → return.
CS Foundations / Core Computer Science

System Design

Read the full topic →

  • Start every design with functional and non-functional requirements, then estimate scale; never start with boxes.
  • 1 million requests per day is about 12 per second; peak is about 2-3x average; a day is about 10^5 seconds.
  • Memory is about 100 ns; 1 MB sequential from RAM is about 10-50 μs (not a few microseconds). SSD random is about 100 μs, same-DC round trip about 0.5 ms, cross-continent about 150 ms.
  • Scale out with stateless services behind load balancers; keep session state in a shared store or token.
  • Report latency as percentiles; fan-out amplifies tail latency.
  • 99.9% availability is about 8.8 hours of downtime per year; 99.99% is about 53 minutes.
  • Serial dependencies multiply availability down; independent redundancy multiplies failure probability down.
  • CAP: during a partition choose consistency or availability; PACELC adds latency vs consistency in normal operation.
  • Quorum reads see the latest write when R + W > N.
  • Pick consistency per feature: strong for money and inventory, eventual for feeds and counts.
  • L4 load balancers route by IP and port; L7 by HTTP content, with TLS termination and path routing.
  • CDNs cache static and cacheable content at the edge; use content-hashed file names instead of purging.
  • Cache-aside is the default; on writes, update the database then delete the cache key.
  • Prevent cache stampedes with request coalescing, TTL jitter and stale-while-revalidate.
  • LRU is O(1) with a hash map plus a doubly linked list.
  • Choose the database from access patterns: relational for transactions and joins, wide-column for massive writes, KV for key lookups.
  • Indexes speed reads to O(log n) but slow writes and use storage; composite indexes follow the leftmost prefix.
  • Replication scales reads and availability; async replication means lag and possible data loss on failover.
  • Sharding scales writes and storage; the shard key must spread load and match the main query.
  • Consistent hashing moves only about 1/N of keys when a node is added; virtual nodes balance load.
  • Queues decouple, buffer spikes and enable retries; they give at-least-once delivery and per-partition ordering.
  • Exactly-once is at-least-once delivery plus idempotent processing.
  • Idempotency keys make retried writes safe; the outbox pattern solves the dual-write problem.
  • Token bucket allows bursts with an average rate; sliding window counter is accurate with O(1) memory.
  • Rate limiter state lives in Redis updated atomically; return 429 with Retry-After.
  • Every remote call needs a timeout; retry only idempotent calls, with exponential backoff and jitter.
  • Circuit breakers fail fast on a sick dependency; bulkheads isolate resource pools.
  • URL shortener: base62 of a unique ID, read-heavy KV store, cache and CDN, async analytics.
  • News feed: hybrid fan-out, push for normal users and pull for celebrities.
  • Chat: WebSocket gateways, persist before acknowledging, per-conversation sequence numbers, push when offline.
  • File sync: content-hashed chunks, deduplication, metadata DB, pre-signed uploads, change cursors.
  • Ride-hailing: in-memory geospatial index (geohash, quadtree, S2/H3) and exclusive trip assignment.
  • OTA: signed payloads, A/B partitions with automatic fallback, staged rollout gated on telemetry, kill switch.
  • Device design treats battery, memory, connectivity and privacy as first-class constraints; batch and defer work.
  • Raft: elect a leader, replicate a log, commit on majority; 2f+1 nodes tolerate f failures. Keep it off the high-QPS data path.
  • Bloom filter: maybe-present, never false-negative, no deletes; ~10 bits/item at 1% false positives. HyperLogLog for distinct counts, Count-Min for heavy hitters.
  • Version vectors detect concurrent writes; do not order distributed events by wall-clock timestamps alone.
  • Senior answers name alternatives, tie choices to requirements, state what is given up, and cover failure, operations and cost.
CS Foundations / Core Computer Science

Linux Kernel & BSP

Read the full topic →

  • BSP = bootloader + device tree + kernel defconfig + drivers that adapt Linux/Android to one SoC and board.
  • Linux is monolithic (all core code in one privileged address space) with loadable modules (.ko, =m in Kconfig).
  • User code runs at EL0, the kernel at EL1; a syscall traps via svc, dispatches through the syscall table and returns with eret.
  • Kernel code must use copy_to_user/copy_from_user for user pointers.
  • The device tree describes non-discoverable hardware; compatible matches a driver's of_match_table, then probe() runs.
  • .dts/.dtsi compile to .dtb; overlays (.dtbo) patch board variants; status = "okay" enables a node.
  • Driver model: bus + device + driver; -EPROBE_DEFER retries when a dependency is not ready; devm_* frees resources automatically.
  • Character drivers expose file_operations; block drivers go through the block layer; network drivers expose net_device, no /dev node.
  • Top half: minimal, cannot sleep. Bottom half: softirq and tasklet cannot sleep; workqueue and threaded IRQ can.
  • Threaded IRQs with IRQF_ONESHOT are the modern default for device drivers.
  • Sleeping allowed only in process context without spinlocks held; otherwise "scheduling while atomic".
  • kmalloc = physically contiguous (DMA-able); vmalloc = virtually contiguous only; GFP_ATOMIC in atomic context.
  • Buddy allocator hands out 2^order pages; SLUB caches small objects on top.
  • DMA on non-coherent ARM needs cache clean before device reads and invalidate before CPU reads; use the DMA API.
  • ION is gone; DMA-BUF heaps share zero-copy buffers by fd.
  • MMU walks page tables; TLB caches translations; ASIDs avoid full TLB flushes on context switch.
  • Page faults: minor (in RAM), major (needs I/O), invalid (SIGSEGV or kernel oops).
  • fork uses copy-on-write; that is why Zygote forking is fast and shares preloaded memory.
  • Android has no disk swap; ZRAM compresses anonymous pages; LMKD kills by oom_score_adj using PSI.
  • Linux schedules tasks (task_struct); threads are tasks sharing an address space via clone flags.
  • CFS picks the task with the smallest vruntime from a red-black tree; EEVDF replaced it in Linux 6.6.
  • RT classes (SCHED_FIFO, SCHED_RR, 1-99) always preempt normal tasks; SCHED_DEADLINE is above them.
  • EAS uses an energy model to place tasks on big.LITTLE cores; uclamp and cpusets steer Android app groups.
  • Spinlock = busy-wait, short, any context; mutex = sleeps, owner, process context; semaphore = counter, no owner.
  • Share data with an IRQ handler using spin_lock_irqsave; RCU for read-mostly data.
  • volatile is not synchronization; use atomics, barriers, READ_ONCE/WRITE_ONCE or locks.
  • Deadlock needs mutual exclusion, hold-and-wait, no preemption and circular wait; lock ordering breaks it; lockdep detects it.
  • Priority inversion is fixed by priority inheritance (rt_mutex, PI futex, Binder priority propagation).
  • Binder: one copy, kernel-verified caller identity, reference counting, death notification, thread pool.
  • Android filesystems: f2fs for /data, erofs for read-only partitions, ext4 still common; fsync for durability.
  • Suspend: freeze tasks, suspend devices, CPUs off, wake on wake IRQ; wakeup sources block it.
  • GKI 1.0 optional in Android 11; GKI 2.0 required for new Android 12+ devices on kernel 5.10+. Google-built core kernel, stable KMI, vendor modules in vendor_boot/vendor_dlkm.
  • Android typically sets /proc/sys/kernel/panic_on_oops to 1, so an oops becomes a panic; the kernel default is to keep running.
  • Emulated /sdcard is MediaProvider FUSE since Android 11; sdcardfs was the Android 6–10 in-kernel wrapper and is gone.
  • Probe order: pinctrl, regulators, clocks, then talk to the device. Debug with clk_summary, regulator_summary and pinctrl debugfs.
  • Prefer wait_event_interruptible / wait_event_killable or a completion; plain wait_event can leave a task unkilleable in D state.
  • Treble: stable AIDL HALs between system and vendor partitions, checked by VINTF manifests and compatibility matrices.
  • Debug kit: dmesg, pstore, ftrace, Perfetto, perf, addr2line, crash/T32 ramdumps, KASAN, lockdep, kmemleak.
Mobile & Embedded Systems / Android Platform

Android Boot Process

Read the full topic →

  • Boot chain: PMIC → PBL → XBL/SBL → ABL → kernel → init → Zygote → system_server → SystemUI/Launcher → BOOT_COMPLETED.
  • Trust chain: OEM PK hash in QFPROM → PBL verifies XBL → XBL verifies TZ/hyp/ABL → ABL runs AVB on vbmeta/boot/dtbo → kernel dm-verity on system/vendor.
  • The PMIC sequences the rails, starts the clock, releases CPU reset and records the power-on reason.
  • PBL lives in SoC mask ROM, runs at EL3 from IMEM/SRAM on the 19.2 MHz XO, and can never be updated - that is why it is the root of trust.
  • PBL uses SRAM because DDR is untrained; DDR training is too large and board-specific for ROM.
  • QFPROM eFuses hold secure-boot enable, OEM PK hash, anti-rollback version and JTAG disable.
  • PBL finds XBL via strap pins and fuses: UFS boot LUN or eMMC boot partition.
  • Verification failure in PBL means halt or EDL (QDLoader 9008, USB 05c6:9008), which still requires a signed Firehose programmer.
  • eMMC: parallel, half duplex, ~400 MB/s HS400. UFS: serial M-PHY/UniPro, full duplex, deep SCSI queue, ~2.1 GB/s (3.1) to ~4.2 GB/s (4.0).
  • XBL's key job is DDR training; it also sets up PMIC/clocks and loads TrustZone/QTEE, the hypervisor, AOP firmware and ABL.
  • TrustZone splits normal and secure worlds; the kernel reaches the TEE through SMC calls; EL3 is the secure monitor.
  • ABL (UEFI app, older LK/aboot) picks boot mode and A/B slot, runs AVB, passes boot state to the TEE, builds bootconfig and loads kernel + ramdisks + DTB.
  • fastboot = bootloader flashing; fastbootd = user-space flashing for super; recovery = OTA/sideload/reset; EDL = ROM-level rescue.
  • vbmeta contains hash descriptors (small images), hashtree descriptors (dm-verity root hashes), chain descriptors (delegated keys) and a rollback index.
  • AVB rollback indexes live in RPMB and are raised only after the new slot is marked successful.
  • dm-verity checks each 4 KB block on read against a Merkle tree whose root hash comes from signed vbmeta; modes restart/eio/logging.
  • Boot states: GREEN locked + OEM key, YELLOW locked + custom key, ORANGE unlocked, RED verification failed.
  • Unlocking requires OEM unlocking enabled, wipes userdata, and is reported to KeyMint attestation.
  • GKI split: boot = generic kernel, init_boot = generic ramdisk (Android 13+), vendor_boot = vendor ramdisk + DTB + early modules, vendor_dlkm = other modules.
  • super holds logical partitions (system, vendor, product ...) mapped with dm-linear from LP metadata.
  • A/B: update inactive slot, setActiveBootSlot, tries counter decremented by the bootloader, markBootSuccessful by user space, automatic rollback.
  • Virtual A/B keeps one copy of dynamic partitions and writes COW snapshots, merged after a successful boot; rollback is impossible once the merge starts.
  • Kernel: head.S enables MMU, start_kernel() parses DT and sets up scheduler/IRQs, initcalls probe drivers (with deferred probe), ramdisk unpacked, /init runs as PID 1.
  • init phases: first stage (mount, dm-verity, super) → selinux_setup (load policy, enforce) → second stage (properties, .rc, services).
  • Trigger order: early-init, init, late-init (fs, post-fs, post-fs-data, zygote-start, boot).
  • critical service: more than 4 crashes in 4 minutes reboots to bootloader/recovery.
  • ueventd creates /dev nodes from ueventd.rc, does coldboot and loads firmware; servicemanager is Binder handle 0.
  • Zygote: the init service is named zygote (binary app_process64); a 32-bit helper is zygote_secondary. It preloads ART/classes, forks system_server, then forks apps (copy-on-write, optional USAP pool).
  • system_server: startBootstrapServices, startCoreServices, startOtherServices, boot phases 100 to 1000, then systemReady starts SystemUI and Home.
  • Boot animation is native; WMS sets service.bootanim.exit=1 when Home is drawn. Endless animation means a framework-level failure.
  • FBE: LOCKED_BOOT_COMPLETED for Direct Boot apps with DE storage; BOOT_COMPLETED after the user unlocks CE storage.
  • Debug by stage: no logo = bootloader, logo only = kernel/early init, endless animation = Zygote/system_server, recovery prompt = Rescue Party.
  • Previous-boot kernel log: pstore/ramoops; framework: logcat -b all, boot_progress events, tombstones, dropbox, sys.boot.reason.
  • Boot-time work: measure per stage (dmesg, ro.boottime, bootchart, Perfetto), then async probe, defer non-critical work, trim preloads and PMS work.
  • Wear OS: same chain plus AON co-processor coordination, eMMC, smaller image, charger mode and power-gated OTAs.
  • Userspace reboot (Android 11+): init re-execs and restarts all userspace; the kernel stays up. Different from a system_server crash (soft reboot of the Java world only) and from a full kernel reboot.
Mobile & Embedded Systems / Android Platform

Android Frameworks Internals

Read the full topic →

  • Stack: apps → (Binder) → system_server services → native daemons → (stable AIDL / HIDL) vendor HALs → kernel drivers.
  • system_server hosts AMS, ATMS, WMS, PMS, PowerMS, DisplayMS, InputMS, SensorService and about 100 more; if it dies the framework soft-reboots.
  • SystemServer starts services in waves: bootstrap, core, other (and APEX); services react to boot phases up to PHASE_BOOT_COMPLETED (1000).
  • ATMS (activities, tasks) was split from AMS (processes, services, broadcasts, providers) in Android 10.
  • SurfaceFlinger is a separate native process; WMS sets window policy and sends SurfaceControl transactions to it.
  • Zygote preloads ART, classes and resources, then forks every app process; pages are shared copy-on-write.
  • Zygote uses a Unix socket, not Binder, because forking a multithreaded process is unsafe.
  • New app process runs ActivityThread.main(), prepares the main Looper, calls attachApplication(); AMS calls back through IApplicationThread.
  • Cold start = no process; warm = process alive, activity recreated; hot = activity just resumed.
  • Measure startup with am start -W, the "Displayed" log (TTID), reportFullyDrawn() (TTFD) and Perfetto.
  • ContentProvider onCreate runs before Application.onCreate; provider data calls run on Binder threads.
  • Activity lifecycle: onCreate, onStart, onResume, onPause, onStop, onDestroy (onRestart when returning); config change recreates by default.
  • Services: started, foreground (must call startForeground() within a few seconds; the timeout is 5 s or 10 s depending on the release), bound (returns an IBinder).
  • onReceive() runs on the main thread; use goAsync() for short async work, WorkManager for long work.
  • AMS computes oom_score_adj (0 foreground ... 900-999 cached) in OomAdjuster; bindings raise the server's priority.
  • lmkd is a userspace daemon using PSI (/proc/pressure/memory) to decide when to kill; kills the highest adj first.
  • Cached apps can be frozen with the cgroup v2 freezer (feature in Android 11, default-on later); a sync Binder call to a frozen process fails with BR_FROZEN_REPLY and does not unfreeze it; oneway is queued. The kernel OOM killer is the last resort.
  • Looper loops over a time-sorted MessageQueue; next() sleeps in epoll_wait on an eventfd, so no busy-waiting.
  • Sync barriers let Choreographer's asynchronous frame messages jump ahead of normal messages.
  • Binder calls run on the Binder thread pool (15 + 1 by default, 31 in system_server), never on the main Looper.
  • ANR timeouts: input 5 s, broadcast 10 s fg / 60 s bg, service 20 s fg / 200 s bg, provider publish 10 s.
  • Input ANRs are detected by native InputDispatcher; broadcast/service ANRs by AMS.
  • On ANR the system sends SIGQUIT and writes stacks to /data/anr/anr_* (formerly traces.txt); read the main thread first.
  • Main causes of ANRs: main-thread I/O, lock contention/deadlock, slow synchronous Binder calls, system starvation.
  • The Watchdog checks key system_server threads and monitor locks every 30 s; blocked 60 s means it kills system_server.
  • PMS parses manifests, verifies signatures, assigns UID = userId * 100000 + appId, and uses installd/artd for files and dexopt.
  • Permission levels: normal, dangerous (runtime), signature, privileged (allowlisted), special/appop.
  • Services check callers with Binder.getCallingUid(); wrap system work in clearCallingIdentity()/restoreCallingIdentity().
  • Sandbox = per-app UID (DAC) plus SELinux domains (MAC); avc: denied logs show policy blocks.
  • Treble (8.0) separates system and vendor via stable HAL interfaces; HIDL (hwbinder) is legacy, stable AIDL (binder) is current.
  • Binder domains: /dev/binder (framework, AIDL HALs), /dev/hwbinder (HIDL), /dev/vndbinder (vendor to vendor).
  • VINTF manifests and compatibility matrices must match; lazy HAL declared in VINTF without an init.rc entry gives a null proxy forever.
  • Key dumps: activity, window, input, SurfaceFlinger, gfxinfo, package, power, batterystats, meminfo; Perfetto for timelines.
  • Launch modes: standard, singleTop, singleTask, singleInstance (plus singleInstancePerTask); Intent flags can override; taskAffinity names the preferred task.
  • Application context has no window token; dialogs and addView need an Activity token or you get BadTokenException.
  • Background timeline: 8 implicit-broadcast / background-service limits; 12 FGS-from-background and PendingIntent mutability; 13 notification permission; 14 FGS types.
  • PendingIntent on apps targeting 12+ must set FLAG_IMMUTABLE or FLAG_MUTABLE; prefer immutable.
  • Adding a system service: AIDL, SystemService, publishBinderService, SystemServer start site, manager + SystemServiceRegistry, SELinux service_contexts, permission checks.
  • Looper.loop() sleeps in epoll_wait when idle; the infinite loop is not an ANR. ANR is a missed deadline on work the system handed you.
  • onTrimMemory is the real cache-trim callback (with levels); onLowMemory is a legacy last-ditch hook and is not guaranteed before a kill.
  • ApplicationExitInfo (Android 11+) is the first place to look for why a process died (LMK, ANR, crash, dependency, self-exit).
Mobile & Embedded Systems / Android Platform

Binder IPC & AIDL

Read the full topic →

  • Apps and services live in separate processes with separate address spaces, so calls between them need IPC; on Android that is Binder.
  • Binder over sockets/pipes: one copy, kernel-stamped caller UID/PID, object handles as capabilities, reference counting and death notifications.
  • Binder is a kernel driver used through open, mmap and ioctl(BINDER_WRITE_READ) with BC_* commands and BR_* returns.
  • Three domains: /dev/binder (servicemanager: framework, apps, AIDL HALs), /dev/hwbinder (hwservicemanager: HIDL), /dev/vndbinder (vndservicemanager: vendor to vendor).
  • Handles are per-process integers referring to nodes in other processes; handle 0 is the context manager.
  • One copy: the driver copies the sender's data straight into pages mapped read-only into the receiver.
  • The receive buffer is about 1 MB per process, shared by all in-flight transactions; oneway may use half.
  • Incoming calls run on Binder pool threads: 15 extra by default (16 total), 31 in system_server; BR_SPAWN_LOOPER grows the pool.
  • Pool exhaustion: all threads stuck in slow calls, new callers block and may ANR.
  • Recursive callbacks go to the waiting thread; the driver passes caller priority to the server thread.
  • .aidl generates an interface, Stub (server, onTransact) and Stub.Proxy (client, transact).
  • asInterface() returns the local object in the same process, otherwise a Proxy.
  • Method codes start at FIRST_CALL_TRANSACTION in declaration order; the interface token is checked with enforceInterface.
  • Direction tags in, out, inout; prefer in.
  • Parcel is a sequential, order-dependent IPC container; not for persistence.
  • Binder objects and fds inside a Parcel are translated by the driver into receiver-local handles and fds.
  • Parcelable is fast hand-written/generated marshalling; Serializable uses reflection and is slow.
  • oneway: returns once queued, no reply or exceptions, ordered per object, calling PID is 0.
  • servicemanager: addService / getService / waitForService, SELinux checks, VINTF checks for HALs, lazy start via init.
  • Apps expose Binders via bound services; AMS brokers bindService and links priorities.
  • linkToDeath gives binderDied(); calls to a dead process throw DeadObjectException.
  • Binder.getCallingUid() for permission checks; clearCallingIdentity() / restoreCallingIdentity() in finally.
  • SELinux checks binder_call, binder_transfer, and service_manager add/find.
  • Radio HAL: oneway request interface with serials, Response interface for solicited replies, Indication interface for unsolicited events.
  • RIL registers callbacks with setResponseFunctions, tracks requests by serial, holds a wakelock while pending, fails pending requests on serviceDied.
  • An outgoing call crosses Dialer → system_server (Telecom) → com.android.phone → vendor radio daemon → modem.
  • Stable AIDL (@VintfStability) replaced HIDL; versions are frozen snapshots and only append changes are allowed.
  • Clients use getInterfaceVersion(); calling a missing method gives UNKNOWN_TRANSACTION.
  • TransactionTooLargeException: payload or concurrent usage exceeded the buffer; send IDs, paginate or use shared memory.
  • Debug with /sys/kernel/debug/binder/*, Perfetto Binder tracks, am trace-ipc, stack dumps and logcat failures.
  • Slow Binder calls are almost always slow server work or lock contention, not driver overhead.
  • Frozen target: sync fails with BR_FROZEN_REPLY and does not unfreeze; oneway is queued. AMS thaws on importance. Freezer was opt-in in 11, default-on later.
  • Messenger is Binder with a Handler and Messages (serialized on one thread). AIDL is typed RPC. ContentProvider is for data/URI grants. Intent is one-shot.
Mobile & Embedded Systems / Android Platform

Trace a Path Through the Android Stack

Read the full topic →

  • The layer cake: app, framework (system_server), native daemons and HALs, kernel, hardware and firmware.
  • App to framework uses Binder; framework to vendor uses stable AIDL or HIDL HALs checked by VINTF; user space to kernel uses syscalls, ioctl and interrupts.
  • Voice call: Dialer, Telecom, TelephonyConnectionService, ImsPhone and ImsService, RIL, Radio HAL, vendor RIL process, modem, IMS core.
  • On modern Qualcomm SoCs, QMI rides QRTR rather than only shared-memory SMD.
  • VoLTE has two planes: SIP signalling through the IMS core, and RTP media on a dedicated QoS bearer (QCI 1 or 5QI 1).
  • If IMS is unavailable, calls fall back to circuit-switched (CSFB) or, on 5G, EPS fallback to LTE.
  • Mobile data has a setup phase (setupDataCall through RIL, PDN or PDU session, IP assigned) and a transfer phase.
  • Uplink packet: app socket, bionic, kernel TCP/IP, eBPF/netfilter, fwmark routing, optional CLAT, rmnet/QMAP, IPA, modem, RAN, P-GW/UPF, Internet.
  • Downlink is not a mirror of interrupts: IPA/NAPI batch packets so the AP sees aggregated IRQs, then eBPF, TCP, socket, app recv.
  • netd programs routes, eBPF/firewall, DNS and fwmark; ConnectivityService picks and validates networks.
  • Vendor data-path offload (IPA on Qualcomm) does routing, filtering, NAT and aggregation in hardware so the AP can sleep during transfers.
  • Sensor data is sampled and batched on the always-on sensor hub; the AP wakes only when the FIFO fills or report latency expires.
  • Sensors HAL 2.x and AIDL deliver events through a Fast Message Queue, not a Binder call per event.
  • High-rate, unbatched sensor use is a classic wearable battery bug.
  • Input: IRQ, kernel driver, evdev, EventHub, InputReader, InputDispatcher, InputChannel socket, app main thread, View.
  • An unacknowledged input event for about 5 seconds triggers an input dispatch ANR.
  • Audio: app track, AudioFlinger in audioserver, AudioPolicyService, Audio HAL, DSP or codec; prefer compressed offload or AAudio MMAP when the AP should sleep.
  • Camera: CameraX/Camera2, CameraService, HAL3 request/result, ISP; preview Surface to SurfaceFlinger, still to ImageReader.
  • Location: fused provider, LocationManagerService, GNSS HAL (batched) and network location; interval and displacement decide power.
  • Frame: invalidate, Choreographer on vsync, RenderThread and GPU, BufferQueue / BLAST, SurfaceFlinger, HWC, display.
  • BufferQueue: app is the producer, SurfaceFlinger the consumer; buffers are passed by handle with fences. BLASTBufferQueue is the usual Android 10+ path.
  • Frame budget is about 16.6 ms at 60 Hz, 11.1 ms at 90 Hz, 8.3 ms at 120 Hz; missing it drops a frame.
  • HWC chooses device composition (overlay planes) or client composition (GPU), which affects power.
  • Binder: Parcel, ioctl, single copy into the receiver's mmap'd buffer, thread pool, onTransact, reply.
  • Binder transaction buffer is about 1 MB per process; oneway calls do not block the caller.
  • Cold start: AMS, Zygote fork (copy-on-write), ActivityThread, bindApplication, Activity lifecycle, first frame.
  • TTID is time to initial display; TTFD is time to full display, marked by reportFullyDrawn().
  • Data Layer: DataItems are persistent and synced; messages are fire-and-forget; channels stream large data.
  • Data Layer transport falls back automatically between Bluetooth, Wi-Fi and cloud or LTE.
  • A/B OTA: update_engine writes the inactive slot, boot control HAL switches slots, rollback if the new slot fails.
  • Virtual A/B stores only a copy-on-write snapshot for dynamic partitions and merges it after a successful boot.
  • AVB verifies vbmeta at boot; dm-verity verifies partition blocks at read time.
  • For any broken flow: name it, reproduce, tap each layer, bisect, hand off with evidence.
Mobile & Embedded Systems / Telephony & Wireless

Android Telephony, RIL & Modem

Read the full topic →

  • Stack: apps → Telecom (system_server) + Telephony (com.android.phone) → RIL.java → Radio HAL → vendor daemon → modem → Uu air interface.
  • Telecom routes calls with PhoneAccount/ConnectionService/InCallService; Telephony owns the radio and plugs in as TelephonyConnectionService.
  • PhoneFactory creates one RIL and one GsmCdmaPhone per slot; ImsPhone handles IMS calls when registered.
  • ServiceStateTracker reacts to networkStateChanged by polling operator, voice reg, data reg and selection mode, then notifies via TelephonyRegistry.
  • DataNetworkController (Android 13+) replaced DcTracker/DataConnection/ApnContext; DataNetwork = one connection, DataProfile = APN.
  • SubscriptionManagerService (Android 14+) replaced SubscriptionController and owns DDS and default voice/SMS subscriptions.
  • UICC tree: UiccController → UiccSlot → UiccCard/UiccPort → UiccProfile → applications (USIM, ISIM) → records.
  • CarrierConfigLoader merges platform defaults, carrier-config app values by carrier ID and carrier-app overrides; broadcasts ACTION_CARRIER_CONFIG_CHANGED.
  • Solicited = request with serial + response callback + wakelock; unsolicited = indication fanned out through RegistrantList.
  • RILJ log: [serial]> request, [serial]< response, [UNSL]< indication; a missing response means the problem is below RIL.
  • RIL holds the *telephony-radio* partial wakelock with its own count and a timeout; an ack wakelock covers the acknowledgement protocol.
  • On HAL death, RIL fails every pending request with RADIO_NOT_AVAILABLE, resets proxies and reconnects when the service returns.
  • serviceDied (vendor process died) is not the same as modem SSR / md1 exception (baseband crashed); timestamps decide cause and effect.
  • Radio HAL: HIDL radio@1.0–1.6 monolithic IRadio → stable AIDL (Android 13) split into Network, Data, Voice, Sim, Modem, Messaging, Ims, Config.
  • Each HAL domain has request, Response and Indication interfaces; setResponseFunctions hands callback binders to the vendor.
  • HAL discovery: VINTF declaration → init starts daemon → registers with servicemanager → RIL waitForDeclaredService; declared-but-not-started gives a permanent null proxy.
  • Qualcomm: qcrild → QMI (NAS, WDS, VOICE, UIM, WMS, DMS, IMSA...) over QRTR on GLINK/SMEM or MHI/PCIe.
  • MediaTek: rild (Rfx/Rmm) → MIPC/AT over CCCI; data on ccmni; crashes are md1 exceptions.
  • AT commands (TS 27.007/27.005): +CFUN, +COPS, +CEREG, +CGDCONT, +CGACT, ATD.
  • NAS (EMM/ESM, 5GMM/5GSM) talks to the core; RRC, PDCP, RLC, MAC, PHY talk to the base station; SDAP maps QoS flows in 5G.
  • PDCP = ciphering, integrity, ROHC; RLC = segmentation + ARQ; MAC = scheduling + HARQ; PHY = coding, modulation, MIMO.
  • RRC states: IDLE, CONNECTED, and INACTIVE on NR; RLF triggers include T310 expiry, max RLC retransmissions, random-access failure.
  • LTE attach creates a default bearer (always-on IP); 5G registration and PDU session establishment are separate steps.
  • Authentication is AKA using the SIM's secret key; 5G hides the IMSI as SUCI.
  • Control plane sets up the PDN/PDU and returns ifname, IP, DNS, MTU, P-CSCF; user plane uses rmnet_data* + QMAP + IPA offload so the CPU can sleep.
  • Android routes per network with fwmark and per-network routing tables; IPv6-only networks use clatd (464XLAT).
  • The IMS PDN stays up with mobile data off; QCI 5 carries SIP, QCI 1 carries voice (QCI 1 is typically started from the SDP answer around 183, not strictly at PRACK).
  • DSDS shares one RF chain; DSDA runs both SIMs concurrently; PhoneSwitcher + setPreferredDataModem implement data switching.
  • eSIM: eUICC (EID) + ISD-R + LPA (EuiccService) + SM-DP+; Android 13+ supports multiple enabled profiles.
  • Debug order: chipset/log family → RF environment → modem health → cause codes at timestamp → framework evidence → hypothesis matrix.
  • Key causes: EMM #11 PLMN not allowed (adds EF_FPLMN), EMM #13/#15 forbid a tracking area (not the whole PLMN), ESM #27 unknown APN, ESM #33 not subscribed, SIP 403 forbidden, SIP 488 codec mismatch.
  • EMM-REGISTERED is not the same as ECM-CONNECTED: a camped LTE phone is usually registered and idle until paging or a Service Request (T3417).
  • Typical 3GPP NAS timer defaults: T3410/T3430 15 s, T3411 10 s, T3412/T3512 54 min, T3417 5 s, T3402 12 min (network can override).
  • AKA MAC failure means the AUTN is not for this USIM; sync failure (AUTS) is an SQN desync the HSS can recover.
  • X2 handover: source prepares target, UE gets RRC handover command, target path-switches through the MME, source releases.
  • Carrier privileges come from the UICC access-rule applet, checked by hasCarrierPrivileges(); they are not ordinary app permissions.
  • Voice can be OOS while data is in service on LTE-only without IMS; read both ServiceState domains plus IMS registration.
  • Back-off timers T3346 (mobility congestion), T3396 (per APN) and T3402 (after repeated failures) must be honoured.
  • Big telephony dump: dumpsys activity service com.android.phone/.TelephonyDebugService; also telephony.registry, isub, carrier_config.
Mobile & Embedded Systems / Telephony & Wireless

Android Data Call: Control, netd, eBPF & Packets

Read the full topic →

  • Control plane leases the PDN/PDU; data plane is every packet after the iface is configured.
  • Camped + registered is not RRC connected; idle uplink needs Service Request.
  • LTE Attach creates a default EPS bearer; 5G Registration does not create a PDU session by itself.
  • Default bearer is always-on IP; dedicated bearers are extra QoS (IMS voice), not how Chrome gets HTTPS.
  • APN (LTE) = DNN (5G). Types: default, ims, mms, dun, fota, emergency, hipri, plus carrier extras.
  • ESM PDN Connectivity / Activate Default EPS Bearer vs 5GSM PDU Session Establishment.
  • GTP-U carries user packets in the core; GTP-C/PFCP builds sessions. The UE speaks NAS, not GTP.
  • DcTracker (pre-13) vs DataNetworkController + DataNetwork + DataRetryManager + AccessNetworksManager.
  • DataService is the pluggable setup API; IWLAN is a DataService, not "just Wi-Fi".
  • Gates: SIM, PS service, data enabled, roaming, DDS, restricted caps, throttle/T3396.
  • Mobile data off must not tear down the IMS data call.
  • IRadioData.setupDataCall / deactivateDataCall; unsolicited dataCallListChanged.
  • SetupDataCallResult: cause, cid, ifname, addresses, dnses, gateways, mtu(V4/V6), pduSessionId, trafficDescriptors.
  • Vendor maps HAL to QMI WDS (Qualcomm example) or the SoC equivalent — do not invent TLVs.
  • ifname comes from the vendor/kernel, not from Java constructing a string.
  • rmnet_data / ccmni / generic wwan: one netdev per data call; QMAP mux+agg is a Qualcomm-example header.
  • IPA (Qualcomm example) offloads route/filter/NAT/aggregation; other SoCs have equivalents.
  • netd applies addresses, rules, routes, firewall/eBPF; Java uses INetd AIDL, not Runtime.exec("ip").
  • ndc is a remnant text interface; still useful to read, not the modern control path.
  • DnsResolver owns stub DNS, Private DNS (DoT), and often DNS64 synthesis.
  • ConnectivityService: NetworkAgent, capabilities, score, validation, VPN overlay.
  • VALIDATED is not the same as "iface has an IP".
  • bindProcessToNetwork vs Network.bindSocket vs default Network.
  • Restricted Networks (IMS/MMS) never become the ordinary app default.
  • eBPF: bpfloader, cgroup skb in/out, UID firewall, stats, tethering, CLAT — check current AOSP names.
  • TrafficStats API stayed; xt_qtaguid is history; counters live in eBPF maps now.
  • Doze / App Standby / Data Saver are netd+eBPF policy, not NAS.
  • fwmark + ip rule select the per-Network routing table; dump table all, not only main.
  • VPN protect mark keeps tunnel sockets off the TUN table.
  • OEM reserved UID ranges can have extra policy rules; quote android_filesystem_config.h for the build.
  • Uplink: OkHttp → ART → bionic → tcp_sendmsg → skb → nft/eBPF → route → CLAT? → rmnet → offload → PDCP… → GTP-U → NAT → server.
  • TCP SYN uses the same path as later data; QUIC uses udp_sendmsg.
  • MSS = MTU − IP − TCP (40 v4, 60 v6, minus options).
  • Downlink: offload aggregation, NAPI, GRO, softirq, then socket queue / epoll — not per-packet hardirq.
  • IRQ rate ≈ pps / aggregation depth when offload is healthy.
  • 464XLAT = CLAT on the UE + PLAT (NAT64) in the network; DNS64 synthesises AAAA from A.
  • IPv4 literals on v6-only need CLAT even if DNS64 exists.
  • Private DNS (DoT) can fail validation independently of the bearer.
  • Tethering may require a DUN APN; hardware may forward without the AP stack.
  • Chatty sockets kill modem DRX and AP suspend; batch and multiplex.
  • Bisect no-internet: CID/IP → rules/routes → validation → DNS → UID → CLAT → tcpdump vs modem → offload.
  • MTU blackhole: handshake works, large segments die; ICMP often filtered on cellular.
  • Shell ping ≠ app UID. Always ask which UID you just tested.
Mobile & Embedded Systems / Telephony & Wireless

CS & IMS Call Flows

Read the full topic →

  • CS voice uses a dedicated circuit in the 2G/3G CS domain; IMS voice is a SIP session with RTP media over an IP bearer.
  • VoLTE, VoNR and VoWiFi are all IMS; only the access network differs (LTE, NR SA, Wi-Fi via ePDG).
  • Shared top path: Dialer → TelecomManager.placeCall() → TelecomServiceImpl → CallsManager → ConnectionServiceWrapper → TelephonyConnectionService → Phone.dial().
  • Three processes: Dialer app, system_server (Telecom), com.android.phone (Telephony); plus the vendor rild and ImsService processes.
  • Telecom → phone process is IConnectionService, not ITelephony; PhoneInterfaceManager is not on the dial path.
  • The CS/IMS fork is GsmCdmaPhone.dial() using useImsForCall() (IMS enabled, VoLTE/WFC on, IMS in service).
  • CS path: GsmCdmaCallTracker → GsmCdmaConnection → RIL.dial() → IRadioVoice.dial() → rild → modem.
  • CS over the air: CM SERVICE REQUEST → SETUP → CALL PROCEEDING → ALERTING → CONNECT → CONNECT ACK; release is DISCONNECT → RELEASE → RELEASE COMPLETE.
  • callStateChanged carries no details; the CS tracker polls getCurrentCalls() and diffs DriverCalls in handlePollCalls().
  • dialResponse success means the modem accepted the request, not that the call connected.
  • IMS path: ImsPhone → ImsPhoneCallTracker → ImsCall → IImsCallSession → vendor MmTelFeature → SIP INVITE.
  • IMS MO ladder: INVITE → 100 → 183 (SDP answer) → PRACK → UPDATE → 180 → 200 OK → ACK → RTP.
  • SIP rides QCI 5 / 5QI 5; voice RTP rides a network-initiated GBR QCI 1 / 5QI 1 bearer, typically started from the SDP answer around 183, not strictly at PRACK.
  • Preconditions reserve the voice bearer before alerting so the callee never answers into silence.
  • The VoLTE dial does not use IRadioVoice.dial; it goes through the ImsService and vendor IMS path.
  • ImsResolver binds the vendor ImsService; ImsManager is the framework handle; ImsRegistrationImplBase reports registration.
  • State returns via TelephonyConnection.setDialing()/setActive() → Telecom → InCallService.onCallAdded/onStateChanged.
  • Telecom has no alerting state; ALERTING still shows as STATE_DIALING.
  • MT: tracker creates INCOMING connection → notifyNewRingingConnection() → PstnIncomingCallNotifier → addNewIncomingCall() → Telecom → onCreateIncomingConnection() → ring UI.
  • Answer: CS RIL.acceptCall() → CC CONNECT; IMS ImsCall.accept() → SIP 200 OK → ACK.
  • Radio CSFB (ESR, redirect to 2G/3G) is invisible to AOSP; framework IMS → CS fallback uses Phone.CS_FALLBACK and silent redial.
  • Synchronous fallback: ImsPhone throws CallStateException(CS_FALLBACK), caught in GsmCdmaPhone.dial().
  • Asynchronous fallback: onCallStartFailed(CODE_LOCAL_CALL_CS_RETRY_REQUIRED) → initiateSilentRedial(), posted to the main executor.
  • Android 14+ domain selection service centralises the CS vs PS choice for normal and emergency calls.
  • Voice can be OOS while LTE data is in service if there is no CS domain and IMS is not registered; useImsForCall() then fails and a CS dial fails too.
  • Emergency: EmergencyNumberTracker, radio-on helper, any SIM or no SIM, emergencyDial (CS EMERGENCY SETUP) or IMS urn:service:sos, then ECBM.
  • MTK vs QCOM diverge only below the HAL: rild + MIPC/AT + MD1 vs qcrild + QMI + MPSS; ImsService com.mediatek.ims vs org.codeaurora.ims.
  • Debug order: path, last successful step, cause at the timestamp, modem health, correlation.
  • Tools: logcat -b radio, dumpsys telecom, dumpsys telephony.registry, QXDM/QCAT or ELT, Wireshark for SIP/RTP.
  • SIP error + healthy modem means IMS/operator layer; RADIO_NOT_AVAILABLE after serviceDied means the HAL or modem went away.
Mobile & Embedded Systems / Telephony & Wireless

IMS, VoLTE & VoWiFi

Read the full topic →

  • IMS is the SIP-based service core that separates access (LTE, NR, Wi-Fi), transport (EPC/5GC) and service control.
  • P-CSCF: the UE's only SIP peer, IPsec endpoint, identity asserter, and the node that asks PCRF/PCF for the voice bearer.
  • I-CSCF: home-network entry; uses Cx UAR (registration) and LIR (terminating calls) to find the S-CSCF.
  • S-CSCF: registrar, authenticates with MAR, downloads the profile with SAR, runs iFC to invoke application servers.
  • HSS (UDM in 5G) stores IMPI/IMPU, AKA vectors, iFC, S-CSCF name, STN-SR and C-MSISDN.
  • TAS runs MMTel supplementary services; MRF handles announcements and conference mixing; BGCF/MGCF/IMS-MGW break out to the PSTN; IBCF guards interconnects.
  • IMS runs on a separate APN (ims): default bearer QCI 5 for SIP, dedicated GBR QCI 1 for voice, QCI 2 for video.
  • The P-CSCF address comes from the PCO (cellular) or the IKEv2 configuration payload (Wi-Fi).
  • Registration: REGISTER, 401 with AKA challenge, IPsec SA, protected REGISTER, 200 OK, third-party REGISTER, SUBSCRIBE reg event.
  • The IPsec SA is set up after the 401 using CK/IK; the P-CSCF strips CK/IK from the 401 before forwarding.
  • SQN out of range leads to a sync failure with AUTS and a fresh challenge; a MAC failure means the UE rejected the network.
  • MO ladder: INVITE, 100, 183 (SDP answer), PRACK, 200, bearer reserved, UPDATE, 200, 180, 200 OK, ACK, RTP, BYE, 200.
  • Preconditions stop the phone ringing before both sides have a QCI 1 bearer; failure gives 580 Precondition Failure.
  • PRACK makes a provisional response reliable (100rel, RSeq/RAck); UPDATE changes session state before answer.
  • MT differs by paging first and by the UE sending 183, then 180 after resources are ready, then 200 on answer.
  • Codecs: AMR-NB (narrowband), AMR-WB (HD Voice, mandatory for VoLTE), EVS (super-wideband/fullband, channel-aware mode).
  • CANCEL aborts an unanswered INVITE (callee returns 487); BYE ends an established dialog.
  • Hold is a re-INVITE with sendonly or inactive; resume uses sendrecv.
  • Call forwarding and barring are configured over Ut using XCAP (HTTP), often authenticated with GBA; the TAS enforces them.
  • Conference: INVITE to the conference factory URI, then REFER each party; transfer uses REFER.
  • SRVCC moves an active IMS call to 2G/3G CS via MME, Sv, MSC server and SCC-AS using STN-SR and C-MSISDN.
  • eSRVCC anchors media at the ATCF/ATGW in the serving network, cutting the voice gap to under about 300 ms.
  • aSRVCC covers the alerting phase, bSRVCC the pre-alerting phase, vSRVCC video, rSRVCC CS back to LTE.
  • VoWiFi: IKEv2/IPsec tunnel over SWu to the ePDG (EAP-AKA), S2b to the P-GW, then the same IMS; N3IWF in 5G.
  • LTE to Wi-Fi handover keeps the call because the P-GW keeps the same IMS IP when the request is handover type.
  • VoWiFi has no GBR bearer; it relies on DSCP EF and Wi-Fi WMM marking.
  • Emergency: emergency PDN, emergency registration, urn:service:sos, E-CSCF, LRF location, PSAP; 380 tells the UE to redial as emergency.
  • ViLTE adds an m=video line and a QCI 2 bearer; upgrade and downgrade are re-INVITEs.
  • RCS chat uses MSRP set up by SIP INVITE; capability discovery uses OPTIONS or presence; file transfer uses HTTP.
  • Transaction = request plus responses (Via branch + CSeq); dialog = Call-ID + From tag + To tag.
  • IMS T1 is 2 s on the UE, so Timer B and F are 128 s; large SIP requests should go over TCP.
  • PCC chain: P-CSCF Rx/N5 to PCRF/PCF, Gx/N7 to P-GW/SMF, dedicated bearer or QoS flow down to the eNB/gNB.
  • QCI 1 is conversational voice (GBR, 100 ms, 10-2 loss); QCI 5 is IMS signalling on the default bearer, with high scheduling priority relative to other default bearers such as QCI 9; ARP controls admission and pre-emption only.
  • On Android, the vendor ImsService provides MmTelFeature and RcsFeature; ImsPhone handles IMS calls in the framework.
  • G.114 mouth-to-ear target is about 150 ms; the QCI 1 PDB is only the network slice of that path.
  • Jitter buffer trades delay for smoothness; PLC hides isolated loss; MOS/E-model scores listening quality.
  • CMR (in-band) and ANBR (RAN bitrate recommendation) step the codec down at the cell edge.
  • VoLTE radio: ROHC, SPS, TTI bundling, C-DRX aligned to 20 ms, RLC UM (no ARQ stall).
  • VoNR keeps the same SIP ladder; QoS becomes a 5QI 1 flow, policy is N5/N7, coverage exit is EPS fallback or VoNR-to-VoLTE HO.
  • Voice-centric UE leaves a RAT that cannot provide voice; data-centric stays. Usage setting is not the IMS VoPS bit.
  • S8HR is the common IMS roaming model (home P-GW and P-CSCF). LBO is specified but not "the" GSMA model. S8HR costs: emergency, LI, no local ATCF, trombone delay.
Mobile & Embedded Systems / Telephony & Wireless

5G NR & 5G Core

Read the full topic →

  • 5G targets three families: eMBB (throughput), URLLC (about 1 ms, 99.999%), mMTC (1 million devices per km²).
  • NSA Option 3/3a/3x uses the EPC with an LTE anchor and NR as secondary via EN-DC; SA Option 2 uses the 5GC with NR only.
  • In EN-DC the eNB is Master Node (MCG) and the en-gNB is Secondary Node (SCG), linked by X2.
  • Option 3 splits at the eNB PDCP, 3a splits in the core, 3x splits at the gNB PDCP (most common).
  • NR addition in NSA: LTE attach, B1 report, SgNB Addition over X2, RRC reconfiguration with NR config, RACH on PSCell, E-RAB modification.
  • An SCG failure keeps LTE up; the UE sends SCGFailureInformationNR.
  • 5GC NFs: AMF mobility and NAS, SMF sessions and UPF control, UPF user plane, AUSF auth, UDM subscription, PCF policy, NRF discovery, NSSF slice selection, NEF exposure.
  • Key interfaces: N1 NAS, N2 NGAP, N3 GTP-U, N4 PFCP, N6 to data network, N9 UPF to UPF, N11 AMF to SMF, N16 V-SMF to H-SMF, N26 AMF to MME, Xn gNB to gNB.
  • SBI is HTTP/2 + JSON with NRF-based discovery; CUPS lets UPFs sit at the edge.
  • Registration (AMF) replaces Attach and does not create a data session; PDU Session Establishment (SMF) does.
  • Registration types: initial, mobility update, periodic (T3512), emergency.
  • Registration Accept carries 5G-GUTI, TAI list, Allowed NSSAI and the IMS voice over PS indicator.
  • PDU session: DNN, S-NSSAI, type (IPv4/IPv6/IPv4v6/Ethernet/Unstructured), SSC mode 1/2/3.
  • SUCI hides the SUPI with ECIES and the home network public key; the SIDF in the UDM recovers it.
  • 5G-AKA lets the home AUSF verify RES*; user-plane integrity protection is new in 5G.
  • QoS flows (QFI) live inside one PDU session tunnel; SDAP maps flows to DRBs.
  • 5QI 1 GBR conversational voice, 5QI 2 video, 5QI 5 IMS signalling (not voice), 5QI 9 default internet, 82 to 86 delay-critical GBR. Do not invert 1 and 5.
  • ARP is admission and pre-emption priority, not packet treatment.
  • SCS = 15 × 2µ kHz; slots per subframe = 2µ; 14 symbols per slot.
  • FR1 is 410 MHz to 7.125 GHz (100 MHz carriers); FR2-1 is 24.25 to 52.6 GHz (400 MHz carriers, beamforming); FR2-2 is 52.6 to 71 GHz in Release 17.
  • Up to 4 BWPs per direction per cell, one active; default BWP saves power.
  • SSB = PSS + SSS + PBCH over 4 symbols and 20 RBs; 1008 PCIs; up to 64 beams in FR2.
  • SSB-to-RACH mapping tells the gNB the UE's best beam; beam failure recovery is faster than RLF.
  • LDPC for data, Polar for control; no always-on CRS in NR.
  • DSS shares one LTE carrier with NR dynamically at the cost of overhead.
  • SDAP is new; PDCP adds duplication; RLC drops concatenation; HARQ is asynchronous both ways.
  • RRC_INACTIVE keeps context in UE and anchor gNB, core sees CM-CONNECTED, resume with I-RNTI, RAN paging in the RNA.
  • VoNR needs SA, IMS PDU session, IMS VoPS indicator, 5QI 1 flows, gNB voice support (ROHC, C-DRX, configured grant, PUSCH repetition, RLC UM).
  • Voice-centric UE leaves 5GS when IMS voice is not available; data-centric stays. Usage setting is not the VoPS bit.
  • IMS roaming on 5G is usually home-routed (V-SMF/H-SMF on N16, N9 between UPFs), the S8HR analogue; LBO is uncommon for the IMS DNN.
  • EPS fallback: gNB rejects the 5QI 1 flow, moves the UE to LTE by handover (N26) or redirection, call completes as VoLTE.
  • EPS fallback is at call setup and stays IMS; SRVCC moves an active call to CS.
  • S-NSSAI = SST (8 bits) + SD (24 bits); up to 8 in an NSSAI; URSP maps apps to slices.
  • Xn handover ends with a Path Switch; N2 handover goes through the AMF.
  • The TAI list is the registration area; leaving it triggers a mobility registration update.
  • On Android NSA the network type is LTE and the 5G icon is an override from NetworkTypeController and 5g_icon_configuration_string.
Mobile & Embedded Systems / Wearables & Platform Integration

Wear OS Platform

Read the full topic →

  • Wear OS is AOSP with a wearable framework layer; kernel, HALs, Binder, ART, SELinux, Treble and AVB are all the same as a phone.
  • The defining hardware trait is an AP plus an always-on low-power co-processor (sensor hub).
  • Battery life is roughly capacity divided by average current; average current is dominated by how much time the AP is suspended.
  • The co-processor runs step counting, heart rate, sensor fusion, gestures and AOD help so the AP can sleep.
  • Ambient mode is a dim, low-color, roughly once-a-minute face with the AP mostly asleep; burn-in protection shifts pixels.
  • Hybrid designs run a small RTOS on the co-processor for the face and notifications while Wear OS sleeps.
  • Wear OS 3 (2021, Android 11 base) was the Google and Samsung reset; Wear OS 4 (Android 13) added Watch Face Format; Wear OS 5 is on Android 14, 5.1 on Android 15, Wear OS 6 on Android 16.
  • Watch Face Push (Wear OS 5) installs a WFF package without a watch-side APK.
  • Tiles, complications and Watch Face Format faces are declarative and rendered by the system, so no app process stays alive.
  • A Tile is a swipeable glanceable card; a complication is a data slot on the face; a notification is an alert in the stream.
  • Tile freshness is throttled (many minutes); requestUpdate is rate-limited; use timeline entries for predictable changes.
  • Standalone apps set com.google.android.wearable.standalone true; they must not depend on MessageClient for core features.
  • All-day body sensors need BODY_SENSORS_BACKGROUND (API 33); foreground spot checks use BODY_SENSORS.
  • The Data Layer is a Play services API; AOSP-only builds need an OEM sync path.
  • RemoteActivityHelper starts an activity on the paired phone without a custom message protocol.
  • Complication data sources update on a long period or by push request; keep updates rare.
  • PPG measures heart rate and SpO2 optically; accelerometer and gyroscope handle steps, sleep and gestures.
  • Sensor path: silicon, hub firmware (fusion and FIFO batching), Sensors HAL, SensorService or Health Services, app.
  • Health Services clients: ExerciseClient (workouts), PassiveMonitoringClient (all-day), MeasureClient (spot checks).
  • High-rate raw sensor access without batching destroys battery; check dumpsys sensorservice.
  • Health Connect on the phone is the shared on-device store for health records.
  • DataClient syncs persistent DataItems (small, about 100 KB) and Assets; MessageClient is fire-and-forget; ChannelClient streams large data; CapabilityClient discovers nodes.
  • Phone and watch apps must share package name and signing certificate to use the Data Layer.
  • Transport preference: Bluetooth, then Wi-Fi, then cloud relay or LTE; sync must be idempotent and transport-agnostic.
  • LTE watches use an eUICC with profiles downloaded via GSMA RSP (LPA on device, SM-DP+ server).
  • Standalone calls use IMS: VoLTE (or VoNR), SIP signalling, RTP media on a QoS bearer.
  • Modem power is tamed with DRX, eDRX and power saving mode, plus a Bluetooth-first policy.
  • Boot chain is identical to a phone; co-processor firmware must be loaded and version-matched.
  • Resume latency and idle residency matter more than cold boot time on a watch.
  • Watch OTA uses A/B or virtual A/B and is gated on battery, charging and Wi-Fi, often overnight.
  • A wearable image adds sensor-hub firmware and power tuning to the usual framework plus BSP plus HALs.
  • Power, stability and health accuracy are release gates, checked on every integration drop.
  • Never measure battery with USB attached; use a lab power monitor or pure battery runs.
  • Watch-phone bugs need logs from both devices plus Bluetooth HCI snoop on a shared timeline.
Mobile & Embedded Systems / Wearables & Platform Integration

Power, Thermal & Battery

Read the full topic →

  • Battery life is roughly capacity divided by average current; average current is dominated by suspend residency.
  • Suspend-to-RAM freezes tasks, suspends devices, offlines CPUs and puts DDR in self-refresh; only wake interrupts resume it.
  • One driver's failing suspend callback aborts the whole suspend; check suspend_stats and last_failed_dev.
  • Modern Android triggers suspend from the SystemSuspend service using the wakeup_count handshake and /sys/power/state.
  • Suspend blockers were Android's original (2009) mechanism. Mainline wakeup sources arrived in kernel 2.6.37 (2010); autosleep and /sys/power/wake_lock followed in 3.5 (2012).
  • cpuidle is per-CPU idling between tasks; system suspend is the whole device sleeping.
  • DVFS scales frequency and voltage together; dynamic power is roughly C·V2·f.
  • cpufreq governors: historic Android interactive, modern default schedutil (frequency from scheduler utilization).
  • cpuidle governors (menu, TEO) pick a C-state by predicted idle time versus target residency.
  • EAS places tasks using a per-CPU energy model; it yields to ordinary load balancing when the system is overutilized.
  • Android steers EAS with uclamp (min boosts, max caps) and cpusets (top-app versus background).
  • The Power HAL (AIDL from Android 11) implements short setBoost hints (interaction, launch) and longer setMode (interactive, low power, sustained performance).
  • ADPF (Android 12+) hint sessions close the loop: target work duration versus actual, plus thermal headroom so apps back off before throttle.
  • A partial wakelock keeps the CPU on with the screen off; it is the number one invisible drain.
  • Always acquire wakelocks with a timeout and release in a finally block; tag them as app:component.
  • Kernel wakeup sources are visible in /sys/kernel/debug/wakeup_sources; app wakelocks in dumpsys power.
  • Light Doze starts soon after screen off on battery; deep Doze requires the device to be stationary.
  • In deep Doze, network, jobs, syncs, standard alarms and Wi-Fi scans are deferred and app wakelocks are ignored.
  • Exact allow-while-idle alarms, alarm-clock alarms, high-priority FCM and allowlisted apps get through Doze.
  • App Standby buckets: active, working set, frequent, rare, restricted (Restricted added in Android 11, API 30).
  • Android 12 adds app hibernation and cached-app freezer; Android 13 adds Low Power Standby for long unused periods.
  • Allow-while-idle exact alarms are rate-limited to roughly one every nine minutes while idle.
  • WorkManager is the default for deferrable guaranteed work; periodic minimum is 15 minutes.
  • Exact alarms wake the AP and need special permission; use them only for user-visible times.
  • Handler.postDelayed uses uptime, which stops during suspend.
  • The sensor hub runs always-on sensing with FIFO batching so the AP stays suspended.
  • Other offloads: CHRE nanoapps, audio DSP hotword, Wi-Fi packet filtering, BLE scan filtering, GNSS batching, modem DRX.
  • Skin temperature, not junction temperature, is usually the binding thermal limit on phones and watches.
  • Thermal zones have trip points; cooling devices (cpufreq, GPU, charge current, modem) are applied by governors.
  • The thermal HAL reports severities from NONE to SHUTDOWN; apps use thermal status listeners and headroom.
  • Heat increases leakage, which increases power, which increases heat.
  • batterystats uses Power Stats HAL rails when present, otherwise a device power_profile.xml model.
  • Fuel gauges combine voltage models and coulomb counting, corrected for temperature and aging.
  • Lithium-ion charging is constant current then constant voltage; JEITA zones limit charging when hot or cold.
  • Voltage droop under load plus an aging cell causes shutdowns at non-zero percent.
  • Battery Historian turns a bugreport into a drain timeline; run it locally and keep bugreports internal.
  • Perfetto with power rails shows energy per rail alongside scheduling, frequency and suspend events.
  • Never measure power with USB attached; use a power monitor or pure battery runs in a controlled setup.
  • Triage: quantify, check suspend, find the waker, attribute, bisect, fix, re-measure, gate.
  • Gate each integration drop on power, thermal, stability, performance and functionality KPIs against a baseline and noise band.
Mobile & Embedded Systems / Wearables & Platform Integration

Platform Integration & Release

Read the full topic →

  • HLOS is the high-level OS on the application processor: Linux kernel plus Android or Wear OS user space.
  • Non-HLOS is firmware on other processors: modem (MPSS), ADSP and CDSP, sensor hub (SLPI), TrustZone, XBL boot loaders, power management processor.
  • The HLOS image is AOSP framework plus vendor BSP (kernel, device tree, drivers), vendor HALs, proprietary libraries and OEM customisation.
  • A meta build pins one tested version of every HLOS and non-HLOS component into a flashable release.
  • HLOS and non-HLOS meet at HALs, remote-processor drivers, shared memory and protocols such as QMI.
  • AOSP source is published on android.googlesource.com; public tag cadence has changed over the years — check current release notes. The monthly security bulletin is a separate patch list.
  • Upstream integration lands each AOSP tag onto vendor trees and per-chipset baselines while carrying vendor deltas forward.
  • Treble, VINTF, stable AIDL HALs and GKI are what make frequent upstream integration feasible.
  • Within the same frozen KMI, GKI kernel updates do not require vendor modules to rebuild; a new ACK/KMI does.
  • Soong + Android.bp is the primary userspace build; Make still composes products (PRODUCT_PACKAGES). Userspace Bazel migration was halted ~2023; Kleaf/Bazel is for kernel builds.
  • lunch selects product-release-variant; user ships, userdebug is the usual debug image. A built module is not on the image unless it is in PRODUCT_PACKAGES.
  • Common conflict sources: HAL version bumps, framework API changes, SELinux policy, build system changes, kernel branch (KMI) moves, toolchain updates.
  • Upstream-first reduces the carried delta and the cost of every future drop.
  • Branching models: trunk-based with feature flags, release branches, feature branches, per-SoC and per-OEM branches.
  • Freeze stages: feature freeze, code freeze, release candidate, golden build.
  • Merge preserves history and SHAs; rebase keeps a clean, linear patch stack but rewrites SHAs.
  • Cherry-pick ports individual fixes; use -x to record the source commit.
  • Semantic conflicts apply cleanly but are wrong; only builds, tests and informed review catch them.
  • git rerere reuses conflict resolutions; git bisect finds the introducing change.
  • CTS tests app-facing APIs against the CDD; VTS tests HALs, kernel and vendor interfaces; GTS tests GMS requirements; CTS-Verifier covers manual hardware checks; STS tests security patches.
  • GMS is licensed (MADA); GTS is the licensee test suite. Compatibility tests are not the same as a GMS licence.
  • Carrier cert (PTCRB/GCF plus operator labs) is independent of xTS; many per-SIM differences are CarrierConfig.
  • CTS-on-GSI proves the vendor side works with a generic AOSP system image.
  • Vendor API level plus a GRF-style freeze window can let a vendor image pair with newer system images; confirm current CDD/release notes rather than memorising a year count.
  • OTA: update_engine applies a signed payload to the inactive A/B slot (virtual A/B snapshots dynamic partitions). Full payloads are source-flexible; incremental payloads are smaller but source-specific.
  • SoC / ODM / OEM / carrier own different layers; the integration lead owns the contracts and the one status of record, not the org chart.
  • Crash triage at scale: symbolize, cluster by stable stack signature, rank by volume × impact, one DRI per top cluster, done when the signature is gone.
  • xTS runs on Tradefed; run subsets continuously and full suites on promotion candidates.
  • Promotion gates: build health, boot, VINTF, smoke, compliance, power, stability, performance, open defects.
  • Gates must be measurable, compared against last good, owned, and stable under schedule pressure.
  • Platform CI: presubmit build and quick tests, postsubmit full builds, nightly xTS, KPI and soak runs.
  • Keep bisection possible: stage large drops, store manifest snapshots, automate culprit finding.
  • Systemic triage: own it, reproduce, instrument, localise, drive, decide, close.
  • The owner of a bug is the team whose code must change, not the team that saw the symptom.
  • RCA tools: 5 Whys, fishbone diagrams, defect timelines, blameless post-mortems with dated actions.
  • An issue is closed only after verification, a regression test or gate, and tracked corrective actions.
  • Release practices: change control board, go/no-go against published criteria, release notes, staged OTA rollout, hotfix path.
  • Scope, date and quality: with fixed people, make the trade-off explicit.
  • Distributed delivery: async-first, single status of record, DRIs, follow-the-sun handoffs, early escalation.
  • Track product KPIs (power, stability, performance, connectivity) and process metrics (lead time, build health, escaped defects, MTTR).
Artificial Intelligence / AI Foundations

Math & Statistics for ML

Read the full topic →

  • Mean uses every value and is pulled by outliers; the median is robust and preferred for skewed data such as income or latency.
  • Quartiles split sorted data into four parts; IQR = Q3 − Q1; the box-plot outlier rule flags points beyond Q1 − 1.5·IQR or Q3 + 1.5·IQR.
  • Variance is the mean squared deviation; standard deviation is its square root, in the data's units.
  • Sample variance divides by n − 1 (Bessel's correction) because deviations from the sample mean are systematically too small.
  • Right skew: long right tail, mean > median. Excess kurtosis > 0: heavier tails than a normal distribution.
  • z = (x − μ)/σ; fit scalers on training data only to avoid leakage.
  • P(A ∪ B) = P(A) + P(B) − P(A ∩ B); P(A ∩ B) = P(A)P(B | A); independence means P(A ∩ B) = P(A)P(B).
  • P(A | B) ≠ P(B | A); Bayes: posterior = likelihood × prior / evidence.
  • 1% prevalence, 95% sensitivity, 5% false-positive rate: a positive test means only about 16% chance of disease (base-rate fallacy).
  • Linearity of expectation holds even for dependent variables; Var(aX + b) = a2Var(X).
  • Var(X + Y) = Var X + Var Y + 2Cov(X, Y); the variance of a mean of n i.i.d. values is σ2/n.
  • Correlation is covariance scaled to [−1, 1]; zero correlation does not imply independence (X and X2).
  • Bernoulli (p, p(1−p)), binomial (np, np(1−p)), Poisson (λ, λ), exponential (1/λ, 1/λ2), uniform ((a+b)/2, (b−a)2/12).
  • Normal: 68-95-99.7 within 1, 2, 3 standard deviations; 1.96 for a two-sided 95% interval.
  • Beta is the conjugate prior for a Bernoulli/binomial rate; Dirichlet for categorical; posterior = prior pseudo-counts + observed counts.
  • LLN: sample means converge to the true mean. CLT: sample means are approximately normal with standard error σ/√n, whatever the data's shape.
  • A 95% CI means 95% of intervals built this way contain the true value, not a 95% probability for this interval.
  • A p-value is P(data at least this extreme | H0), not P(H0 | data).
  • Type I = false positive (α); Type II = false negative (β); power = 1 − β, increased by larger n, larger effects, lower variance.
  • Use Welch's t-test for two means, paired tests for the same units, chi-square for categorical association, z-test for proportions.
  • A/B pitfalls: peeking, multiple comparisons, underpowered tests, sample-ratio mismatch, novelty effects.
  • Bootstrap puts error bars on a statistic (resample with replacement); CV estimates how well a training procedure generalises (retrain on folds). They are not interchangeable.
  • Student-t variance is ν/(ν−2) only for ν > 2; the mean exists only for ν > 1.
  • Label smoothing: true class gets (1 − ε) + ε/K, every other class gets ε/K.
  • Matrix product (m×n)(n×p) = (m×p); not commutative but associative; (AB)T = BTAT.
  • Only square, full-rank matrices (det ≠ 0) are invertible; rank = number of independent rows or columns.
  • Normal equation w = (XTX)−1XTy; in practice use lstsq/QR/SVD, never an explicit inverse.
  • Av = λv; trace = sum of eigenvalues, determinant = product; symmetric matrices have real eigenvalues and orthogonal eigenvectors.
  • SVD A = UΣVT works for any matrix; truncated SVD is the best low-rank approximation.
  • PCA: center/standardize, covariance, eigenvectors = directions of max variance, keep top k by explained variance.
  • Broadcasting compares shapes right to left; dimensions must match or be 1; (N,) minus (N, 1) silently becomes (N, N).
  • Cosine similarity ignores magnitude; for unit vectors, squared Euclidean distance = 2 − 2cos θ.
  • The gradient points in the direction of steepest ascent; gradient descent steps along −∇L.
  • Chain rule: multiply local derivatives along a path, sum over paths; backprop applies it from the loss backwards.
  • σ'(x) = σ(1 − σ) ≤ 0.25; the softmax + cross-entropy gradient with respect to logits is p − y.
  • Hessian eigenvalues all positive = local minimum; mixed signs = saddle point; learning rate must satisfy η < 2/λmax.
  • Vanishing gradients: fix with ReLU-family activations, He/Xavier init, residuals, normalization; exploding: gradient clipping.
  • Momentum averages gradients; RMSProp scales by recent gradient magnitude; Adam combines both with bias correction; AdamW decouples weight decay.
  • Warmup avoids early divergence; cosine decay lets the model settle; tune learning rates on a log scale.
  • Convex losses have one global minimum; neural network losses are non-convex but mostly have good minima and many saddles.
  • Entropy H = −Σp log p; cross-entropy = entropy + KL; KL ≥ 0 and is not symmetric.
  • Perplexity = ecross-entropy per token; a loss of 2.0 nats means perplexity about 7.4.
  • MSE = Gaussian MLE, MAE = Laplace MLE, cross-entropy = categorical MLE; L2 = Gaussian prior, L1 = Laplace prior (MAP).
  • Stable softmax subtracts the max; log-sum-exp = m + log Σex − m; work with log-probabilities; pass logits to fused losses.
  • BF16 has FP32's range with less precision, so LLMs train in BF16 without loss scaling; FP16 needs loss scaling.
  • Temperature divides logits before softmax: T < 1 sharpens, T > 1 flattens; it never changes the argmax.
  • Attention = softmax(QKT/√dk)V; dividing by √dk keeps score variance at 1 so softmax does not saturate.
  • Attention is order-blind; sinusoidal, learned, ALiBi or RoPE encodings add position; RoPE rotates Q and K so scores depend on relative distance.
Artificial Intelligence / AI Foundations

Python & Data Tools for ML

Read the full topic →

  • Python is the orchestration layer; speed comes from keeping work inside compiled libraries (NumPy, BLAS, cuDNN) rather than Python loops.
  • Use one isolated environment per project (venv or conda) with pinned versions; use %pip inside notebooks.
  • Restart the kernel and run all cells before sharing a notebook, to expose hidden state.
  • Variables are names bound to objects; assignment never copies.
  • Lists, dicts and sets are mutable; ints, floats, strings, tuples and frozensets are immutable.
  • == compares values, is compares identity; use is only with None and other singletons.
  • Never use a mutable default argument; default to None and create the object inside.
  • Dict and set lookups are O(1) on average; list membership is O(n). Keys must be hashable.
  • *args collects extra positional arguments into a tuple; **kwargs collects extra keyword arguments into a dict.
  • A closure remembers variables from its enclosing scope; closures bind late, so loop variables are read at call time.
  • A decorator is a function that takes a function and returns a wrapped one; always use functools.wraps.
  • Generators (yield) produce values lazily, use constant memory, and can be consumed only once.
  • Dunder methods (__len__, __getitem__, __call__, __enter__) plug your classes into Python syntax and into frameworks.
  • Type hints are not enforced at runtime; tools such as mypy and libraries such as Pydantic read them.
  • with guarantees cleanup via __exit__, even when an exception is raised.
  • The GIL allows one thread to run Python bytecode at a time: threads or asyncio for I/O-bound work, processes for CPU-bound Python work.
  • A full PyTorch checkpoint (state_dict plus epoch or config) needs torch.load(..., weights_only=False) and must be trusted; tensor-only loads can keep the safer default.
  • NumPy arrays are homogeneous, typed and contiguous, which enables vectorized C loops.
  • Broadcasting aligns shapes from the right; dimensions must match or be 1.
  • Basic slicing returns a view; fancy and boolean indexing return copies.
  • axis=0 collapses rows (one result per column); axis=1 collapses columns (one result per row).
  • * is element-wise, @ is matrix multiplication.
  • Subtract the maximum before exponentiating in softmax for numerical stability.
  • NumPy's var/std default to ddof=0; pandas defaults to ddof=1.
  • loc selects by label with an inclusive end; iloc selects by position with an exclusive end.
  • Combine pandas masks with &, |, ~ and parentheses.
  • groupby().agg() returns one row per group; transform() returns a result aligned to the original rows.
  • Validate merges (validate=, row counts) to catch duplicate keys that explode joins.
  • Vectorized column operations beat apply, which beats iterrows.
  • Assign with a single df.loc[mask, col] = value; chained assignment does not modify the original.
  • Shrink pandas memory with downcasting, category dtype, usecols, Parquet and chunking.
  • EDA checklist: structure, quality, univariate, target, bivariate, multivariate, leakage, decisions.
  • scikit-learn contract: fit learns, transform/predict apply; learned attributes end with an underscore.
  • Put every data-dependent step inside a Pipeline so cross-validation cannot leak.
  • Use CV to choose models; use a paired bootstrap or McNemar on a sealed test set to compare two finished models.
  • ColumnTransformer applies different preprocessing to numeric and categorical columns.
  • Stratify classification splits; use group splits for repeated entities and time-series splits for temporal data.
  • Tune on training data with CV; touch the test set once at the end.
  • Choose metrics by error cost: recall when misses are costly, precision when false alarms are costly, PR-AUC for rare positives.
  • Pick the decision threshold on validation or out-of-fold predictions, not the test set.
  • Scale features for linear models, SVM, kNN, PCA and neural networks; trees do not need it.
  • PyTorch loop: zero_grad, forward, loss, backward, step.
  • model.eval() changes dropout and BatchNorm behaviour; torch.no_grad() stops gradient tracking; inference needs both.
  • CrossEntropyLoss takes raw logits and integer class labels; BCEWithLogitsLoss takes logits and float targets.
  • Save state_dicts; weights_only=True is the safe default, but a checkpoint that also stores epoch or config needs weights_only=False from a trusted file.
  • Hugging Face: pipeline for quick tasks, AutoTokenizer plus AutoModelFor... for control; tokenizer and model must match.
  • Profile before optimising; vectorize first, compile or parallelize second, change engine last.
Artificial Intelligence / AI Foundations

Machine Learning Fundamentals

Read the full topic →

  • ML learns rules from data plus answers; traditional programming applies hand-written rules to data.
  • AI contains ML, which contains deep learning, which contains generative AI.
  • Supervised learning uses labels (classification or regression); unsupervised finds structure; self-supervised makes labels from raw data; RL learns from rewards.
  • Parameters are learned during training; hyperparameters are set before training and tuned on validation data.
  • Train fits parameters, validation chooses hyperparameters and thresholds, test is used once for the final estimate.
  • Use stratified splits for classification, time-based splits for temporal data and group splits for repeated entities.
  • Leakage is information unavailable at prediction time reaching training; fit every preprocessing step on training data only, inside a Pipeline.
  • Do not fill missing values with 0 unless zero is meaningful; use median, most-frequent, model-based imputation or a missing flag.
  • One-hot for nominal low-cardinality features, ordinal for ordered ones, out-of-fold target encoding for high cardinality.
  • kNN, k-means, SVM, PCA, regularized linear models and neural nets need scaling; tree models do not.
  • Loss is per example, cost is the average (plus penalties); MSE for regression, cross-entropy for classification.
  • Cross-entropy punishes confident wrong predictions hardest: −ln 0.05 ≈ 3.0 versus −ln 0.9 ≈ 0.1.
  • Gradient descent updates θ ← θ − η∇J; too small a learning rate is slow, too large diverges.
  • Linear and logistic regression have convex costs with one global minimum; neural networks do not.
  • L2 (Ridge) shrinks all weights; L1 (Lasso) zeroes some (feature selection); Elastic Net mixes both.
  • In scikit-learn, C = 1/λ: smaller C means stronger regularization.
  • Logistic regression models log-odds linearly; ew is the odds ratio per unit of a feature.
  • kNN: small k overfits, large k underfits; suffers from the curse of dimensionality.
  • Naive Bayes assumes conditional independence; use Laplace smoothing to avoid zero probabilities.
  • Gini = 1 − Σp²; entropy = −Σp log2p; trees choose the split with the largest impurity decrease.
  • Unpruned trees overfit; control with max_depth, min_samples_leaf or cost-complexity pruning.
  • Bagging reduces variance with parallel deep trees; boosting reduces bias with sequential shallow trees.
  • Random forests decorrelate trees by sampling features at each split; about 36.8% of rows are out-of-bag per tree.
  • Gradient boosting fits each new tree to the negative gradient (residuals for MSE) with a small learning rate.
  • LightGBM grows leaf-wise with histograms; XGBoost uses second-order gains and regularized leaves; CatBoost handles categoricals with ordered statistics.
  • Stacking trains a meta-model on out-of-fold predictions of diverse base models.
  • SVM maximises the margin 2/‖w‖; C trades margin width for violations; the kernel trick replaces dot products.
  • RBF γ large means wiggly, overfit boundaries; small means smooth ones.
  • k-means minimises within-cluster squared distance, needs k, assumes spherical clusters and finds only a local optimum.
  • DBSCAN finds arbitrary shapes and labels noise; GMM gives soft, elliptical clusters via EM.
  • PCA projects onto orthogonal directions of maximum variance; standardize first; keep 90–95% variance or tune k.
  • t-SNE and UMAP are for visualization; t-SNE distances between clusters are not meaningful.
  • Expected error = bias² + variance + irreducible noise.
  • Underfitting: both errors high. Overfitting: low train error, much higher validation error.
  • k-fold CV gives a mean and a spread; use nested CV for an unbiased estimate of tuning plus training.
  • CV estimates a training procedure's performance; the bootstrap puts error bars on a statistic of a frozen model. Do not substitute a training-set bootstrap for held-out CV.
  • Leakage patterns: target proxies, future-looking windows, in-sample target encoding, IDs, fit-on-all preprocessing, duplicates, SMOTE-before-split, random splits of grouped or temporal data.
  • Random search usually beats grid search for the same budget; search rates and penalties on a log scale.
  • Precision = TP/(TP+FP); recall = TP/(TP+FN); F1 is their harmonic mean.
  • With TP 80, FP 20, FN 10, TN 890: precision 0.80, recall 0.89, F1 0.84, accuracy 0.97.
  • A 99%-accurate model on a 99:1 dataset can have zero recall; prefer recall, precision, F1 or PR-AUC.
  • ROC-AUC is the probability a random positive outranks a random negative; PR-AUC is better when positives are rare.
  • Cost-optimal threshold for calibrated probabilities is CFP/(CFP + CFN); tune thresholds on validation data only.
  • Calibration means predicted probabilities match observed frequencies; fix with Platt scaling or isotonic regression.
  • Macro averaging treats classes equally; weighted averaging can hide a failing rare class.
  • RMSE ≥ MAE; a large gap means a few big errors. R² can be negative on test data.
  • Imbalance toolkit: right metric, stratification, class weights, resampling inside CV (SMOTE), threshold moving.
  • SHAP values add up to prediction minus base value; importance is not causation.
  • Drift types: data (P(X)), concept (P(y|X)), label (P(y)); PSI above 0.25 signals significant shift.
  • A feature store is one definition of each feature, served offline (point-in-time training joins) and online (low-latency inference) to prevent training–serving skew.
  • Ship the full preprocessing-plus-model pipeline as one artifact to avoid training–serving skew.
  • Q-learning: Q ← Q + α[r + γ max Q′ − Q]; RLHF uses a reward model and PPO with a KL penalty.
Artificial Intelligence / AI Foundations

Deep Learning & Neural Networks

Read the full topic →

  • Deep learning is machine learning with multi-layer neural networks that learn their own features from raw data (representation learning).
  • It took off because of large datasets, GPUs, and algorithmic fixes: ReLU, good initialization, normalization, residual connections, dropout and Adam.
  • A neuron computes z = w·x + b and outputs f(z); weights live on connections, activations live in nodes.
  • A single perceptron only separates linearly separable data; XOR needs a hidden layer.
  • Without non-linear activations, any stack of layers collapses into one linear layer.
  • The universal approximation theorem guarantees existence of a one-hidden-layer approximator, not that training finds it; depth is more parameter-efficient.
  • Dense layer parameters = inputs × outputs + outputs.
  • Sigmoid saturates and its derivative is at most 0.25; tanh is zero-centered; ReLU has derivative 1 for positive inputs but can die.
  • GELU and SiLU are smooth ReLU-like activations used in Transformers and modern LLMs.
  • Output activations follow the task: identity for regression, sigmoid for binary or multi-label, softmax for single-label multi-class.
  • Loss is per example, cost is the average (plus regularization); libraries call both "loss".
  • MSE penalizes large errors quadratically and is outlier-sensitive; MAE is robust; Huber mixes both.
  • Cross-entropy is negative log-likelihood; confident wrong predictions are punished hardest (−ln 0.05 ≈ 3.0).
  • Softmax plus cross-entropy has the gradient p − y with respect to logits.
  • Feed raw logits to CrossEntropyLoss and BCEWithLogitsLoss; never softmax twice.
  • Focal loss and class weights handle imbalance; label smoothing and temperature scaling improve calibration.
  • Gradient descent: θ ← θ − η∇L; mini-batch SGD is the practical default.
  • An epoch is one pass over the data; an iteration is one update on one mini-batch.
  • Backpropagation is reverse-mode chain rule; it computes all gradients in about the cost of one or two extra forward passes but must store activations.
  • Backprop computes gradients; the optimizer applies them; both happen only in training.
  • Momentum smooths and accelerates; RMSProp adapts per-parameter step sizes; Adam combines both with bias correction.
  • AdamW decouples weight decay from the adaptive update and is the default for Transformers.
  • Learning rate is the most important hyperparameter; too high diverges, too low crawls.
  • Warmup stabilizes early training; cosine decay is the common schedule; scale the learning rate with batch size.
  • Zero or equal initialization leaves neurons symmetric; Xavier suits tanh/sigmoid, He (variance 2/fan_in) suits ReLU.
  • Overfitting: low training loss, high validation loss; underfitting: both high.
  • Regularizers: weight decay, dropout, early stopping, data augmentation, label smoothing, more data, transfer learning.
  • Inverted dropout scales kept units by 1/(1 − p) in training and is disabled at inference.
  • BatchNorm normalizes over the batch per channel and uses running statistics at inference; the forward pass uses biased batch variance, the running variance is updated unbiased; LayerNorm normalizes each example and suits Transformers; RMSNorm drops the mean.
  • Label smoothing in PyTorch is y' = (1 − ε)y + ε/K, not (1 − ε) and ε/(K − 1).
  • Vanishing or exploding gradients come from multiplying many factors below or above 1 through depth or time.
  • Residual connections (y = F(x) + x) give gradients a direct path and solved the degradation problem. The residual stream is the vector every block reads and adds to.
  • Pre-LN (x + F(Norm(x))) keeps a clean residual path and is the modern default; Post-LN (Norm(x + F(x))) needs warmup.
  • SwiGLU is W2(SiLU(W1x) ⊙ W3x); hidden width ≈ 8d/3 keeps the parameter count equal to a 4d MLP.
  • Clip gradients by norm (typically 1.0) for RNNs and Transformers.
  • CNNs exploit locality and parameter sharing; conv output size = ⌊(W + 2P − K)/S⌋ + 1.
  • Conv parameters = (Cin × k × k + 1) × Cout, independent of image size. Dilated output size uses effective kernel 1 + D(K − 1).
  • Pooling downsamples and adds shift tolerance but discards fine positional detail.
  • Stacked 3×3 convolutions grow the receptive field cheaply; 1×1 convolutions mix channels.
  • Architecture lineage: LeNet, AlexNet, VGG, Inception, ResNet, DenseNet, MobileNet, EfficientNet, ViT.
  • Transfer learning: replace the head, freeze then gradually unfreeze, use a much smaller learning rate.
  • RNNs share weights across time and carry a hidden state; BPTT unrolls them and suffers vanishing gradients.
  • LSTM gates (forget, input, output) control an additively updated cell state; GRU merges them into update and reset gates.
  • Bidirectional RNNs see both past and future context and cannot be used for streaming generation.
  • Seq2seq compresses the source into a context vector; attention removes this bottleneck and leads to Transformers.
  • Embeddings are learned dense vectors; word2vec (CBOW, skip-gram) learns them from context; contextual models give per-occurrence vectors.
  • Autoencoders compress and reconstruct; VAEs add a KL term and the reparameterization trick to make the latent space sampleable.
  • GANs train a generator against a discriminator and risk mode collapse; diffusion models learn to predict added noise and denoise step by step.
  • Mixed-precision Adam needs about 16 bytes per parameter before activations; BF16 avoids FP16's loss scaling.
  • Gradient accumulation simulates big batches; activation checkpointing trades compute for memory.
  • DDP replicates the model and all-reduces gradients; FSDP/ZeRO shard states; tensor and pipeline parallelism split the model.
  • Debug by overfitting one small batch, checking the initial loss (about ln K), and watching loss, learning rate and gradient norms.
  • The PyTorch rhythm per batch: zero_grad, forward, loss, backward, step; use train/eval modes and no_grad for evaluation.
Artificial Intelligence / AI Foundations

NLP Fundamentals

Read the full topic →

  • NLP pipeline: normalize → tokenize → map to ids → represent as vectors → model → post-process and evaluate.
  • Language is hard because of ambiguity, word order and composition, long-range dependencies, noise, world knowledge, and a long tail of rare words (Zipf's law).
  • Aggressive cleaning (lowercase, stop words, stemming) helps bag-of-words models and hurts pretrained Transformers.
  • Keep negations ("not", "no", "never") when removing stop words for sentiment tasks.
  • Stemming chops suffixes by rule ("studies" → "studi"); lemmatization maps to a dictionary form using part of speech ("better" → "good").
  • Distinct one-hot vectors always have cosine similarity 0, so one-hot encodes identity but no similarity.
  • Bag-of-words ignores order: "dog bites man" equals "man bites dog"; n-grams restore some local order.
  • TF-IDF = term frequency × log(N / document frequency); a term in every document gets idf 0; fit it on training data only.
  • Cosine similarity compares direction and ignores length; on unit vectors it ranks identically to Euclidean distance and dot product.
  • BM25 adds term-frequency saturation and length normalization to TF-IDF and is the standard lexical retriever.
  • Distributional hypothesis: words in similar contexts have similar meanings; embeddings are learned purely from co-occurrence.
  • CBOW predicts the center word from context (faster, frequent words); skip-gram predicts context from the center word (better for rare words).
  • Negative sampling replaces the full-vocabulary softmax with k binary real-vs-random classifications, sampled from unigram3/4.
  • GloVe fits word vectors to log global co-occurrence counts; FastText sums character n-gram vectors, so it can embed unseen words.
  • Embedding arithmetic works because consistent relations become roughly constant offset directions (king − man + woman ≈ queen).
  • Static embeddings give one vector per word type; contextual models (ELMo, BERT) give one vector per occurrence and resolve polysemy.
  • Antonyms are often close in embedding space because they share contexts.
  • Raw BERT [CLS] vectors are poor for similarity; SBERT and contrastively trained embedding models fix this.
  • Bi-encoders embed separately and scale to millions of documents; cross-encoders score pairs jointly and are used for re-ranking.
  • BPE repeatedly merges the most frequent adjacent pair (deterministic, GPT default); WordPiece merges the pair that most increases likelihood; Unigram prunes a large vocabulary top-down and can sample segmentations.
  • SentencePiece trains on raw text with the space as a symbol, so it needs no language-specific pre-tokenizer.
  • Byte-level BPE starts from 256 bytes and never produces an unknown token.
  • English averages about 1.3 tokens per word; many non-Latin-script languages need several times more tokens, raising cost.
  • Always use the exact tokenizer, version and chat template the model was trained with.
  • A language model factorizes P(sequence) into next-token probabilities by the chain rule.
  • n-gram LMs use the Markov assumption and need smoothing (Laplace, backoff, Kneser-Ney) for unseen n-grams.
  • Perplexity = exp(average negative log-likelihood per token); lower is better; a uniform model over V tokens has perplexity V; compare only across identical tokenizers.
  • Vanilla RNNs suffer vanishing and exploding gradients through backpropagation through time; clip gradients.
  • The LSTM's additive cell-state update with a forget gate near 1 lets gradients flow over long distances.
  • GRU uses update and reset gates, fewer parameters than LSTM, similar accuracy.
  • Bidirectional RNNs suit tagging and classification, not left-to-right generation.
  • Seq2seq squeezes the source into one vector; attention lets the decoder look back at every encoder state.
  • Scaled dot-product attention: softmax(QKT / √dk) V; the scaling prevents softmax saturation.
  • Transformers replaced RNNs because of parallel training, 1-hop paths between any two tokens, and no fixed-size bottleneck; they need positional encodings.
  • NER is token classification with BIO tags; evaluate with entity-level F1, not token accuracy.
  • Aspect-based sentiment extracts (aspect, opinion, polarity) and is what makes sentiment actionable.
  • LDA models documents as mixtures of topics and topics as distributions over words; BERTopic works better for short text.
  • Extractive summarization is faithful but choppy; abstractive is fluent but can hallucinate.
  • BLEU is clipped n-gram precision with a brevity penalty; ROUGE is n-gram or LCS recall.
  • BERTScore matches tokens by contextual embedding similarity and rewards paraphrases; METEOR uses stems and synonyms.
  • In the IMDB study, linear SVM and logistic regression (F1 0.875) beat all neural models; cross-attention was the best neural model (0.863).
  • IMDB lessons: build baselines, clip gradients, do not trust training loss, use pretrained embeddings, and inspect precision and recall separately.
  • Code-mixed text (for example Hinglish) needs language ID, transliteration handling, multilingual encoders and in-domain labels.
  • LLM extraction needs a schema, constrained output, evidence spans verified against the source, normalization and F1 evaluation.
  • Entity linking maps mentions to knowledge-base IDs via candidate generation plus contextual disambiguation, with a NIL option.
Artificial Intelligence / Generative AI & LLMs

Transformers & Large Language Models

Read the full topic →

  • RNNs are sequential (no parallel training), have O(n) paths between distant tokens and suffer vanishing/exploding gradients; seq2seq RNNs also squeeze the whole input into one context vector.
  • Attention removed the bottleneck by letting the decoder take a fresh weighted average of all encoder states at every step; the Transformer (2017) removed recurrence entirely.
  • Self-attention gives O(1) path length and full parallelism at the cost of O(n2d + nd2) time and O(n2) score-matrix memory (O(n) with FlashAttention).
  • Text becomes subword tokens (BPE, WordPiece, Unigram/SentencePiece); English averages about 4 characters or 0.75 words per token; other languages often need more.
  • APIs bill per input and output token; output tokens cost more and are generated sequentially; context window = input + output.
  • The embedding table is vocab × d; input embeddings are static, hidden states after each layer are contextual.
  • Attention is permutation-equivariant, so position must be injected: sinusoidal (added, fixed), learned absolute (added, trainable), RoPE (rotates Q and K, relative), ALiBi (distance penalty on scores).
  • RoPE makes the q·k score depend only on relative distance, adds no parameters and can be extended with interpolation, NTK scaling or YaRN plus fine-tuning.
  • Attention(Q,K,V) = softmax(QKT/√dk + mask)V; Q, K, V are linear projections of the same input in self-attention.
  • Scaling by √dk keeps dot-product variance near 1 so softmax does not saturate and gradients survive.
  • Masked positions get −∞ (not 0) before softmax; causal masks block the future, padding masks block filler tokens.
  • Attention weights are computed per input; only WQ, WK, WV, WO are learned.
  • Multi-head attention splits d into h heads of size d/h, each learning a different relationship; concatenation plus WO merges them at the same parameter cost as one head.
  • Cross-attention takes Q from the decoder and K, V from the final encoder layer; decoder-only models do not use it.
  • A block is attention + FFN, each wrapped with a residual connection and normalization; modern LLMs use pre-norm with RMSNorm.
  • Residuals give gradients an identity path and let each layer learn an additive update to the residual stream.
  • Pre-LN is x + F(Norm(x)) (clean residual path, modern default); Post-LN is Norm(x + F(x)) (needs warmup, unstable when very deep).
  • LayerNorm normalises each token over features, so it is independent of batch size and sequence length, unlike BatchNorm.
  • The FFN expands d → 4d (or about 8d/3 with SwiGLU), applies a non-linearity and projects back; it holds about two-thirds of parameters and acts per token.
  • One block has about 12d2 parameters; GPT-2 small = 124M, GPT-3 = 175B can be derived from this. Tying the embedding table to the LM head saves a second |V|·d matrix.
  • Encoder-only (BERT) = bidirectional understanding; decoder-only (GPT, Llama) = causal generation; encoder-decoder (T5, BART, Whisper) = sequence transformation.
  • CLM predicts the next token at every position (loss on all tokens); MLM predicts 15% masked tokens with the 80/10/10 rule; T5 uses span corruption with sentinel tokens.
  • Perplexity = exp(average cross-entropy per token); lower is better.
  • Training compute ≈ 6ND FLOPs; inference ≈ 2N FLOPs per generated token.
  • Chinchilla: compute-optimal training uses about 20 tokens per parameter; modern small models are deliberately overtrained for cheaper inference.
  • Emergent abilities such as in-context learning appear with scale, but some apparent jumps are artefacts of all-or-nothing metrics.
  • The pipeline is pretraining → supervised fine-tuning on instructions → preference alignment (RLHF or DPO) → optional RL for reasoning.
  • RLHF trains a reward model on human comparisons and optimises the policy with PPO plus a KL penalty to the SFT model to prevent reward hacking.
  • DPO optimises the same objective directly on preference pairs with a classification-style loss, without a reward model or RL loop.
  • Greedy and beam search are deterministic; beam search suits translation and summarisation but produces bland open-ended text.
  • Temperature divides logits before softmax: below 1 sharpens, above 1 flattens, and the ranking never changes.
  • Top-k keeps a fixed number of tokens, top-p keeps the smallest set reaching probability p, min-p keeps tokens above a fraction of the top probability.
  • Repetition penalty scales logits of seen tokens; frequency penalty subtracts per occurrence; presence penalty subtracts once.
  • Prefill is compute-bound (time to first token); decode is memory-bandwidth-bound (tokens per second).
  • The KV cache stores past keys and values; size = 2 × layers × KV heads × head dim × tokens × batch × bytes.
  • GQA shares K/V heads among groups of query heads (MQA uses one), shrinking the KV cache several-fold with little quality loss; MLA compresses K/V into a latent.
  • FlashAttention is exact attention that tiles computation in on-chip memory: O(n) memory and faster, but the same O(n2d) FLOPs.
  • Speculative decoding: a draft model proposes several tokens; the target verifies them in one parallel pass (rejection sampling leaves the output distribution unchanged).
  • Sliding-window attention limits each token to recent tokens; stacked layers still spread information further.
  • Long context is limited by quadratic attention, KV cache memory, position generalisation and "lost in the middle" effects.
  • MoE replaces the FFN with many experts and a router choosing top-k per token: huge total parameters, small active parameters, but all experts must fit in memory.
  • ViT turns an image into patch tokens; CLIP aligns image and text embeddings with a contrastive loss; VLMs feed projected visual tokens into an LLM.
  • Open weights give control, privacy and fine-tuning; closed APIs give frontier capability with no infrastructure; behaviour differences come mainly from data and post-training.
  • Reasoning models spend test-time compute on long chains of thought trained with RL on verifiable rewards; the cost is latency and tokens.
  • Hallucination comes from a plausibility objective, knowledge gaps, incentives to guess, sycophancy and decoding; grounding and verification reduce it, temperature 0 does not eliminate it.
  • Known limitations: tokenization quirks, stale knowledge, finite context, no persistent memory, bias, prompt injection and non-determinism.
Artificial Intelligence / Generative AI & LLMs

Prompt Engineering

Read the full topic →

  • An LLM predicts the next token given all previous tokens; the prompt is the conditioning context, so it shapes the distribution but adds no new knowledge or weights.
  • Chat roles are flattened by a chat template into one token stream with role markers; roles work because instruction tuning taught the model to prioritise them.
  • The same single-stream design is why prompt injection is possible: there is no hard boundary between instructions and data.
  • A strong prompt contains role, task, context, delimited input, constraints, output format, optional examples and an output indicator.
  • Be specific, explain why a rule exists, prefer positive instructions and always give an escape hatch for missing information.
  • System messages hold stable rules; user messages hold the request and data; assistant messages hold history, staged examples or a prefill; tool messages return function results.
  • APIs are stateless: the application must resend history on every call.
  • Delimit all data with tags or fences; number multi-step procedures; specify exact output schemas with enums and null rules.
  • For machine-read output, use structured outputs or function calling, validate with a typed schema and retry on failure.
  • In JSON outputs, put the reasoning field before the answer field; generation order matters.
  • Structural length limits (bullets, sentences) are followed better than word counts; max_tokens truncates rather than shortens.
  • Zero-shot uses instructions only; few-shot adds input-output demonstrations; a long detailed prompt without examples is still zero-shot.
  • In-context learning changes activations, not weights; examples teach format, label space and task identity.
  • Good examples are diverse, label-balanced, include edge cases and share one identical format; order matters because of recency bias.
  • With large example pools, retrieve the most similar and diverse examples per query (dynamic few-shot).
  • Temperature near 0 gives stable, repeatable output but does not cure hallucination; grounding does.
  • Chain-of-thought gives the model extra computation through generated tokens; benefits emerged with scale and are largest on multi-step problems.
  • Zero-shot CoT uses a trigger like "Let's think step by step"; few-shot CoT uses worked examples; Auto-CoT clusters questions and generates demonstrations automatically.
  • Self-consistency samples several reasoning paths at temperature above 0 and majority-votes the final answer; agreement is a confidence signal; cost is N times.
  • Written chains can be unfaithful: they improve accuracy but are not proof of how the model decided.
  • Least-to-most and plan-and-solve decompose problems; program-aided prompting offloads maths to code.
  • Tree of thoughts proposes, evaluates and searches over partial thoughts with pruning and backtracking; graph of thoughts adds merging.
  • Self-consistency votes on complete answers; tree of thoughts evaluates intermediate nodes.
  • Self-correction works best with external feedback (tests, tools, evidence); pure introspection can make answers worse.
  • ReAct interleaves Thought, Action and Observation; it grounds reasoning in real tool results and underlies agents.
  • The model never executes tools; it emits a structured call and your code runs it; use "Observation:" as a stop sequence in text ReAct.
  • Tool descriptions are prompts: state purpose, when to use, when not to use and argument format.
  • Personas shift style and depth, not factual knowledge; the audience specification is often more powerful.
  • Negative instructions can prime the forbidden behaviour and give no target; pair them with a positive alternative and a reason.
  • Directional stimulus prompting adds hint keywords, in research generated by a small RL-trained policy model steering a frozen LLM.
  • Meta prompting gives an abstract solution structure, or asks a model to write and improve prompts.
  • Mode collapse comes from preference tuning and early-token lock-in; verbalized sampling asks for several responses with probabilities to recover diversity.
  • Templates separate static instructions from variables; put static content first to benefit from prompt caching.
  • Prompt chaining splits tasks into validated sequential calls; routing sends each request to the right prompt, model or settings.
  • Treat prompts as versioned, tested artifacts: semantic versions, changelogs, minimal diffs, regression sets, A/B tests, monitoring.
  • Automatic optimizers: APE searches instructions, ProTeGi applies textual gradients, OPRO uses scored history in a meta-prompt, evolutionary methods mutate populations, DSPy compiles programs against a metric.
  • Prompt compression (LLMLingua, summarization, retrieval-side selection) cuts cost and latency; compress context more than instructions.
  • Evaluation hierarchy: deterministic checks, reference metrics, model-based judges, human review; calibrate judges and control position, verbosity and self-preference bias.
  • Attacks: direct and indirect injection, jailbreaks, prompt leaking, data exfiltration, tool abuse and backdoors.
  • Defense in depth: instruction hierarchy, spotlighting, sandwich reminders, input and output filters, least privilege, human confirmation, isolation, no secrets in prompts, red teaming.
  • Reasoning models need goals and constraints, not step-by-step scaffolding; tune reasoning effort instead.
  • Place long documents first and the question last; ask for quotes before answers to reduce hallucination.
  • Every model or version change is effectively a prompt change: re-run the evaluation set.
  • Prompt engineering designs the instruction; context engineering assembles the full window (retrieval, memory, tools, budget, trust).
  • Multimodal prompts specify what to look at, what to ignore and a schema with nulls; image tokens scale with resolution.
  • Logprobs, vote share and verbalized percentages are uncalibrated confidence signals; plot a reliability curve before using them as thresholds.
  • Prompt tuning / soft prompts learn continuous prefix vectors with the model frozen; they are PEFT, not wording, and do not transfer across models.
  • Prompts are sensitive to wording, example order and model version; measure mean and variance across paraphrases, not one lucky run.
Artificial Intelligence / Generative AI & LLMs

LLM APIs, Structured Outputs & Tool Calling

Read the full topic →

  • An LLM API call sends a model name, a list of role-tagged messages and parameters, and returns generated message(s), a finish_reason and token usage. Chat Completions is the common teaching shape; some providers also have a Responses-style API — check current docs for field names.
  • Roles: system (or developer) sets rules, user is the input, assistant is prior model output or few-shot examples, tool carries function results.
  • Roles are flattened into one token sequence by the model's chat template; they are learned conventions, not separate channels.
  • APIs are stateless: conversation memory means resending history every call, so input cost grows with each turn.
  • Temperature divides logits before softmax; near 0 is focused, higher is more random.
  • Top-p keeps the smallest token set reaching cumulative probability p; top-k keeps a fixed count.
  • Temperature 0 is not a determinism guarantee; GPU numerics, batching and model updates cause variation.
  • max_tokens (or max_completion_tokens / max_output_tokens on some APIs) caps output, bounding cost and latency; finish_reason "length" means truncation. Confirm the accepted name for your model.
  • Frequency penalty scales with repeat count, presence penalty is a flat penalty once a token appears.
  • logprobs give per-token confidence, useful for classification thresholds and routing.
  • logit_bias adjusts specific token IDs; -100 effectively bans a token.
  • 1 token is about 4 English characters or 0.75 words; tokenizers differ by model and language.
  • Input and output tokens are priced separately; output is usually several times more expensive.
  • Reasoning tokens are billed as output even when hidden.
  • Cost = uncached input x input price + cached input x cached price + output x output price.
  • Biggest cost levers: model choice and routing, output length, input trimming, caching, batch APIs.
  • Context window covers input plus output; reserve space for the answer.
  • Context strategies: truncation, sliding window, rolling summary, retrieval, map-reduce, structured state.
  • Streaming uses server-sent events; it cuts time to first token, not total generation time.
  • Latency is roughly TTFT plus output tokens times per-token time; shorter outputs are faster.
  • Async with a semaphore gives concurrency without blowing rate limits; batch APIs trade hours of latency for about half price.
  • Rate limits come as RPM, TPM, daily quotas and concurrency; exceeding them returns 429.
  • Retry 429, timeouts and 5xx with exponential backoff plus jitter, honoring Retry-After; never retry 400/401/403 or quota exhaustion.
  • Make side-effecting operations idempotent so retries cannot double-charge or double-send.
  • Prompt caching reuses the provider's computed prefix; put stable content first and keep it byte-identical.
  • Response caching skips the model; semantic caches risk false hits and must be tenant-scoped.
  • Structured-output ladder: prompt, validate-and-retry, JSON mode, forced tool call, strict JSON Schema.
  • JSON mode guarantees parseable JSON, not your schema; strict schema mode guarantees both, for a supported schema subset.
  • Pydantic validates, safely coerces and generates JSON Schema; dataclasses do none of this.
  • Validation catches wrong shape; only evaluation and grounding catch wrong values.
  • Constrained decoding masks invalid tokens to minus infinity at every step using an automaton compiled from a schema, regex or grammar.
  • Hosted APIs do not expose logits for custom processors; local engines (Transformers, vLLM, llama.cpp, Ollama) support grammars or schemas.
  • Tool calling: the model proposes a function name and JSON arguments; your code validates, executes and returns a tool message; loop until a final answer.
  • Custom tools run in your environment, not on the provider's servers.
  • tool_choice can be auto, none, required or a specific function; forcing a function is a structured-output trick.
  • Parallel tool calls need one tool message per tool_call_id before the next request.
  • Treat model tool calls as untrusted input: validate, authorize in code, least privilege, confirm irreversible actions, sandbox code.
  • Multimodal content is a list of parts; image tokens scale with resolution.
  • Embeddings map text to vectors; compare with cosine similarity; never mix embedding models in one index.
  • Choose models by eval quality, latency, cost, context, features, governance and licensing; route easy tasks to small models.
  • Ollama is easy local running; vLLM is high-throughput serving with PagedAttention and continuous batching; llama.cpp runs quantized models on modest hardware.
  • Weight memory is roughly parameters times bytes per parameter; 4-bit quantization makes a 7B model fit in about 4-5 GB.
  • Keys live in environment variables or a secrets manager, never in code or client apps; rotate on exposure.
  • Minimize and redact PII, check provider retention and training terms, and log with redaction and access control.
  • Observe every call: prompt version, model, tokens, cost, latency, finish_reason, validation outcome, quality signals.
Artificial Intelligence / Generative AI & LLMs

Retrieval-Augmented Generation (RAG)

Read the full topic →

  • RAG retrieves external evidence at inference time and conditions generation on it; it separates knowledge (index) from reasoning (LLM).
  • It addresses knowledge cutoff, hallucination, private data and verifiability; it does not change the model's weights.
  • Knowledge problem: RAG. Behaviour, tone or format problem: fine-tuning. Both: fine-tune style, RAG facts.
  • Long context complements RAG; if the whole corpus fits comfortably in context, you may not need RAG at all.
  • Offline path: ingest, parse, clean, chunk, embed, index with metadata and ACLs. Online path: shape, route, retrieve, filter, rerank, build context, generate, cite, log.
  • Naive RAG is one retrieve-then-generate pass; advanced RAG adds pre-retrieval (query transformation) and post-retrieval (reranking, filtering) steps.
  • Retrieval quality caps answer quality, and parsing and chunking cap retrieval quality.
  • Scanned PDFs have no text layer; probe for text and route to OCR, or you silently index empty chunks.
  • Tables need table-aware extraction and structured serialisation (HTML or Markdown), or they become word salad.
  • RAG retrieves chunks, not documents; each chunk should hold one self-contained idea.
  • Smaller chunks are precise but lack context; larger chunks carry context but blur embeddings.
  • Recursive splitting at a few hundred tokens with 10-20 percent overlap is the usual baseline; count tokens with the embedder's tokenizer.
  • Parent-child retrieval matches small chunks and returns their larger parent for context.
  • Semantic chunking cuts at topic shifts via consecutive-sentence similarity; best for unstructured text, not always worth its cost.
  • Code chunks by function or class; logs by event; tables by row with headers repeated.
  • Bi-encoders embed query and document independently, so document vectors are precomputed; this enables fast first-stage retrieval.
  • Embedding dimension is fixed by the model and unrelated to chunk length; bigger is not automatically better.
  • Asymmetric embedders need their query and passage prefixes; missing prefixes silently cut recall.
  • Use the same embedding model and version for queries and documents; changing models means re-embedding everything.
  • Matryoshka embeddings can be truncated to shorter prefixes with graceful quality loss; ordinary embeddings cannot.
  • For unit vectors cosine equals dot product and L22 = 2 − 2cos, so all three rank identically.
  • Match the similarity metric to how the model was trained; database defaults are often L2.
  • Flat search is exact and the ground truth for measuring ANN recall.
  • IVF clusters vectors with k-means and scans only nprobe cells; nlist ≈ √N is a common start.
  • PQ splits vectors into m sub-vectors and stores one byte per sub-vector; ADC turns distance into table lookups.
  • IVF-PQ combines pruning and compression and encodes residuals; it is the billion-scale, RAM-constrained choice.
  • HNSW is a layered proximity graph with about log N search; M sets memory and baseline recall, efSearch is the query-time dial.
  • nprobe and efSearch trade latency for recall; they can never beat exact search.
  • FAISS is a library (no persistence, metadata or text); vector databases add storage, filtering, CRUD and operations.
  • Always store chunk text and source metadata with or alongside vectors for citation, filtering and debugging.
  • BM25 adds term-frequency saturation (k1) and length normalisation (b) to TF-IDF; it excels at exact terms and IDs.
  • Hybrid search fuses dense and sparse results; RRF sums 1/(k + rank) across lists (typically k = 60) and needs no score calibration.
  • Dense retrievers are trained contrastively (InfoNCE) with in-batch and hard negatives; beware false negatives.
  • Cross-encoders read query and document together and output a score; accurate but only affordable on a shortlist.
  • ColBERT stores token vectors and scores with MaxSim: near cross-encoder accuracy for more storage.
  • A reranker cannot recover a document the first stage never retrieved; fix recall first.
  • Query transformations: rewriting, follow-up condensing, multi-query, decomposition, step-back, HyDE, expansion, self-query filters.
  • Routing sends queries to the right source, index, model and prompt; the hybrid waterfall is rule, then embedding, then LLM.
  • Lost in the middle and distractor chunks mean fewer, better-ordered chunks beat more chunks.
  • Grounded prompts demand source-only answers, citations and an explicit "I don't know".
  • The oracle-context ablation separates retrieval errors from generation errors.
  • Retrieval metrics: hit rate, recall@k, precision@k, MRR, nDCG. Generation metrics: faithfulness, answer relevance, context precision and recall, EM and F1.
  • Faithfulness = supported claims / total claims; it catches unsupported details that relevance metrics miss.
  • Build a golden set from real questions with labelled evidence, include unanswerable ones, and calibrate LLM judges against humans.
  • Access control belongs in the retrieval layer as identity-derived metadata filters, never in the system prompt.
  • Semantic cache thresholds should be biased high because false hits cost far more than misses; scope caches by permissions.
  • LLM inference usually dominates RAG running cost; send fewer tokens and route easy queries to smaller models.
  • Freshness needs incremental re-indexing, stable chunk IDs, deletions, version and status metadata, and blue/green rebuilds.
  • Retrieved text is untrusted input: defend against prompt injection and poisoning.
  • Advanced patterns: conversational, iterative (chain-of-retrieval), Self-RAG, CRAG, adaptive, GraphRAG, multimodal, text-to-SQL, agentic.
Artificial Intelligence / Generative AI & LLMs

Agentic AI & Multi-Agent Systems

Read the full topic →

  • An agent is an LLM plus tools, memory and planning inside a control loop that repeats until a goal or a limit is reached.
  • The deciding question between workflow and agent: does code or the model choose the next step?
  • Use the least autonomy that works: single call, then chain, then router, then tool loop, then multi-agent. Do not use an agent when the path is known, latency is tight, volume is high and value is low, or every path must be pre-approved.
  • The agent loop: observe, think, act, observe the result, update memory, check stop conditions.
  • Stop conditions must be enforced in code: final answer, step limit, budget, no progress, needs human, fatal error.
  • ReAct interleaves Thought, Action and Observation; observations ground reasoning and prevent the compounding errors of closed-book chain-of-thought.
  • On multi-hop questions, ReAct beats standard prompting, CoT alone and act-only baselines because each observation can correct the next thought.
  • Text-based ReAct parses "Action:" lines and uses a stop sequence at "Observation:"; modern agents use native structured tool calls.
  • The LLM never executes tools; it emits a name and JSON arguments, and the application validates, authorizes and executes.
  • Models choose tools mainly from descriptions: say when to use, when not to, inputs, outputs and errors.
  • Return tool errors as observations so the model can self-correct; cap observation size.
  • Single agents degrade past roughly 15-20 tools (tool hallucination); fix with better tools, tool retrieval or specialist agents.
  • Plan-and-execute separates a planner (strong model, no tool calls) from executors (cheaper models that call tools), improving accuracy, cost and fault isolation.
  • Re-planning feeds step results back so the plan adapts when a step fails or reveals new facts.
  • ReWOO plans all steps up front with placeholders to cut LLM calls; DAG planners run independent steps in parallel.
  • Reflexion stores verbal lessons from failed attempts in memory and reuses them; reflection works best with an external signal such as tests.
  • Tree of Thoughts and tree search explore multiple reasoning branches at multiplied cost; self-consistency votes over several samples.
  • LLMs are stateless; memory is whatever you inject into the prompt.
  • Short-term memory is the thread's messages and state, persisted by a checkpointer under a thread id.
  • Long-term memory is usually a vector store the agent writes to: effectively RAG over self-written documents.
  • Cognitive memory types: working, episodic (experiences), semantic (facts), procedural (skills and rules).
  • Memory failures: state bloat, stale or contradictory facts, needle-in-a-haystack dilution, poisoning and cross-user leakage.
  • Timestamp every memory, provide update and delete tools, and retrieve only the top few relevant memories.
  • Agentic RAG makes retrieval a decision: route, plan queries, grade results, rewrite or fall back, check groundedness, all bounded by a hop limit.
  • Corrective RAG grades retrieved documents and falls back to web search; Self-RAG uses reflection tokens; Adaptive RAG routes by query complexity.
  • Core design patterns: prompt chaining, routing, parallelization (sectioning and voting), orchestrator-workers, evaluator-optimizer, autonomous agent, plus human-in-the-loop.
  • Multi-agent systems exist to keep each agent's tools and context small, specialize prompts, parallelize and isolate permissions.
  • Topologies: supervisor, hierarchical, network, swarm with hand-offs, sequential, debate, role-based crews, blackboard.
  • In hierarchical ReAct the supervisor's actions are delegations; specialists return synthesized observations, never raw payloads or credentials.
  • Multi-agent failure modes: delegation loops, state bloat, lost context in hand-offs, error propagation, groupthink and cost explosion.
  • State machines put deterministic tracks under a non-deterministic model; LangGraph uses typed state, nodes, edges, conditional edges and reducers.
  • Graphs with cycles enable think-act-observe loops that one-directional chains (DAGs) cannot express.
  • Checkpointing enables pause and resume, crash recovery, multi-turn threads, streaming and time-travel debugging.
  • Human-in-the-loop interrupts pause before irreversible actions; serialized state lets the wait last hours or days.
  • MCP (Anthropic, 2024; now Linux Foundation) connects hosts to servers via clients over JSON-RPC; servers expose tools (model-controlled), resources (application-controlled) and prompts (user-controlled).
  • MCP transports are stdio for local servers and streamable HTTP with OAuth-based authorization for remote ones.
  • A2A (Google, 2025; now Linux Foundation) lets opaque agents collaborate: Agent Cards for discovery, tasks with lifecycle states, messages with parts, and artifacts. MCP gives an agent hands; A2A lets agents hire each other.
  • Citation architectures: inline prompting, structured output with exact quotes, post-hoc attribution, and agentic state provenance; always verify cited ids in code.
  • Evaluate outcome, trajectory, tool-call accuracy, efficiency, safety and consistency; prefer final-state checks over judging text alone.
  • pass@k rewards any success in k tries; pass^k requires all k to succeed and measures reliability.
  • Know the benchmarks: SWE-bench (repo issues), GAIA (general assistant), tau-bench (tool-agent-user with policies), WebArena, OSWorld, AgentBench; beware contamination.
  • Compounding error: per-step success p over n steps gives pn; 0.95 over 20 steps is about 0.36, so shorten chains and verify steps.
  • Reliability tools: step and token budgets, timeouts, retries with backoff, idempotency keys, repeated-call detection, fallbacks and deterministic code paths.
  • Security: indirect prompt injection is the top agent risk; never combine private data, untrusted content and external communication without controls.
  • Defences: least privilege, authorization outside the model, validation and allowlists, approval gates, sandboxing, guardrails, audit logs and red teaming.
  • Trace every LLM call, tool call, retrieval, hand-off and approval as spans; alert on cost spikes and loop-limit hits.
  • Agent cost is dominated by re-sent context; use model tiering, prompt caching, trimming, summarization, caching and budgets, and measure cost per successful task.
Artificial Intelligence / Generative AI & LLMs

Fine-Tuning & Alignment

Read the full topic →

  • Fine-tuning continues gradient training of a pretrained model on your data; inference and prompting never change weights.
  • Rule of thumb: new knowledge calls for RAG, new behaviour calls for fine-tuning, and many systems use both.
  • Always baseline with the best prompt you can write and build an evaluation set before fine-tuning.
  • Continued pretraining uses raw domain text; SFT uses prompt–response pairs; preference tuning uses chosen/rejected pairs or rewards.
  • SFT is next-token cross-entropy with teacher forcing, usually on response tokens only (prompt labels set to -100).
  • Logits at position t predict token t+1, so shift logits and labels (and the mask) by one.
  • Use the model's own chat template via apply_chat_template; mismatched templates degrade quality.
  • Append the end-of-turn/EOS token to every target, or the model never learns to stop.
  • Tokenize prompt + answer once and use offsets to locate the answer boundary.
  • Quality beats quantity: a thousand clean, diverse examples can beat a hundred thousand noisy ones.
  • Deduplicate, scrub personal data, decontaminate against test sets, and split by group to avoid leakage.
  • Set maximum sequence length near the 95th percentile of tokenized length plus template overhead.
  • Packing concatenates short examples to cut padding waste; block cross-example attention when possible.
  • Full fine-tuning with mixed-precision AdamW needs about 16 bytes per parameter before activations: 112 GB for 7B.
  • Activations scale with batch × sequence × hidden × layers; gradient checkpointing trades roughly a third more compute for large savings.
  • Large vocabularies make the logits tensor a hidden memory hog; chunked or fused cross-entropy helps.
  • PEFT freezes the base and trains a small set of parameters: adapters, prefix/prompt tuning, IA3, BitFit, LoRA.
  • LoRA: h = W0x + (α/r)BAx, with A random and B zero so training starts from the pretrained model.
  • LoRA adds r·(din + dout) parameters per adapted matrix; rank 16 on all linear layers of a 7B model is about 40M parameters.
  • Targeting all linear layers usually matters more than increasing rank.
  • Keep α/r fixed when sweeping rank; rsLoRA uses α/√r for stability at high rank.
  • Merged LoRA adds zero inference latency; unmerged adapters add a small overhead but can be swapped.
  • DoRA separates magnitude and direction, training LoRA on the direction, and often helps at low rank.
  • QLoRA = NF4 4-bit frozen base + double quantization + paged optimizers + 16-bit LoRA adapters; compute is in bf16/fp16.
  • NF4 places 16 levels at normal-distribution quantiles with blockwise (64) absmax scaling.
  • Double quantization cuts scale-constant overhead from 0.5 to about 0.127 bits per parameter.
  • prepare_model_for_kbit_training upcasts norms, enables checkpointing and input gradients for k-bit training.
  • Typical LRs: full FT about 1e-5, LoRA about 2e-4, DPO about 5e-7 to 5e-6 (full) with β about 0.1.
  • 1–3 epochs, warmup 3–10%, cosine or linear decay, clipping at 1.0, and keep the best validation checkpoint.
  • RLHF = SFT, then a Bradley–Terry reward model from human comparisons, then PPO maximizing reward minus β·KL to the reference.
  • PPO-based RLHF keeps four models in memory: policy, reference, reward and value. The clip applies to ρ then multiplies by A; the objective is maximized.
  • DPO loss: −log σ(β [log(πθ(yw)/πref(yw)) − log(πθ(yl)/πref(yl))]); no reward model or sampling; smaller β allows more deviation from the reference.
  • ORPO and SimPO are reference-free; KTO uses unpaired thumbs-up/down data; IPO regularizes DPO's margins.
  • GRPO keeps PPO's clip but replaces the critic with group-standardized rewards over G samples; two models in memory instead of four; stalls when a group has zero reward variance.
  • RLAIF and Constitutional AI replace human preference labels with AI judgements guided by written principles.
  • Reward hacking, sycophancy and verbosity are classic preference-tuning failure modes; the KL penalty limits them.
  • Evaluate on held-out task data, with a validated LLM judge, on general benchmarks, safety suites and a regression suite, all against the base model.
  • Catastrophic forgetting is mitigated by PEFT, data replay, lower LR, fewer epochs, regularization and model merging.
  • Even benign fine-tuning can weaken safety refusals, so re-run safety evaluations on every release.
  • Merge adapters into a 16-bit base, never directly into 4-bit weights, then re-quantize if needed.
  • Model merging (linear, SLERP, task arithmetic, TIES, DARE) only works for models sharing a base.
  • Distillation trains a smaller student on teacher outputs or soft logits with temperature; check the teacher's terms of use.
  • Serve one merged model for high-traffic single tasks, or multi-LoRA over a shared base for many variants.
  • For local/edge deployment: merge, convert to GGUF, quantize (for example Q4_K_M), and re-evaluate with the right template.
Artificial Intelligence / Generative AI & LLMs

LLM Evaluation, Safety & Responsible AI

Read the full topic →

  • LLM evaluation is hard because outputs are open-ended, non-deterministic, multi-dimensional and subjective, and systems change constantly.
  • "Evaluation" covers output quality (correctness, faithfulness, tone, safety) and system performance (latency, cost, reliability).
  • Axes: offline vs online, reference-based vs reference-free, human vs automatic, component vs end-to-end, pointwise vs pairwise.
  • Use the cheapest reliable evaluator first: deterministic checks, then statistical, model-based, LLM judge, humans.
  • Human evaluation needs rubrics, blinding, multiple raters and chance-corrected agreement (Cohen's kappa, Fleiss' kappa, Krippendorff's alpha).
  • Kappa = (po − pe) / (1 − pe) with pe = ∑k p1,k p2,k; 80% raw agreement can be only moderate kappa.
  • Precision = TP/(TP+FP), recall = TP/(TP+FN), F1 is their harmonic mean; accuracy misleads on imbalanced data; exact match suits short answers, token F1 gives partial credit, Levenshtein counts character edits.
  • BLEU = clipped n-gram precision (geometric mean) times a brevity penalty; clip to the max count in any one reference; r is closest-reference length; use corpus-level scores.
  • ROUGE is recall-oriented n-gram (ROUGE-N) or LCS (ROUGE-L) overlap; for summarisation coverage.
  • METEOR adds stem and synonym matching, recall weighting and a fragmentation penalty.
  • BERTScore matches contextual token embeddings by cosine similarity; handles paraphrase but not factual errors.
  • Perplexity = exp(mean negative log-likelihood); lower is better; only comparable with the same tokenizer; measures fluency, not truth.
  • No n-gram metric suits reasoning tasks; use final-answer extraction, execution or a judge.
  • Key benchmarks: MMLU (knowledge), HellaSwag (commonsense), GSM8K (maths), HumanEval/MBPP (code, pass@k), TruthfulQA, BIG-bench, MT-Bench, Chatbot Arena, HELM.
  • pass@k unbiased estimator: 1 − C(n−c,k)/C(n,k).
  • Chatbot Arena ranks models from pairwise human votes using Elo / Bradley-Terry; style and length can inflate ratings.
  • Contamination (test data in training data) inflates benchmark scores; your private eval set is the real test.
  • LLM-as-a-judge: pointwise, pairwise, reference-guided, context-grounded; one criterion per call; reasoning before score; structured JSON.
  • Judge biases: position, verbosity, self-preference / same-family, style, limited knowledge, injection, authority/sycophancy, compassion/sentiment; mitigate with order swapping, rubrics, other-family judges, references.
  • Calibrate judges against expert labels (kappa, correlation) on a held-out set; re-validate after judge upgrades.
  • G-Eval: auto-generated evaluation steps plus form filling, optionally probability-weighted scores.
  • RAG triad: context relevance, faithfulness (groundedness), answer relevance; plus recall@k, MRR, correctness, citation accuracy, abstention.
  • Agent evals: task success verified by end state, tool correctness, trajectory efficiency, safety, reliability across repeated runs.
  • Golden datasets mix representative, edge, adversarial, unanswerable and regression cases, tagged by slice and versioned.
  • Synthetic eval data is fast but risks low diversity, unrealism and circularity; review a sample by hand.
  • SE of a pass rate = √(p(1−p)/n); 100 items gives about ±8 points at 95% confidence.
  • Eval-driven development: define evals first, compare baseline vs candidate on the same items, gate merges in CI, pin versions.
  • Use paired tests (McNemar, bootstrap) and read flipped examples before declaring a prompt better.
  • Online evaluation: shadow, canary, A/B tests with a primary metric plus guardrail metrics; explicit and implicit feedback.
  • Observability: traces as span trees; monitor latency, cost, reliability, quality, safety and drift; redact logs.
  • Hallucination types: factual, faithfulness, input-conflicting, self-contradictory, fabricated references, tool/field hallucination.
  • Hallucination is reduced by grounding, abstention, citations with verification, low temperature, validation, tools and human review; never eliminated.
  • Safety prevents harm the system causes; security protects the system from attackers; jailbreaks bridge both.
  • Risks: harmful content, bias, privacy/PII leakage, misinformation, over-reliance, IP, unsafe autonomy and over-refusal.
  • CIA triad: confidentiality, integrity, availability; ML attacks include membership inference, extraction, poisoning, backdoors, adversarial examples.
  • GCG finds gibberish adversarial suffixes via gradients that force an affirmative prefix; perplexity filters help against it.
  • PAIR uses an attacker LLM and judge to refine readable jailbreaks in about 20 queries.
  • Prompt injection works because instructions and data share one channel; indirect injection hides instructions in retrieved content.
  • Jailbreak families: language strategies, rhetoric, imaginary worlds, operational exploitation; plus low-resource languages, tense shifts and many-shot.
  • OWASP LLM Top 10 (2025): prompt injection, sensitive information disclosure, supply chain, poisoning, improper output handling, excessive agency, system prompt leakage, vector/embedding weaknesses, misinformation, unbounded consumption. 2023 had model DoS, insecure plugins, overreliance and model theft instead of the last four.
  • Defence in depth: least privilege, authorisation outside the model, human approval, input/output rails, egress control, monitoring.
  • Guardrails: rules, moderation models, Llama Guard-style classifiers, programmable rails, schema validation, grounding checks, action controls.
  • Red teaming (offence) plus blue teaming (defence) run continuously; attack success rate is paired with false-refusal rate.
  • Alignment: SFT, RLHF (reward model plus PPO with KL penalty), DPO, Constitutional AI (principles plus AI feedback).
  • Governance: model cards, impact assessments, EU AI Act tiers (unacceptable, high, limited, minimal), NIST AI RMF (Govern, Map, Measure, Manage).
Artificial Intelligence / Generative AI & LLMs

GenAI System Design & Case Studies

Read the full topic →

  • A GenAI system is a distributed system with a slow, costly, probabilistic, instructable component in the request path. Design to contain it.
  • Framework: clarify use case and users, then metrics, data, approach, architecture, deep-dives, evaluation, safety, cost and latency, iteration.
  • State success metrics early: one business metric, quality metrics, guardrail metrics and operational SLOs.
  • Approach ladder: prompting, then RAG, then fine-tuning, then agent, then multi-agent. Climb only when a requirement forces it.
  • RAG supplies knowledge that changes, with citations and permissions. Fine-tuning supplies behaviour, format, style and a cheaper small model.
  • Workflows are predictable and testable. Agents are flexible but compound errors: five 95%-reliable steps give about 77% end to end.
  • Reference stack: UI, API gateway, orchestrator, retrieval, tools, memory, model gateway, serving, guardrails, observability, LLMOps.
  • The model gateway centralises routing, failover, caching, keys, cost metering and data policy.
  • Routing sends each request to the cheapest adequate model. A cascade escalates on low confidence. A fallback chain covers outages.
  • Cascade expected cost = Csmall + pescalate × Clarge.
  • Prefill is compute-bound and sets TTFT. Decode is memory-bandwidth-bound and sets tokens per second.
  • KV bytes per token = 2 × layers × KV heads × head dimension × bytes. That is about 128 KB for an 8B model and 320 KB for a 70B model in FP16 with grouped-query attention.
  • Weight memory ≈ parameters × bytes per parameter (2 for BF16, 1 for FP8 or INT8, 0.5 for INT4).
  • Prefill FLOPs ≈ 2 × parameters × input tokens. Decode step time ≈ (weight bytes + active KV bytes) / bandwidth.
  • Continuous batching adds and removes sequences at every step, giving several times the throughput of static batching.
  • Paged attention stores the KV cache in blocks, removing fragmentation and fitting more concurrent sequences.
  • Paged attention stores KV in fixed-size blocks mapped by a block table. Contiguous waste is (Tmax − Tactual) × KV per token; paged waste is at most one block per sequence. Shared blocks enable prefix caching.
  • Prefix caching reuses the KV of identical prompt prefixes (saves about P / (P + U) of prefill). Put static content first; a timestamp at the top kills the cache.
  • Speculative decoding: expected tokens per target pass = (1 − αk+1) / (1 − α); speed-up ≈ that / (1 + k · c). Helps most at low batch sizes when draft acceptance is high.
  • INT8 / FP8 weights are about 2× smaller than BF16 and usually near-lossless. INT4 is about 4× smaller and faster for decode, but quality loss is larger on reasoning and maths; always re-evaluate and keep sensitive layers higher.
  • MoE models: size memory by total parameters and compute by active parameters.
  • Little's law: concurrency = arrival rate × time in system. Use it to check KV capacity.
  • End-to-end latency ≈ pre-processing + TTFT + (Nout − 1) × TPOT. The first token is already in TTFT; output length still dominates long answers.
  • Stream responses, parallelise independent steps, shrink prompts and outputs, and design to p95 rather than the average.
  • Cost per request = input tokens × input price + output tokens × output price. Output tokens usually cost several times more.
  • Conversation history is re-sent every turn, so summarise or window it.
  • Cheapest cost levers first: caching and prompt trimming. Then routing, batch APIs, distillation, self-hosting.
  • Self-hosting pays off at high, steady utilisation, or when compliance requires it.
  • Rate-limit by tokens, not just requests, with per-tenant guaranteed slices plus a shared burst pool.
  • Retry with exponential backoff and jitter. Never retry write tools without idempotency keys.
  • Fallback prompts must be evaluated per model, and failover targets must respect data residency.
  • The ingestion pipeline must handle incremental updates, fast deletes, permission sync and embedding version switches.
  • The feedback flywheel: production failures become evaluation cases, then fixes, then CI gates, then canary releases.
  • Version prompts, models, retrieval configuration, tool schemas, guardrail policies and evaluation sets.
  • Calibrate LLM-as-judge against human labels, and watch for length, position and self-preference bias.
  • Gate releases on per-slice metrics, because averages hide regressions in a language or a customer tier.
  • Never rely on the model for security. Enforce permissions, limits and approvals in code.
  • RAG access control means filtering inside the vector query before retrieval, never instructing the model to refuse.
  • Indirect prompt injection arrives through documents, web pages and emails. Treat that content as data and apply least privilege.
  • Semantic cache keys must be scoped by tenant and permission, or they leak data.
  • Hallucination control: ground, allow abstention, cite, constrain, verify, and route to a human.
  • "Zero hallucination" is achieved by making unverified output non-actionable, not by claiming the model never errs.
  • In regulated decisions the LLM produces evidence and suggestions, while deterministic rules or humans decide.
  • Industry cases are mostly extraction plus analytics plus workflow. Use cheap specialised models for the bulk and LLMs for the hard remainder.
  • Agents should suggest, and policy gates plus human approval should execute consequential actions.
Artificial Intelligence / Edge AI

Edge AI Fundamentals

Read the full topic →

  • Edge AI runs models near the data source, on a continuum from MCUs to devices, edge boxes, edge servers and the cloud; on-device AI is the extreme end.
  • Drivers: latency, privacy, cost, offline use, bandwidth, data sovereignty and personalization; costs: compute, memory, bandwidth, power, thermal, storage and fragmentation.
  • Most products are hybrid: a small local model for common cases with a clear rule for cloud fallback and a defined offline behaviour.
  • Phone DRAM bandwidth is ~50-100 GB/s versus 2-8 TB/s on data-centre GPUs; bandwidth sets a hard latency floor of bytes moved divided by bandwidth.
  • Moving data from DRAM costs orders of magnitude more energy than computing on it, so reuse, fusion and smaller data types save power.
  • CPU is flexible, GPU is parallel and FP16-friendly, NPU gives the best performance per watt for low-precision tensor maths, DSP suits always-on low-power tasks, MCUs run TinyML at milliwatts.
  • NPUs are efficient because of low-precision MAC arrays, on-chip SRAM reuse, fixed dataflow and ahead-of-time compilation, which is also why they want static shapes and static quantization.
  • Peak TOPS = MACs x 2 x clock; real utilization is often 20-50% because of bandwidth, operator coverage, batch size 1 and throttling.
  • Roofline: attainable performance = min(peak compute, bandwidth x arithmetic intensity); the ridge point is peak / bandwidth.
  • Batch-1 matrix-vector work (LLM decode) has ~2 ops/byte at INT8 and is deeply memory-bound; prefill with hundreds of tokens is compute-bound.
  • Model size is roughly parameters times bytes per parameter: 3B params is ~6 GB FP16, ~3 GB INT8, ~1.5-1.8 GB INT4.
  • A dense layer costs about 2 x inputs x outputs FLOPs; a transformer costs about 2 x parameters FLOPs per token plus attention; depthwise-separable convs cut 3x3 conv cost ~8-9x.
  • FP16 has more precision but overflows above 65,504; BF16 has FP32's range with less precision; FP8 comes as E4M3 (precision) and E5M2 (range).
  • Block formats (MXFP4, NVFP4, GGUF blocks) share one scale per small group; effective bits = element bits + scale bits / group size.
  • Quantization maps reals to integers with a scale and zero-point: x is approximately scale x (q - zero_point); symmetric sets zero_point to 0.
  • Rounding error is about scale squared / 12 in variance; each extra bit adds ~6 dB SQNR; calibration balances rounding error against clipping error.
  • Per-channel weight scales are standard for INT8; per-group scales (32-128) are standard for INT4 LLM weights; one outlier ruins a per-tensor scale.
  • Calibration methods: min-max, percentile, MSE and KL/entropy; calibration data must match production.
  • Static quantization fixes activation scales at compile time (NPU-friendly); dynamic computes them at runtime (CPU-friendly, no calibration).
  • Integer matmul accumulates in INT32 and requantizes with a fixed-point multiplier M = s_w x s_x / s_y; weight-only W4A16 dequantizes on the fly and wins on bandwidth.
  • PTQ needs only a calibration set; QAT uses fake quantization with the straight-through estimator and recovers accuracy at low bits. Try PTQ first.
  • Accuracy loss usually comes from outliers, sensitive layers or bad calibration; fix with per-channel scales, clipping, equalization, mixed precision, advanced PTQ or QAT, guided by layer-wise error analysis.
  • GPTQ rounds column by column with Hessian-based error compensation; AWQ scales up salient channels; SmoothQuant moves activation outliers into weights; rotations spread outliers.
  • GGUF is llama.cpp's single-file format; Q4_K_M (~4.8 bits) is a popular balance; imatrix improves low-bit quants.
  • CPU paths commonly use 4-bit group-wise weights with 8-bit dynamic activations; NPUs commonly use 4- or 8-bit weights with 16-bit static activations.
  • Unstructured pruning compresses but rarely speeds up mobile hardware; structured and N:M sparsity give real speed-ups.
  • Distillation trains a small student on a big teacher's soft outputs (temperature-scaled KL); most good small LLMs are distilled, often after pruning.
  • Low-rank factorization turns m x n parameters into r(m + n); LoRA adapters let one base model serve many features.
  • Hardware-aware NAS searches architectures using measured latency on the target device.
  • Operator fusion and BatchNorm folding remove memory round trips at no accuracy cost; static shapes, layout choice and memory planning matter as much.
  • Delegates/execution providers hand subgraphs to accelerators; every CPU fallback boundary adds copies and sync; check partition logs.
  • LiteRT is the new name of TensorFlow Lite; ExecuTorch is PyTorch's on-device runtime; ONNX Runtime is framework-agnostic; QNN/QAIRT is Qualcomm's NPU SDK; TensorRT serves Jetson; llama.cpp and MLC-LLM serve LLMs.
  • NNAPI is deprecated from Android 15 because of inconsistent drivers and fragmentation; use LiteRT delegates or vendor backends instead.
  • The pipeline is: export, optimize, quantize, convert, validate on host, deploy, run on accelerator, profile, iterate.
  • LLM prefill is compute-bound and sets time to first token (~2 x params x prompt tokens FLOPs); decode is memory-bound and sets tokens/s.
  • Decode tokens/s ceiling is roughly effective memory bandwidth divided by (weight bytes + KV bytes at the current context). Long chats slow down even with fixed weights.
  • KV cache = 2 x layers x KV heads x head_dim x context x bytes; at long context it can exceed the weights; GQA, KV quantization, windows and shorter context shrink it.
  • Paged KV: contiguous waste is (T_max - T_actual) x KV per token; a block table wastes at most one block per sequence and can share prefix blocks.
  • Prefix caching reuses KV for a byte-identical prefix; savings about P / (P + U). Put static tokens first.
  • Speculative decoding drafts k tokens and verifies them in one pass: expected tokens per pass = (1 - alpha^(k+1)) / (1 - alpha); speed-up ≈ that / (1 + k c). Unchanged output distribution; helps most at low batch when acceptance is high.
  • INT8 weights are ~2x smaller than FP16 and usually near-lossless. INT4 is ~4x smaller and the usual on-device decode lever, but quality loss is larger on reasoning and maths; keep embeddings and the LM head higher and always re-evaluate the task.
  • Realistic phone LLMs are about 0.5-4B parameters in INT4; 7-8B needs 12-16 GB+ RAM.
  • TinyML's binding constraint is usually peak activation memory in SRAM (the tensor arena), not weight size in flash.
  • Always-on features use cascades: hardware trigger, tiny model on DSP/MCU, verifier on NPU, heavy work last; false triggers cost energy.
  • Energy per inference (power x time) matters more than peak power; racing to idle often wins; plan for the sustained, throttled clock.
  • Use thermal headroom APIs to degrade gracefully (smaller model, lower fps) before the OS throttles hard.
  • Federated learning shares updates, not data (FedAvg); secure aggregation hides individual updates; differential privacy bounds what any individual's data can reveal.
  • Benchmark with warm-up, hundreds of runs, p50/p95/p99, sustained 10-30 minute runs, baseline rows, device tiers and real silicon only.
  • Pre- and post-processing can cost more than the model; profile the whole pipeline inside the real app.
  • Ship per-device-tier model variants, monitor latency and fallback in production, and keep a remote kill switch.
Artificial Intelligence / Edge AI

Edge AI Deployment & Projects

Read the full topic →

  • The deployment loop is choose, export, convert, quantize, compile for target, integrate, benchmark on device, monitor; stages 3-7 iterate many times.
  • Export captures a static graph (torch.export, ONNX); conversion produces a runtime format (.pte, .tflite, .gguf, .onnx/.ort, .mlpackage, QNN context binary).
  • NNAPI is deprecated from Android 15; new work uses LiteRT delegates/accelerators, ExecuTorch backends or ONNX Runtime execution providers. Confirm current LiteRT CompiledModel / Interpreter package names in official docs; do not invent flags.
  • ExecuTorch swaps backends by changing the partitioner (XNNPACK, Vulkan, QNN, MediaTek, Core ML) while keeping one export flow.
  • llama.cpp with GGUF is the fastest way to a working on-device LLM baseline; Q4_0 is repacked for fast Arm kernels, Q4_K_M is a quality-per-byte default.
  • For ExecuTorch LLM export, use_kv_cache and use_sdpa_with_kv_cache are essential; 8da4w means 8-bit dynamic activations, 4-bit weights.
  • KleidiAI kernels in XNNPACK add more than 20% prefill speed on Arm CPUs; always build Release.
  • Validate numerics after every conversion: FP32 differences should be around 1e-5 to 1e-4.
  • Never benchmark on an emulator: no realistic memory system, DVFS, thermal model or NPU.
  • Report load time (cold and warm), TTFT, prefill tok/s, decode tok/s, per-token p50/p90/p99, peak RSS, file size, energy per request and sustained/peak ratio.
  • Always state prompt length and generated length with LLM numbers; prefill and decode differ by 5-20x.
  • A fair protocol fixes the environment, cools down between runs, discards warm-up runs, repeats, and records device, build and runtime versions.
  • Peak RSS comes from VmHWM in /proc/<pid>/status; app memory breakdown from dumpsys meminfo.
  • Energy = integral of (power minus idle power) over time; use power rails via Perfetto, fuel gauge sampling, batterystats, or an external monitor.
  • A 10-30 minute thermal soak reveals throttling; phones typically sustain 60-85% of peak.
  • Quantization quality needs both perplexity and task-level evaluation on a fixed set including hard cases.
  • Layer-wise error analysis compares each intermediate tensor to an FP32 reference using cosine similarity, MAE, max error and SQNR.
  • SQNR = 10 log10(signal power / error power); each bit adds about 6 dB; low SQNR on the residual stream predicts quality loss.
  • Single-layer sensitivity (quantize one layer at a time) separates fragile layers from layers receiving bad inputs.
  • Typical sensitive parts: outlier activation channels, MLP down projection, LM head, first/last blocks, norms and softmax.
  • Mixed precision promotes only the most sensitive layers to 8-bit or FP16 and prices each promotion in size and latency.
  • Host fake-quant matching FP32 but device diverging means a device implementation issue (rounding, accumulators, FP16 overflow), not the scheme.
  • NPUs need static quantization: ranges fixed at compile time from calibration; LLMs on Hexagon commonly use W4A16 or W8A16.
  • NPUs need static shapes; export separate prefill (chunked) and decode (one token) graphs that share weights.
  • One unsupported op mid-graph creates partitions and CPU round trips that can make the NPU slower than the CPU.
  • Context binaries are finalized graphs for one Hexagon architecture and SDK version; loading them avoids on-device compilation.
  • Hexagon library folders (v73, v75, v79) must match the SoC; set ADSP_LIBRARY_PATH for DSP-side libraries.
  • Large models are split into several context binaries because of NPU session memory limits.
  • KV bytes = 2 x layers x KV heads x head dim x tokens x bytes; Llama-3.2-1B uses 32 KiB per token in FP16 (verified: 2 × 16 × 8 × 64 × 2).
  • At long context the KV cache exceeds the weights (about 22-35k tokens for 1B INT4, about 16-20k for 3B, under 8k for models without GQA).
  • Decode speed falls as context grows because each step reads the weights plus the whole cache: tok/s ≤ BW / (W + KV(t)).
  • INT8 weights are usually near-lossless; INT4 is the decode lever but needs a task eval, plus higher precision on embeddings and the LM head.
  • Quantize keys per channel (outlier channels) and values per token; INT8 KV is near-lossless.
  • Sliding-window attention bounds memory; keep attention-sink tokens to avoid collapse on long streams.
  • Paged KV: contiguous waste is (T_max - T_actual) x KV per token; paged waste is at most one block. Prefix caching saves about P / (P + U) of prefill if the prefix is byte-identical.
  • Speculative decoding: E[tokens per pass] = (1 - alpha^(k+1)) / (1 - alpha); speed-up ≈ E / (1 + k c). Helps on-device when decode is bandwidth-bound and the draft is accurate.
  • lmkd kills by oom_score_adj under memory pressure (PSI); anonymous memory is not reclaimable, mmapped clean file pages are.
  • Store model assets uncompressed (noCompress) so they can be memory-mapped; downloaded models need resume, checksum, atomic install and a smoke test.
  • Create sessions once, off the main thread, warm them up, reuse buffers, make generation cancellable and respond to onTrimMemory.
  • ONNX Runtime's QNN EP needs a QDQ model; disable CPU fallback during development and enable EP context caching. Provider option keys (backend_path, HTP performance mode) are version-specific: check the current ORT QNN EP docs.
  • QAIRT converter and Genie binary/API names move between SDK releases. Copy flags, JSON keys and C symbols from the samples for the version you link; do not invent them.
  • Profile at three levels: system (Perfetto), runtime (per-op profiles, AI Hub, QNN profiler, ETDump), native (simpleperf).
  • Accuracy drift is bisected: original, converted FP32, FP16, quantized host, quantized device, full app pipeline; pre-processing bugs are the most common cause.
  • Shipping needs device tiering, versioned manifests, staged rollout with halt thresholds, A/B tests, a kill switch and a fallback chain (NPU, GPU, CPU, cloud, refuse).
  • Monitor load success, backend used, fallback rate, latency by tier, low-memory kills and thermal state, segmented by SoC, RAM and OS build.
Leadership & Career / Leadership & Behavioral

Leadership, Delivery & Behavioral

Read the full topic →

  • Senior loops are weighted towards design and leadership; every story is scored on scope, scale, ambiguity and impact.
  • Frame yourself as the DRI for delivery: capacity, sequencing, quality gates and status of record.
  • Be honest about budget ownership; talk cost as people-weeks, cost of delay and cost of escapes.
  • Resource playbook: map demand vs capacity, cut coordination tax, limit WIP, build slack, measure, grow the bench.
  • Real capacity is roughly 60-70% of nominal after meetings, support and leave; reserve some for escalations.
  • Cross-functional squads with one backlog beat narrow silos when load is high and variable.
  • Publish a prioritization stack: safety and regulatory, ship blockers, platform milestones, strategic debt, nice-to-haves.
  • Defects found late cost far more than those found early; quality gates are cheaper than escapes.
  • Decision loop: name it and the deadline, classify Type 1/2, two or three options, evidence, roles, commit in public, review.
  • Type 1 decisions are hard to reverse, so slow down; Type 2 decisions are reversible, so decide fast with a tested rollback.
  • RACI clarifies task ownership (one Accountable); RAPID clarifies who Decides on contested calls.
  • Use a pre-mortem to surface hidden risks and ADRs to record why decisions were made.
  • With incomplete data: directional evidence, stated assumptions, rollback plan and a parallel mitigation.
  • Status reports: RAG against dates and KPIs, what changed, what is blocked, the explicit ask. Bad news the same day.
  • Escalate early, jointly where possible, with the problem, impact, options, recommendation and deadline.
  • Managing up: no surprises, frame updates in their goals, agree decision rights, disagree privately and support publicly.
  • Distributed teams: async first, written decisions, rotated meeting times, follow-the-sun handoffs.
  • A handoff note has hypothesis, eliminated paths, artifacts, next action and the named next DRI.
  • With fixed resources you hold two of scope, date and quality; never let quality be the hidden variable.
  • MoSCoW and phased delivery control scope; safety paths are never deferred.
  • A risk register tracks likelihood, impact, owner, mitigation and trigger; responses are avoid, mitigate, transfer, accept.
  • The critical path is the longest dependent chain; protect it and put buffers at the end, not on every task.
  • Systemic issue playbook: own, reproduce, instrument, localize, drive, decide, close with regression test and post-mortem.
  • Mentoring ramp: shadow, reverse-shadow, own; explain why in reviews; give credit to the mentee.
  • Feedback with SBI: situation, behaviour, impact; then listen and agree next steps.
  • Diagnose underperformance by clarity, capability, capacity and motivation before acting.
  • Disagree with data, not ego; once decided, commit as if it were your idea.
  • Say no by returning with options (full, phased, workaround), quantified impact and a recommendation.
  • Influence without authority comes from the map, clear asks, credibility, reciprocity and pilots.
  • STAR plus Learning: short Situation and Task, long Action with "I", quantified Result, one-line Learning.
  • Build 10-12 stories, each covering two or three prompts, with at least two stories for common themes.
  • Fill in real numbers and remove confidential details before the interview; rehearse aloud to about three minutes.
  • High-frequency gaps: fake failures, "we" with no "I", mentoring with no mentee outcome, refusing in the room, win/lose disagreement, waiting for a spec, overpromising dates, watermelon status, no "I changed my mind" story.
  • Strong shapes: real impact plus a gate you added; 48 hours and three options; restate their constraint then data; structure first in ambiguity; same-day amber; public course-correct.
  • Inclusion is a mechanism (written comments, rotated facilitation, stop interruptions), not a slogan.
  • Ethics answers need a real pressure to hide risk or waive a safety gate, and a schedule or scope cost you accepted.
  • Prepare two stories each for failure, conflict, ambiguity and people-development; do not recycle one plot all morning.