--- name: os-dev-scratch description: OS development from scratch skill for bootloader through context switching. Use when building a minimal x86-64 OS, setting up GDT/IDT/page tables, writing keyboard/serial drivers, or using QEMU for kernel boot. Activates on queries about bootloader, long mode, page tables, IDT, PIC/APIC, xv6, or x86_64-elf-gcc. --- # OS Development from Scratch ## Purpose Guide agents through building a minimal operating system from scratch: bootloader stages (BIOS/GRUB vs UEFI/limine), 64-bit long mode setup with GDT and page tables, IDT and interrupt handlers, PIC/APIC configuration, basic keyboard and serial drivers, physical and virtual memory managers, context switching, with xv6-RISC-V as a reference architecture. ## When to Use - Learning how an OS boots from power-on to `main()` - Implementing protected/long mode transitions on x86-64 - Writing a physical memory allocator (bitmap) and page table manager - Handling timer, keyboard, and page fault interrupts - Testing with QEMU `-kernel` and cross-compiler `x86_64-elf-gcc` - Porting concepts from xv6 to a custom x86 or RISC-V kernel ## Workflow ### 1. Boot stages overview ``` BIOS path (legacy) ├── BIOS POST ├── MBR (512 bytes) → boot sector loads stage2 ├── GRUB/multiboot → loads kernel ELF └── kernel entry (_start) UEFI path (modern) ├── UEFI firmware ├── EFI bootloader (limine, systemd-boot) ├── Loads kernel + initrd from ESP └── kernel entry (handoff with memory map) ``` ### 2. Toolchain setup ```bash # Cross-compiler for bare metal brew install x86_64-elf-gcc x86_64-elf-binutils # macOS # or build from source / apt install gcc-x86-64-elf x86_64-elf-gcc --version # QEMU for testing qemu-system-x86_64 --version ``` Linker script essentials: ```ld /* linker.ld */ ENTRY(_start) SECTIONS { . = 0x100000; /* 1MB — typical kernel load address */ .text : { *(.text .text.*) } .rodata : { *(.rodata .rodata.*) } .data : { *(.data .data.*) } .bss : { *(.bss .bss.*) } } ``` ```bash x86_64-elf-gcc -ffreestanding -nostdlib -c kernel.c -o kernel.o x86_64-elf-ld -T linker.ld kernel.o -o kernel.elf ``` ### 3. Multiboot/limine boot ```bash # QEMU direct kernel boot (no disk) qemu-system-x86_64 \ -kernel kernel.elf \ -serial stdio \ -m 128M \ -no-reboot -no-shutdown # With limine (UEFI) qemu-system-x86_64 \ -bios /usr/share/ovmf/OVMF.fd \ -drive file=disk.img,format=raw \ -serial stdio ``` ### 4. Long mode setup ``` Protected mode (32-bit) → enable PAE → setup 4-level page tables → enable long mode ``` ```c // Minimal GDT entry (64-bit flat segments) struct gdt_entry { uint16_t limit_low; uint16_t base_low; uint8_t base_mid; uint8_t access; uint8_t granularity; uint8_t base_high; } __attribute__((packed)); // Page table setup (4KB pages, identity map first 1GB) uint64_t pml4[512] __attribute__((aligned(4096))); uint64_t pdpt[512] __attribute__((aligned(4096))); uint64_t pd[512] __attribute__((aligned(4096))); void setup_paging(void) { for (int i = 0; i < 512; i++) pd[i] = (i * 0x200000) | 0x83; // 2MB huge pages pdpt[0] = (uint64_t)pd | 0x03; pml4[0] = (uint64_t)pdpt | 0x03; __asm__ volatile("mov %0, %%cr3" :: "r"(pml4)); } ``` ### 5. IDT and interrupt handlers ```c struct idt_entry { uint16_t offset_low; uint16_t selector; uint8_t ist; uint8_t type_attr; uint16_t offset_mid; uint32_t offset_high; uint32_t zero; } __attribute__((packed)); // ISR stub (assembly) → common handler → dispatch by vector void interrupt_handler(struct trap_frame *frame) { if (frame->vector == 14) // page fault handle_page_fault(frame->cr2, frame->error_code); else if (frame->vector == 33) // keyboard IRQ remapped keyboard_handler(); } ``` ```bash # Test page fault # QEMU monitor: info registers ``` ### 6. PIC and APIC ```c // Legacy PIC remapping (8259) // Remap IRQ 0-15 to vectors 32-47 outb(0x20, 0x11); outb(0xA0, 0x11); outb(0x21, 0x20); outb(0xA1, 0x28); // vector offsets // ... // Modern: use APIC/IOAPIC (ACPI MADT parsing) // LAPIC timer for preemption ``` ### 7. Serial and keyboard drivers ```c // COM1 serial (0x3F8) void serial_putc(char c) { while ((inb(0x3F8 + 5) & 0x20) == 0); outb(0x3F8, c); } // PS/2 keyboard scancode → ASCII lookup table void keyboard_handler(void) { uint8_t scancode = inb(0x60); char c = scancode_to_ascii[scancode]; if (c) serial_putc(c); outb(0x20, 0x20); // EOI to PIC } ``` ```bash qemu-system-x86_64 -kernel kernel.elf -serial stdio # printk output appears in terminal ``` ### 8. Physical memory manager ```c // Bitmap allocator over usable RAM regions // From multiboot memory map or UEFI memory map #define PAGE_SIZE 4096 uint8_t *frame_bitmap; uint64_t total_frames; uint64_t alloc_frame(void) { for (uint64_t i = 0; i < total_frames; i++) { if (!test_bit(frame_bitmap, i)) { set_bit(frame_bitmap, i); return i * PAGE_SIZE; } } return 0; // OOM } ``` ### 9. Context switching ```c struct context { uint64_t rax, rbx, rcx, rdx, rsi, rdi, rbp, rsp; uint64_t r8, r9, r10, r11, r12, r13, r14, r15; uint64_t rip; }; void switch_context(struct context *old, struct context *new); // Assembly: save callee-saved regs to old, restore from new, ret to new->rip ``` Cooperative scheduling first; add timer IRQ preemption later. ### 10. xv6-RISC-V reference ```bash git clone https://github.com/mit-pdos/xv6-riscv cd xv6-riscv && make qemu ``` | xv6 component | x86 equivalent | |---------------|----------------| | `kernel/vm.c` | Page table management | | `kernel/trap.c` | IDT/interrupt dispatch | | `kernel/proc.c` | Context switch, scheduler | | `kernel/plic.c` | PIC/APIC interrupt controller | | `user/usys.pl` | System call stubs | ## Common Problems | Symptom | Cause | Fix | |---------|-------|-----| | Triple fault on boot | Invalid GDT/IDT or stack | Set up stack before enabling interrupts | | QEMU black screen | No serial output configured | `-serial stdio`; early `serial_init` | | Page fault in kernel | Unmapped address | Identity-map kernel; check CR3 | | IRQ never fires | PIC mask or IDT not loaded | `lidt`; unmask IRQ in PIC | | Timer doesn't tick | LAPIC not initialized | Parse ACPI; calibrate LAPIC timer | | Linker relocation error | Wrong load address | Match `linker.ld` with bootloader expectation | ## Related Skills - `skills/low-level-programming/assembly-x86` — x86-64 assembly for ISR stubs - `skills/low-level-programming/assembly-riscv` — xv6-RISC-V reference ISA - `skills/platform/riscv-privileged` — RISC-V trap handling and page tables - `skills/virtualization/qemu-kvm` — QEMU flags for kernel development - `skills/kernel/kernel-internals` — Linux implementation of these concepts - `skills/low-level-programming/linux-kernel-modules` — graduate to Linux once basics work