# gVisor

### What is gVisor?

gVisor is a userspace kernel used for sandboxing untrusted programs. It is
Linux-like kernel implementation in Go.

NOTE: The project uses [Bazel](https://bazel.build/) as a build system, so paths
starting with `//` refer to the repository root, and `//some/path:some_target`
refers to target `some_target` defined inside `some/path/BUILD`.

gVisor is made of multiple components:

-   The gVisor Sentry is the main component implementing a Linux-like kernel in
    Go. Its implementation is at `//pkg/sentry`.
-   Gofers are I/O proxies for the gVisor Sentry. There are multiple Gofers, but
    an implementation of one is at `//runsc/fsgofer`.
-   `runsc` is a Docker-compatible OCI-compliant container runtime that uses
    gVisor under the hood. It is useful to run Docker images in gVisor.
    Location: `//runsc`.

### Important gVisor subpackages

-   The gVisor "platform" acts as a syscall interception mechanism for sandboxed
    applications, and which is responsible for redirecting control to the gVisor
    sentry for handling the system call. There are multiple platform
    implementations; the name "platform" refers to the abstraction. Location:
    `//pkg/sentry/platform`
-   The gVisor `kernel` package implements the bulk of the kernel logic.
    Location: `//pkg/sentry/kernel`
-   The Linux kernel ABI is redefined as a Go package so that we can use it in
    the gVisor kernel. Location: `//pkg/abi`.
-   The `runsc` CLI is the main entry-point into gVisor for most usage.
-   gVisor may also be used programmatically using a subprocess-like API via
    language bindings with the `sandboxexec` API; see `//sandboxexec`.

### gVisor code conventions

See `//.gemini/styleguide.md` for a general style guide.

In the gVisor codebase, all Go packages use the same package name as the name of
their parent directory. For example:

-   The package at `//pkg/errors/linuxerr` (which is the same as the BUILD
    target `//pkg/errors/linuxerr:linuxerr`) has Go import path
    `gvisor.dev/gvisor/pkg/errors/linuxerr`.
-   The package at `//pkg/sentry/vfs` has Go import path
    `gvisor.dev/gvisor/pkg/sentry/vfs`.
-   This implies that new libraries you may create must be under their own new
    subfolder. For example, if you wish to create a library called `foo` under
    an existing directory `//pkg/bar`, you should create the directory
    `//pkg/bar/foo` and then create a file `//pkg/bar/foo/foo.go` within it. The
    `BUILD` file at `//pkg/bar/foo/BUILD` would contain the `go_library` rule
    with name `foo`, and the `package` line within `//pkg/bar/foo/foo.go` would
    be `package foo`. The import statement for this package in other Go
    libraries would be `gvisor.dev/gvisor/pkg/bar/foo`, and the dependency
    target would be `//pkg/bar/foo`. Other Go source files within
    `//pkg/bar/foo` generally also belong to the same Go package, though it is
    also acceptable to use the `foo_test` package name for test files in the
    same `//pkg/bar/foo` directory, if having a separate package name is
    desirable.
-   gVisor's Bazel rules are wrappers over the usual ones, especially for
    `go_library`, `go_binary`, and `go_test`. They should be imported into
    `BUILD` files from `//tools:defs.bzl`, e.g. `load("//tools:defs.bzl",
    "go_library")` for the `go_library` wrapper. These wrapper rules create
    auto-generated Go source files that are added to the source code of each Go
    package, enabling code generation for Go code that gets injected into every
    package, such as automated serialization. You generally do not need to
    inspect these autogenerated Go source files, but be aware that they exist,
    and that what you see in the repository is not the entire picture that the
    Go compiler actually sees.
-   When writing tests that need to read external files as input, you should use
    the `testutil.FindFile` function defined in `//pkg/test/testutil` with
    import path `gvisor.dev/gvisor/pkg/test/testutil`. It takes in a path as
    argument, which is rooted from the `//` directory. For example, to access
    file `//foo/bar/baz.txt`, you would call
    `testutil.FindFile("foo/bar/baz.txt")`. Every external file that you expect
    to be able to find using `testutil.FindFile` must also be declared as a
    `data` dependency in the `BUILD` target of the `go_test` target you are
    editing. For example, to access file `//foo/bar/baz.txt`, the `go_test`
    BUILD rule would need to have `data = ["//foo/bar:baz.txt"]`. If you want an
    example, look at existing usages of this function to understand how it
    works, e.g. `dumpfilter_test.go` in `//runsc/boot/filter/dumpfilter`.
-   Despite being a bazel project, gVisor uses a Docker-based wrapper over bazel
    using `make`. The `Makefile` defines wrapper rules over `bazel` subcommands,
    which run these commands inside an automatically-managed Docker container.
    This container is useful to smooth over Bazel quirks and version differences
    across machines. For example, building a target is `make build
    TARGETS=//path/to:target`, and running a test is `make test
    TARGETS=//path/to:some_test`.
-   Comments must describe the code as it is for the future codebase readers,
    not for present-time PR reviewers. A comment that talks about a bug that the
    PR at hand fixes is counter-productive, because it leaves behind irrelevant
    historical context that future readers of the codebase should no longer have
    to worry about.
-   Function/struct/field comments should generally not refer to their callers
    (e.g. "Used by", "Useful for"). If this is not already clear from the
    function name or context, then the function name and/or context is the
    problem, and the comment is a band-aid.
-   Prose (such as comments, documentation, other markdown files) should mostly
    follow ASD-STE100 Simplified Technical English. Some liberties are expected
    (and welcome in some contexts, such as website/blog post contents), but
    technical content should be crisp, clear, and to-the-point.
-   gVisor is packaged as a fileset at `//:release`, and any consumers of this
    must treat this as an opaque, all-or-nothing set of files to install or set
    up. With narrow exceptions, they should not enumerate the set of files
    within.

### General approach to resolving gVisor issues

#### Modifying gVisor kernel behavior

Use the `gvisor-kernel-development` skill.

#### Security first

All changes are reviewed for security. gVisor's design and architecture ensures
that a Sentry compromise, however unlikely, is still tightly contained.
Nonetheless, any changes to the Sentry are very security-sensitive and must be
reviewed carefully to avoid introducing new escalation vectors. See
`SECURITY.md` for various aspects of gVisor security.

#### Compatibility in practice

gVisor aims to match broad Linux-like behavior, but this is a moving target and
some incompatibilities matter more than others in practice. If there exists no
popular real-world application that runs into the compatibility problem that a
PR is addressing, the PR review will be de-prioritized against those that do
have popular real-world applications.

The pecking order, from most important to least, with compatibility code:

-   `COMPATIBILITY:REGRESSION-GVISOR`: Regressions in real-world application
    compatibility of a previously-supported application, due to past changes in
    gVisor
-   `COMPATIBILITY:REGRESSION-APP`: Regressions in real-world application
    compatibility of a previously-supported application, due to changes/updates
    by the application
-   `COMPATIBILITY:UPCOMING-KERNEL`: Incompatibilities not yet encountered, but
    known to later be encountered due to an upcoming change in the Linux kernel.
-   `COMPATIBILITY:NEW-APP`: Incompatibilities encountered in real-world
    applications that never worked in gVisor before, in their default
    configuration/use-case.
-   `COMPATIBILITY:NEW-APP-CONFIG`: Incompatibilities encountered in real-world
    applications that never worked in gVisor before, in a specific non-default
    configuration/use-case.
-   `COMPATIBILITY:PREVENTIVE-APP`: Syscall-test-only change that verifies a
    piece of **working** gVisor compatibility that isn't yet covered by syscall
    tests, which is known to be relied upon by a real-world application.
-   `COMPATIBILITY:UPCOMING-APP`: Incompatibilities not yet encountered, but
    known to later be encountered due to an upcoming change in an application
    that currently works in gVisor.
-   `COMPATIBILITY:SYNTHETIC`: Incompatibilities exercisable only via syscall
    tests, which no known real-world application encounters.

#### Performance sensitivity

As a kernel, gVisor performance matters almost as much as security. As such, any
change that may impact performance is required to come with benchmark
measurements to measure its benefit and to avoid regressions.

The following code is particularly performance-sensitive:

-   The system call hot path: entry and dispatch in `pkg/sentry/kernel`
    (`task_run.go`, `task_syscall.go`), the handlers in
    `pkg/sentry/syscalls/linux`, and the layers that copy data to and from
    application memory (`pkg/usermem`, `pkg/safemem`, `pkg/safecopy`,
    `pkg/marshal`).
-   Platform code (`pkg/sentry/platform`, `pkg/ring0`).
-   Serialization code (`pkg/state`).
-   Sandbox startup (`runsc/boot`, `runsc/container`, `runsc/sandbox`).
-   Netstack (`pkg/tcpip`).
-   Checkpoint/restore (`pkg/state`, `pkg/sentry/state`).
-   File I/O (`pkg/sentry/vfs`, `pkg/sentry/fsimpl`).
-   Memory management (`pkg/sentry/mm`, `pkg/sentry/pgalloc`).
-   Inter-process communication with the gofer (`pkg/lisafs`, `runsc/fsgofer`).
-   Go `init()` functions that run on every binary startup.

Load the `gvisor-performance` skill whenever any of these paths show up, or when
a change has any possibility of performance implications. Test-only changes do
not have performance implications.

## Git / GitHub / PR Guidelines

Before sending PRs, check the following:

-   Compatibility fixes must come with a compatibility code (see "Compatibility
    in practice" section), along with an explanation for this code.
-   Syscall behavior must be verified to match Linux behavior by way of syscall
    tests, not Go tests.
-   Everything must build; use `make build TARGETS=...` building affected paths.
-   Relevant tests must pass; use `make test TARGETS=...` or `make syscall-tests
    TARGETS=...` as appropriate.
-   All PRs must pass `make lint` and `make lint-cc`.
-   When benchmarks are required, they must come with statistically-significant
    figures. See section on performance changes.
-   Website changes should come with before/after screenshots (use `make
    website-server` to start an HTTP server serving the site locally).
-   Test-only changes that improve stability/flakiness are very much welcome.