# Running wasi workload natively on kubernetes using crun

Crun natively supports running wasm/wasi workload on using `wasmedge`, `wasmer`, `wasmtime` and `wamr`.
Each one of them (`wasmedge`, `wasmer`, `wasmtime` and `wamr`) comes with their own set of unique features.
For instance `wasmer` can compile your `.wat` on the fly. Similarly `wasmedge` has its own perks. `wamr` has a layered JIT architecture which can tier up during runtime.
Crun can support only one of them at a time. Please build crun with whatever runtime suits you the best.

#### How does crun detects if is a wasm workload ?
* Make sure oci config propagated by your CRI implementation contains annotations `run.oci.handler: wasm` or `module.wasm.image/variant=compat`.
* Entrypoint must point to a valid **.wat** (webassembly text) `**wasmer-only**` or **.wasm** (webassembly binary).
* If your kubernetes infrastructure is using service-mesh, proxy or in general side-cars pattern then in such cases recommended annotations would be
  `module.wasm.image/variant=compat-smart` or `run.oci.handler: wasm-smart`.


So spec generated by CRI implementation must contain annotation something like.

 ```json
...
"annotations": {
  "run.oci.handler": "wasm"
},
...
```

### Building `wasm` images
* Read here: https://github.com/containers/crun/blob/main/docs/wasm-wasi-example.md#running-oci-wasm-compat-images-with-buildah-and-podman

### Supported and tested CRI implementations

##### CRI-O

* Following features works completely out if the box once `cri-o` is using `crun` built with `wasm` support.
* Configure `cri-o` to use `crun` instead of `runc` by editing config at `/etc/crio/crio.conf` read more about it here https://docs.openshift.com/container-platform/3.11/crio/crio_runtime.html#configure-crio-use-crio-engine
* As of `cri-o` version `1.31` it defaults to `crun`, but the bundled `crun` may not have been built with `wasm` support.
* Restart `cri-o` by `sudo systemctl restart crio`
* `cri-o` automatically propagates pod annotations to container spec. So we don't need to do anything.

```yaml
apiVersion: v1
kind: Pod
metadata:
  name: pod-with-wasm-workload
  namespace: mynamespace
  annotations:
    module.wasm.image/variant: compat
spec:
  containers:
  - name: wasm-container
    image: myrepo/mywasmimage:latest
```

##### Containerd
* `Containerd` supports switching container runtime via custom config defined at `/etc/containerd/config.toml`
* Configure `containerd` to use `crun` by making sure runtime binary points to `crun` read more about config here https://github.com/containerd/containerd/blob/main/docs/cri/config.md
* Configure `containerd` to whilelist `wasm` annotations so they could propagated to OCI spec. By setting `pod_annotations` in the config.
```toml
pod_annotations = ["*.wasm.*", "wasm.*", "module.wasm.image/*", "*.module.wasm.image", "module.wasm.image/variant.*"]
```
* Restart `containerd` by `sudo systemctl start containerd`
* Now `containerd` should propagate `wasm` pod annotations to containers.

```yaml
apiVersion: v1
kind: Pod
metadata:
  name: pod-with-wasm-workload
  namespace: mynamespace
  annotations:
    module.wasm.image/variant: compat
spec:
  containers:
  - name: wasm-container
    image: myrepo/mywasmimage:latest
```

### Known Issues:
* ~~CRI implementations as of now propagates annotations to `side-car` as well so if `side-car` has a non `wasm` workload it can break. Read on going discussion here: https://github.com/containers/crun/issues/829~~, issue still exists with CRI but crun supports a workaround with feature added here: https://github.com/containers/crun/pull/886