--- title: fault-injection keywords: - Apache APISIX - API Gateway - Plugin - Fault Injection - fault-injection description: This document contains information about the Apache APISIX fault-injection Plugin. --- ## Description The `fault-injection` Plugin can be used to test the resiliency of your application. This Plugin will be executed before the other configured Plugins. The `abort` attribute will directly return the specified HTTP code to the client and skips executing the subsequent Plugins. The `delay` attribute delays a request and executes the subsequent Plugins. ## Attributes | Name | Type | Requirement | Default | Valid | Description | |-------------------|---------|-------------|---------|------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------| | abort.http_status | integer | required | | [200, ...] | HTTP status code of the response to return to the client. | | abort.body | string | optional | | | Body of the response returned to the client. Nginx variables like `client addr: $remote_addr\n` can be used in the body. | | abort.headers | object | optional | | | Headers of the response returned to the client. The values in the header can contain Nginx variables like `$remote_addr`. | | abort.percentage | integer | optional | | [0, 100] | Percentage of requests to be aborted. | | abort.vars | array[] | optional | | | Rules which are matched before executing fault injection. See [lua-resty-expr](https://github.com/api7/lua-resty-expr) for a list of available expressions. | | delay.duration | number | required | | | Duration of the delay. Can be decimal. | | delay.percentage | integer | optional | | [0, 100] | Percentage of requests to be delayed. | | delay.vars | array[] | optional | | | Rules which are matched before executing fault injection. See [lua-resty-expr](https://github.com/api7/lua-resty-expr) for a list of available expressions. | :::info IMPORTANT To use the `fault-injection` Plugin one of `abort` or `delay` must be specified. ::: :::tip `vars` can have expressions from [lua-resty-expr](https://github.com/api7/lua-resty-expr) and can flexibly implement AND/OR relationship between rules. For example: ```json [ [ [ "arg_name","==","jack" ], [ "arg_age","==",18 ] ], [ [ "arg_name2","==","allen" ] ] ] ``` This means that the relationship between the first two expressions is AND, and the relationship between them and the third expression is OR. ::: ## Enable Plugin You can enable the `fault-injection` Plugin on a specific Route as shown below: :::note You can fetch the `admin_key` from `config.yaml` and save to an environment variable with the following command: ```bash admin_key=$(yq '.deployment.admin.admin_key[0].key' conf/config.yaml | sed 's/"//g') ``` ::: ```shell curl http://127.0.0.1:9180/apisix/admin/routes/1 -H "X-API-KEY: $admin_key" -X PUT -d ' { "plugins": { "fault-injection": { "abort": { "http_status": 200, "body": "Fault Injection!" } } }, "upstream": { "nodes": { "127.0.0.1:1980": 1 }, "type": "roundrobin" }, "uri": "/hello" }' ``` Similarly, to enable a `delay` fault: ```shell curl http://127.0.0.1:9180/apisix/admin/routes/1 -H "X-API-KEY: $admin_key" -X PUT -d ' { "plugins": { "fault-injection": { "delay": { "duration": 3 } } }, "upstream": { "nodes": { "127.0.0.1:1980": 1 }, "type": "roundrobin" }, "uri": "/hello" }' ``` You can also enable the Plugin with both `abort` and `delay` which can have `vars` for matching: ```shell curl http://127.0.0.1:9180/apisix/admin/routes/1 -H "X-API-KEY: $admin_key" -X PUT -d ' { "plugins": { "fault-injection": { "abort": { "http_status": 403, "body": "Fault Injection!\n", "vars": [ [ [ "arg_name","==","jack" ] ] ] }, "delay": { "duration": 2, "vars": [ [ [ "http_age","==","18" ] ] ] } } }, "upstream": { "nodes": { "127.0.0.1:1980": 1 }, "type": "roundrobin" }, "uri": "/hello" }' ``` ## Example usage Once you have enabled the Plugin as shown above, you can make a request to the configured Route: ```shell curl http://127.0.0.1:9080/hello -i ``` ``` HTTP/1.1 200 OK Date: Mon, 13 Jan 2020 13:50:04 GMT Content-Type: text/plain Transfer-Encoding: chunked Connection: keep-alive Server: APISIX web server Fault Injection! ``` And if we configure the `delay` fault: ```shell time curl http://127.0.0.1:9080/hello -i ``` ``` HTTP/1.1 200 OK Content-Type: application/octet-stream Content-Length: 6 Connection: keep-alive Server: APISIX web server Date: Tue, 14 Jan 2020 14:30:54 GMT Last-Modified: Sat, 11 Jan 2020 12:46:21 GMT hello real 0m3.034s user 0m0.007s sys 0m0.010s ``` ### Fault injection with criteria matching You can enable the `fault-injection` Plugin with the `vars` attribute to set specific rules: ```shell curl http://127.0.0.1:9180/apisix/admin/routes/1 -H "X-API-KEY: $admin_key" -X PUT -d ' { "plugins": { "fault-injection": { "abort": { "http_status": 403, "body": "Fault Injection!\n", "vars": [ [ [ "arg_name","==","jack" ] ] ] } } }, "upstream": { "nodes": { "127.0.0.1:1980": 1 }, "type": "roundrobin" }, "uri": "/hello" }' ``` Now, we can test the Route. First, we test with a different `name` argument: ```shell curl "http://127.0.0.1:9080/hello?name=allen" -i ``` You will get the expected response without the fault injected: ``` HTTP/1.1 200 OK Content-Type: application/octet-stream Transfer-Encoding: chunked Connection: keep-alive Date: Wed, 20 Jan 2021 07:21:57 GMT Server: APISIX/2.2 hello ``` Now if we set the `name` to match our configuration, the `fault-injection` Plugin is executed: ```shell curl "http://127.0.0.1:9080/hello?name=jack" -i ``` ``` HTTP/1.1 403 Forbidden Date: Wed, 20 Jan 2021 07:23:37 GMT Content-Type: text/plain; charset=utf-8 Transfer-Encoding: chunked Connection: keep-alive Server: APISIX/2.2 Fault Injection! ``` ## Delete Plugin To remove the `fault-injection` Plugin, you can delete the corresponding JSON configuration from the Plugin configuration. APISIX will automatically reload and you do not have to restart for this to take effect. ```shell curl http://127.0.0.1:9180/apisix/admin/routes/1 -H "X-API-KEY: $admin_key" -X PUT -d ' { "uri": "/hello", "plugins": {}, "upstream": { "type": "roundrobin", "nodes": { "127.0.0.1:1980": 1 } } }' ```