# strfry - a nostr relay

![strfry logo](docs/strfry.svg)

strfry is a relay for the [nostr protocol](https://github.com/nostr-protocol/nostr)

* Supports most applicable NIPs: 1, 2, 4, 9, 11, 28, 40, 42, 45, 70, 77
* No external database required: All data is stored locally on the filesystem in LMDB
* Hot reloading of config file: No server restart needed for many config param changes
* Zero downtime restarts, for upgrading binary without impacting users
* Websocket compression using permessage-deflate with optional sliding window, when supported by clients. Optional on-disk compression using zstd dictionaries.
* Durable writes: The relay never returns an `OK` until an event has been confirmed as committed to the DB
* Built-in support for real-time streaming (up/down/both) events from remote relays, and bulk import/export of events from/to jsonl files
* [negentropy](https://github.com/hoytech/negentropy)-based set reconcilliation for efficient syncing with clients or between relays, accurate counting of events between relays, and more
* Prometheus metrics endpoint for monitoring relay activity (client/relay messages by verb, events by kind)

If you are using strfry, please [join our telegram chat](https://t.me/strfry_users). Hopefully soon we'll migrate this to nostr.

<hr>

<!-- TOC FOLLOWS -->
<!-- START OF TOC -->
<!-- DO NOT EDIT! Auto-generated by md-toc: https://github.com/hoytech/md-toc -->

* [Setup](#setup)
    * [Compile](#compile)
* [Operating](#operating)
    * [Running a relay](#running-a-relay)
    * [Configuration](#configuration)
    * [Selecting and Deleting Events](#selecting-and-deleting-events)
    * [Importing data](#importing-data)
    * [Exporting data](#exporting-data)
        * [Fried Exports](#fried-exports)
    * [Upload](#upload)
    * [Download](#download)
    * [Sync](#sync)
* [Monitoring](#monitoring)
    * [Prometheus Metrics](#prometheus-metrics)
* [Advanced](#advanced)
    * [DB Upgrade](#db-upgrade)
    * [DB Compaction](#db-compaction)
    * [Zero Downtime Restarts](#zero-downtime-restarts)
    * [Plugins](#plugins)
    * [Router](#router)
    * [Syncing](#syncing)
    * [Compression Dictionaries](#compression-dictionaries)
* [Learn More](#learn-more)
* [Author and Copyright](#author-and-copyright)

<!-- END OF TOC -->

## Setup

### Compile

A C++20 compiler is required, along with a few other common dependencies.

On Debian/Ubuntu use these commands:

    sudo apt install -y git g++ make libssl-dev zlib1g-dev liblmdb-dev libflatbuffers-dev libsecp256k1-dev libzstd-dev
    git clone https://github.com/hoytech/strfry && cd strfry/
    git submodule update --init
    make setup-golpe
    make -j4

FreeBSD has slightly different commands (warning: possibly out of date):

    pkg install -y gcc gmake cmake git perl5 openssl lmdb flatbuffers libuv libinotify zstr secp256k1 zlib-ng
    git clone https://github.com/hoytech/strfry && cd strfry/
    git submodule update --init
    gmake setup-golpe
    gmake -j4

To upgrade strfry, do the following:

    git pull
    make update-submodules
    make -j4


## Operating

### Running a relay

Here is how to run the relay:

    ./strfry relay

For dev/testing, the config file `./strfry.conf` is used by default. It stores data in the `./strfry-db/` directory.

By default, it listens on port 7777 and only accepts connections from localhost. In production, you'll probably want a systemd unit file and a reverse proxy such as nginx to support SSL and other features.

### Configuration

In order to operate, strfry needs a `strfry.conf` config file. It searches in the following locations, using the first one it finds:

1. The `--config` option, for example: `strfry --config /path/to/strfry.conf relay`
1. The `STRFRY_CONFIG` environment variable
1. `/etc/strfry.conf`
1. `./strfry.conf`

The `strfry.conf` distributed in the repo shows the default values for all options. All parameters are optional, except for `db`. In fact, this is a perfectly valid `strfry.conf`:

    db = "/path/to/db/"

However, in addition to `db` these are commonly used config parameters:

* `relay.bind`: By default it only listens for connections from localhost. This is fine during development and testing, or if you are putting a reverse proxy in front of strfry, but if you want the outside world to connect directly to your relay, you should change this to `0.0.0.0` (which means any IP can connect).
* `relay.port`: By default this is `7777`, but you may want to listen on a different port.
* `relay.auth.serviceUrl`: This is the external address of your relay to the outside world. It must be set for NIP-42 AUTH to work, typically something like `wss://my-relay.com`.
* `relay.info.name`, `relay.info.description`, etc: These are shown in the NIP-11 output and the HTML landing page served by strfry. They provide users extra information about who runs this relay, and its policies.

### Selecting and Deleting Events

Because strfry uses a custom LMDB schema, there is no SQL interface for managing the DB. Instead, regular nostr filters (as described in [NIP-01](https://nips.nostr.com/1)) can be used for basic tasks.

For example, `strfry scan` can be used to select all events matching a particular nostr filter:

    ./strfry scan '{"kinds":[0,1]}'

Each matching event will be printed on its own line (in other words, in JSONL format).

The `strfry delete` command can be used to delete events from the DB that match a specified nostr filter. For example, to delete all events from a particular pubkey, use the following command:

    ./strfry delete --filter '{"authors":["4c7a4fa1a6842266f3f8ca4f19516cf6aa8b5ff6063bc3ec5c995e61e5689c39"]}'

### Importing data

The `strfry import` command reads line-delimited JSON (jsonl) from its standard input and imports events that validate into the DB in batches of 10,000 at a time:

    cat my-nostr-dump.jsonl | ./strfry import

* By default, it will verify the signatures and other fields of the events. If you know the messages are valid, you can speed up the import a bit by passing the `--no-verify` flag.

### Exporting data

The `strfry export` command will print events from the DB to standard output in jsonl, ordered by their `created_at` field (ascending).

Optionally, you can limit the time period exported with the `--since` and `--until` flags. Normally exports will be in ascending order by `created_at` (oldest first). You can reverse this with `--reverse`.

#### Fried Exports

If you pass the `--fried` argument to `strfry export`, then the outputed JSON lines will include `fried` elements. This is precomputed data that strfry can use to re-import these events more quickly. To take advantage of this, use the `--fried` flag on import as well.

This can be especially useful for upgrading strfry to a new, incompatible database version. See the [fried exports](https://github.com/hoytech/strfry/blob/master/docs/fried.md) documentation for more details on the format.



### Upload

This command uses the regular nostr websocket protocol to upload events to a remote relay. The events are read from stdin, and are assumed to be in minified JSONL format (one event per line).

For example:

    cat my-events.jsonl | ./strfry upload wss://relay.example.com

This command will try to maintain 50 events in-flight at once. This "pipeline depth" can be increased or decreased with the `--pipeline` argument.

If you want to send events from your own local strfry DB, you can use either [`strfry export`](#exporting-data) or the `strfry scan` command if you want to use an arbitrary filter. For example:

    ./strfry scan '{"kinds":[0]}' | ./strfry upload wss://relay.example.com

Note that `strfry upload` will exit once it has completed reading the full input stream. If you want a process that stays running and continuously uploads events added to your DB, use [`strfry router`](#router).

### Download

This command is the complement of upload. It connects to a remote relay, performs a `REQ` request to retrieve events matching a filter, and then prints them to stdout.

For example:

    ./strfry download wss://relay.example.com > my-events.jsonl

The above uses the default "everything" filter (`{}`). To specify a more narrow filter, use the `--filter` flag:

    ./strfry download wss://relay.example.com --filter '{"kinds":[0]}' > my-events.jsonl

Note that many relays will put an implicit limit on the number of events you can retrieve with a single REQ. To retrieve more, typically you would do "pagination" by setting the `until` field to the oldest event's `created_at` and then performing another query.

`strfry download` also has a `--range` that can be used to conveniently specify `since/until` filter flags. Some examples:

    --range 2h-    ## since 2 hours ago until present
    --range 1Y-6M  ## since 1 year ago until 6 months ago

The units are: s=seconds, m=minutes, h=hours, d=days, w=weeks, M=months, Y=years

Note that `strfry download` will exit once it has retrieved all of the stored events from the remote relay. If you want a process that stays running and continuously downloads new events as they come in, use [`strfry router`](#router).

### Sync

This command uses the [negentropy](https://github.com/hoytech/negentropy) protocol and performs a set reconcilliation between the local DB and the specified relay's remote DB.

That is a fancy way of saying that it figures out which events the remote relay has that it doesn't, and vice versa. Assuming that both sides have some events in common, it does this more efficiently than simply transferring the full set of events (or even just their ids). You can read about the algorithm used in our [article on Range-Based Set Reconciliation](https://logperiodic.com/rbsr.html).

In addition to the C++ implementation used by strfry, negentropy has also been implemented in Javascript, Rust, Go, and more.

Here is how to perform a "full DB" set reconcilliation against a remote server:

    ./strfry sync wss://relay.example.com

This will download all missing events from the remote relay and insert them into your DB. Similar to `stream`, you can also sync in the `up` or `both` directions:

    ./strfry sync wss://relay.example.com --dir both

`both` is especially efficient, because performing the set reconcilliation automatically determines the missing members on each side.

Instead of a "full DB" sync, you can also sync the result of a nostr filter (or multiple filters, use a JSON array of them):

    ./strfry sync wss://relay.example.com --filter '{"authors":["..."]}'

`strfry sync` also supports the `--range` flag documented in [`strfry download`](#download).

Warning: Syncing can consume a lot of memory and bandwidth if the DBs are highly divergent (for example if your local DB is empty and your filter matches many events).

By default strfry keeps a precomputed BTree to speed up full-DB syncs. You can also cache BTrees for arbitrary filters, see the [syncing](#syncing) section for more details.



## Monitoring

### Prometheus Metrics

strfry includes built-in Prometheus metrics support for monitoring relay activity. Metrics are exposed via HTTP at the `/metrics` endpoint on the same port as the relay WebSocket server.

For example, if your relay is running on `localhost:7777`, you can access metrics at `http://localhost:7777/metrics`

The following metrics are available:

* **`nostr_client_messages_total{verb}`** - Total number of messages received from clients, broken down by verb (EVENT, REQ, CLOSE, NEG-OPEN, NEG-MSG, NEG-CLOSE)
* **`nostr_relay_messages_total{verb}`** - Total number of messages sent to clients, broken down by verb (EVENT, OK, EOSE, NOTICE, NEG-MSG, NEG-ERR)
* **`nostr_events_total{kind}`** - Total number of events processed, broken down by event kind (0, 1, 3, 4, etc.)

To scrape these metrics with Prometheus, add a job to your `prometheus.yml`:

```yaml
scrape_configs:
  - job_name: 'strfry'
    static_configs:
      - targets: ['localhost:7777']
    metrics_path: '/metrics'
```

Note that the prometheus metrics are global only per strfry process. So, if you are running multiple strfry processes on the same port (`REUSE_PORT`) then the metrics will not be comprehensive. In addition, metrics are reset whenever the strfry process is restarted.



## Advanced

### DB Upgrade

In the past, incompatible changes have been made to the DB format. If you try to use a `strfry` binary with an incompatible DB version, an error will be thrown. Only the `strfry export` command will work.

In order to upgrade the DB, you should export and then import again using [fried exports](#fried-exports):

    ./strfry export --fried > dbdump.jsonl
    mv strfry-db/data.mdb data.mdb.bak
    ./strfry import --fried < dbdump.jsonl

After you have confirmed everything is working OK, the `dbdump.jsonl` and `data.mdb.bak` files can be deleted.


### DB Compaction

The `strfry compact` command creates a raw dump of the LMDB file (after compaction) and stores in the specified file (use `-` to print to stdout). It cannot be used for DB upgrade purposes. It can however be useful for reclaiming space caused by fragmentation, or for migrating a DB to a new server that is running the same version of strfry.

To reclaim space, it is recommended to actually stop strfry for a compaction:

    ## ... stop strfry ...
    ./strfry compact - > strfry-db/data.mdb.compacted
    mv strfry-db/data.mdb.compacted strfry-db/data.mdb
    ## ... start strfry ...

For migration purposes, no restart is required to perform the compaction.


### Zero Downtime Restarts

strfry can have multiple different running instances simultaneously listening on the same port because it uses the `REUSE_PORT` linux socket option. One of the reasons you may want to do this is to restart the relay without impacting currently connected users. This allows you to upgrade the strfry binary, or perform major configuration changes (for the subset of config options that require a restart).

If you send a `SIGUSR1` signal to a strfry process, it will initiate a "graceful shutdown". This means that it will no longer accept new websocket connections, and after its last existing websocket connection is closed, it will exit.

So, the typical flow for a zero downtime restart is:

* Record the PID of the currently running strfry instance.

* Start a new relay process using the same configuration as the currently running instance:

      strfry relay

  At this point, both instances will be accepting new connections.

* Initiate the graceful shutdown:

      kill -USR1 $OLD_PID

  Now only the new strfry instance will be accepting connections. The old one will exit once all its connections have been closed.


### Plugins

When hosting a relay, you may not want to accept certain events. To avoid having to encode that logic into strfry itself, we have a plugin system. Any programming language can be used to build a plugin using a simple line-based JSON interface.

In addition to write-policy plugins, plugins can also be used inside [strfry router](#router) to determine which events to stream up/down to other relays.

See the [plugin documentation](https://github.com/hoytech/strfry/blob/master/docs/plugins.md) for details and examples.

See the file [docs/community.md](https://github.com/hoytech/strfry/blob/master/docs/community.md) for a list of community plugins. Please create a PR to add your own!




### Router

If you are building a "mesh" topology of routers, or mirroring events to neighbour relays (up and/or down), you can use `strfry router`. The router system handles many streams in one process, supports pre-filtering events using nostr filters and/or [plugins](#plugins), and more. See the [router documentation](https://github.com/hoytech/strfry/blob/master/docs/router.md) for more details.



### Syncing

The most original feature of strfry is a set reconcillation protocol based on [negentropy](https://github.com/hoytech/negentropy). This is implemented over a [nostr protocol extension](https://github.com/hoytech/strfry/blob/master/docs/negentropy.md) that allows two parties to synchronise their sets of stored messages with minimal bandwidth overhead. Negentropy can be used by both clients and relays.

The results of arbitrary nostr filter expressions can be synced. Relays can maintain BTree data-structures for pre-configured filters, improving the efficiency of commonly synced queries (such as the full DB). Whenever two parties to the sync share common subsets of identical events, then there will be significant bandwidth savings compared to downloading the full set. In addition to syncing, negentropy can also be used to compute accurate event counts for a filter across multiple relays, without having to download the entire filter results from each relay.

The `strfry negentropy` command can be used to manage the pre-configured queries to sync.

`negentropy list` will list the current BTrees. Here we see we have one filter, `{}` which matches the full DB:

    $ strfry negentropy list
    tree 1
      filter: {}
      size: 483057
      fingerprint: 9faaf0be1c25c1b4ee7e65f18cf4b352

This filter will be useful for full-DB syncs, and for syncs that use only `since`/`until`.

To add a new filter, use `negentropy add`. For example:

    $ strfry negentropy add '{"kinds":[0]}'
    created tree 2
      to populate, run: strfry negentropy build 2

Note that the tree starts empty. To populate it, use the `negentropy build` command with the newly created tree ID:

    $ strfry negentropy build 2
    $ strfry negentropy list
    tree 1
      filter: {}
      size: 483057
      fingerprint: 9faaf0be1c25c1b4ee7e65f18cf4b352
    tree 2
      filter: {"kinds":[0]}
      size: 33245
      fingerprint: 37c005e6a1ded72df4b9d4aa688689db

Now negentropy queries for kind 0 (optionally including `since`/`until`) can be performed efficiently and statelessly.



### Compression Dictionaries

Although nostr events are compressed during transfer using websocket compression, they are stored uncompressed on disk by default. In order to attempt to reduce the size of the strfry DB, the `strfry dict` command can be used to compress these events while still allowing them to be efficiently served via a relay. Only the raw event JSON itself is compressed: The indices needed for efficient retrieval are not. Since the indices are often quite large, the relative effectiveness of this compression depends on the type of nostr events stored.

`strfry dict` uses [zstd dictionaries](https://facebook.github.io/zstd/#small-data) to compress events. First you must build one or more dictionaries with `strfry dict train`. You can provide this command a nostr filter and it will select just these events. You may want to use custom dictionaries for certain kinds of events, or segment based on some other criteria. If desired, dictionary training can happen entirely offline without interfering with relay operation.

After building dictionaries, selections of events can be compressed with `strfry dict compress` (events also selected with nostr filters). These events will be compressed with the indicated dictionary, but will still be served by the relay. Use the compress command again to re-compress with a different dictionary, or use `dict decompress` to return it to its uncompressed state.

`strfry dict stats` can be used to print out stats for the various dictionaries, including size used by the dataset, compression ratios, etc.



## Learn More

For details on strfry's architecture, see the [architecture.md](https://github.com/hoytech/strfry/blob/master/docs/architecture.md) document.

To report issues or submit pull requests, please visit the [strfry github page](https://github.com/hoytech/strfry).

To chat with the strfry devs and community, please [join our telegram chat](https://t.me/strfry_users).

There is a list of community contributions in [docs/community.md](https://github.com/hoytech/strfry/blob/master/docs/community.md). Please create a PR to add your own!



## Author and Copyright

strfry © 2023-2026 Doug Hoyte and contributors.

GPLv3 license. See the LICENSE file.