# RESTful API in Sync & Async Rust _11 May 2021 · #rust · #diesel · #rocket · #sqlx · #actix-web_ **Table of Contents** - [Intro](#intro) - [General](#general) - [Project Setup](#project-setup) - [Loading Environment Variables w/dotenv](#loading-environment-variables-wdotenv) - [Handling Dates & Times w/chrono](#handling-dates--times-wchrono) - [Logging w/fern](#logging-wfern) - [JSON Serialization w/serde](#json-serialization-wserde) - [Domain Modeling](#domain-modeling) - [Sync Implementation](#sync-implementation) - [SQL Schema Migrations w/diesel-cli](#sql-schema-migrations-wdiesel-cli) - [Executing SQL Queries w/Diesel](#executing-sql-queries-wdiesel) - [Mapping DB Enums to Rust Enums](#mapping-db-enums-to-rust-enums) - [Fetching Data](#fetching-data) - [Inserting Data](#inserting-data) - [Updating Data](#updating-data) - [Deleting Data](#deleting-data) - [Using a Connection Pool w/r2d2](#using-a-connection-pool-wr2d2) - [Refactoring DB Operations Into a Module](#refactoring-db-operations-into-a-module) - [HTTP Routing w/Rocket](#http-routing-wrocket) - [Routing Basics](#routing-basics) - [GET Requests](#get-requests) - [POST & PATCH Requests](#post--patch-requests) - [DELETE Requests](#delete-requests) - [Refactoring API Routes Into a Module](#refactoring-api-routes-into-a-module) - [Authentication](#authentication) - [Async Implementation](#async-implementation) - [SQL Schema Migrations w/sqlx-cli](#sql-schema-migrations-wsqlx-cli) - [Executing SQL Queries w/sqlx](#executing-sql-queries-wsqlx) - [Fetching Data](#fetching-data-1) - [Inserting Data](#inserting-data-1) - [Updating Data](#updating-data-1) - [Deleting Data](#deleting-data-1) - [Compile-Time Verification of SQL Queries](#compile-time-verification-of-sql-queries) - [Using a Connection Pool w/sqlx](#using-a-connection-pool-wsqlx) - [Refactoring DB Operations Into a Module](#refactoring-db-operations-into-a-module-1) - [HTTP Routing w/actix-web](#http-routing-wactix-web) - [Routing Basics](#routing-basics-1) - [GET Requests](#get-requests-1) - [POST & PATCH Requests](#post--patch-requests-1) - [DELETE Requests](#delete-requests-1) - [Refactoring API Routes Into a Module](#refactoring-api-routes-into-a-module-1) - [Authentication](#authentication-1) - [Benchmarks](#benchmarks) - [Servers](#servers) - [Methodology](#methodology) - [Measuring Resource Usage](#measuring-resource-usage) - [Results](#results) - [Read-Only Workload](#read-only-workload) - [Reads + Writes Workload](#reads--writes-workload) - [Concluding Thoughts](#concluding-thoughts) - [Diesel vs sqlx](#diesel-vs-sqlx) - [Rocket vs actix-web](#rocket-vs-actix-web) - [Sync Rust vs Async Rust](#sync-rust-vs-async-rust) - [Rust vs JS](#rust-vs-js) - [In Summary](#in-summary) - [Discuss](#discuss) - [Further Reading](#further-reading) - [Notifications](#notifications) ## Intro Let's implement a RESTful API server in Rust for an imaginary Kanban-style project management app. A popular real-world example of such an app is Trello: ![trello board](../assets/trello-board.png) On its surface Kanban is pretty simple: there's a board and cards. The board represents a project. The cards represent tasks. The position of the cards on the board represents the state and progress of the tasks. The simplest boards have 3 columns for tasks which are: queued (to do), in progress (doing), and done (done). Despite being simple on its surface, Kanban, and all kinds of project management software in general, is a literal bottomless pit of complexity. There's a million things we could implement, and after we finish the first million things there would be a million more. However, since I'm trying to write a single article and not an entire book series let's keep the feature scope tiny. The server should support the ability to: - Create boards - Boards have names - Get a list of all boards - Delete boards - Create cards - Can associate cards with boards - Cards have descriptions and statuses - Get a list of all the cards on a board - Get a board summary: count of all the cards on the board grouped by their status - Update cards - Delete cards And that's it! To make this project slightly more interesting let's also include token-based authentication for all of the server's endpoints, but let's keep it simple: as long as a request contains a valid token it has access to all of the boards and cards. Furthermore, to satisfy my own curiosity, and to maximize the educationalness of this article, we're going to write two implementations together: one using sync Rust and the other using async Rust. The first implementation will use r2d2, Diesel, and Rocket. The second implementation will use sqlx, and actix-web. Here's a quick preview of the crates we'll be using for this project: General crates - dotenv (loading environment variables) - log + fern (logging) - chrono (date & time handling) - serde + serde_json (JSON de/serialization) Sync crates - diesel-cli (DB schema migrations) - diesel + diesel-derive-enum (ORM / building SQL queries) - r2d2 (DB connection pool) - rocket + rocket_contrib (HTTP routing) Async crates - sqlx-cli (DB schema migrations) - sqlx (executing SQL queries & DB connection pool) - actix-web (HTTP routing) - futures (general future-related utilities) After finishing both sync and async implementations we'll run some benchmarks to see which has better performance, because everyone loves benchmarks. ## General ### Project Setup All of the boring instructions for setting this project up, like installing Docker and running locally, are in the [companion code repository](https://github.com/pretzelhammer/kanban). For this article let's focus entirely on the fun part: the Rust! After the initial setup we have this empty `Cargo.toml` file: ```toml # Cargo.toml [package] name = "kanban" version = "0.1.0" edition = "2018" ``` And this empty main file: ```rust // src/main.rs fn main() { println!("Hello, world!"); } ``` ### Loading Environment Variables w/dotenv crates - dotenv ```diff # Cargo.toml [package] name = "kanban" version = "0.1.0" edition = "2018" +[dependencies] +dotenv = "0.15" ``` This crate does one small simple job: it loads variables from an `.env` in the current working directory and adds them to the program's environment variables. Here's the general `.env` file we'll be using: ```bash # .env LOG_LEVEL=INFO LOG_FILE=server.log DATABASE_URL=postgres://postgres@localhost:5432/postgres ``` Updated main file which uses `dotenv`: ```rust // src/main.rs type StdErr = Box; fn main() -> Result<(), StdErr> { // loads env variables from .env dotenv::dotenv()?; // example assert_eq!("INFO", std::env::var("LOG_LEVEL").unwrap()); Ok(()) } ``` ### Handling Dates & Times w/chrono crates - chrono ```diff # Cargo.toml [package] name = "kanban" version = "0.1.0" edition = "2018" [dependencies] dotenv = "0.15" +chrono = "0.4" ``` Rust's go-to library for handling dates & times is chrono. We're not using the dependency in our project _just yet_ but will very soon after we add a few more dependencies. ### Logging w/fern crates - log - fern ```diff # Cargo.toml [package] name = "kanban" version = "0.1.0" edition = "2018" [dependencies] dotenv = "0.15" chrono = "0.4" +log = "0.4" +fern = "0.6" ``` Log is Rust's logging facade library. It provides the high-level logging API but we still need to pick an implementation, and the implementation we're going to use is the fern crate. Fern allows us to easily customize the logging format and also chain multiple outputs so we can log to stderr and a file if we wanted to. After adding log and fern let's encapsulate all of the logging configuration and initialization into its own module: ```rust // src/logger.rs use std::env; use std::fs; use log::{debug, error, info, trace, warn}; pub fn init() -> Result<(), fern::InitError> { // pull log level from env let log_level = env::var("LOG_LEVEL").unwrap_or("INFO".into()); let log_level = log_level .parse::() .unwrap_or(log::LevelFilter::Info); let mut builder = fern::Dispatch::new() .format(|out, message, record| { out.finish(format_args!( "[{}][{}][{}] {}", chrono::Local::now().format("%H:%M:%S"), record.target(), record.level(), message )) }) .level(log_level) // log to stderr .chain(std::io::stderr()); // also log to file if one is provided via env if let Ok(log_file) = env::var("LOG_FILE") { let log_file = fs::File::create(log_file)?; builder = builder.chain(log_file); } // globally apply logger builder.apply()?; trace!("TRACE output enabled"); debug!("DEBUG output enabled"); info!("INFO output enabled"); warn!("WARN output enabled"); error!("ERROR output enabled"); Ok(()) } ``` And then add that module to our main file: ```diff // src/main.rs +mod logger; type StdErr = Box; fn main() -> Result<(), StdErr> { dotenv::dotenv()?; + logger::init()?; Ok(()) } ``` If we run the program now, since `INFO` is the default logging level, here's what we'd see: ```bash $ cargo run [08:36:30][kanban::logger][INFO] INFO output enabled [08:36:30][kanban::logger][WARN] WARN output enabled [08:36:30][kanban::logger][ERROR] ERROR output enabled ``` ### JSON Serialization w/serde crates - serde - serde_json ```diff # Cargo.toml [package] name = "kanban" version = "0.1.0" edition = "2018" [dependencies] dotenv = "0.15" - chrono = "0.4" + chrono = { version = "0.4", features = ["serde"] } log = "0.4" fern = "0.6" + serde = { version = "1.0", features = ["derive"] } + serde_json = "1.0" ``` Pro-tip: when adding a new dependency to a project it's good to look through existing dependencies to see if they have the new dependency as a feature flag. In this case chrono has serde as a feature flag, which if enabled, adds `serde::Serialize` and `serde::Deserialize` impls to all of chrono's types. This will allow us to use chrono types in our own structs later which we will also derive `serde::Serialize` and `serde::Deserialize` impls for. ### Domain Modeling Okay, let's start modeling our domain. We know we will have boards so: ```rust #[derive(serde::Serialize)] #[serde(rename_all = "camelCase")] pub struct Board { pub id: i64, pub name: String, pub created_at: chrono::DateTime, } ``` Unpacking the new stuff: - `#[derive(serde::Serialize)]` derives a `serde::Serialize` impl for `Board` which will allow us to serialize it to JSON using the `serde_json` crate. - `#[serde(rename_all = "camelCase")]` renames all of the snake_case member identifiers to camelCase when serializing (or vice versa when deserializing). This is because it's a convention to use snake_case names in Rust but JSON is often produced and consumed by JS code and the JS convention is to use camelCase for member identifiers. - Making `id` an `i64` instead of an `u64` might seem like an odd choice but since we're using PostgreSQL as our DB we have to do this because PostgreSQL only supports signed integer types. - A `created_at` member is always useful to have, if for no other reason than to be able to sort entities by chronological order when no better sort order is available. Okay, let's add cards and statuses: ```rust #[derive(serde::Serialize)] #[serde(rename_all = "camelCase")] pub struct Card { pub id: i64, pub board_id: i64, pub description: String, pub status: Status, pub created_at: chrono::DateTime, } #[derive(serde::Serialize, serde::Deserialize)] #[serde(rename_all = "camelCase")] pub enum Status { Todo, Doing, Done, } ``` Since we'd also like to support returning a "board summary" which contains the count of all of the cards on a board grouped by their status here's the model for that: ```rust #[derive(serde::Serialize)] pub struct BoardSummary { pub todo: i64, pub doing: i64, pub done: i64, } ``` When using the API to create a new board users can provide the board name but not its id, since that will be set by the DB, so we need a model for that as well: ```rust #[derive(serde::Deserialize)] pub struct CreateBoard { pub name: String, } ``` Likewise users can also create cards. When creating a card let's assume we only want users to provide the new card's description and what board it should be associated with. The new card will get the default todo status and will get its id set by the DB: ```rust #[derive(serde::Deserialize)] #[serde(rename_all = "camelCase")] pub struct CreateCard { pub board_id: i64, pub description: String, } ``` When updating a card let's assume we want users to only be able to update the description or the status. It would be pretty weird if we allowed them to move cards between boards which is a pretty unusual feature in most project management apps: ```rust #[derive(serde::Deserialize)] pub struct UpdateCard { pub description: String, pub status: Status, } ``` Throw all of those into their own module and we get: ```rust // src/models.rs // for GET requests #[derive(serde::Serialize)] #[serde(rename_all = "camelCase")] pub struct Board { pub id: i64, pub name: String, pub created_at: chrono::DateTime, } #[derive(serde::Serialize)] #[serde(rename_all = "camelCase")] pub struct Card { pub id: i64, pub board_id: i64, pub description: String, pub status: Status, pub created_at: chrono::DateTime, } #[derive(serde::Serialize, serde::Deserialize)] #[serde(rename_all = "camelCase")] pub enum Status { Todo, Doing, Done, } #[derive(serde::Serialize)] pub struct BoardSummary { pub todo: i64, pub doing: i64, pub done: i64, } // for POST requests #[derive(serde::Deserialize)] pub struct CreateBoard { pub name: String, } #[derive(serde::Deserialize)] #[serde(rename_all = "camelCase")] pub struct CreateCard { pub board_id: i64, pub description: String, } // for PATCH requests #[derive(serde::Deserialize)] pub struct UpdateCard { pub description: String, pub status: Status, } ``` And the updated main file: ```diff // src/main.rs mod logger; +mod models; type StdErr = Box; fn main() -> Result<(), StdErr> { dotenv::dotenv()?; logger::init()?; Ok(()) } ``` ## Sync Implementation ### SQL Schema Migrations w/diesel-cli crates - diesel-cli ```bash cargo install diesel_cli ``` If the above command doesn't work at first, it's likely because we don't have all the development libraries for all of diesel-cli's supported databases. Since we're just using PostgreSQL, we can make sure the development libraries are installed with these commands: ```bash # macOS brew install postgresql # ubuntu apt-get install postgresql libpq-dev ``` And then we can tell cargo to only install diesel-cli with support for PostgreSQL: ```bash cargo install diesel_cli --no-default-features --features postgres ``` Once we have diesel-cli installed we can use it to create new migrations and execute pending migrations. diesel-cli figures out which DB to connect to by checking the `DATABASE_URL` environment variable, which it will also load from an `.env` file if one exists in the current working directory. Assuming the DB is currently running and a `DATABASE_URL` environment variable is present, here's the first diesel-cli command we'd run to bootstrap our project: ```bash diesel setup ``` With this diesel-cli creates a `migrations` directory where we can generate and write our DB schema migrations. Let's generate our first migration: ```bash diesel migration generate create_boards ``` This will create a new directory, e.g. `migrations/---