**[Docs](https://docs.cortexlabs.com)** • **[Slack](https://community.cortexlabs.com)**

<br>

<img src='https://cortex-public.s3.us-west-2.amazonaws.com/logo.png' height='32'>

<br>

Note: This project is no longer actively maintained by its original authors.

# Production infrastructure for machine learning at scale

Deploy, manage, and scale machine learning models in production.

<br>

## Serverless workloads

**Realtime** - respond to requests in real-time and autoscale based on in-flight request volumes.

**Async** - process requests asynchronously and autoscale based on request queue length.

**Batch** - run distributed and fault-tolerant batch processing jobs on-demand.

<br>

## Automated cluster management

**Autoscaling** - elastically scale clusters with CPU and GPU instances.

**Spot instances** - run workloads on spot instances with automated on-demand backups.

**Environments** - create multiple clusters with different configurations.

<br>

## CI/CD and observability integrations

**Provisioning** - provision clusters with declarative configuration or a Terraform provider.

**Metrics** - send metrics to any monitoring tool or use pre-built Grafana dashboards.

**Logs** - stream logs to any log management tool or use the pre-built CloudWatch integration.

<br>

## Built for AWS

**EKS** - Cortex runs on top of EKS to scale workloads reliably and cost-effectively.

**VPC** - deploy clusters into a VPC on your AWS account to keep your data private.

**IAM** - integrate with IAM for authentication and authorization workflows.