---
title: "ML Ops"
center: true
theme: black
transition: slide
margin: 0.04
revealOptions:
transition: 'slide'
css:
- ./css/custom.css
---
# ML Ops Examples
#### [`saforem2/mlops`](https://www.github.com/saforem2/mlops)
Sam Foreman
May, 2022
::: block
[](https://github.com/saforem2)
[](https://samforeman.me)
[](https://twitter.com/saforem2)
:::
---
# 📊 ML Ops
**Goal**: Allow researchers to focus on their science / model development without all the boilerplate.
---
### MLOps
---
[](https://comet.ml)
```python
# For Comet to start tracking a training run,
# just add these two lines at the top of your training script:
import comet_ml
experiment = comet_ml.Experiment(
api_key="API_KEY",
project_name="PROJECT_NAME"
)
# Metrics from this training run will now be visible in the Comet UI
```
---
# Why Comet?
::: block
::: block
> While some ML platform vendors offer stand-alone experiment tracking or model production monitoring systems, Comet offers both.
>
— [comet.ml](https://comet.ml)
:::
- Features:
- Experiment tracking and management
- Dataset versioning
- Model registry
- Model production monitoring
- Code panels
- Reports
:::
---
# Comet
---
# First Steps
1. Setup a Comet account
2. Get an API key
- Make sure you're logged into comet.ml
- Go to [settings](https://www.comet.ml/api/my/settings)
- Developer Information --> **Generate API Key**
3. Install Comet
```bash
python3 -m pip install comet_ml
```
4. Create an Experiment and log to Comet
```python
from comet_ml import Experiment
experiment = Experiment(
api_key="API_KEY",
project_name="PROJECT_NAME",
workspace="WORKSPACE_NAME",
)
params = {'batch_size': 32, 'lr': 0.001}
metrics = {'accuracy': 0.9, 'loss': 0.01}
experiment.log_metrics(metrics, step=1)
experiment.log_parameters(parameters)
```
---
# Comet +
- [comet-ml/comet-examples](https://github.com/comet-ml/comet-examples)
- [`saforem2/mlops`](https://github.com/saforem2/mlops)
- **PyTorch**:
- [`ddp.py`](https://github.com/saforem2/mlops/src/comet/torch/lightning.py)
- ️️[`lightning.py`](https://github.com/saforem2/mlops/src/comet/torch/lightning.py)
- **TensorFlow**:
- [`main.py`](https://github.com/saforem2/mlops/src/comet/tensorflow)
- [dashboard](https://www.comet.ml/saforem2/mlops/2cc1b07491554afcb42af1c7f040353e?experiment-tab=chart&showOutliers=true&smoothing=0&transformY=smoothing&xAxis=step)
---
note:
- Background iframe
---
---
# ⚡️ Pytorch Lightning
- Tutorial on [Tracking and Visualizing Experiments](https://pytorch-lightning.readthedocs.io/en/latest/visualize/logging_intermediate.html) from PyTorch Lightning
- Built-in support for multiple different "loggers"
- [`base`](https://pytorch-lightning.readthedocs.io/en/latest/api/pytorch_lightning.loggers.base.html#module-pytorch_lightning.loggers.base)
- [`comet`](https://pytorch-lightning.readthedocs.io/en/latest/api/pytorch_lightning.loggers.comet.html#module-pytorch_lightning.loggers.comet)
- [`csv_logs`](https://pytorch-lightning.readthedocs.io/en/latest/api/pytorch_lightning.loggers.csv_logs.html#module-pytorch_lightning.loggers.csv_logs)
- [`mlflow`](https://pytorch-lightning.readthedocs.io/en/latest/api/pytorch_lightning.loggers.mlflow.html#module-pytorch_lightning.loggers.mlflow)
- [`neptune`](https://pytorch-lightning.readthedocs.io/en/latest/api/pytorch_lightning.loggers.neptune.html#module-pytorch_lightning.loggers.neptune)
- [`tensorboard`](https://pytorch-lightning.readthedocs.io/en/latest/api/pytorch_lightning.loggers.tensorboard.html#module-pytorch_lightning.loggers.tensorboard)
- [`wandb`](https://pytorch-lightning.readthedocs.io/en/latest/api/pytorch_lightning.loggers.wandb.html#module-pytorch_lightning.loggers.wandb)
---
# ⚡️ Pytorch Lightning
- **Loggers**:
```python
from pytorch_lightning import loggers as pl_loggers
```
- **Tensorboard**:
```python
tensorboard = pl_loggers.TensorBoardLogger(save_dir='')
trainer = Trainer(logger=tensorboard)
def training_step(self):
tensorboard = self.logger.experiment
tensorboard.add_image()
tensorboard.add_histogram()
tensorboard.add_figure()
```
- [**Comet.ml**](https://comet.ml): (docs on [`CometLogger`](https://pytorch-lightning.readthedocs.io/en/latest/extensions/generated/pytorch_lightning.loggers.CometLogger.html#pytorch_lightning.loggers.CometLogger))
```python
comet_logger = pl_loggers.CometLogger("YOUR_API_KEY")
trainer = Trainer(logger=comet_logger)
class LitModel(LightningModule):
def any_lightning_module_function_or_hook(self):
comet = self.logger.experiment
fake_images = torch.Tensor(32, 3, 28, 28)
comet.add_image('generated_images', fake_images, 0)
```
---
# ⚡️ Pytorch Lightning
- **MLflow**:
```python
from pytorch_lightning.loggers import MLFlowLogger
mlf_logger = MLFlowLogger(experiment_name='experiment', tracking_uri='file:./ml-runs')
trainer = Trainer(logger=mlf_logger)
class LitModel(LightningModule):
def any_lightning_module_function_or_hook(self):
mlf_logger = self.logger.experiment
fake_images = torch.Tensor(32, 3, 28, 28)
mlf_logger.add_aimge('generated_images', fake_images, 0)
```
---