# 🍃 MongoOps: MongoDB Backup & Restore Manager
**A robust, interactive command-line utility for managing, backing up, restoring, and verifying MongoDB databases across multiple environments.**
[](https://www.python.org/)
[](https://www.mongodb.com/)
[](LICENSE)
[](https://github.com/spyberpolymath/mongoops)
*Designed to simplify database operations, connection health monitoring, and secure backup verification with deep BSON validation.*
[Features](#-features) • [Prerequisites](#-prerequisites) • [Installation](#-installation) • [Architecture](#-system-architecture) • [Configuration](#-configuration) • [Usage](#-usage-guide) • [Project Structure](#-project-structure) • [Technical Reference](#-technical-reference)
---
## 🎯 Project Overview
Managing MongoDB backups across multiple developmental, staging, and production environments can easily become cluttered, error-prone, and risky.
**MongoOps** is an interactive, console-based manager built in Python that wraps MongoDB's powerful `mongodump` and `mongorestore` command-line tools into a unified management panel. It empowers database administrators, developers, and DevOps teams to seamlessly monitor active MongoDB cluster connections, automate individual or batch database backups, perform zero-hassle database restorations (with optional auto-drop targets), and perform **Deep BSON Verification** to ensure backups are corruption-free and restorable.
---
## ✨ Features
### 📡 Multi-Cluster Connection Management
- Automatically dynamically loads and parses multiple indexed connection strings (`MONGO_URL_1`, `MONGO_URL_2`, etc.) from environment files.
- Orders connection keys logically, handling numeric suffixes properly.
- Performs real-time server response pings using a custom timeout threshold (3000ms) to mark connections as either **ACTIVE** or **FAILED**.
### 📥 Automated Backup Operations
- **Single Database Backup**: Scans a selected active cluster, filters out default system databases (`admin`, `local`, `config`), allows interactive selection, and stores BSON data locally.
- **Bulk Cluster Backup**: Iterates over all active connection URIs in parallel-safe order, dumping all user-created databases into timestamped archive directories.
### 📤 Seamless Restoration & Synchronization
- **Single Database Restore**: Rebuilds/syncs a chosen database directory back to your preferred active MongoDB cluster with an automatic drop-before-restore mechanism (`--drop`) to prevent conflicts.
- **Bulk Cluster Restore**: Re-deploys a full multitenant database snapshot back onto all operational servers.
### 🔍 Deep Backup Verification Engine
- Decodes raw binary BSON files on the fly utilizing Python's `bson` module.
- Validates the structural integrity of collections, counting exact document counts.
- Displays raw decoded documents directly in the terminal for instant validation.
- Parses and formats database/collection schema metadata (`*.metadata.json`) and prelude config files (`prelude.json`) with syntax highlights.
### 🎨 Human-Centric Terminal Interface
- High-contrast, beautiful console formatting provided by `colorama`.
- Graceful shutdown handles interrupted user operations (`Ctrl+C`, EOF) without leaving ghost background processes.
---
## 🏗️ System Architecture
MongoOps functions by combining high-level connection diagnostics with low-level execution of Native MongoDB Database Tools. Below is a simplified visual representation of the operational flow:
```mermaid
graph TD
A[".env Configuration (MONGO_URL_N)"] --> B["MongoOps CLI Dashboard"]
B --> C["1. Show/Ping Connections"]
B --> D["2. mongodump (Backup Engine)"]
B --> E["3. mongorestore (Restore Engine)"]
B --> F["4. Deep BSON Verification"]
C -->|Connection Health Check| G["pymongo Ping Check (3000ms Timeout)"]
D -->|Executes mongodump Subprocess| H["Local backups/ Directory"]
E -->|Executes mongorestore Subprocess| I["MongoDB Cluster Target"]
H -->|Select Backup Folder| F
F -->|BSON Parsing| J["Decode all BSON documents using bson.decode_all"]
F -->|JSON Parsing| K["Format prelude & metadata JSON templates"]
F -->|Integrity Check| L["Generate VerificationSummary statistics"]
```
---
## 🚀 Installation & Prerequisites
### 📋 Prerequisites
Before running MongoOps, you must install the **MongoDB Database Tools** (which provide the vital `mongodump` and `mongorestore` command-line binaries).
#### Windows Installation:
1. Download the Database Tools ZIP/MSI installer from [MongoDB Database Tools Download Page](https://www.mongodb.com/try/download/database-tools).
2. Install the package or extract the binaries to a directory of your choice (e.g., `C:\Program Files\MongoDB\Tools\bin`).
3. Add that path to your User or System **PATH** environment variable:
- Search for **Edit the system environment variables** in Windows.
- Click **Environment Variables**, select **Path** under System Variables, and click **Edit**.
- Add the path to the directory containing `mongodump.exe` and `mongorestore.exe`.
#### macOS Installation (via Homebrew):
```bash
brew tap mongodb/brew
brew install mongodb-database-tools
```
#### Linux Installation (Ubuntu/Debian):
```bash
wget https://fastdl.mongodb.org/tools/db/mongodb-database-tools-ubuntu2204-x86_64-100.9.4.tgz
tar -zxvf mongodb-database-tools-*.tgz
sudo cp mongodb-database-tools-*/bin/* /usr/local/bin/
```
Verify your installation by running:
```bash
mongodump --version
mongorestore --version
```
---
### 🛠️ Local Setup
**1️⃣ Clone the repository:**
```bash
git clone https://github.com/spyberpolymath/mongoops.git
cd mongoops
```
**2️⃣ Install Python Dependencies:**
```bash
pip install -r requirements.txt
```
> [!NOTE]
> The primary package dependencies include:
> - `pymongo` (For server connection, database listings, and state verification)
> - `colorama` (For cross-platform colorful console feedback)
> - `python-dotenv` (For local environment configuration parsing)
> - `bson` (Supplied automatically by `pymongo`, used for parsing backup documents)
---
## ⚙️ Configuration
Set up your MongoDB connection strings in a `.env` file located in the root of the project folder:
```env
MONGO_URL_1=mongodb+srv://admin:secure_pass@production-cluster.mongodb.net/
MONGO_URL_2=mongodb://local_user:local_pass@127.0.0.1:27017/
MONGO_URL_3=mongodb+srv://staging_user:staging_pass@staging-cluster.mongodb.net/
```
> [!IMPORTANT]
> MongoOps matches variables starting with `MONGO_URL_`. They are loaded and sorted logically (e.g., `MONGO_URL_1`, `MONGO_URL_2`, ..., `MONGO_URL_10`). Keep passwords URL-encoded if they contain special characters (like `@`, `:`, `/`).
---
## 📖 Usage Guide
To launch the dashboard, execute:
```bash
python mongo_manager.py
```
### 🎮 Main CLI Dashboard
The terminal-based menu organizes your tasks into easy selections:
```text
===================================================
MongoDB Backup Restore Manager
===================================================
1. Backup Single Database
2. Backup All Databases
3. Restore Single Database
4. Restore All Databases
5. Update Backup
6. Update Restore
7. Show MongoDB Status
8. Verify Backup
9. Exit
Choose Option :
```
---
### 🔍 Detailed Operation Guide
#### 1. Backup Single Database
- Prompts you to pick an active cluster from the connection list.
- Fetches all user databases on that cluster (filtering out `admin`, `local`, and `config`).
- Spawns `mongodump` to backup only that selected database.
- Saved under: `backups///` (containing your `.bson` and `.metadata.json` files).
#### 2. Backup All Databases (Bulk)
- Connects and pings all databases listed in your `.env`.
- If a database server is reachable, it initiates a full backup of all databases on that cluster.
- Saved under: `backups/_all_databases_/`.
#### 3. Restore Single Database
- Prompts you to choose a target server cluster.
- Displays available backup directories in your local `backups/` directory.
- Re-imports the chosen directory with the `--drop` flag (clearing the old database collection first to ensure a clean restoration).
#### 4. Restore All Databases
- Deploys full bulk database folders back to all active connections listed in your environment config.
#### 5. Update Backup & 6. Update Restore
- Fast shortcuts to quickly run single-backup/single-restore pipelines.
#### 7. Show MongoDB Status
- Instantly checks and lists the status of all configured connections, showing the connection status (`ACTIVE` or `FAILED`) alongside their host addresses.
#### 8. Deep Backup Verification
- Reads local backup folders and lets you verify a backup's integrity.
- **Single Verification**: Recursively processes a chosen single database backup. Decodes all binary BSON files, counts documents, formats collection metadata, and prints validation diagnostics.
- **Bulk Verification**: Loops through all databases inside a bulk backup folder, checking everything and compiling a global statistics report.
---
## 🏗️ Project Structure
```text
mongoops/
│
├── backups/ # Local backups directory (Automatically generated)
│ ├── production_db/ # Standard backup output directory
│ │ └── production_db/ # Nested database files
│ │ ├── users.bson
│ │ └── users.metadata.json
│ └── all_databases_.../ # Timestamped bulk directory
│
├── .env # Local configuration of MongoDB connection strings
├── mongo_manager.py # Core interactive console application file
├── requirements.txt # Dependencies (pymongo, colorama, python-dotenv)
└── README.md # Project documentation and user guide
```
---
## 🛠️ Technical Reference
### The Verification Summary Class
The structural health of backups is monitored through a designated `VerificationSummary` data class that records structural and content data points:
```python
@dataclass
class VerificationSummary:
database_name: str
collection_count: int = 0
bson_count: int = 0
metadata_count: int = 0
prelude_count: int = 0
document_count: int = 0
backup_size_bytes: int = 0
success: bool = False
errors: list[str] = field(default_factory=list)
collection_names: list[str] = field(default_factory=list)
```
### Key Functions Breakdown
- `show_connections()`: Spawns short-lived `pymongo.MongoClient` pings, ensuring the client has a 3-second connection limit (`serverSelectionTimeoutMS=3000`) before declaring failure.
- `decode_bson_file(full_path)`: Uses `bson.decode_all` to translate raw binary sequences into readable Python dictionaries.
- `is_single_backup_folder(folder_name)`: Checks for nested directories, as single-database backups generated by `mongodump --db= --out=` are structured inside `/`.
---
## 🤝 Contributing
1. Fork this project repo.
2. Create your feature branch (`git checkout -b feature/cool-new-feature`).
3. Commit your changes (`git commit -m 'Add awesome feature'`).
4. Push to your branch (`git push origin feature/cool-new-feature`).
5. Open a Pull Request.
---
## 📝 License
This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.
---
## 🏷️ Footer
**👨💻 Aman Anil** (aka **SpyberPolymath**)
*Crafting digital experiences with passion and precision*
[](https://github.com/spyberpolymath)
[](https://kaggle.com/spyberpolymath)
[](https://linkedin.com/in/spyberpolymath)
[](https://t.me/spyberpolymath)
**📧 [aman@spyberpolymath.com](mailto:aman@spyberpolymath.com)** | **🌐 [spyberpolymath.com](https://spyberpolymath.com)**
---
*Made with ❤️ by Aman Anil aka (SpyberPolymath)* ✨