# Memoir+: Enhanced Persona Extension for Text Generation Web UI
![Image of an android writing in a journal with futuristic city out of window: Generated by Dalle](https://raw.githubusercontent.com/brucepro/Memoir/main/images/ai_gen_memoir.jpg)

Important: I recommend you go to the qdrant dashboard: http://localhost:6333/dashboard and create and download a snapshot of your agents vector store. This will allow you to restore in case something happens where the docker zaps it. Next release has a few features to fix this, but good to have the backup.

## Introduction

Memoir is an AI-powered plugin designed to enrich your existing AI companions within the Text Generation Web UI. With advanced memory capabilities and emotional intelligence, Memoir transforms your interactions with AI into a more nuanced and human-like experience.

## Table of Contents

- [Introduction](#introduction)
- [Key Features](#key-features)
- [Configuration](#configuration)
- [Future Development](#future-development)
- [Installation](#installation)
- [Usage](#usage)
- [Support](#support)
- [Contributing](#contributing)
- [License](#license)

## Key Features

- **Short-Term Memory:** Stores recent conversations for enhanced contextual awareness.
- **Long-Term Memory:** Utilizes a vector database for creating and recalling durable memories. (You can manage and edit entries on the qdrant dashboard: http://localhost:6333/dashboard )
- **Emotion Tracking:** Monitors and adjusts the AI's emotional responses over time.
- **Command Handling:** A modular system for executing custom commands within the AI environment. [GET_URL=url,output]
- **RAG System:** Ability to ingest urls and files. Uses langchain community loaders for supported filetypes. For .epub support you may need to install (https://github.com/jgm/pandoc)
Command Structure:

YES:

[FILE_LOAD=https://arxiv.org/pdf/2402.10790.pdf]

[FILE_LOAD=C:/pdfs/2402.10790.pdf]

[FILE_LOAD=C:/pdfs/] - Will use unstructured loader. Better to use the pdf loader.

[GET_URL=https://www.npr.org/sections/world/, output]

NO:

[FILE_LOAD=https://www.npr.org/sections/world/] - Have not added the logic for no file extension on urls for file loader yet, use the [GET_URL=url,output] command.




## Configuration

Memoir offers detailed configuration options for personalizing your AI's memory and personality traits:

### Memory Settings

- **Long Term Memory Result Count:** Adjust the number of memories to incorporate into the current context for a richer interaction.
- **Short Term Memory Processing Interval:** Set the frequency of converting short-term to long-term memories to optimize performance and relevance.

### Ego Configuration

- **Ego Name:** Customize the ego name to correspond with your AI's identity, enhancing recognition and personalization.
- **Ego Persona Details:** Craft a detailed persona for your AI's subconscious mind to guide its summarization and understanding of conversations.
- **Ego Thinking Statement:** Direct your AI on how to synthesize conversations and identify key points, allowing for creative experimentation.

### Memory in Bot Prefix

- **Enable/Disable Saving Context:** Control whether memories are added to the bot's prefix, trading off between preserving context and conversational depth.

### Narration and Roleplay

- **Activate Narrator:** Omit character names from emotes to better set the scene during narrative passages.
- **Activate Roleplay Flag:** Indicate to the system when it is summarizing roleplay sessions to adjust the handling of memories.

### Debugging and Memory Management

- **Memory Saving Toggle:** Swiftly enable or disable memory recording for troubleshooting or adjusting privacy settings.

### Character Management

- **Delete Characters:** Efficiently manage and delete character data, including all associated memories and emotional data.

## Future Development

- **Docker/Shell Access:** Upcoming feature for advanced control and system automation.
- **Topic Research:** Enhanced capabilities for information gathering and utilization.
- **Messaging Integrations:** For direct notifications and updates across various messaging platforms.

## Installation

### Prerequisites
1. Install the Text Generation Web UI as per instructions on [GitHub](https://github.com/oobabooga/text-generation-webui).
2. **Qdrant Vector Database** (localhost:6333) - Choose ONE option:

   **Option A - Docker (Recommended):**
   - Install [Docker Desktop](https://www.docker.com/products/docker-desktop/)
   - Run: `docker run -p 6333:6333 qdrant/qdrant`

   **Option B - Standalone Binary:**
   - Download from [Qdrant Releases](https://github.com/qdrant/qdrant/releases)
   - Run the binary (automatically starts on port 6333)

   **Note:** Memoir+ will check Qdrant availability on startup and provide helpful guidance if not found.

### Installation Steps
3. Clone the Memoir repository:
   ```bash
   git clone https://github.com/brucepro/Memoir
   ```

4. Move the Memoir folder into the extensions directory of your TextGenWebUI installation (folder must be named 'Memoir').

5. **Install Python dependencies:**

   **For portable Text-generation-webui:**
   ```bash
   cd text-generation-webui-3.17
   portable_env\python.exe -m pip install -r user_data\extensions\Memoir\requirements.txt
   ```

   **For standard installation:**
   ```bash
   cd text-generation-webui/extensions/Memoir
   pip install -r requirements.txt --upgrade
   ```

   **Or use the update wizard:**
   - Run the update_wizard bat for your OS
   - Select B) Install/update extensions requirements
   - Select Memoir from the list

6. Restart Text Generation Web UI, go to 'Session' tab → check Memoir → 'Apply flags/extensions and restart'.

7. Verify Memoir loaded successfully in the Text Generation Web UI console.

### Qdrant Dashboard
Access the Qdrant dashboard at http://localhost:6333/dashboard to:
- View and manage your AI's vector memories
- Create backups (snapshots) of your collections
- Monitor memory usage and performance
## Usage

Configure your AI character's description if you like in the TextGen UI character tab. Engage in conversation, and watch as Memoir+ begins to store and utilize the generated memories, enhancing the interaction with your AI.

### Memory Management Utility

Memoir+ includes a command-line utility for managing your character memories:

```bash
# List all collections and databases
python memoir_utils.py list

# View detailed stats for a character
python memoir_utils.py stats <character_name>

# Create a backup (saves Qdrant snapshots)
python memoir_utils.py backup <character_name>

# Export memories to human-readable text file
python memoir_utils.py export <character_name> output.txt

# Use custom Qdrant address
python memoir_utils.py --qdrant http://remote-server:6333 list
```

**Examples:**
```bash
python memoir_utils.py stats AI
python memoir_utils.py backup AI
python memoir_utils.py export AI ai_memories.txt
```

**Backup Location:**
- Qdrant snapshots: Accessible via Qdrant dashboard at http://localhost:6333/dashboard
- SQLite backups: Saved to `backups/` folder in Memoir+ directory
 
## Support

If Memoir adds value to your AI experience and you'd like to show your appreciation, consider supporting the project:

- [Buy me a coffee](https://www.buymeacoffee.com/brucepro)
- [Ko-fi](https://ko-fi.com/F1F7U45XV)

## Contributing

Contributions, suggestions, and feedback are always welcome. Please submit issues or pull requests on GitHub, or contact us directly with your ideas and suggestions.

## Changelog

### Version 2024-11-22 - Stability & Bug Fixes
**Critical Fixes:**
- Fixed Qdrant API compatibility (updated to `query_points` API) - Resolves GitHub Issue #92
- Fixed path concatenation bug in directory file loading
- Added CUDA memory protection (force CPU for embeddings)
- Removed Docker dependency - Qdrant can now run standalone
- Fixed module import paths for text-generation-webui 3.17+ compatibility

**New Features:**
- **Memory Management Utility** (`memoir_utils.py`) - Command-line tool for:
  - Listing all collections and databases
  - Viewing detailed character statistics
  - Creating backups (Qdrant snapshots + SQLite)
  - Exporting memories to human-readable text files

**Improvements:**
- Added comprehensive error handling and logging throughout
- Implemented retry logic with exponential backoff for Qdrant operations
- Replaced random integer IDs with UUIDs for better uniqueness
- All operations now gracefully degrade on failure (no crashes)
- Added startup Qdrant availability check with helpful error messages
- Improved installation documentation with portable text-gen-webui instructions

**Backward Compatibility:**
- All changes are backward compatible - existing installations work without modification
- New features are opt-in with sensible defaults

## License

Memoir is made available under the MIT License. For more details, see the LICENSE file in the repository.