# Glean 拾灵

**[English](./README.md)** | **[中文](./README.zh-CN.md)**

> [!IMPORTANT]
> Join our [Discord](https://discord.gg/KMKC4sRVSJ) to stay updated on the latest developments and get support.
>
> This project is still in development and is not ready for production use.

A self-hosted RSS reader and personal knowledge management tool.

> **Glean** (拾灵) helps information-heavy consumers efficiently manage their reading through intelligent RSS aggregation.

![Glean](asset/Screenshot.png)

## Features

### Core Features
- 📰 **RSS Subscription** - Subscribe and organize RSS/Atom feeds with OPML import/export
- 📚 **Smart Reading** - Clean reading experience with content filtering
- 🔖 **Read Later** - Save articles for later reading with auto-cleanup
- 📁 **Folders & Tags** - Multi-level folders and tags for organization
- ⭐ **Bookmarks** - Save articles from feeds or external URLs
- 🔧 **Background Sync** - Automatic feed updates every 15 minutes
- 🔒 **Self-hosted** - Full data ownership with Docker deployment
- 🎨 **Modern UI** - Beautiful, responsive warm dark theme interface
- 👨‍💼 **Admin Dashboard** - User management and system monitoring

### Planned Features (WIP)
- 🧠 **Smart Recommendations** - AI-powered preference learning and article scoring
- ⚙️ **Rule Engine** - Automated processing with Jinja2-style conditions
- 🤖 **AI Features** - Summary generation, auto-tagging, keyword extraction (BYOK)
- 📄 **Full Content Fetch** - Fetch complete article content for RSS summaries
- 🔌 **Chrome Extension** - One-click bookmarking from browser
- 📱 **Mobile PWA** - Progressive Web App for mobile devices

## Quick Start

### One-Command Deployment

```bash
# Download docker-compose.yml
curl -fsSL https://raw.githubusercontent.com/LeslieLeung/glean/main/docker-compose.yml -o docker-compose.yml

# Start Glean (full deployment with Milvus)
docker compose up -d

# Access:
# - Web App: http://localhost
# - Admin Dashboard: http://localhost:3001 (default: admin/Admin123!)
```

**Default Admin Account**: An admin account is automatically created with:
- Username: `admin`
- Password: `Admin123!`
- ⚠️ **Change this password in production!**

**Lite Deployment** (without Milvus, if you don't need Phase 3 features):

```bash
# Download lite version
curl -fsSL https://raw.githubusercontent.com/LeslieLeung/glean/main/docker-compose.lite.yml -o docker-compose.yml

# Start Glean
docker compose up -d

# Admin Dashboard: http://localhost:3001 (default: admin/Admin123!)
```

### Customize Admin Account (Optional)

To use custom admin credentials instead of the defaults, create a `.env` file **before** starting:

```bash
# Set custom admin credentials in .env
cat > .env << EOF
ADMIN_USERNAME=admin
ADMIN_PASSWORD=YourSecurePassword123!
SECRET_KEY=$(openssl rand -base64 32)
EOF

# Start services
docker compose up -d
```

To disable auto-creation and create admin manually:

```bash
# Disable auto-creation in .env
echo "CREATE_ADMIN=false" >> .env

# Start services
docker compose up -d

# Create admin manually
docker exec -it glean-backend /app/scripts/create-admin-docker.sh
```

## Configuration

For production, customize your deployment with environment variables. Download the example file:

```bash
curl -fsSL https://raw.githubusercontent.com/LeslieLeung/glean/main/.env.example -o .env
```

**Important settings to change:**

| Variable            | Description          | Default                             |
| ------------------- | -------------------- | ----------------------------------- |
| `SECRET_KEY`        | JWT signing key      | **Must change in production!**      |
| `POSTGRES_PASSWORD` | Database password    | `glean` (**Change in production!**) |
| `ADMIN_PASSWORD`    | Admin password       | `Admin123!` (**Change!**)           |
| `WEB_PORT`          | Web interface port   | `80`                                |
| `ADMIN_PORT`        | Admin dashboard port | `3001`                              |
| `CREATE_ADMIN`      | Auto-create admin    | `true` (set `false` to disable)     |

For all configuration options, see [.env.example](.env.example).

## Docker Images

Pre-built images are available on GitHub Container Registry:

- `ghcr.io/leslieleung/glean-backend:latest` - API server & worker
- `ghcr.io/leslieleung/glean-web:latest` - Web frontend
- `ghcr.io/leslieleung/glean-admin:latest` - Admin dashboard

Supported architectures: `linux/amd64`, `linux/arm64`

### Testing Pre-release Versions

Want to test upcoming features? Use pre-release versions (alpha/beta/rc):

**Method 1: Using environment variable (recommended)**

```bash
# Set the IMAGE_TAG in .env file
echo "IMAGE_TAG=v0.3.0-alpha.1" >> .env

# Or export it directly
export IMAGE_TAG=v0.3.0-alpha.1

# Start with pre-release images
docker compose up -d
```

**Method 2: Inline environment variable**

```bash
IMAGE_TAG=v0.3.0-alpha.1 docker compose up -d
```

**Note**: Pre-release versions are for testing only. They won't trigger auto-updates for Electron apps and are not recommended for production use.

See available pre-release versions on the [Releases page](https://github.com/LeslieLeung/glean/releases).

## Deployment

The default deployment includes all services (full version):
- **Web App** (port 80) - Main user interface
- **Admin Dashboard** (port 3001) - User management and system monitoring
- **Backend API** - FastAPI server
- **Worker** - Background task processor (feed fetching, cleanup)
- **PostgreSQL** - Database
- **Redis** - Task queue
- **Milvus** - Vector database for smart recommendations and preference learning (Phase 3)

**Lite deployment** (without Milvus) is also available using `docker-compose.lite.yml`.

For detailed deployment instructions and configuration, see [DEPLOY.md](DEPLOY.md).

## Tech Stack

**Backend:**
- Python 3.11+ / FastAPI / SQLAlchemy 2.0
- PostgreSQL / Redis / arq (task queue)

**Frontend:**
- React 18 / TypeScript / Vite
- Tailwind CSS / Zustand / TanStack Query

## Development

See **[DEVELOPMENT.md](./DEVELOPMENT.md)** for complete development setup instructions.

Quick start:

```bash
# Clone and setup
git clone https://github.com/LeslieLeung/glean.git
cd glean
npm install

# Start infrastructure
make up

# Initialize database (first time only)
make db-upgrade

# Install pre-commit hooks (optional but recommended)
make pre-commit-install

# Start all services
make dev-all

# Access:
# - Web: http://localhost:3000
# - Admin: http://localhost:3001
# - API Docs: http://localhost:8000/api/docs
```

### Pre-commit Hooks

The project uses pre-commit hooks to ensure code quality:

```bash
# Install hooks (one-time setup)
make pre-commit-install

# Run hooks manually on all files
make pre-commit-run

# Uninstall hooks (if needed)
make pre-commit-uninstall
```

Hooks automatically run on commit and check:
- Backend: ruff format, ruff linter, pyright type checking
- Frontend: ESLint, Prettier formatting
- General: trailing whitespace, file endings, YAML/JSON/TOML validation

## Roadmap

| Phase                     | Status    | Features                                                       |
| ------------------------- | --------- | -------------------------------------------------------------- |
| **Phase 1: MVP**          | ✅ Done    | User system, RSS subscription, reader, admin dashboard         |
| **Phase 2: Organization** | ✅ Done    | Bookmarks, folders, tags, read later                           |
| **Phase 3: Preferences**  | 🚧 WIP     | Embedding pipeline, preference learning, smart recommendations |
| **Phase 4: Rules**        | 📋 Planned | Rule engine, Jinja2 conditions, automated actions              |
| **Phase 5: AI**           | 📋 Planned | AI summaries, auto-tagging, keyword extraction, BYOK support   |
| **Phase 6: Extensions**   | 📋 Planned | Chrome extension, PWA, web snapshots                           |

See **[Product Requirements](./docs/glean-prd-v1.2.md)** for detailed feature specifications.

## Documentation

- **[Development Guide](./DEVELOPMENT.md)** - Set up your development environment
- **[Deployment Guide](./deploy/README.md)** - Production deployment details

## Contributing

Contributions are welcome! Please read our [Development Guide](./DEVELOPMENT.md) first.

1. Fork the repository
2. Create a feature branch
3. Make your changes
4. Run tests and linting
5. Submit a Pull Request

## License

This project is licensed under the **AGPL-3.0 License** - see the [LICENSE](LICENSE) file for details.