# OpenMined Syft Space [![Python](https://img.shields.io/badge/python-3.12%2B-blue)](https://www.python.org/downloads/) [![Node.js](https://img.shields.io/badge/node.js-20%2B-green)](https://nodejs.org/) [![FastAPI](https://img.shields.io/badge/FastAPI-0.116-009688)](https://fastapi.tiangolo.com/) [![Vue](https://img.shields.io/badge/Vue-3-4FC08D)](https://vuejs.org/) [![License](https://img.shields.io/github/license/OpenMined/syft-space)](LICENSE) > **Make your knowledge queryable without giving it up.** Syft Space lets you create AI-powered endpoints that others can query while your data stays under your control. ![Syft Space Dashboard](homepage.png) ## What is Syft Space? A Syft Space is a decentralized node operated by anyone with knowledge to share: a publisher, a research lab, a journalist, an institution, or just an individual. You add your documents, connect an AI model, and create endpoints that others can query. You control who gets access, what they can ask, and whether they pay. **Key Benefits:** - **๐Ÿ”’ Privacy-First**: Your data never leaves your control - **๐ŸŽฏ Selective Sharing**: Share insights, not raw data - **๐Ÿ’ฐ Monetization**: Set your own pricing and access rules - **๐Ÿ” AI-Powered**: Make any document collection searchable with AI - **๐ŸŒ Decentralized**: Part of a growing network while maintaining independence Think of it like creating an API for your knowledge - others get answers without seeing your source material. ## ๐Ÿ—๏ธ Architecture Syft Space is a full-stack application that creates a bridge between your private data and AI models, allowing controlled access through secure endpoints. ![Syft Space Architecture](fullsetup.png) **Core Architecture:** - **Frontend**: Vue 3 web interface for management and configuration - **Backend**: FastAPI server handling requests, policies, and data processing - **Vector Databases**: Local or remote databases (Weaviate, Qdrant, ChromaDB) for document indexing - **AI Models**: Integration with OpenAI, Anthropic, Ollama, vLLM, and other providers - **SyftHub Integration**: Optional publishing to the decentralized knowledge network The system automatically provisions Docker containers for local vector databases and provides a unified API for querying across different data sources and AI models. ## How It Works Build your knowledge hub with four core components: ### ๐Ÿ“š **Datasets** Upload documents or connect vector databases. Files are automatically indexed for AI search. ### ๐Ÿค– **Models** Connect OpenAI, Anthropic, Ollama, vLLM, or any OpenAI-compatible provider. ### ๐Ÿ”— **Endpoints** Combine datasets and models into queryable RAG endpoints anyone can use. ### ๐Ÿ›ก๏ธ **Policies** Control access, set rate limits, and configure pricing for your endpoints. ``` [Your Data] + [AI Model] โ†’ [Queryable Endpoint] + [Your Rules] = [Controlled Knowledge Sharing] ``` ## Use Cases - **๐Ÿ“Š Publishers & Creators** โ€” Make your content AI-queryable for your audience while maintaining attribution and control - **๐ŸŽ“ Researchers** โ€” Turn your papers and notes into a searchable knowledge assistant that others can query - **๐Ÿข Organizations** โ€” Build AI-powered customer support from your documentation without exposing internal details - **๐Ÿ‘ฅ Teams** โ€” Create searchable knowledge hubs from scattered wikis, docs, and guides - **๐Ÿ’ก Data Monetization** โ€” Share valuable insights from your data without exposing the underlying information ## ๐Ÿš€ Getting Started ### Step 1: Install Syft Space **Prerequisites:** 4GB RAM minimum, 8GB recommended Choose your installation method: #### Option A: Desktop App (Recommended for beginners) - **macOS**: Download `.dmg` from [Releases](https://github.com/OpenMined/syft-space/releases) - **Linux**: Download `.AppImage`, `.deb`, or `.rpm` from [Releases](https://github.com/OpenMined/syft-space/releases) **Features:** One-click setup, system tray integration, auto-updates #### Option B: Docker (Production ready) ```bash docker run -d \ --name syft-space \ --restart unless-stopped \ -p 8080:8080 \ -v /var/run/docker.sock:/var/run/docker.sock \ -v /dev/null:/root/.docker/config.json \ -v syft-space-data:/data \ ghcr.io/openmined/syft-space:latest ``` #### Option C: From Source (Development) ```bash git clone https://github.com/OpenMined/syft-space.git cd syft-space ./run.sh ``` ### Step 2: Register Your Account 1. Open [http://localhost:8080](http://localhost:8080) in your browser 2. Click **"Register"** and create your account 3. **Important**: Add your developer token (for local) or public IP (for VM deployment) ### Step 3: Add Your Data 1. Go to **Datasets** โ†’ **Add Dataset** 2. Upload documents, files, or connect a vector database 3. Wait for automatic processing and indexing ### Step 4: Create an Endpoint 1. Go to **Endpoints** โ†’ **Add Endpoint** 2. Select your dataset and choose an AI model 3. Configure output type (search, AI summary, or both) 4. Set access policies and pricing ### Step 5: Test & Publish 1. Test your endpoint with the built-in query interface 2. Click **Publish** โ†’ Choose visibility (Organization or Public) 3. Your endpoint is now live at [syfthub.openmined.org](https://syfthub.openmined.org) **Example API Query:** ```bash curl -X POST http://localhost:8080/api/v1/endpoints/my-docs/query \ -H "Content-Type: application/json" \ -H "Authorization: Bearer your-token" \ -d '{"messages": [{"role": "user", "content": "What are the main topics?"}]}' ``` ## ๐Ÿ”‘ Developer Token **Required for local deployments only.** If you're running Syft Space locally (not on a VM with public IP), you'll need a developer token to publish endpoints. | Scenario | Token Required? | |----------|----------------| | ๐Ÿ  Local machine | โœ… Yes - get from OpenMined | | โ˜๏ธ Cloud VM/Server | โŒ No - uses public IP | | ๐Ÿงช Local development only | โŒ No - for testing only | **Getting a token:** Contact OpenMined directly (currently in beta) **Configure:** During onboarding or in Settings โ†’ Network ## โœจ Key Features - **๐Ÿ—„๏ธ Multi-database support**: Weaviate, Qdrant, ChromaDB, or remote data sources - **๐Ÿค– Multi-model support**: OpenAI, Anthropic, vLLM, Ollama, and other providers - **โš™๏ธ Auto-provisioning**: Automatically manages Docker containers for local databases - **๐Ÿ’ป Desktop & web interface**: Run it as an app or deploy as a service - **๐Ÿ“Š Usage analytics**: Monitor costs, tokens, and performance - **๐ŸŒ Marketplace integration**: Publish to SyftHub for discovery ## ๐Ÿ“ Project Structure ``` syft-space/ โ””โ”€โ”€ packages/ โ””โ”€โ”€ spaces/ # the Syft Space package โ”œโ”€โ”€ backend/ # Python FastAPI server โ”‚ โ””โ”€โ”€ syft_space/ # Main application code โ”œโ”€โ”€ frontend/ # Vue 3 web interface โ”œโ”€โ”€ docs/ # Documentation โ”œโ”€โ”€ Dockerfile # Container build โ””โ”€โ”€ docker-compose.yml ``` ## ๐Ÿ“š Documentation - **[Quick Start](http://syft.docs.openmined.org/quickstart)** โ€” Get started in 3 steps: add data, create endpoint, publish - **[Introduction](http://syft.docs.openmined.org/space)** โ€” Understand the concepts - **[Components Overview](http://syft.docs.openmined.org/space/components/overview)** โ€” Datasets, models, endpoints, and policies - **[API Reference](http://syft.docs.openmined.org/space/api)** โ€” REST API documentation - **[Installation Guide](http://syft.docs.openmined.org/installation)** โ€” Complete setup instructions ### Component Guides - [Datasets](http://syft.docs.openmined.org/space/components/datasets) โ€” Managing your data sources - [Models](http://syft.docs.openmined.org/space/components/models) โ€” AI providers and configuration - [Endpoints](http://syft.docs.openmined.org/space/components/endpoints) โ€” Queryable RAG services - [Policies](http://syft.docs.openmined.org/space/components/policies) โ€” Access control and rate limiting **Interactive API docs:** [http://localhost:8080/docs](http://localhost:8080/docs) (when running locally) ## ๐ŸŒ Part of the Syft Network A Space connects to **[SyftHub](https://syfthub.openmined.org)** โ€” a decentralized registry where knowledge providers meet knowledge seekers. **How it works:** - ๐Ÿ“ **Discovery**: Others find your published endpoints through SyftHub - ๐Ÿ” **Querying**: Users can search and query across multiple Spaces - ๐Ÿ“ **Attribution**: Every contribution is tracked and credited automatically - ๐Ÿ—๏ธ **Open Architecture**: Built on open protocols โ€” no single point of control The network grows stronger as more Spaces join, but each Space maintains full autonomy over its data and policies. ## ๐Ÿค Contributing We welcome contributions! Here's how to get started: 1. **Fork the repository** and clone it locally 2. **Set up development environment** (see [Development Guide](http://syft.docs.openmined.org)) 3. **Create a feature branch** for your changes 4. **Submit a pull request** with a clear description **Development setup:** ```bash git clone https://github.com/your-fork/syft-space.git cd syft-space ./run.sh ``` See the [Development Documentation](http://syft.docs.openmined.org) for detailed setup and contribution guidelines. ## ๐Ÿ“„ License This project is part of the OpenMined ecosystem and is licensed under the Apache License 2.0. See the [LICENSE](LICENSE) file for details. ---
Built with โค๏ธ by the OpenMined community
Making AI safer through privacy-preserving technology