# SmartStudy Agent > The AI study agent that **learns how you learn** โ€” a reinforcement-learning policy decides *what* you study next, an FSRS memory model decides *when* you review it, and an LLM generates the quizzes in between. In your browser, in your terminal, or inside Claude via MCP. [![Python 3.10+](https://img.shields.io/badge/python-3.10+-blue.svg)](https://www.python.org/downloads/) [![Claude API](https://img.shields.io/badge/Claude-opus--4--6-purple.svg)](https://www.anthropic.com/) [![Streamlit](https://img.shields.io/badge/Streamlit-1.30+-red.svg)](https://streamlit.io/) [![Chrome Extension](https://img.shields.io/badge/Chrome-Extension_(MV3)-4A90E2.svg)](chrome-extension/) [![FSRS](https://img.shields.io/badge/Scheduler-FSRS-orange.svg)](https://github.com/open-spaced-repetition/py-fsrs) [![Anki Export](https://img.shields.io/badge/Anki-.apkg_export-29B6F6.svg)](anki_export.py) [![MCP](https://img.shields.io/badge/MCP-server-black.svg)](mcp_server.py) [![License: MIT](https://img.shields.io/badge/License-MIT-green.svg)](LICENSE) [![Hugging Face Space](https://img.shields.io/badge/๐Ÿค—_Live_Demo-Hugging_Face-yellow.svg)](https://huggingface.co/spaces/HumphreySun98/smart-study-agent) **[English](#-live--two-ways-to-use-it)** ยท **[ไธญๆ–‡็ฎ€ไป‹](#-ไธญๆ–‡็ฎ€ไป‹)** ยท **[๐Ÿ“š Technical Deep Dive](docs/DEEP_DIVE.md)** โ€” the POMDP framing, the math behind both policies, and the experiments (including the ones the heuristic won) ## ๐ŸŒ Live โ€” Two Ways to Use It | | Where it runs | How to try it | |---|---|---| | **Web app** | Hugging Face Spaces (free Kimi-K2 backend) | [Open in browser](https://huggingface.co/spaces/HumphreySun98/smart-study-agent) | | **Chrome extension** | Your browser โ€” works on any page, PDF, or YouTube video | [**Install from the Chrome Web Store**](https://chromewebstore.google.com/detail/edbjkpfjonahanfkamlcbobmnplihmik) ยท [Source](chrome-extension/) (MV3) |

SmartStudy Agent live on the Chrome Web Store
Live on the Chrome Web Store โ€” click to install.

SmartStudy extension running on a YouTube ML course โ€” side panel shows extracted topics and belief state
Side panel running on a YouTube ML course โ€” topics auto-extracted from captions, belief state updating in real time.

The web app runs on Hugging Face Spaces using free HF Inference Providers (Kimi-K2). The Chrome extension calls the Anthropic API directly from your browser โ€” same agent core, zero backend. For local development, plug in your own Anthropic key to get Claude's higher-quality reasoning. SmartStudy Agent is a goal-based, partially observable AI agent that turns any lecture material into a fully personalized learning experience. Unlike a chatbot, it maintains a persistent belief state about student knowledge and uses an adaptive policy to decide what to study next. --- ## Why SmartStudy? Traditional study tools are static. They show you the same content regardless of what you already know. SmartStudy Agent solves this by closing the loop: | Problem | SmartStudy's Solution | |---------|----------------------| | Generic study materials | Topics extracted and prioritized per student | | No feedback on weak areas | Quiz answers update a persistent belief state | | Same recommendations for everyone | Q-learning policy (or Contextual Bandit) adapts per student trajectory | | Forgetting without practice | **FSRS** spaced repetition โ€” the same modern memory model family as Anki (SM-2 fallback) | | Out-of-order topics | Topological sort over a concept dependency graph | | Locked into one app | **Anki .apkg export**, **MCP server** for Claude, Chrome extension, web app | ### How it compares | | SmartStudy | DeepTutor | OpenTutor | Anki | |---|---|---|---|---| | Runs where you read (browser extension) | โœ… Chrome Web Store | โŒ web workspace | โŒ self-hosted app | โŒ | | RL policy decides next action | โœ… Q-learning + LinUCB, honest benchmark | โŒ | โŒ | โŒ | | Spaced repetition | โœ… FSRS | โŒ | โœ… FSRS | โœ… FSRS | | Quiz generation from any material | โœ… | โœ… | โœ… | โŒ | | Anki export | โœ… .apkg | โŒ | โŒ | โ€” | | Drive it from Claude (MCP) | โœ… | โŒ | โŒ | via 3rd-party | | Footprint | ~3k LOC, SQLite, runs on free tier | Full platform (FastAPI + Next.js) | FastAPI + Next.js | Desktop app | SmartStudy is deliberately **not** an all-in-one learning platform โ€” it's the lightweight agent core: observe โ†’ plan โ†’ quiz โ†’ evaluate โ†’ adapt, with real learning-science scheduling. If you want a full workspace, [DeepTutor](https://github.com/HKUDS/DeepTutor) is excellent. If you want the decision loop embedded where you already study โ€” this repo. --- ## Architecture SmartStudy implements the **OPEAA loop** โ€” a five-phase adaptive agent cycle: ``` โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” โ”‚ Lecture Materials โ”‚ โ”‚ PDF ยท TXT ยท MD ยท DOCX ยท PPTX ยท VTT ยท SRT โ”‚ โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ โ–ผ โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” โ”‚ Claude API ยท claude-opus-4-6 โ”‚ โ”‚ thinking: { type: "adaptive" } โ”‚ โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ โ–ผ โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” โ”‚ OBSERVE โ”‚โ”€โ–ถโ”‚ PLAN โ”‚โ”€โ–ถโ”‚ ACT โ”‚โ”€โ–ถโ”‚ EVALUATEโ”‚ โ”‚ โ”‚ โ”‚ + DAG โ”‚ โ”‚ quizzesโ”‚ โ”‚ + LLM โ”‚ โ”‚ extract โ”‚ โ”‚ sort โ”‚ โ”‚ 3 MCQs โ”‚ โ”‚feedback โ”‚ โ”‚ topics โ”‚ โ”‚ โ”‚ โ”‚ โ”‚ โ”‚ โ”‚ โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ โ””โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”˜ โ–ฒ โ”‚ โ”‚ โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ–ผโ” โ”‚ โ”‚ ADAPT โ”‚ โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ค Heuristic OR Q-learning โ”‚ โ”‚ StudentProfile updated โ”‚ โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ โ–ผ โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” โ”‚ Persistent Belief State โ”‚ โ”‚ (JSON storage ยท per student) โ”‚ โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ โ”‚ โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ผโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” โ–ผ โ–ผ โ–ผ Spaced Repetition Concept Graph Interfaces (FSRS) (DAG topo sort) Streamlit ยท Chrome ext MCP server ยท Agent Skill ``` The agent is modeled as a **POMDP** (partially observable Markov decision process): - **State** โ€” student's true knowledge (hidden) - **Belief state** โ€” `StudentProfile` (mastered topics, weak areas, quiz history) - **Actions** โ€” `advance` ยท `reinforce` ยท `review` - **Observations** โ€” student answers to generated quizzes - **Reward** โ€” improvement in quiz scores over time --- ## Features ### Core Agent - **5-phase OPEAA loop** โ€” Observe โ†’ Plan โ†’ Act โ†’ Evaluate โ†’ Adapt - **Claude integration** with `thinking: {type: "adaptive"}` for internal reasoning - **Goal-based agent design** following Russell & Norvig's PEAS framework - **POMDP belief state** persisted across sessions - **Two adaptive policies** โ€” heuristic (Bloom's 70% mastery threshold) and tabular Q-learning ### Knowledge & Memory - **Concept dependency graph** โ€” Kahn's algorithm topological sort over a topic prerequisite DAG, rendered as an **interactive draggable graph** (pyvis) - **FSRS spaced repetition** โ€” per-topic memory model (stability ยท difficulty ยท recall probability) via [py-fsrs](https://github.com/open-spaced-repetition/py-fsrs); automatic SM-2 fallback - **Anki export** โ€” one click turns your generated quiz bank into a styled `.apkg` deck - **Persistent SQLite storage** โ€” student profiles survive across sessions - **Multi-student support** with peer comparison dashboard ### Integrations - **MCP server** โ€” Claude Desktop / Claude Code can query your review queue, generate quizzes, record results (training the RL policy), and export decks - **Claude Agent Skill** โ€” `skills/smartstudy/` turns any Claude Code session into an adaptive study coach - **Multi-provider LLM** โ€” Claude, HF Inference, or **any OpenAI-compatible endpoint** (Ollama, LM Studio, vLLM, DeepSeek) โ€” fully local studying is supported ### Input & Evaluation - **7 input formats** โ€” PDF, TXT, MD, DOCX, PPTX, VTT, SRT - **Quantitative evaluation** โ€” Monte Carlo simulation of adaptive vs random baselines - **Mock client** โ€” `MockAnthropic` lets you run the entire system offline without an API key ### User Interface - **Streamlit web app** with 8 pages (premium glassmorphism theme) - **Chrome extension (MV3)** โ€” run the full OPEAA loop on any web page, Q-table persisted in `chrome.storage.local` - **Interactive terminal UI** powered by `rich` - **Auto-demo mode** for video recording --- ## Installation ```bash git clone https://github.com/HumphreySun98/Smart-Study-Agent.git cd Smart-Study-Agent pip install -r requirements.txt ``` The agent supports four LLM backends and picks one automatically: | Backend | Env variable | Cost | Quality | |---------|--------------|------|---------| | **Anthropic Claude** | `ANTHROPIC_API_KEY` | Pay as you go | โญโญโญโญโญ Best โ€” supports adaptive thinking | | **Any OpenAI-compatible endpoint** (Ollama, LM Studio, vLLM, DeepSeekโ€ฆ) | `SMARTSTUDY_LLM_BASE_URL` + `SMARTSTUDY_LLM_MODEL` | Free if local | โญโญโญโ€“โญโญโญโญ your choice | | **HF Inference (Kimi-K2)** | `HF_TOKEN` | **Free** | โญโญโญโญ Great | | **Mock** | _(no env vars)_ | Free | โญโญ Canned responses for offline demos | ```bash # Option 1 โ€” Claude (premium quality) export ANTHROPIC_API_KEY=sk-ant-... # Option 2 โ€” fully local & private with Ollama export SMARTSTUDY_LLM_BASE_URL=ollama # shortcut for http://localhost:11434/v1 export SMARTSTUDY_LLM_MODEL=llama3.1 # ...or any other OpenAI-compatible server export SMARTSTUDY_LLM_BASE_URL=https://api.deepseek.com/v1 export SMARTSTUDY_LLM_MODEL=deepseek-chat export SMARTSTUDY_LLM_API_KEY=sk-... # Option 3 โ€” Hugging Face (completely free) export HF_TOKEN=hf_... # Option 4 โ€” Mock mode (no setup) # just run the agent without any keys ``` Get a Claude key from [console.anthropic.com](https://console.anthropic.com) ($5 free credit) or a free HF token from [huggingface.co/settings/tokens](https://huggingface.co/settings/tokens). --- ## Quick Start ### Hosted Demo (zero install) ๐Ÿ‘‰ **[https://huggingface.co/spaces/HumphreySun98/smart-study-agent](https://huggingface.co/spaces/HumphreySun98/smart-study-agent)** ### Web App (local) ```bash streamlit run app.py ``` Open **http://localhost:8501**, create a student in the sidebar, then go to **๐Ÿ“– Study Session** to run the full OPEAA loop on a sample ML lecture or your own PDF. ### Terminal demo (interactive) ```bash python demo.py python demo.py --pdf path/to/lecture.pdf python demo.py --mock # offline mode, no API key needed ``` ### Auto demo (for screen recording) ```bash python demo_auto.py ``` ### Chrome extension (run the agent on any web page, PDF, or YouTube video) **Now live on the Chrome Web Store** โ€” [install in one click](https://chromewebstore.google.com/detail/edbjkpfjonahanfkamlcbobmnplihmik). Prefer to run the source directly? Load it unpacked in < 60 seconds: ``` 1. chrome://extensions โ†’ enable Developer mode 2. Load unpacked โ†’ select the chrome-extension/ folder 3. Pin the SmartStudy icon โ†’ clicking it opens the Side Panel 4. Settings โ†’ pick a backend (Anthropic or free HF) โ†’ paste key โ†’ Save 5. Open any article / PDF / YouTube page โ†’ "Observe this page" ``` Full install + architecture notes in [chrome-extension/README.md](chrome-extension/). ### MCP server (drive SmartStudy from Claude) ```bash pip install "mcp[cli]" # Claude Code claude mcp add smartstudy -- python /path/to/mcp_server.py # Claude Desktop โ€” add to claude_desktop_config.json: # {"mcpServers": {"smartstudy": {"command": "python", # "args": ["/path/to/mcp_server.py"]}}} ``` Then just ask Claude: *"What's due for review today?"*, *"Quiz me on backpropagation and record my score"*, *"Export my question bank to Anki"*. Every recorded result trains the Q-learning policy and reschedules the topic under FSRS. ### Claude Agent Skill (adaptive study coach in your terminal) ```bash cp -r skills/smartstudy ~/.claude/skills/ ``` Open Claude Code anywhere and say "study this PDF with me" โ€” the skill runs the full OPEAA loop, one question at a time, with real FSRS scheduling underneath. ### Anki export Every generated quiz is saved to a per-student question bank. Export it from the **๐Ÿƒ Anki Export** page (or `python -c "from anki_export import export_student_deck; export_student_deck('alice')"`) and import the `.apkg` into Anki โ€” cards are tagged by topic and styled. --- ## Programmatic API ```python from smartstudy_agent import SmartStudyAgent agent = SmartStudyAgent() # uses ANTHROPIC_API_KEY env var # Phase 1 โ€” Observe observed = agent.observe("Lecture text about Machine Learning...") # {'topics': [...], 'descriptions': {...}, 'summary': '...'} # Phase 2 โ€” Plan plan = agent.plan(observed) print(plan.sequence) # ['Linear Algebra', 'Neural Networks', ...] # Phase 3 โ€” Act topic = plan.sequence[0] questions = agent.act(topic, observed["descriptions"][topic], n=3) # Phase 4 โ€” Evaluate result = agent.evaluate(questions, answers=["B", "A", "C"]) print(f"Score: {result['score']:.0%}") print(result["feedback"]) # Phase 5 โ€” Adapt adaptation = agent.adapt(topic, result) print(adaptation["action"]) # 'advance' | 'reinforce' | 'review' print(agent.profile.summary()) ``` ### Supporting modules ```python import storage from concept_graph import ConceptGraph from spaced_repetition import get_review_queue from rl_policy import QLearningPolicy from evaluation import compare # Persistent storage record = storage.load_student("alice") storage.add_session("alice", {"topic": "Neural Networks", "score": 0.9}) # Concept dependency graph (topological sort) g = ConceptGraph() g.topological_sort(["Backpropagation", "Linear Algebra", "Neural Networks"]) # -> ['Linear Algebra', 'Neural Networks', 'Backpropagation'] # Spaced repetition scheduler (SM-2) due_today = get_review_queue(record["quiz_history"]) # Q-learning adaptive policy policy = QLearningPolicy() action = policy.choose_action(score=0.55) # 'reinforce' policy.update(prev_score=0.55, action=action, new_score=0.80) # Quantitative evaluation vs random baseline results = compare(n_runs=30, n_sessions=20) print(f"Adaptive beats baseline by {results['improvement_pct']:.1f}%") ``` --- ## Web App Pages | Page | Purpose | |------|---------| | ๐Ÿ  **Dashboard** | Mastered topics, weak areas, due reviews, and key metrics | | ๐Ÿ“– **Study Session** | Upload a lecture and run the full OPEAA loop step-by-step | | ๐Ÿ” **Spaced Review** | FSRS memory state per topic โ€” recall %, stability, next due date | | ๐Ÿƒ **Anki Export** | Build a styled `.apkg` deck from your generated question bank | | ๐Ÿง  **Concept Graph** | Interactive draggable prerequisite DAG โ€” mastered/weak topics color-coded | | ๐Ÿ“Š **Progress History** | Personal score trajectory across all attempts | | ๐Ÿ‘ฅ **Peer Comparison** | Multi-student leaderboard ranked by average score | | ๐ŸŽฏ **RL Policy** | Inspect the Q-table and train it on simulated episodes | | ๐Ÿงช **Baseline Evaluation** | Adaptive vs random topic-selection simulation results | | ๐Ÿ“‹ **Pilot Study** | Real usage metrics, engagement analysis, learning progression report | --- ## Project Structure ``` smartstudy-agent/ โ”œโ”€โ”€ smartstudy_agent.py # Core agent โ€” 5 OPEAA phases โ”œโ”€โ”€ mock_claude.py # Offline mock client โ”œโ”€โ”€ hf_client.py # Hugging Face Inference adapter (free LLM backend) โ”œโ”€โ”€ app.py # Streamlit web app (8 pages) โ”œโ”€โ”€ demo.py # Interactive terminal demo โ”œโ”€โ”€ demo_auto.py # Automated demo (no input needed) โ”‚ โ”œโ”€โ”€ storage.py # SQLite persistent storage (+ question bank) โ”œโ”€โ”€ concept_graph.py # Topic prerequisite DAG with cross-course linking โ”œโ”€โ”€ pilot_study.py # Pilot study data collection and analysis โ”œโ”€โ”€ rl_policy.py # Tabular Q-learning policy โ”œโ”€โ”€ bandit_policy.py # Contextual Bandit (LinUCB) โ€” alternative to RL โ”œโ”€โ”€ spaced_repetition.py # FSRS review scheduler (SM-2 fallback) โ”œโ”€โ”€ anki_export.py # Question bank โ†’ Anki .apkg deck (genanki) โ”œโ”€โ”€ mcp_server.py # MCP server โ€” drive the agent from Claude โ”œโ”€โ”€ skills/smartstudy/ # Claude Agent Skill โ€” study coach for Claude Code โ”œโ”€โ”€ multi_format.py # PDF/TXT/MD/DOCX/PPTX/VTT/SRT loader โ”œโ”€โ”€ evaluation.py # Adaptive vs baseline simulation โ”œโ”€โ”€ experiments/ # Reproducible studies (scheduler comparison, ...) โ”œโ”€โ”€ docs/DEEP_DIVE.md # POMDP framing, policy math, experiment analysis โ”‚ โ”œโ”€โ”€ generate_visuals.py # Generates architecture diagrams โ”œโ”€โ”€ requirements.txt # Python dependencies โ”œโ”€โ”€ README.md # This file โ”‚ โ”œโ”€โ”€ chrome-extension/ # Chrome MV3 extension โ€” OPEAA loop in the browser โ”‚ โ”œโ”€โ”€ manifest.json โ”‚ โ”œโ”€โ”€ popup.{html,css,js} # Gradient popup UI + full agent logic โ”‚ โ”œโ”€โ”€ content.js # Active-tab text extractor โ”‚ โ”œโ”€โ”€ options.{html,js} # API key + model settings โ”‚ โ”œโ”€โ”€ background.js # Service worker โ”‚ โ””โ”€โ”€ icons/ # 16/48/128 PNG โ”‚ โ”œโ”€โ”€ data/ # Created at runtime โ”‚ โ”œโ”€โ”€ smartstudy.db # SQLite database (student profiles + sessions) โ”‚ โ”œโ”€โ”€ qtable.json # Q-learning policy state โ”‚ โ””โ”€โ”€ concept_graph.json # User-defined graph edges โ”‚ โ””โ”€โ”€ visuals/ # Generated PNG diagrams โ”œโ”€โ”€ adaptive_loop.png โ”œโ”€โ”€ system_architecture.png โ”œโ”€โ”€ performance_dashboard.png โ””โ”€โ”€ ai_techniques.png ``` --- ## Tech Stack | Layer | Technology | |-------|-----------| | LLM | Claude (adaptive thinking) ยท any OpenAI-compatible endpoint ยท HF Inference | | Web UI | Streamlit | | RL | Tabular Q-learning over discretized score buckets + LinUCB bandit | | Knowledge Graph | NetworkX + Kahn's algorithm + pyvis (interactive) | | Spaced Repetition | FSRS via [py-fsrs](https://github.com/open-spaced-repetition/py-fsrs) (SM-2 fallback) | | Flashcards | genanki โ†’ Anki `.apkg` | | Agent Interop | MCP server (FastMCP) + Claude Agent Skill | | Storage | SQLite (auto-migrates from JSON, scales to >1k students) | | Document Parsing | pypdf, python-docx, python-pptx | | Terminal UI | rich | --- ## How the Agent Decides The ADAPT phase uses a **two-layer decision system**: the RL policy chooses the action, and the LLM explains the decision to the student in natural language. ### Q-Learning Policy (decides the action) The action (`advance` / `reinforce` / `review`) is chosen by a tabular Q-learning agent โ€” **not** by the LLM. This runs every time a student finishes a quiz. | Component | Value | |-----------|-------| | **State** | Quiz score discretized into 5 buckets: `very_low` / `low` / `medium` / `high` / `very_high` | | **Actions** | `review` ยท `reinforce` ยท `advance` | | **Reward** | Score change between attempts: `r = (new_score โˆ’ prev_score) ร— 10` | | **Learning rate (ฮฑ)** | 0.2 | | **Discount factor (ฮณ)** | 0.8 | | **Exploration (ฮต)** | 0.15 (epsilon-greedy) | Update rule: ``` Q(s, a) โ† Q(s, a) + ฮฑ ยท [r + ฮณ ยท max(Q(s', a')) โˆ’ Q(s, a)] ``` The Q-table is **persisted to disk** (`data/qtable.json`) and trains on every real quiz attempt. It can also be inspected and manually trained in the **๐ŸŽฏ RL Policy** page. ### LLM Layer (explains the decision) After the RL policy picks the action, Claude (or Kimi-K2) generates a natural-language explanation of *why* that action makes sense for the student. The LLM cannot override the RL decision โ€” it only produces the recommendation text. ``` Student takes quiz โ†’ score = 55% โ†’ RL policy: Q("medium", "reinforce") = 0.42 (highest) โ†’ action = "reinforce" โ†’ Q-table updated with reward = (0.55 - 0.40) ร— 10 = 1.5 โ†’ LLM generates: "You're close! Practice the same topic one more time..." ``` ### Heuristic Fallback The Q-table is initialized with values informed by Bloom's 1968 mastery learning threshold (70%). As real data accumulates, the learned policy diverges from the heuristic and adapts to actual student behavior patterns. --- ## Why RL (and not just a Contextual Bandit)? **Context.** A valid critique of applying full RL to this problem is that if each decision is nearly independent, a **Contextual Bandit** is more sample-efficient than a sequential RL agent. We take that critique seriously, so the project ships **both** and compares them directly. **When RL is justified here.** The student's mastery state depends on the *sequence* of actions, not just the current context: 1. **Prerequisite coupling.** Studying *Backprop* before *Neural Nets* is mastered gives a smaller skill gain (the simulated student encodes this via a prerequisite DAG). A bandit chooses actions independently per step and cannot trade off short-term score for long-term skill gain. 2. **Forgetting.** Topics not practiced decay each step, so *when* you schedule a review matters โ€” a classic sequential credit-assignment problem. 3. **Action latency.** `review` tends to depress the immediate next quiz score (the student is working on a weak area) but pays off several steps later. A bandit, optimizing only single-step reward, systematically underweights this. **When a Bandit is better.** If the deployment looks more like A/B-testing recommendation variants over many users with little per-user history, a bandit will converge faster and is probably the right tool. We added `bandit_policy.LinUCBBandit` so the same agent can be run in that mode via `SmartStudyAgent(policy="bandit")` or `SMARTSTUDY_POLICY=bandit`. ### Empirical comparison Run `python evaluation.py`. Each policy is evaluated on 30 simulated students, all facing the **same** student trajectories for a fair paired comparison: | Policy (30 sessions) | Avg. observed score | Final mean skill | vs. random | |---------------|---------------------|------------------|------------| | Random | 0.34 ยฑ 0.02 | 0.29 ยฑ 0.01 | +0.0 % | | Rule-based (Bloom 70 %) | **0.46 ยฑ 0.02** | **0.53 ยฑ 0.02** | **+36.6 %** | | Contextual Bandit (LinUCB) | 0.43 ยฑ 0.02 | 0.47 ยฑ 0.02 | +27.7 % | | Q-learning (tabular) | 0.40 ยฑ 0.04 | 0.43 ยฑ 0.06 | +19.1 % | **Reading the result honestly.** The rule-based heuristic wins in this simulator โ€” at 30 sessions *and* at 100 (we previously reported Q-learning catching up by ~100 sessions; a larger fresh run does not reproduce that, so we've corrected the claim). The Bandit is the best learned policy and essentially ties the rule on final skill by 100 sessions (0.51 vs 0.51); Q-learning trails with the highest variance โ€” the bootstrap-variance tax is real at this sample size. **The honest claim is "learned policies are competitive and keep training on real quiz outcomes," not "RL beats everything."** Full analysis with both horizons in the [Technical Deep Dive](docs/DEEP_DIVE.md). ### Does the review scheduler matter? (new experiment) We isolated the *scheduler* (same student, same actions, only the timing differs) across two regimes โ€” a 6-topic course with uniform forgetting, and a 24-topic corpus with per-topic decay rates:

FSRS vs SM-2 vs rotation vs random scheduling across two regimes

**Finding:** with few topics and ample review capacity, *scheduling doesn't matter* โ€” round-robin matches FSRS. With 24 topics and scarce capacity, due-date scheduling retains **4.5ร— more** than rotation (which collapses below random), and FSRS โ‰ˆ SM-2 within noise on this simulator. Spaced repetition is a **triage algorithm โ€” it pays off exactly when material volume ร— decay variance exceeds review capacity.** Design, caveats and numbers: [docs/DEEP_DIVE.md ยง4](docs/DEEP_DIVE.md). ### Simulated Student Model Following the evaluation feedback, we replaced the earlier noise-only simulator with a small cognitive model (`evaluation.SimulatedStudent`): per-topic hidden skills, prerequisite-gated learning gain, diminishing returns as skill โ†’ 1, and per-step forgetting on unpracticed topics. This is what makes the rule-based vs. bandit vs. RL comparison meaningful โ€” a purely-random simulated student would flatten the differences. --- ## Roadmap - [x] Core 5-phase OPEAA loop with Claude - [x] Heuristic adaptive policy (Bloom 70%) - [x] Persistent multi-student storage - [x] Concept dependency graph + topological sort - [x] Q-learning adaptive policy - [x] SM-2 spaced repetition - [x] Streamlit web app with 8 pages - [x] Multi-format input loader - [x] Quantitative baseline evaluation - [x] Concept graph editor in the UI - [x] Cross-course prerequisite linking (4 courses: AI, Data Science, NLP, Computer Vision) - [x] Pilot study dashboard with engagement analysis and progression tracking - [x] SQLite storage backend (replaces JSON, handles >1k students) - [x] Deployed as hosted SaaS on [Hugging Face Spaces](https://huggingface.co/spaces/HumphreySun98/smart-study-agent) - [x] Contextual Bandit (LinUCB) policy as an alternative to full RL - [x] 4-way evaluation against Rule-based baseline + Simulated Student Model (per professor feedback) - [x] **Chrome extension (MV3)** โ€” same OPEAA loop on any web page, client-side Q-learning - [x] Migrate extension to `chrome.sidePanel` for persistent belief-state display - [x] Chrome Web Store listing โ€” [live](https://chromewebstore.google.com/detail/edbjkpfjonahanfkamlcbobmnplihmik) - [x] **FSRS scheduler** (py-fsrs) replacing SM-2 โ€” per-topic memory state with recall probability - [x] **Anki .apkg export** from the generated question bank - [x] **MCP server** โ€” use SmartStudy from Claude Desktop / Claude Code - [x] **Claude Agent Skill** โ€” adaptive study coach in any Claude Code session - [x] **Multi-provider LLM** โ€” Ollama / LM Studio / vLLM / DeepSeek via OpenAI-compatible API - [x] Interactive concept graph (pyvis) - [ ] FSRS parameter optimization from real review logs - [ ] Import existing Anki decks as topics - [ ] Bayesian Knowledge Tracing as a third mastery model --- ## ๐Ÿ‡จ๐Ÿ‡ณ ไธญๆ–‡็ฎ€ไป‹ **SmartStudy Agent โ€” ไผšๅญฆไน "ไฝ ๆ€Žไนˆๅญฆ"็š„ AI ๅญฆไน ๆ™บ่ƒฝไฝ“ใ€‚** ๅ’Œๆ™ฎ้€š AI ๅญฆไน ๅทฅๅ…ท็š„ๅŒบๅˆซ:ๅ†ณๅฎš"ไธ‹ไธ€ๆญฅๅญฆไป€ไนˆ"็š„ไธๆ˜ฏ LLM,่€Œๆ˜ฏไธ€ไธช**ๅผบๅŒ–ๅญฆไน ็ญ–็•ฅ**(Q-learning / LinUCB ่€่™Žๆœบ,้™„ๅธฆ่ฏšๅฎž็š„ๅŸบๅ‡†ๅฏนๆฏ”);ๅ†ณๅฎš"ไป€ไนˆๆ—ถๅ€™ๅคไน "็š„ๆ˜ฏ **FSRS ่ฎฐๅฟ†ๆจกๅž‹**(ไธŽๆ–ฐ็‰ˆ Anki ๅŒๆบ็ฎ—ๆณ•);LLM ๅช่ดŸ่ดฃๅ‡บ้ข˜ๅ’Œ่ฎฒ่งฃใ€‚ - ๐ŸŒ **ๅœจ็บฟไฝ“้ชŒ**:[Hugging Face Space](https://huggingface.co/spaces/HumphreySun98/smart-study-agent)(ๅ…่ดน,ๆ— ้œ€ๆณจๅ†Œ) - ๐Ÿงฉ **Chrome ๆ’ไปถ**:[ๅทฒไธŠๆžถ Chrome ๅ•†ๅบ—](https://chromewebstore.google.com/detail/edbjkpfjonahanfkamlcbobmnplihmik),ๅœจไปปๆ„็ฝ‘้กต / PDF / YouTube ไธŠ็›ดๆŽฅๅญฆ - ๐Ÿƒ **Anki ๅฏผๅ‡บ**:็”Ÿๆˆ็š„้ข˜ๅบ“ไธ€้”ฎๅฏผๅ‡บ `.apkg` - ๐Ÿค– **MCP server**:ๅœจ Claude Desktop / Claude Code ้‡Œ็›ดๆŽฅ้—ฎ"ไปŠๅคฉ่ฏฅๅคไน ไป€ไนˆ" - ๐Ÿ  **ๅฎŒๅ…จๆœฌๅœฐ**:ๆ”ฏๆŒ Ollama / LM Studio ็ญ‰ OpenAI ๅ…ผๅฎน็ซฏ็‚น,ๆ•ฐๆฎไธๅ‡บๆœฌๆœบ ```bash pip install -r requirements.txt streamlit run app.py # ้›ถ้…็ฝฎ็ฆป็บฟ Mock ๆจกๅผๅณๅฏไฝ“้ชŒ ``` ่ง‰ๅพ—ๆœ‰็”จ่ฏท็‚นไธช โญ! --- ## License MIT License โ€” see [LICENSE](LICENSE) for details. ``` Copyright ยฉ 2026 Haofei Sun Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction... ``` --- ## Author **Haofei Sun** If you find this project useful, please consider giving it a โญ on GitHub. For questions, suggestions, or collaboration: open an [issue](../../issues) or start a [discussion](../../discussions).