--- name: chatkit-integration description: Foundation skill for integrating OpenAI ChatKit framework with custom backends. This skill should be used for initial ChatKit setup including server implementation, React component integration, authentication, context injection, and database persistence. For streaming UI patterns use chatkit-streaming. For interactive widgets and actions use chatkit-actions. --- # ChatKit Integration Skill (Tier 1: Foundation) ## Overview This skill provides the foundation for ChatKit integration - getting the basic chat working end-to-end with your custom backend and agent. It covers: - **Backend**: ChatKitServer setup, database persistence, agent wiring - **Frontend**: useChatKit configuration, authentication, context injection - **Infrastructure**: Script loading, httpOnly cookie proxy (Next.js) **For advanced capabilities, see related skills:** - `chatkit-streaming` (Tier 2): Response lifecycle, progress updates, client effects - `chatkit-actions` (Tier 3): Interactive widgets, entity tagging, composer tools ## Persona You are a full-stack engineer integrating OpenAI ChatKit framework with a custom backend and AI agents. You understand that ChatKit provides standardized conversation UI/UX, but requires custom integration to work with domain-specific agents and context. ## Questions to Ask Before Implementing 1. **Backend Integration**: - What agent framework are you using? (OpenAI Agents SDK, LangChain, custom) - What tools does your agent need? (RAG search, custom functions) - What context does your agent need? (user profile, page context, conversation history) - What database are you using? (PostgreSQL, MongoDB, Redis) 2. **Frontend Integration**: - What frontend framework? (React, Next.js, Docusaurus) - How is authentication handled? (OAuth, JWT, session cookies, **httpOnly cookies**) - Are auth tokens stored in httpOnly cookies? (Requires server-side proxy) - What context can you extract client-side? (page URL, title, DOM content) - Do you need custom UI features? (text selection, personalization menu) 3. **Context Requirements**: - What user information is available? (name, email, role, preferences) - What page context is needed? (URL, title, headings, content) - How should context be transmitted? (headers, metadata, query params) - Should context be included in every request or only when needed? 4. **Persistence Requirements**: - Do conversations need to persist across sessions? - What's the expected conversation volume? (affects database choice) - Do you need multi-tenancy? (organization isolation) - What's the retention policy? (how long to keep conversations) ## Principles ### Backend Principles 1. **Extend ChatKit Server, Don't Replace** - Inherit from `ChatKitServer[RequestContext]` - Override only `respond()` method for agent execution - Let base class handle read-only operations (threads.list, items.list) - **Rationale**: ChatKit handles protocol, you handle agent logic 2. **Context Injection in Prompt** - Include conversation history as string in system prompt (CarFixer pattern) - Include user context (name, profile) in system prompt - Include page context (current page) in system prompt - **Rationale**: Agent SDK receives single prompt, history must be in prompt 3. **User Isolation via RequestContext** - All operations scoped by `user_id` in `RequestContext` - Store operations filter by `user_id` automatically - Never expose data across users - **Rationale**: Multi-tenant safety, data privacy 4. **Graceful Degradation** - System starts even if database unavailable (ChatKit disabled) - RAG search can fail without blocking ChatKit - Log warnings but don't crash - **Rationale**: Partial functionality better than no functionality 5. **Connection Pool Warmup** - Pre-warm database connections on startup - Avoids 7+ second first-request delay - Test connections before use (`pool_pre_ping=True`) - **Rationale**: Production-ready performance ### Frontend Principles 1. **Custom Fetch Interceptor** - Provide custom `fetch` function to `useChatKit` config - Intercept all ChatKit requests - Add authentication headers (`X-User-ID`) - Add metadata (userInfo, pageContext) to request body - **Rationale**: ChatKit doesn't handle auth natively, you must inject it 2. **Script Loading Detection** - Check for ChatKit custom element before rendering - Listen for script load events - Only render ChatKit component when script ready - Handle script load failures gracefully - **Rationale**: External script required, component fails without it 3. **Page Context Extraction** - Extract client-side (DOM, window.location) - Include: URL, title, path, headings, meta description - Send with every message (in metadata) - **Rationale**: Agent needs to know what user is viewing 4. **Build-Time Configuration** - Read env vars in `docusaurus.config.ts` (build-time) - Add to `customFields` for client-side access - Don't use `process.env` in browser code - **Rationale**: Static sites bake config at build time 5. **Authentication Gate** - Require login before allowing chat access - Show login prompt if not authenticated - Redirect to OAuth flow - **Rationale**: User ID required for conversation persistence 6. **httpOnly Cookie Proxy (Next.js)** - httpOnly cookies cannot be accessed from JavaScript (security feature) - Create server-side API route to read cookies and add Authorization header - Proxy forwards requests to backend with JWT token - **Rationale**: Secure token storage requires server-side access 7. **Next.js Script Loading Strategy** - Use `beforeInteractive` for web component scripts in `layout.tsx` - Place Script in `` element - Script must load before React hydration for custom elements - **Rationale**: Web components must be defined before React renders them ## Implementation Patterns ### Pattern 1: ChatKit Server with Custom Agent **When**: Integrating ChatKit with OpenAI Agents SDK **Implementation**: ```python from chatkit.server import ChatKitServer from agents import Agent, Runner from chatkit.agents import stream_agent_response class CustomChatKitServer(ChatKitServer[RequestContext]): """Extend ChatKit server with custom agent.""" async def respond( self, thread: ThreadMetadata, input_user_message: UserMessageItem | None, context: RequestContext, ) -> AsyncIterator[ThreadStreamEvent]: # Only handle user messages (let base class handle read-only ops) if not input_user_message: return # Load conversation history previous_items = await self.store.load_thread_items( thread.id, after=None, limit=10, order="desc", context=context ) # Build history string for prompt history_str = "\n".join([ f"{item.role}: {item.content}" for item in reversed(previous_items.data) ]) # Extract context from metadata user_info = context.metadata.get('userInfo', {}) page_context = context.metadata.get('pageContext', {}) # Build context strings user_context_str = f"\nUser: {user_info.get('name')}\n" page_context_str = f"\nPage: {page_context.get('title')}\n" # Create agent with tools agent = Agent( name="Assistant", tools=[your_search_tool], instructions=f"{history_str}\n{user_context_str}{page_context_str}\n{system_prompt}", ) # Convert message to agent input converter = YourThreadItemConverter() agent_input = await converter.to_agent_input(input_user_message) # Run agent with streaming agent_context = YourAgentContext( thread=thread, store=self.store, request_context=context, ) result = Runner.run_streamed(agent, agent_input, context=agent_context) # Stream results async for event in stream_agent_response(agent_context, result): yield event ``` **Evidence**: `rag-agent/chatkit_server.py:100-270` ### Pattern 2: Custom Fetch Interceptor **When**: Adding authentication and context to ChatKit requests **Implementation**: ```typescript const { control, sendUserMessage } = useChatKit({ api: { url: `${backendUrl}/chatkit`, domainKey: domainKey, // Custom fetch to inject auth and context fetch: async (url: string, options: RequestInit) => { // Check authentication if (!isLoggedIn) { throw new Error('User must be logged in'); } const userId = session.user.id; const pageContext = getPageContext(); const userInfo = { id: userId, name: session.user.name, email: session.user.email, // ... other user fields }; // Modify request body to add metadata let modifiedOptions = { ...options }; if (modifiedOptions.body && typeof modifiedOptions.body === 'string') { const parsed = JSON.parse(modifiedOptions.body); if (parsed.type === 'threads.create' && parsed.params?.input) { parsed.params.input.metadata = { userId, userInfo, pageContext, ...parsed.params.input.metadata, }; modifiedOptions.body = JSON.stringify(parsed); } else if (parsed.type === 'threads.run' && parsed.params?.input) { if (!parsed.params.input.metadata) { parsed.params.input.metadata = {}; } parsed.params.input.metadata.userInfo = userInfo; parsed.params.input.metadata.pageContext = pageContext; modifiedOptions.body = JSON.stringify(parsed); } } // Add authentication header return fetch(url, { ...modifiedOptions, headers: { ...modifiedOptions.headers, 'X-User-ID': userId, 'Content-Type': 'application/json', }, }); }, }, }); ``` **Evidence**: `robolearn-interface/src/components/ChatKitWidget/index.tsx:197-240` ### Pattern 3: Script Loading Detection **When**: ChatKit requires external script, component must wait **Implementation**: ```typescript const [scriptStatus, setScriptStatus] = useState<'pending' | 'ready' | 'error'>( isBrowser && window.customElements?.get('openai-chatkit') ? 'ready' : 'pending' ); useEffect(() => { if (scriptStatus !== 'pending') return; // Check if already loaded if (window.customElements?.get('openai-chatkit')) { setScriptStatus('ready'); return; } // Listen for script load events const handleLoaded = () => setScriptStatus('ready'); const handleError = () => setScriptStatus('error'); window.addEventListener('chatkit-script-loaded', handleLoaded); window.addEventListener('chatkit-script-error', handleError); // Timeout after 5 seconds const timeoutId = setTimeout(() => { if (scriptStatus === 'pending') { setScriptStatus('error'); } }, 5000); return () => { window.removeEventListener('chatkit-script-loaded', handleLoaded); window.removeEventListener('chatkit-script-error', handleError); clearTimeout(timeoutId); }; }, [scriptStatus]); // Only render ChatKit when script ready {isOpen && scriptStatus === 'ready' && ( )} ``` **Evidence**: `robolearn-interface/src/components/ChatKitWidget/index.tsx:67-113` ### Pattern 4: Page Context Extraction **When**: Agent needs to know what page user is viewing **Implementation**: ```typescript const getPageContext = useCallback(() => { if (typeof window === 'undefined') return null; // Extract meta tags const metaDescription = document.querySelector('meta[name="description"]') ?.getAttribute('content') || ''; // Find main content const mainContent = document.querySelector('article') || document.querySelector('main') || document.body; // Extract headings const headings = Array.from(mainContent.querySelectorAll('h1, h2, h3')) .slice(0, 5) .map(h => h.textContent?.trim()) .filter(Boolean) .join(', '); return { url: window.location.href, title: document.title, path: window.location.pathname, description: metaDescription, headings: headings, timestamp: new Date().toISOString(), }; }, []); ``` **Evidence**: `robolearn-interface/src/components/ChatKitWidget/index.tsx:121-151` ### Pattern 5: Text Selection "Ask" Feature **When**: Users want to ask questions about selected content **Implementation**: ```typescript // Detect text selection useEffect(() => { const handleSelection = () => { const selection = window.getSelection(); if (!selection || selection.rangeCount === 0) { setSelectedText(''); setSelectionPosition(null); return; } const selectedText = selection.toString().trim(); if (selectedText.length > 0) { setSelectedText(selectedText); // Get selection position const range = selection.getRangeAt(0); const rect = range.getBoundingClientRect(); setSelectionPosition({ x: rect.left + rect.width / 2, y: rect.top - 10, }); } }; document.addEventListener('selectionchange', handleSelection); document.addEventListener('mouseup', handleSelection); return () => { document.removeEventListener('selectionchange', handleSelection); document.removeEventListener('mouseup', handleSelection); }; }, []); // Send selected text const handleAskSelectedText = useCallback(async () => { const pageContext = getPageContext(); const messageText = `Can you explain this from "${pageContext.title}":\n\n"${selectedText}"`; if (!isOpen) { setIsOpen(true); await new Promise(resolve => setTimeout(resolve, 300)); } await sendUserMessage({ text: messageText, newThread: false, }); // Clear selection window.getSelection()?.removeAllRanges(); setSelectedText(''); setSelectionPosition(null); }, [selectedText, isOpen, sendUserMessage, getPageContext]); ``` **Evidence**: `robolearn-interface/src/components/ChatKitWidget/index.tsx:153-187`, `273-331` ### Pattern 6: httpOnly Cookie Proxy (Next.js App Router) **When**: Authentication tokens stored in httpOnly cookies (cannot be read by JavaScript) **Implementation**: ```typescript // app/api/chatkit/route.ts import { NextRequest, NextResponse } from "next/server"; import { cookies } from "next/headers"; const API_BASE = process.env.NEXT_PUBLIC_API_URL || "http://localhost:8000"; export async function POST(request: NextRequest) { const cookieStore = await cookies(); // Read httpOnly cookie (only accessible server-side) const idToken = cookieStore.get("taskflow_id_token")?.value; if (!idToken) { return NextResponse.json({ error: "Not authenticated" }, { status: 401 }); } // Build target URL - note: ChatKit endpoint is at /chatkit, NOT /api/chatkit const url = new URL("/chatkit", API_BASE); try { const body = await request.text(); // Forward request with Authorization header const response = await fetch(url.toString(), { method: "POST", headers: { Authorization: `Bearer ${idToken}`, "Content-Type": "application/json", // Forward custom headers "X-User-ID": request.headers.get("X-User-ID") || "", "X-Page-URL": request.headers.get("X-Page-URL") || "", }, body: body || undefined, }); // Handle SSE streaming responses if (response.headers.get("content-type")?.includes("text/event-stream")) { return new Response(response.body, { status: response.status, headers: { "Content-Type": "text/event-stream", "Cache-Control": "no-cache", "Connection": "keep-alive", }, }); } // Return JSON for non-streaming responses const data = await response.json().catch(() => null); return NextResponse.json(data, { status: response.status }); } catch (error) { console.error("[ChatKit Proxy] Error:", error); return NextResponse.json({ error: "ChatKit proxy request failed" }, { status: 500 }); } } ``` **Evidence**: `web-dashboard/src/app/api/chatkit/route.ts` ### Pattern 7: Next.js Script Loading for Web Components **When**: ChatKit script must load before React hydration **Implementation**: ```tsx // app/layout.tsx import Script from "next/script"; export default function RootLayout({ children }: { children: React.ReactNode }) { return ( {/* MUST be in with beforeInteractive for web components */}