#part.content#
"); } // Image content else if (left(part.contenttype, 6) == "image/") { img = imageRead(part.content); writeDump(img); // Or save to file: fileWrite("./generated.png", part.content); } // Other content types else { writeDump(part); } } } else { // Simple text response (backward compatible) writeOutput(response); } ``` #### Response Structure Each part in the multipart response array contains: | Property | Type | Description | Example | |----------|------|-------------|---------| | **contenttype** | string | MIME type of the content | `"text/plain"`, `"image/png"`, `"image/jpeg"` | | **type** | string | Content category | `"text"` or `"binary"` | | **content** | string or binary | The actual content | Text string or Java byte array | #### String Conversion The multipart response array has special string conversion behavior for convenience: ```javascript response = inquiryAISession(aiSession, "Create an image and describe it"); // Array acts as string - concatenates all text/plain parts echo(response); // Outputs only text portions writeOutput(toString(response)); // Same as above // Still works as array for detailed handling if (isArray(response)) { writeOutput("Response contains #arrayLen(response)# parts"); } ``` **Note**: This string conversion is unique to multipart AI responses - regular Lucee arrays cannot be used this way. #### Provider Support for Multipart Responses - **Gemini (Google)**: ✅ Full support for multipart responses - **Claude (Anthropic)**: 🔄 Coming soon - **ChatGPT (OpenAI)**: 🔄 Coming soon - **Ollama**: 🔄 Coming soon ### Structured JSON Responses (Gemini Beta) **New in Lucee 7** - With Gemini's beta API, you can enforce structured JSON responses by defining a `generationConfig` with a response schema. This ensures the AI returns data in a specific, predictable format. #### Configuration Add `generationConfig` to your Gemini endpoint configuration to specify the desired response structure: ```json "recipes": { "class": "lucee.runtime.ai.google.GeminiEngine", "custom": { "connectTimeout": "2000", "beta": true, "message": "Keep all answers concise and accurate", "model": "gemini-2.5-flash", "apikey": "${GEMINI_API_KEY}", "socketTimeout": "120000", "temperature": "0.7", "conversationSizeLimit": "10", "generationConfig": { "responseMimeType": "application/json", "responseSchema": { "type": "ARRAY", "items": { "type": "OBJECT", "properties": { "recipeName": { "type": "STRING" }, "calories": { "type": "INTEGER" } }, "required": ["recipeName", "calories"] } } } } } ``` **Configuration Options:** - **`responseMimeType`**: Set to `"application/json"` to enforce JSON responses - **`responseSchema`**: Define the expected JSON structure using a schema - **`generationConfig`**: All other Gemini generation config options are supported and passed directly to the API #### Usage Example ```javascript aiSessionJson = createAISession( name: "recipes", limit: 100, temperature: 0.2, systemMessage: "Keep your answers as short as possible" ); response = inquiryAISession(aiSessionJson, "List 3 healthy breakfast ideas."); dump(response); ``` **Response:** ```json [ {"recipeName":"Oatmeal with berries","calories":300}, {"recipeName":"Greek yogurt with fruit","calories":250}, {"recipeName":"Scrambled eggs with spinach","calories":350} ] ``` **Note**: This feature requires Gemini's beta API (`"beta": true` or URL set to `https://generativelanguage.googleapis.com/v1beta/`). ### Provider Passthrough *(since 7.0.4.3)* All three AI engines (Claude, OpenAI, Gemini) support a **passthrough mechanism** for forwarding additional configuration directly to the provider's API. This allows you to use provider-specific features — including beta features — without needing Lucee to explicitly support them. #### How It Works When initializing an AI engine, Lucee extracts all known configuration properties (`model`, `apikey`, `temperature`, etc.) from the `custom` block. Any **remaining unknown keys** are collected and forwarded directly into the HTTP request body sent to the provider. Additionally, a `headers` key allows forwarding custom HTTP headers. This means: - New provider features work immediately via config, no Lucee update required - Beta APIs and experimental parameters are fully accessible - The passthrough respects existing keys — it will never overwrite fields Lucee sets explicitly (like `model` or `messages`) #### Passthrough Behavior by Provider | Provider | Unknown body fields | Custom headers | |----------|-------------------|----------------| | Claude (Anthropic) | Forwarded ✅ | Forwarded ✅ | | Gemini (Google) | Forwarded ✅ | Forwarded ✅ | | OpenAI / ChatGPT | Forwarded ✅ | Forwarded ✅ | **Important**: While Claude silently ignores truly unknown fields, Gemini and OpenAI will return a `400` error for fields their API does not recognize. Only forward fields that are valid for the target provider's API. #### Custom Headers Use the `headers` key inside `custom` to forward arbitrary HTTP headers: ```json "custom": { "headers": { "my-custom-header": "value" } } ``` #### Example: Claude with MCP MCP (Model Context Protocol) is an open standard that allows AI models to connect to external tool servers — databases, APIs, or any custom logic you expose. Since Anthropic's MCP support requires a beta header and specific request body fields, it is configured entirely via the passthrough mechanism — no special Lucee support is needed. See [[mcp]] for Lucee's three MCP servers and provider-specific configuration. ```json "ai": { "myclaude": { "class": "lucee.runtime.ai.anthropic.ClaudeEngine", "custom": { "apikey": "${CLAUDE_API_KEY}", "model": "claude-sonnet-4-6", "message": "You are a helpful assistant.", "headers": { "anthropic-beta": "mcp-client-2025-11-20" }, "mcp_servers": [ { "type": "url", "url": "https://your-mcp-server.com/mcp", "name": "my-tools", "authorization_token": "${MCP_TOKEN}" } ], "tools": [ { "type": "mcp_toolset", "mcp_server_name": "my-tools" } ] } } } ``` Usage in CFML is identical to any other AI session — the MCP integration is transparent: ```javascript aiSession = createAISession(name: 'myclaude'); answer = inquiryAISession(aiSession, "What tools do you have available?"); writeOutput(answer); ``` Claude will automatically discover the tools exposed by your MCP server and use them when relevant to answer the query. **Notes:** - The `anthropic-beta` header version (`mcp-client-2025-11-20`) may change as Anthropic promotes MCP out of beta. Update your config accordingly — no Lucee code change is needed. - Anthropic handles all MCP communication (tool discovery, tool calls, result handling) on their side. Lucee simply passes the server reference through. - For Gemini and OpenAI, MCP configuration format differs from Claude's. Refer to each provider's API documentation for the correct field names and structure. ### Session Serialization Save and restore conversation state across requests. See [AI Session Serialization](https://github.com/lucee/lucee-docs/blob/master/docs/recipes/ai-serialisation.md) for details. ```javascript // Serialize session serializedData = serializeAISession(myAISession); fileWrite("session.json", serializedData); // Load session later restoredAISession = loadAISession('mychatgpt', fileRead("session.json")); // Continue conversation inquiryAISession(restoredAISession, "Can you remind me what we were discussing?"); ``` ### Streaming Responses Stream responses in real-time for better user experience: ```javascript aiSession = createAISession(name:'mygemini'); inquiryAISession(aiSession, "Write a short story about AI", function(chunk) { writeOutput(chunk); cfflush(throwonerror=false); }); ``` ### Temperature Control Control randomness/creativity of responses (0.0 = deterministic, 1.0 = creative): ```javascript // More deterministic responses (good for factual questions) conservativeAISession = createAISession( name: 'mychatgpt', temperature: 0.2 ); // More creative responses (good for brainstorming) creativeAISession = createAISession( name: 'mychatgpt', temperature: 0.9 ); ``` ### Retrieval-Augmented Generation (RAG) Enhance AI responses with your own data using Lucene search integration. See [AI Augmentation with Lucene](https://github.com/lucee/lucee-docs/blob/master/docs/recipes/ai-augmentation.md) for complete details. ```javascript // Search your indexed content search contextpassages=5 contextBytes=4000 name="searchResults" collection="knowledge_base" criteria=userQuery; // Add search results as context to AI query augmentedQuery = "Query: #userQuery#\n\nContext: #serializeJSON(searchResults)#"; answer = inquiryAISession(aiSession, augmentedQuery); ``` ## Available Functions Lucee 7 provides the following AI functions (all available with or without the `Lucee` prefix): ### Core Functions - **`createAISession()`** / `LuceeCreateAISession()` - Create a new AI session - Arguments: `name`, `systemMessage`, `temperature`, `limit`, `connectTimeout`, `socketTimeout` - **`inquiryAISession()`** / `LuceeInquiryAISession()` - Send a query to an AI session - Arguments: `session`, `question` (string or array for multipart), `listener` (optional callback) - Returns: String (simple text) or Array (multipart response with text/images/files) - **`getAIMetadata()`** / `LuceeGetAIMetadata()` - Get metadata about an AI connection - Arguments: `name`, `refresh` (boolean) ### Serialization Functions - **`serializeAISession()`** - Serialize an AI session to JSON - Arguments: `session`, `maxlength` (optional), `condense` (boolean) - **`loadAISession()`** - Restore an AI session from serialized data - Arguments: `name`, `data` (JSON string or struct) For complete function documentation, refer to the AI functions documentation in the Lucee Administrator. ## Built-in Integration ### Exception Handling AI is integrated into Lucee's exception template. When an exception occurs and you have defined an AI connection with `"default": "exception"`, the AI engine will automatically analyze the exception and provide insights and suggestions for resolving the issue. ```json "ai": { "mychatgpt": { "class": "lucee.runtime.ai.openai.OpenAIEngine", "custom": { "secretKey": "${CHATGPT_SECRET_KEY}", "model": "gpt-4o-mini" }, "default": "exception" } } ``` When an error occurs, the exception screen will include AI-generated analysis and suggestions for fixing the issue. ### Monitor Documentation In the Lucee Administrator's Monitor Documentation tab, AI can answer questions about Lucee using retrieval-augmented generation (RAG) with the available documentation. Configure with `"default": "documentation"`. See [AI for Documentation](https://github.com/lucee/lucee-docs/blob/master/docs/recipes/ai-for-documentation.md) for setup details. ## Security Considerations Security is a top priority when integrating AI into Lucee. ### Local AI Lucee supports local AI hosting through Ollama and other local providers, allowing you to run AI engines without external data transmission. ### Logging Lucee logs all AI interactions to `ai.log`, or to `application.log` if `ai.log` is unavailable. This enables you to review what information is being transmitted to AI services. ### Best Practices - Use environment variables for API keys (never hardcode them) - Review AI logs regularly - Consider using local AI for sensitive data - Be mindful of what data you send to external AI services - Implement rate limiting in production environments - Filter sensitive information before sending to AI (data redaction coming in future releases) ## Migration from Lucee 6.2 If you're upgrading from Lucee 6.2 with the experimental AI features: 1. **Function names**: Your existing code using `LuceeCreateAISession`, `LuceeInquiryAISession`, etc. will continue to work. You can optionally remove the `Lucee` prefix for cleaner code. 2. **Configuration**: Your existing `.CFConfig.json` AI configurations will work without changes. 3. **New features**: You can now use: - Multipart content (images, PDFs) in your AI queries - Multipart responses (generated images and files from AI) - Structured JSON responses with Gemini beta API - Session serialization for persistent conversations - RAG with Lucene integration - Provider passthrough for beta and custom API features Example migration: ```javascript // Lucee 6.2 (still works in Lucee 7) aiSession = LuceeCreateAISession(name:'mychatgpt'); answer = LuceeInquiryAISession(aiSession, "Hello"); // Lucee 7 (cleaner syntax) aiSession = createAISession(name:'mychatgpt'); answer = inquiryAISession(aiSession, "Hello"); // Lucee 7 (new multipart request support) answer = inquiryAISession(aiSession, [ "Analyze this image", fileReadBinary("./photo.jpg") ]); // Lucee 7 (new multipart response support - Gemini with beta API) response = inquiryAISession(aiSession, "Create a logo and describe it"); if (isArray(response)) { // Handle both text and generated images loop array=response item="part" { if (part.contenttype == "text/plain") { echo(part.content); } else if (left(part.contenttype, 6) == "image/") { fileWrite("./generated.png", part.content); } } } ``` ## Examples ### Chatbot ```javascript aiSession = createAISession( name: 'mychatgpt', systemMessage: "You are a helpful customer service assistant." ); form action="" method="post" { textarea name="question" placeholder="Ask a question..."; button type="submit" { writeOutput("Send"); } } if (structKeyExists(form, "question")) { answer = inquiryAISession(aiSession, form.question); writeOutput("
");
}
}
}
else {
writeOutput(response);
}
```
### Process Input and Generate Visual Output
```javascript
aiSession = createAISession(name: 'mygemini');
// Analyze user's image and create a new variation
userImage = fileReadBinary(expandPath("./user_photo.jpg"));
response = inquiryAISession(aiSession, [
"Analyze the style of this image and create a similar one with a mountain landscape theme",
userImage
]);
// Extract and save generated images
if (isArray(response)) {
imageCount = 0;
loop array=response item="part" {
if (left(part.contenttype, 6) == "image/") {
imageCount++;
fileWrite("./result_#imageCount#.png", part.content);
}
}
// Output text explanation
writeOutput(toString(response)); // Auto-extracts text parts
}
```
### Document Summarization
```javascript
aiSession = createAISession(name:'myclaude');
// Process PDF report
pdfPath = expandPath("./reports/Q4-2024.pdf");
summary = inquiryAISession(aiSession, [
"Provide a brief summary of the key findings and recommendations from this report",
pdfPath // Lucee automatically reads the file
]);
writeOutput("#summary#
"); ``` ### Code Review Assistant ```javascript aiSession = createAISession( name: 'mychatgpt', systemMessage: "You are an expert CFML developer. Review code for best practices, security, and performance." ); code = fileRead(expandPath("./mycomponent.cfc")); review = inquiryAISession(aiSession, "Review this CFML code and suggest improvements:\n\n#code#" ); dump(review); ``` ### Knowledge Base Assistant with RAG ```javascript // Create session aiSession = createAISession(name:'myclaude'); // User question userQuery = "How do I configure a datasource in Lucee?"; // Search indexed documentation search contextpassages=5 contextBytes=4000 name="searchResults" collection="lucee_docs" criteria=userQuery; // Build augmented query with search results context = []; loop query=searchResults { arrayAppend(context, { "title": searchResults.title, "content": searchResults.context.passages.original }); } augmentedQuery = "Question: #userQuery#\n\nRelevant Documentation:\n#serializeJSON(context)#"; // Get AI response with documentation context answer = inquiryAISession(aiSession, augmentedQuery); dump(answer); ``` ### Structured Data Extraction ```javascript // Configure Gemini with JSON schema for product data aiSessionJson = createAISession( name: "geminijson", systemMessage: "Extract product information accurately" ); // Extract structured data from unstructured text productText = "The new MacBook Pro 16-inch costs $2499 and is currently in stock. Popular tags include: laptop, apple, professional."; productData = inquiryAISession(aiSessionJson, "Extract the product information from this text: #productText#" ); // Response is guaranteed to match the schema defined in config dump(productData); // Returns: {"productName":"MacBook Pro 16-inch","price":2499,"inStock":true,"tags":["laptop","apple","professional"]} // Direct database insertion is now easy queryExecute( "INSERT INTO products (name, price, in_stock, tags) VALUES (?, ?, ?, ?)", [ productData.productName, productData.price, productData.inStock, serializeJSON(productData.tags) ] ); ``` ### Claude with MCP Tools ```javascript // Configure Claude with an MCP server in .CFConfig.json (see Provider Passthrough section) // then use it like any other session - MCP is transparent to CFML code aiSession = createAISession(name: 'myclaude'); // Claude will automatically use MCP tools when relevant answer = inquiryAISession(aiSession, "Check the current weather in Zurich and summarize it"); writeOutput(answer); // Streaming also works with MCP inquiryAISession(aiSession, "List all open tasks from the project board", function(chunk) { writeOutput(chunk); cfflush(throwonerror=false); }); ``` ## Related Documentation - **[AI Context from Page Content](ai-page-context.md)** - Capture page HTML with a custom tag and feed it to an AI session - **[AI Augmentation with Lucene](ai-augmentation.md)** - Implement RAG to enhance AI with your own data - **[AI Session Serialization](ai-serialisation.md)** - Save and restore conversation state - **[AI for Documentation](ai-for-documentation.md)** - Configure AI in the Monitor Documentation tab - **[Lucene Extension](lucene-3-extension.md)** - Search and indexing functionality for RAG ## Future Development AI integration in Lucee continues to be actively developed. We welcome your feedback and feature requests. ### Planned Features - **Multipart Response Support**: Expanding support to Claude, ChatGPT, and Ollama - **Advanced Data Redaction**: Automatic filtering of sensitive data before sending to AI services - **Additional Providers**: Support for more AI providers and models - **Enhanced RAG Tools**: More built-in tools for retrieval-augmented generation - **Fine-Tuning Support**: More functionality for customizing AI behavior - **Native MCP Support**: Dedicated MCP configuration per provider beyond the current passthrough mechanism ## Getting Help - **Documentation**: Refer to the AI function documentation in the Lucee Administrator - **Community**: Ask questions on the [Lucee Dev forum](https://dev.lucee.org) - **Issues**: Report bugs on the [Lucee GitHub repository](https://github.com/lucee/Lucee/issues) ## See Also - [Monitoring and Debugging](monitoring-debugging.md) - [Lucee Administrator](lucee-administrator.md) - [Extensions](extensions.md)