{"$schema": "https://raw.githubusercontent.com/guitaripod/wwdc-sessions/master/schema/transcript.schema.json", "id": "wwdc2026-232", "language": "eng", "source": "https://developer.apple.com/videos/play/wwdc2026/232", "wordCount": 2032, "segments": [{"start": 7.0, "text": "Hi, I'm Angelos, an engineer on the MLX team."}, {"start": 10.0, "text": "Today I'm going to show you how to build"}, {"start": 12.0, "text": "and run agentic AI workflows entirely on your Mac using MLX."}, {"start": 17.0, "text": "No cloud, no API keys, just your hardware doing the work."}, {"start": 21.0, "text": "Over the past year, AI agents have gone from research prototypes"}, {"start": 25.0, "text": "to everyday productivity tools."}, {"start": 27.0, "text": "But before we talk about agents, let's look at what we had before."}, {"start": 32.0, "text": "Here's the chat experience you're familiar with."}, {"start": 34.0, "text": "You send a prompt to the language model."}, {"start": 36.0, "text": "The model sends a response back."}, {"start": 38.0, "text": "If you need to act on that response, run a command,"}, {"start": 41.0, "text": "check a file, or fix an error, that's on you."}, {"start": 45.0, "text": "But now you're talking to an agent."}, {"start": 48.0, "text": "The agent talks to the model to decide what to do."}, {"start": 51.0, "text": "Then it calls tools to actually do it:"}, {"start": 54.0, "text": "running commands, reading files, hitting APIs —"}, {"start": 57.0, "text": "It observes the results and goes back to the model to figure out the next step."}, {"start": 62.0, "text": "User to agent."}, {"start": 63.0, "text": "Agent to model."}, {"start": 64.0, "text": "Agent to tools."}, {"start": 66.0, "text": "This is the agentic loop."}, {"start": 67.0, "text": "And it keeps cycling until your task is done."}, {"start": 70.0, "text": "What makes this particularly exciting on Apple silicon"}, {"start": 73.0, "text": "is that the entire loop can run locally."}, {"start": 76.0, "text": "Your data stays on your machine; AI is available anywhere at any time"}, {"start": 81.0, "text": "and there are no usage costs."}, {"start": 84.0, "text": "Let me now show you what this looks like in practice."}, {"start": 87.0, "text": "Here I have an agent running locally on my Mac."}, {"start": 91.0, "text": "On my screen you can see the setup: on the left, MLX running the model,"}, {"start": 96.0, "text": "and on the right the OpenCode agent I am interacting with."}, {"start": 100.0, "text": "I asked it to fetch the recent pull requests from our MLX repository,"}, {"start": 103.0, "text": "summarize the changes, and identify anything that needs my attention."}, {"start": 107.0, "text": "The model reasons about the request, calls the GitHub CLI to fetch PR data,"}, {"start": 113.0, "text": "reads through the diffs, and produces a concise summary."}, {"start": 116.0, "text": "All of this is happening locally,"}, {"start": 118.0, "text": "the model runs on my hardware and only the git commands reach the network."}, {"start": 123.0, "text": "Well it seems like I have a lot of work to do after finishing this video."}, {"start": 127.0, "text": "Now that you've seen what's possible,"}, {"start": 129.0, "text": "let me walk you through how we'll get there today."}, {"start": 132.0, "text": "We'll start by introducing the local agentic AI stack,"}, {"start": 135.0, "text": "the four layers that make all of this work,"}, {"start": 138.0, "text": "from MLX at the foundation all the way up to the agent."}, {"start": 142.0, "text": "Then I'll show you step-by-step how to set up your own local agent."}, {"start": 146.0, "text": "After that, we'll look at how MLX gets the most out of your hardware"}, {"start": 150.0, "text": "to make agents fast."}, {"start": 152.0, "text": "And finally, we'll go through more live demos,"}, {"start": 154.0, "text": "including building a SwiftUI app from scratch and fixing a bug in Xcode."}, {"start": 159.0, "text": "Let's start with the stack."}, {"start": 162.0, "text": "The stack that powers local agentic AI on the Mac has four layers."}, {"start": 166.0, "text": "Let me walk you through each one, starting from the bottom."}, {"start": 169.0, "text": "At the bottom is MLX,"}, {"start": 171.0, "text": "our open-source array framework purpose-built for Apple silicon."}, {"start": 175.0, "text": "It handles all the low-level computation, Metal acceleration, and memory management."}, {"start": 180.0, "text": "This is the foundation everything else is built on."}, {"start": 183.0, "text": "One level up, we have the language model layer."}, {"start": 186.0, "text": "MLX-LM provides everything you need to load, run, quantize,"}, {"start": 191.0, "text": "and fine-tune large language models."}, {"start": 193.0, "text": "It supports thousands of models from HuggingFace"}, {"start": 196.0, "text": "and gives you both CLI tools and a Python API."}, {"start": 199.0, "text": "If you saw our sessions last year, this is what we covered in depth."}, {"start": 203.0, "text": "But to serve an agent, we need something more:"}, {"start": 205.0, "text": "a persistent server with a standard API."}, {"start": 208.0, "text": "That's where MLX-LM Server comes in."}, {"start": 211.0, "text": "This is an OpenAI-compatible HTTP server"}, {"start": 213.0, "text": "that exposes your local model through a standard API."}, {"start": 217.0, "text": "It supports structured tool calling so the model can invoke functions reliably,"}, {"start": 221.0, "text": "and reasoning models that can analyze complex problems"}, {"start": 224.0, "text": "step-by-step before responding."}, {"start": 226.0, "text": "It's a drop-in replacement for any cloud LLM API."}, {"start": 230.0, "text": "And at the top of the stack, we have the agent itself."}, {"start": 233.0, "text": "This can be any framework"}, {"start": 234.0, "text": "or tool that speaks the OpenAI chat completions protocol:"}, {"start": 238.0, "text": "Xcode, OpenCode, Pi agent, a custom script, or anything else."}, {"start": 244.0, "text": "Because MLX-LM Server provides a standard interface,"}, {"start": 247.0, "text": "any agent framework works out of the box."}, {"start": 250.0, "text": "And it's not just us building on this stack."}, {"start": 253.0, "text": "Several popular apps and tools build on MLX and MLX-LM."}, {"start": 257.0, "text": "Ollama, LM Studio, and vLLM are just a few of the most popular ones."}, {"start": 262.0, "text": "The ecosystem is broad and growing,"}, {"start": 265.0, "text": "and if you're using one of these tools, chances are you're already running on MLX."}, {"start": 271.0, "text": "So that's the stack."}, {"start": 272.0, "text": "Let me now show you how to set everything up yourself."}, {"start": 276.0, "text": "It only takes three steps to go from zero to a fully local agentic workflow."}, {"start": 281.0, "text": "Step one: install MLX-LM."}, {"start": 283.0, "text": "A single pip install gets you everything you need."}, {"start": 286.0, "text": "Step two: start the server."}, {"start": 289.0, "text": "Run mlx_lm.server with a model that supports tool calling."}, {"start": 293.0, "text": "Starting with a small model to test your set-up is always a good idea."}, {"start": 297.0, "text": "The server starts up, loads the model,"}, {"start": 299.0, "text": "and is ready to accept requests on local host."}, {"start": 302.0, "text": "Step three: point your agent at the local server."}, {"start": 305.0, "text": "In most agent frameworks,"}, {"start": 307.0, "text": "you just set the base URL to your local server's address"}, {"start": 310.0, "text": "and you're done."}, {"start": 311.0, "text": "The agent doesn't know or care"}, {"start": 313.0, "text": "that the model is running on your Mac rather than in the cloud."}, {"start": 318.0, "text": "Let me show you a concrete example."}, {"start": 320.0, "text": "Here's the configuration for OpenCode."}, {"start": 322.0, "text": "We define a local provider."}, {"start": 324.0, "text": "In particular, we set the URL to local host"}, {"start": 327.0, "text": "and set the model name the server expects."}, {"start": 330.0, "text": "We also tell OpenCode to use this local model for everything."}, {"start": 334.0, "text": "That's it."}, {"start": 335.0, "text": "Now every interaction runs through your local model."}, {"start": 338.0, "text": "Now that we have an agent talking to MLX,"}, {"start": 341.0, "text": "let's look at how MLX gets the most out of your hardware"}, {"start": 344.0, "text": "and addresses the key challenges of running agents locally."}, {"start": 348.0, "text": "The first challenge is prompt processing."}, {"start": 351.0, "text": "In an agentic workflow, every time the model receives tool output,"}, {"start": 355.0, "text": "it has to process all that new context before it can reason about the next step."}, {"start": 359.0, "text": "This happens over and over throughout the agentic loop, and it adds up fast."}, {"start": 365.0, "text": "Agentic sessions usually comprise hundreds of thousands of tokens"}, {"start": 368.0, "text": "and most of those are not generated."}, {"start": 372.0, "text": "The M5 chip introduces dedicated Neural Accelerators,"}, {"start": 375.0, "text": "and MLX can target them for exactly this kind of work."}, {"start": 379.0, "text": "Specifically, Neural Accelerators make matrix multiplication"}, {"start": 382.0, "text": "four times faster on M5 compared to M4."}, {"start": 385.0, "text": "And with the specialized multiplication and attention kernels in MLX"}, {"start": 389.0, "text": "this translates almost exactly to prompt processing speedup."}, {"start": 394.0, "text": "Reducing prompt processing time means your agents can read your codebase"}, {"start": 397.0, "text": "or process tool results almost four times faster."}, {"start": 401.0, "text": "And the best part?"}, {"start": 402.0, "text": "Taking advantage of Neural Accelerators requires no special arguments"}, {"start": 406.0, "text": "or code changes on your part,"}, {"start": 408.0, "text": "MLX selects the best kernel for the available hardware and it just works."}, {"start": 413.0, "text": "Let's now talk about the second challenge, concurrency."}, {"start": 417.0, "text": "In practice, agents rarely work alone."}, {"start": 420.0, "text": "A common pattern is for an agent to spawn several subagents,"}, {"start": 424.0, "text": "each tackling a different part of the problem in parallel."}, {"start": 427.0, "text": "One might be reading documentation, another searching code,"}, {"start": 430.0, "text": "and a third writing tests; all at the same time."}, {"start": 434.0, "text": "That means multiple requests hitting your local model simultaneously."}, {"start": 438.0, "text": "MLX-LM Server handles this with continuous batching."}, {"start": 445.0, "text": "Instead of processing requests one at a time,"}, {"start": 447.0, "text": "it dynamically groups incoming requests into batches"}, {"start": 450.0, "text": "and processes them together on the GPU."}, {"start": 453.0, "text": "New requests can join a batch in progress"}, {"start": 455.0, "text": "without waiting for the current one to finish."}, {"start": 457.0, "text": "The result is that your subagents don't stall waiting in a queue."}, {"start": 461.0, "text": "They all get served concurrently,"}, {"start": 463.0, "text": "which keeps the entire agentic workflow moving."}, {"start": 466.0, "text": "Finally, the third challenge is model size."}, {"start": 469.0, "text": "Sometimes a single machine, even one with 512GB of RAM,"}, {"start": 474.0, "text": "just isn't enough because the model is too large to fit in memory."}, {"start": 478.0, "text": "The most recent DeepSeek model for instance has a whopping 1.6 trillion parameters"}, {"start": 483.0, "text": "and requires more than 800GB of memory just for the weights."}, {"start": 488.0, "text": "MLX's distributed support lets you spread a model across multiple Macs"}, {"start": 492.0, "text": "connected over Thunderbolt or Ethernet."}, {"start": 495.0, "text": "For agents, this is powerful in two ways."}, {"start": 497.0, "text": "First, it lets you run much larger, more capable models"}, {"start": 501.0, "text": "that wouldn't fit on a single machine."}, {"start": 503.0, "text": "Second, it parallelizes prompt processing across devices,"}, {"start": 506.0, "text": "which directly speeds up the agentic loop"}, {"start": 508.0, "text": "since the model can process tool results faster."}, {"start": 513.0, "text": "Setting up distributed inference with MLX-LM Server"}, {"start": 515.0, "text": "is fairly straightforward."}, {"start": 517.0, "text": "You launch the server using mlx.launch"}, {"start": 519.0, "text": "and a hostfile that contains information about the nodes and the type of connection."}, {"start": 524.0, "text": "The model is automatically sharded across all available devices"}, {"start": 527.0, "text": "and everything else just works."}, {"start": 530.0, "text": "Starting with macOS 26.2, we have support for Thunderbolt RDMA,"}, {"start": 534.0, "text": "which provides low-latency, high-bandwidth communication over Thunderbolt."}, {"start": 539.0, "text": "As a result, distributed inference with MLX has seen significant speed-ups:"}, {"start": 543.0, "text": "up to three times with four nodes."}, {"start": 545.0, "text": "To learn how to set up your Macs for distributed inference with MLX,"}, {"start": 549.0, "text": "check out our session \"Explore distributed inference and training with MLX\"."}, {"start": 554.0, "text": "Remember our PR summary demo from earlier?"}, {"start": 557.0, "text": "That was a simple read-and-report task."}, {"start": 560.0, "text": "Let's now push things further and see what happens"}, {"start": 562.0, "text": "when we ask an agent to write an entire project from scratch"}, {"start": 565.0, "text": "and then fix a bug in an existing one."}, {"start": 569.0, "text": "In this demo, I'm going to ask the agent"}, {"start": 571.0, "text": "to build a small SwiftUI application from scratch."}, {"start": 575.0, "text": "I have started with a blank Xcode project"}, {"start": 577.0, "text": "and I am asking the agent to build a drawing app for the iPad."}, {"start": 583.0, "text": "And off it goes."}, {"start": 585.0, "text": "The agent first looks at the current directory"}, {"start": 586.0, "text": "to find out the existing project structure,"}, {"start": 589.0, "text": "makes a plan to guide its implementation,"}, {"start": 591.0, "text": "and gets on to writing the code."}, {"start": 593.0, "text": "Using an agent means we don't need to copy anything or even build the project."}, {"start": 597.0, "text": "The agent writes the file then builds the app,"}, {"start": 600.0, "text": "fixing any errors it encounters along the way."}, {"start": 605.0, "text": "And here we are: the model is done,"}, {"start": 607.0, "text": "it only took a couple of minutes to create the first version of the app."}, {"start": 611.0, "text": "At the same time, I have the project open in Xcode"}, {"start": 613.0, "text": "and I am launching the app in the simulator."}, {"start": 616.0, "text": "Let's have a look at what the agent created."}, {"start": 621.0, "text": "It seems that we have a fully functional drawing app."}, {"start": 624.0, "text": "That's really nice for something that was built in 2 minutes."}, {"start": 627.0, "text": "With agentic coding, however, we can keep iterating"}, {"start": 630.0, "text": "until we are happy with the result."}, {"start": 632.0, "text": "For instance, I prefer rounded end caps."}, {"start": 634.0, "text": "I think they look much better."}, {"start": 636.0, "text": "Let's ask the agent to add them."}, {"start": 649.0, "text": "The agent will edit the code and recompile the app"}, {"start": 652.0, "text": "until it compiles without errors."}, {"start": 658.0, "text": "Let's test the new version."}, {"start": 677.0, "text": "We now have rounded end caps."}, {"start": 679.0, "text": "This is cool indeed."}, {"start": 681.0, "text": "It is even more cool that all of this happened locally,"}, {"start": 684.0, "text": "the model ran through MLX-LM server on this Mac"}, {"start": 687.0, "text": "and the agent used standard development tools"}, {"start": 689.0, "text": "like xcodebuild to verify and build its work."}, {"start": 693.0, "text": "For our final demo,"}, {"start": 694.0, "text": "let's look at something that integrates directly"}, {"start": 696.0, "text": "with your development environment."}, {"start": 699.0, "text": "Here I have the same drawing app project open in Xcode."}, {"start": 703.0, "text": "Let's connect Xcode to our already running MLX server."}, {"start": 707.0, "text": "We open the settings and navigate to the Intelligence tab."}, {"start": 710.0, "text": "We click on Add Chat Provider... and select a Locally Hosted provider."}, {"start": 715.0, "text": "We set the Port to 8080 or whichever port we selected when launching our MLX server"}, {"start": 720.0, "text": "and we're done."}, {"start": 721.0, "text": "Now Xcode can talk to our local model."}, {"start": 725.0, "text": "I have introduced a bug to our previously working app"}, {"start": 727.0, "text": "and now we can ask the model to fix it."}, {"start": 744.0, "text": "Within seconds, it identifies the bug and inspects the code around it."}, {"start": 749.0, "text": "Finally, it writes a fix and we can now build and run our app."}, {"start": 767.0, "text": "This shows how a locally running agent"}, {"start": 769.0, "text": "can integrate with your existing development workflow in Xcode,"}, {"start": 773.0, "text": "reading project files, understanding build errors,"}, {"start": 775.0, "text": "and making targeted fixes."}, {"start": 777.0, "text": "Local AI means your code never leaves your Mac."}, {"start": 781.0, "text": "Today, we showed you the full stack for running agentic AI locally on your Mac,"}, {"start": 786.0, "text": "from MLX all the way up to the agent,"}, {"start": 788.0, "text": "and how Neural Accelerators, continuous batching,"}, {"start": 791.0, "text": "and distributed inference make it fast."}, {"start": 794.0, "text": "To get started, install MLX-LM, launch the server,"}, {"start": 797.0, "text": "and point your favorite agent at it."}, {"start": 799.0, "text": "Everything we showed today is open-source and available right now."}, {"start": 804.0, "text": "Thank you for watching and I'm excited to see what you build"}, {"start": 806.0, "text": "with local agentic AI on the Mac."}]}