{"$schema": "https://raw.githubusercontent.com/guitaripod/wwdc-sessions/master/schema/transcript.schema.json", "id": "wwdc2026-233", "language": "eng", "source": "https://developer.apple.com/videos/play/wwdc2026/233", "wordCount": 2973, "segments": [{"start": 7.0, "text": "Hi, I'm Tatiana, research scientist at MLX team."}, {"start": 11.0, "text": "It's been a remarkable time for local LLMs:"}, {"start": 14.0, "text": "models keep getting larger"}, {"start": 16.0, "text": "and gaining new amazing capabilities --"}, {"start": 18.0, "text": "becoming smarter and handling harder problems."}, {"start": 22.0, "text": "And as they improve, we use them for more: longer contexts, harder tasks,"}, {"start": 26.0, "text": "more complex workflows."}, {"start": 28.0, "text": "Eventually, memory, compute, or bandwidth on a single machine becomes a limitation."}, {"start": 35.0, "text": "In our video \"Run local agentic AI on the Mac using MLX\""}, {"start": 39.0, "text": "it is shown how to run AI agents locally."}, {"start": 42.0, "text": "But when you have multiple devices, you can take local AI even further,"}, {"start": 47.0, "text": "running larger LLMs or accelerating them"}, {"start": 50.0, "text": "through distributed inference and training."}, {"start": 53.0, "text": "Today, we'll take a deep dive into scaling across multiple Macs with MLX,"}, {"start": 59.0, "text": "using the hardware right on your desk."}, {"start": 62.0, "text": "We'll start with the command line interface to get models running"}, {"start": 65.0, "text": "on your machines,"}, {"start": 67.0, "text": "move to the Python API for experimentation,"}, {"start": 70.0, "text": "and finish with Swift for embedding these workflows directly into your apps."}, {"start": 75.0, "text": "Let's start!"}, {"start": 77.0, "text": "First, we'll look at the full hardware and software stacks"}, {"start": 80.0, "text": "to make distributed workloads on Apple Silicon possible."}, {"start": 84.0, "text": "Then we'll put everything together: turn four M3 Ultras into a cluster."}, {"start": 89.0, "text": "We'll walk through every step: choosing the right topology to connect machines,"}, {"start": 94.0, "text": "enabling fast communication, and launching distributed jobs."}, {"start": 98.0, "text": "Once the cluster is ready, we'll get to the exciting part:"}, {"start": 102.0, "text": "fast and local distributed LLM inference and finetuning."}, {"start": 106.0, "text": "We'll run it with MLX, compare it side by side against a single Mac,"}, {"start": 111.0, "text": "and look at how MLX distributes the model across the cluster."}, {"start": 116.0, "text": "Having most examples in command line interface,"}, {"start": 120.0, "text": "in the end we'll show how distributed communication"}, {"start": 123.0, "text": "is also exposed to you via Python, Swift and C++ APIs."}, {"start": 129.0, "text": "Let's start by looking at distributed communication for Apple Silicon."}, {"start": 134.0, "text": "To send and receive data fast,"}, {"start": 136.0, "text": "machines need to be connected with a physical link"}, {"start": 139.0, "text": "— an interconnect."}, {"start": 141.0, "text": "On top of that, we also need a transport protocol —"}, {"start": 144.0, "text": "a mechanism that pushes bytes"}, {"start": 146.0, "text": "from one machine's memory to another's."}, {"start": 149.0, "text": "Starting in macOS 26.2,"}, {"start": 152.0, "text": "Remote Direct Memory Access protocol, shortly RDMA,"}, {"start": 156.0, "text": "is supported over Thunderbolt 5."}, {"start": 158.0, "text": "RDMA moves data directly from one machine's memory to another's,"}, {"start": 163.0, "text": "avoiding most CPU and operating system overhead."}, {"start": 167.0, "text": "RDMA over Thunderbolt gives us the high-bandwidth —"}, {"start": 170.0, "text": "low-latency communication"}, {"start": 172.0, "text": "we need for distributed workloads."}, {"start": 174.0, "text": "However, alone, it gives us raw data movement between two machines only."}, {"start": 179.0, "text": "Thus, distributed programs need something higher-level —"}, {"start": 183.0, "text": "a communication backend"}, {"start": 185.0, "text": "which provides communication primitives for sending data"}, {"start": 188.0, "text": "between individual machines"}, {"start": 190.0, "text": "or coordinating across the entire group."}, {"start": 193.0, "text": "These two operations are building blocks of distributed training and inference."}, {"start": 197.0, "text": "And this is where JACCL comes in."}, {"start": 201.0, "text": "JACCL is an open-source collective communication library"}, {"start": 204.0, "text": "built by Apple."}, {"start": 206.0, "text": "It leverages RDMA over Thunderbolt"}, {"start": 208.0, "text": "and gives you collective communication primitives"}, {"start": 211.0, "text": "for sending data between machines"}, {"start": 213.0, "text": "and combining results across the group —"}, {"start": 215.0, "text": "without managing any of the low-level transport yourself."}, {"start": 219.0, "text": "And it's not limited to machine learning — any distributed workload on Apple Silicon"}, {"start": 224.0, "text": "can be built on top of it."}, {"start": 226.0, "text": "And the final piece of the stack is a machine learning framework"}, {"start": 230.0, "text": "that uses the communication backend for distributed inference and training —"}, {"start": 235.0, "text": "that's MLX."}, {"start": 237.0, "text": "MLX is an open-source machine learning library"}, {"start": 240.0, "text": "built by Apple for Apple Silicon."}, {"start": 242.0, "text": "It leverages JACCL for low-latency distributed communication"}, {"start": 246.0, "text": "and provides tools for orchestrating distributed jobs across the cluster."}, {"start": 251.0, "text": "If you're new to MLX, check out our video"}, {"start": 254.0, "text": "\"Getting Started with MLX on Apple Silicon\" from WWDC25."}, {"start": 260.0, "text": "So now we understand the full stack."}, {"start": 262.0, "text": "Let's put it all together and build a cluster —"}, {"start": 265.0, "text": "a group of machines that work together on the same task."}, {"start": 269.0, "text": "We will use 4 M3 Ultras."}, {"start": 272.0, "text": "To setup the cluster, we need to connect the machines with Thunderbolt 5 cables."}, {"start": 277.0, "text": "There are different ways to wire them together,"}, {"start": 280.0, "text": "and the topology directly affects the communication time."}, {"start": 284.0, "text": "So to begin with, we'll look at what defines that time."}, {"start": 288.0, "text": "Next, we'll look at how to actually connect the machines —"}, {"start": 291.0, "text": "which topologies JACCL supports, and the trade-offs between them."}, {"start": 296.0, "text": "After that, we'll show how to enable RDMA on the machines for fast communication."}, {"start": 301.0, "text": "And finally, we'll launch distributed jobs on the cluster using MLX."}, {"start": 308.0, "text": "So, communication time has two components:"}, {"start": 311.0, "text": "latency and transfer time."}, {"start": 313.0, "text": "Latency is the fixed cost paid for each communication operation,"}, {"start": 317.0, "text": "independent of the amount of data being sent."}, {"start": 322.0, "text": "Transfer time is the cost of moving the data though the link;"}, {"start": 326.0, "text": "it grows with message size"}, {"start": 328.0, "text": "and depends on the bandwidth of the link."}, {"start": 332.0, "text": "For small messages, the data movement cost is tiny,"}, {"start": 335.0, "text": "so latency dominates."}, {"start": 339.0, "text": "For large messages, the trade off is opposite."}, {"start": 342.0, "text": "Depending on whether communication is latency-bound or bandwidth-bound,"}, {"start": 346.0, "text": "we may prefer different topologies."}, {"start": 350.0, "text": "JACCL supports two of them: a mesh and a ring."}, {"start": 354.0, "text": "In a full mesh, every machine connects directly to every other,"}, {"start": 358.0, "text": "thus any group communication has the lowest possible latency."}, {"start": 362.0, "text": "In a ring, each node connects only to its two neighbors."}, {"start": 367.0, "text": "Communication between nonadjacent nodes must travel through intermediate machines"}, {"start": 372.0, "text": "which increases latency."}, {"start": 374.0, "text": "However, the ring requires fewer cables and ports per machine,"}, {"start": 378.0, "text": "making it easier to scale to more nodes."}, {"start": 381.0, "text": "And because each node has only two connections,"}, {"start": 384.0, "text": "we can use the extra Thunderbolt ports"}, {"start": 386.0, "text": "to run two or tree cables per neighbor (depending on the Mac)"}, {"start": 390.0, "text": "— thus increasing the bandwidth per link"}, {"start": 391.0, "text": "and reducing transfer time."}, {"start": 394.0, "text": "When machines are connected into a mesh,"}, {"start": 397.0, "text": "we have the flexibility to route each communication"}, {"start": 400.0, "text": "through either a mesh topology or a ring topology."}, {"start": 406.0, "text": "What's nice about JACCL, it automatically picks the best topology"}, {"start": 410.0, "text": "depending on the message size and communication operation —"}, {"start": 414.0, "text": "mesh when latency matters,"}, {"start": 417.0, "text": "ring when bandwidth matters."}, {"start": 419.0, "text": "For this flexibility, let's connect all M3 Ultras into a mesh."}, {"start": 426.0, "text": "As we connected all M3 Ultras together,"}, {"start": 428.0, "text": "now we need to enable RDMA on all machines."}, {"start": 431.0, "text": "Open settings on the machine, search for \"RDMA\","}, {"start": 439.0, "text": "click on \"Enable RDMA over Thunderbolt\","}, {"start": 444.0, "text": "enable RDMA, and reboot."}, {"start": 448.0, "text": "Great!"}, {"start": 449.0, "text": "Macs are connected with Thunderbolt 5 cables,"}, {"start": 452.0, "text": "and RDMA is enabled."}, {"start": 453.0, "text": "Now we need a way to launch distributed programs."}, {"start": 458.0, "text": "One way to do it, is over the local network,"}, {"start": 461.0, "text": "for example, through wifi or ethernet."}, {"start": 464.0, "text": "From any machine with SSH access to the cluster,"}, {"start": 467.0, "text": "for example MacBook in my case,"}, {"start": 470.0, "text": "we connect to each Mac, start the program,"}, {"start": 473.0, "text": "and from that point on,"}, {"start": 474.0, "text": "all machines communicate directly over the Thunderbolt links."}, {"start": 479.0, "text": "MLX provides a launch helper, which exactly does all of this for you!"}, {"start": 485.0, "text": "You run mlx.launch on your MacBook and it orchestrates the cluster."}, {"start": 491.0, "text": "You give it the executable you want to run"}, {"start": 494.0, "text": "and a JSON hostfile describing your cluster."}, {"start": 497.0, "text": "From there, it SSHes into each node using hostnames from provided hostfile"}, {"start": 503.0, "text": "and starts the executable on every machine."}, {"start": 506.0, "text": "Let's see how the hostfile that describes the cluster should look like."}, {"start": 511.0, "text": "It is a JSON array — one entry per node."}, {"start": 515.0, "text": "\"ssh\" is the hostname used by mlx.launch to reach the machine."}, {"start": 520.0, "text": "\"ips\" is the machine's IP on your local network"}, {"start": 523.0, "text": "used by JACCL for initial coordination between nodes."}, {"start": 527.0, "text": "And \"rdma\" is a list of the RDMA device names"}, {"start": 531.0, "text": "for each Thunderbolt peer connection."}, {"start": 534.0, "text": "You can write it manually, but MLX also provides"}, {"start": 536.0, "text": "a helper script `mlx.distributed_config` that generates it for you."}, {"start": 542.0, "text": "You pass the list of hostnames, and an output path."}, {"start": 546.0, "text": "You can also embed environment variables in the config."}, {"start": 550.0, "text": "They will be set automatically on every node at launch time."}, {"start": 554.0, "text": "Here we set MLX_METAL_FAST_SYNCH=1,"}, {"start": 558.0, "text": "which enables faster GPU-to-CPU synchronization."}, {"start": 562.0, "text": "It is critical for distributed tasks because computation runs"}, {"start": 566.0, "text": "on the GPU"}, {"start": 567.0, "text": "while communication runs on the CPU."}, {"start": 570.0, "text": "You can also pass the --auto-setup flag"}, {"start": 572.0, "text": "to configure the Thunderbolt network automatically."}, {"start": 576.0, "text": "Communication --backend argument defines whether it is a mesh or ring:"}, {"start": 581.0, "text": "for a mesh, --backend is set to jaccl, as in this example;"}, {"start": 585.0, "text": "for a ring, we would change it to jaccl-ring."}, {"start": 589.0, "text": "Let's run this command and generate the hostfile for our cluster."}, {"start": 594.0, "text": "First, it checks that all hosts are reachable over SSH."}, {"start": 599.0, "text": "Then it probes each machine's Thunderbolt ports"}, {"start": 602.0, "text": "to discover which machines are physically connected to which"}, {"start": 606.0, "text": "— building a map of the topology."}, {"start": 608.0, "text": "Since we passed --auto-setup, it disables the Thunderbolt Bridge"}, {"start": 612.0, "text": "on all machines"}, {"start": 613.0, "text": "and configures each Thunderbolt link for RDMA."}, {"start": 617.0, "text": "Finally, it writes a JSON hostfile with everything mlx.launch needs."}, {"start": 622.0, "text": "Note, that without --auto-setup flag, script prints the configuration commands,"}, {"start": 627.0, "text": "so you can review them and run yourself."}, {"start": 631.0, "text": "Now, the cluster is ready."}, {"start": 633.0, "text": "Let's move to the exciting part — distributed language model inference"}, {"start": 637.0, "text": "and finetuning."}, {"start": 639.0, "text": "And the easiest way to start is via command line interface"}, {"start": 643.0, "text": "and MLX LM."}, {"start": 645.0, "text": "MLX LM is an open-source Python package"}, {"start": 648.0, "text": "built on top of MLX"}, {"start": 649.0, "text": "that provides command-line tools"}, {"start": 652.0, "text": "and a Python API for running language models locally"}, {"start": 655.0, "text": "on Apple Silicon."}, {"start": 656.0, "text": "Check out our video, \"Explore large language models on Apple Silicon with MLX\""}, {"start": 662.0, "text": "from WWDC25 to get started on a single device."}, {"start": 667.0, "text": "As we showed last year, chatting with a model on a single Mac"}, {"start": 671.0, "text": "can be done via command line interface with mlx_lm.chat."}, {"start": 676.0, "text": "We run it in the terminal, specifying the model we want to use,"}, {"start": 679.0, "text": "for example, Qwen 3.6,"}, {"start": 682.0, "text": "and the maximum number of tokens for the response."}, {"start": 685.0, "text": "Under the hood, MLX LM loads and runs the model on a single machine."}, {"start": 691.0, "text": "To chat with the same model on the cluster via command line interface,"}, {"start": 696.0, "text": "we wrap the command with mlx.launch."}, {"start": 699.0, "text": "On our MacBook, in the terminal we run mlx.launch"}, {"start": 703.0, "text": "with the --hostfile pointing to our cluster configuration."}, {"start": 707.0, "text": "After the double dash, we pass the exact same mlx_lm.chat command —"}, {"start": 711.0, "text": "but using the remote path to the executable on each node."}, {"start": 716.0, "text": "The command is almost identical,"}, {"start": 718.0, "text": "MLX LM shards the model and coordinates the distributed inference for you."}, {"start": 723.0, "text": "Keep in mind that all necessary libraries like MLX must be installed on each Mac"}, {"start": 729.0, "text": "and the executable must be accessible on all machines."}, {"start": 734.0, "text": "One line via command line interface, and we're running a model"}, {"start": 738.0, "text": "spread across the entire cluster!"}, {"start": 741.0, "text": "Let's try both side by side and chat with Qwen 3.6 —"}, {"start": 745.0, "text": "a 27-billion-parameter model —"}, {"start": 747.0, "text": "on a single M3 Ultra and on 4 of them."}, {"start": 752.0, "text": "I've already started mlx_lm.chat on both sides —"}, {"start": 755.0, "text": "on the left, the model is loaded on a single M3 Ultra;"}, {"start": 759.0, "text": "on the right, it's sharded across four machines."}, {"start": 763.0, "text": "Let's prompt both with \"Implement a transformer model in MLX.\""}, {"start": 770.0, "text": "It is a quite impressive speed up!"}, {"start": 772.0, "text": "The cluster generates tokens at nearly three times the rate"}, {"start": 775.0, "text": "of a single machine"}, {"start": 777.0, "text": "for Qwen 3.6 model."}, {"start": 779.0, "text": "As we see, running a model across multiple Macs"}, {"start": 783.0, "text": "can significantly boost inference speed."}, {"start": 786.0, "text": "The exact speedup depends on the model size and architecture."}, {"start": 790.0, "text": "But time improvement is not the only reason to go distributed,"}, {"start": 794.0, "text": "sometimes a model is simply too large for one machine."}, {"start": 798.0, "text": "Kimi 2.6, for example, has 1 trillion total parameters."}, {"start": 803.0, "text": "Even with 8-bit quantization,"}, {"start": 806.0, "text": "the weights alone require about one terabyte of memory."}, {"start": 810.0, "text": "That does not fit on a single M3 Ultra, but it can fit across four."}, {"start": 815.0, "text": "So how do we actually split the weights and computation across machines?"}, {"start": 820.0, "text": "MLX and MLX LM support two approaches: pipeline and tensor parallelism."}, {"start": 827.0, "text": "Pipeline parallelism splits the model by depth."}, {"start": 830.0, "text": "In this case, each machine holds a group of layers,"}, {"start": 833.0, "text": "and data moves through the machines sequentially."}, {"start": 836.0, "text": "It does not speed up the inference, because each token still has to pass"}, {"start": 841.0, "text": "through the layer groups one after another."}, {"start": 844.0, "text": "But the benefit is simple communication: machines only exchange activations"}, {"start": 849.0, "text": "at the boundaries between layer groups."}, {"start": 853.0, "text": "Tensor parallelism splits the model by width."}, {"start": 855.0, "text": "In this case, each machine holds part of every layer,"}, {"start": 859.0, "text": "so all machines process the same token at the same time."}, {"start": 864.0, "text": "It improves inference speed due to parallelized per-layer computation."}, {"start": 868.0, "text": "However the trade-off is much more frequent communication,"}, {"start": 872.0, "text": "that happens at every layer and for every token."}, {"start": 876.0, "text": "This makes low latency important,"}, {"start": 878.0, "text": "and that is why the mesh topology is crucial for this case —"}, {"start": 882.0, "text": "every machine can reach every other machine in a single hop."}, {"start": 888.0, "text": "Tensor paralelism is the default sharding strategy in MLX LM."}, {"start": 892.0, "text": "To shard the model with pipeline parallelism,"}, {"start": 894.0, "text": "we can simply append a flag --pipeline to the command."}, {"start": 898.0, "text": "Note, that not all models support pipeline parallelism."}, {"start": 903.0, "text": "Now, let's chat with a one-trillion-parameter Kimi 2.6"}, {"start": 907.0, "text": "on our cluster."}, {"start": 909.0, "text": "For this we use mlx.launch from our MacBook as before,"}, {"start": 914.0, "text": "pointing to the hostfile."}, {"start": 916.0, "text": "I'm not passing the --pipeline flag, so we're using tensor parallelism."}, {"start": 920.0, "text": "We need to wait a moment — mlx.launch is connecting to every machine,"}, {"start": 924.0, "text": "MLX LM loads and shards the model,"}, {"start": 927.0, "text": "and starts the chat."}, {"start": 929.0, "text": "Great, the model is loaded!"}, {"start": 931.0, "text": "Let's prompt model with:"}, {"start": 933.0, "text": "\"Implement machine learning architecture for GPT in Python with MLX\"."}, {"start": 942.0, "text": "And there we go — with one command, a massive trillion-parameter model"}, {"start": 946.0, "text": "is running locally across your Macs, answering your questions."}, {"start": 954.0, "text": "With MLX and MLX LM, you can not only run language model inference,"}, {"start": 958.0, "text": "you can also fine-tune models on your hardware."}, {"start": 961.0, "text": "Fast, efficient, and fully private — your data never leaves your machines."}, {"start": 967.0, "text": "Let's start with a single Mac, and then scale to our cluster."}, {"start": 971.0, "text": "When fine-tuning or training on a single machine,"}, {"start": 974.0, "text": "we split the training data into batches —"}, {"start": 977.0, "text": "a set of multiple examples."}, {"start": 980.0, "text": "For each batch, the Mac computes gradients"}, {"start": 984.0, "text": "and updates the model weights."}, {"start": 986.0, "text": "We repeat this process for one or more passes over the training dataset,"}, {"start": 990.0, "text": "until the model reaches the desired quality."}, {"start": 993.0, "text": "The faster we process the training data,"}, {"start": 995.0, "text": "the sooner fine-tuning finishes."}, {"start": 997.0, "text": "So how can we use multiple machines to speed this up?"}, {"start": 1001.0, "text": "The idea is straightforward."}, {"start": 1003.0, "text": "We replicate the model on every Mac."}, {"start": 1006.0, "text": "Each machine receives a different batch of data and computes gradients locally."}, {"start": 1011.0, "text": "Then we average the gradients, so the model's update uses information"}, {"start": 1016.0, "text": "from all batches."}, {"start": 1017.0, "text": "This is called data-parallel training because the model is replicated,"}, {"start": 1021.0, "text": "while the data is processed in parallel across machines —"}, {"start": 1025.0, "text": "this is what gives us the speedup."}, {"start": 1027.0, "text": "So with N machines we can process data up to N times faster."}, {"start": 1032.0, "text": "Sounds amazing!"}, {"start": 1033.0, "text": "Lets see how we can use data parallelism with MLX LM."}, {"start": 1038.0, "text": "As before, the only difference from a single device"}, {"start": 1041.0, "text": "is launching the job with mlx.launch"}, {"start": 1043.0, "text": "from your MacBook,"}, {"start": 1044.0, "text": "specifying a path to mlx_lm.lora on remote machines."}, {"start": 1049.0, "text": "Data sharding is handled by MLX LM and the command is almost identical —"}, {"start": 1054.0, "text": "we scale --batch-size by the number of devices"}, {"start": 1057.0, "text": "so each machine still processes"}, {"start": 1059.0, "text": "the same number of samples per step as before."}, {"start": 1063.0, "text": "Let's fine-tune Qwen 3.5 with 9 billion parameters"}, {"start": 1067.0, "text": "on a single machine and on the cluster,"}, {"start": 1070.0, "text": "and compare the number of tokens the model processes per second."}, {"start": 1075.0, "text": "We are launching fine-tuning on a single device on the left"}, {"start": 1078.0, "text": "and on the cluster on the right"}, {"start": 1080.0, "text": "using mlx.launch and hostfile,"}, {"start": 1083.0, "text": "specifying path to mlx_lm.lora on the remote machine."}, {"start": 1087.0, "text": "First, it loads data and model; and then training starts."}, {"start": 1090.0, "text": "Single M3 Ultra is processing around 180 tokens per second,"}, {"start": 1096.0, "text": "while on the cluster we process around 600 tokens per second,"}, {"start": 1101.0, "text": "which gives us more than 3 times speed up for fine-tuning."}, {"start": 1105.0, "text": "Now, with MLX, you can turn your devices into a local training cluster"}, {"start": 1110.0, "text": "for efficient fine-tuning without moving to a cloud."}, {"start": 1114.0, "text": "So far, we used command line interface for distributed inference"}, {"start": 1118.0, "text": "and fine-tuning within MLX LM."}, {"start": 1121.0, "text": "However, MLX provides a fine-grained control"}, {"start": 1124.0, "text": "over sharding and distributed operations,"}, {"start": 1127.0, "text": "via flexible Python, Swift, and C++ APIs."}, {"start": 1131.0, "text": "This allows you to experiment with models in Python and C++"}, {"start": 1135.0, "text": "or embed models into your App with Swift."}, {"start": 1138.0, "text": "Let's look at the examples."}, {"start": 1141.0, "text": "To run distributed inference with Python API and MLX LM,"}, {"start": 1145.0, "text": "we first initialize the distributed group for communication."}, {"start": 1148.0, "text": "Then, define the type of parallelism we want,"}, {"start": 1152.0, "text": "for example, tensor parallelism."}, {"start": 1154.0, "text": "Finally, we shard the model using the sharded_load function."}, {"start": 1158.0, "text": "After that, we use the model exactly as we would on a single device —"}, {"start": 1163.0, "text": "MLX LM handles all distributed communications under the hood."}, {"start": 1168.0, "text": "To have more control over the model and its sharding,"}, {"start": 1171.0, "text": "we can use low-level primitives from MLX itself."}, {"start": 1175.0, "text": "For example, after defining a simple Linear layer,"}, {"start": 1178.0, "text": "we can shard it with tensor parallelism using shard_linear function."}, {"start": 1183.0, "text": "You can even control basic distributed operations like all reduce."}, {"start": 1188.0, "text": "In Python, Swift or C++ after initializing the distributed group via JACCL,"}, {"start": 1193.0, "text": "we perform a collective distributed sum across all Macs for our tensor"}, {"start": 1198.0, "text": "using corresponding MLX primitives."}, {"start": 1201.0, "text": "As we pointed out at the beginning of the session,"}, {"start": 1204.0, "text": "JACCL is available on its own"}, {"start": 1206.0, "text": "and you can leverage it for any applications"}, {"start": 1209.0, "text": "requiring distributed communication,"}, {"start": 1212.0, "text": "even non-ML applications."}, {"start": 1214.0, "text": "JACCL can be built without MLX and it provides a C++ API"}, {"start": 1219.0, "text": "with communication primitives:"}, {"start": 1221.0, "text": "after initializing a JACCL group,"}, {"start": 1223.0, "text": "we again perform a collective distributed sum across all Macs"}, {"start": 1227.0, "text": "for our tensor but via JACCL directly, not MLX."}, {"start": 1233.0, "text": "Now you know both high-level and low-level APIs,"}, {"start": 1236.0, "text": "for distributed inference and training with MLX and JACCL,"}, {"start": 1239.0, "text": "and you are ready to build advanced distributed workflows with MLX."}, {"start": 1245.0, "text": "Throughout this session, we looked at the full stack"}, {"start": 1247.0, "text": "that makes distributed training and inference"}, {"start": 1250.0, "text": "possible on Apple Silicon —"}, {"start": 1252.0, "text": "from RDMA over Thunderbolt,"}, {"start": 1255.0, "text": "all the way up to MLX and MLX LM."}, {"start": 1258.0, "text": "We showed you how easy it is to scale from a single device to multiple devices,"}, {"start": 1263.0, "text": "and the benefits it brings:"}, {"start": 1265.0, "text": "faster inference, the ability to run trillion-parameter models,"}, {"start": 1269.0, "text": "and faster fine-tuning;"}, {"start": 1271.0, "text": "all with minimal changes to your single device code,"}, {"start": 1274.0, "text": "supporting command line interface, Python, Swift and C++ APIs."}, {"start": 1280.0, "text": "With distributed cluster, now you can run local AI agents powered entirely by MLX —"}, {"start": 1286.0, "text": "fast, private, and on the hardware you own."}, {"start": 1291.0, "text": "To know more, check out our WWDC 2026 video"}, {"start": 1294.0, "text": "\"Run local agentic AI on the Mac using MLX\"."}, {"start": 1298.0, "text": "To further dive into advanced distributed features —"}, {"start": 1302.0, "text": "including custom parallelism strategies and training loops,"}, {"start": 1305.0, "text": "check out our documentation."}, {"start": 1307.0, "text": "You can also use MLX LM to serve models distributedly with the built-in server."}, {"start": 1314.0, "text": "We can't wait to see what you build with MLX on Apple Silicon!"}]}