{"$schema": "https://raw.githubusercontent.com/guitaripod/wwdc-sessions/master/schema/transcript.schema.json", "id": "wwdc2026-324", "language": "eng", "source": "https://developer.apple.com/videos/play/wwdc2026/324", "wordCount": 3348, "segments": [{"start": 7.0, "text": "Hi everyone, my name is Ben and I'm an engineer on the Core AI team."}, {"start": 12.0, "text": "Today I'll be giving an introduction to Core AI,"}, {"start": 15.0, "text": "and showing how you can use it to add intelligent features into your apps."}, {"start": 20.0, "text": "AI is advancing faster than ever."}, {"start": 22.0, "text": "New models and capabilities that previously seemed out of reach"}, {"start": 26.0, "text": "are emerging constantly."}, {"start": 28.0, "text": "Core AI is built to help you harness that momentum and build on top of it."}, {"start": 33.0, "text": "Core AI marks the next evolution of on-device AI execution"}, {"start": 37.0, "text": "across Apple platforms."}, {"start": 39.0, "text": "It's built from the ground up for modern workloads,"}, {"start": 41.0, "text": "and delivers the high-performance inference you need"}, {"start": 44.0, "text": "to build advanced AI features."}, {"start": 47.0, "text": "Core AI is the inference framework powering on-device Apple Intelligence."}, {"start": 51.0, "text": "And now, it's available for you to use,"}, {"start": 54.0, "text": "bringing that same power to your app's own intelligence."}, {"start": 57.0, "text": "Core AI is more than just a framework."}, {"start": 59.0, "text": "It's a complete set of technologies, covering the model deployment lifecycle,"}, {"start": 64.0, "text": "from model optimization and conversion"}, {"start": 66.0, "text": "to debugging and integration into your app."}, {"start": 69.0, "text": "All designed to support the fast, iterative cycle"}, {"start": 72.0, "text": "that building great AI features requires."}, {"start": 75.0, "text": "Core AI allows you to leverage all of Apple Silicon."}, {"start": 78.0, "text": "It provides blazing fast inference across the CPU, GPU, and Neural Engine."}, {"start": 84.0, "text": "The framework comes with a modern Swift API."}, {"start": 87.0, "text": "It's an expressive API that delivers the performance your app demands"}, {"start": 91.0, "text": "without compromising on memory safety."}, {"start": 94.0, "text": "The broader set of technologies fit naturally"}, {"start": 97.0, "text": "into common ML engineering workflows,"}, {"start": 99.0, "text": "reusing familiar Python and PyTorch foundations"}, {"start": 103.0, "text": "for model authoring, optimization and conversion."}, {"start": 107.0, "text": "Core AI also supports extensive customization"}, {"start": 110.0, "text": "from fine-grained inference management and model specialization"}, {"start": 113.0, "text": "to custom GPU kernels."}, {"start": 116.0, "text": "And all of this is tightly integrated into a new developer toolchain,"}, {"start": 120.0, "text": "with ahead-of-time compilation, dedicated Core AI Instruments,"}, {"start": 124.0, "text": "and a powerful visual Debugger"}, {"start": 126.0, "text": "to trace tensor values directly back to your original Python source code."}, {"start": 130.0, "text": "Core AI is designed to scale to your needs and available compute."}, {"start": 134.0, "text": "Whether you want your app to identify who's talking in a live meeting"}, {"start": 139.0, "text": "with a small speaker diarization model,"}, {"start": 141.0, "text": "your users to point their camera at anything, ask a question,"}, {"start": 145.0, "text": "and instantly get an answer with a larger vision language model,"}, {"start": 148.0, "text": "or let them hand off complex, multi-step tasks"}, {"start": 151.0, "text": "to a powerful agentic assistant powered by a 70 billion parameter LLM."}, {"start": 156.0, "text": "Core AI has you covered."}, {"start": 158.0, "text": "With all of it running locally on Apple devices,"}, {"start": 161.0, "text": "with no server and no cost per token."}, {"start": 164.0, "text": "In this talk, I'll start by showing how to get your model into the Core AI format."}, {"start": 169.0, "text": "Then I'll go over how to integrate the converted model into your app."}, {"start": 174.0, "text": "I'll then dive a little deeper"}, {"start": 175.0, "text": "into optimizing the performance of your model and app."}, {"start": 179.0, "text": "And lastly I'll highlight some additional features of Core AI"}, {"start": 182.0, "text": "and its associated tools that you may find useful."}, {"start": 185.0, "text": "Let's get started."}, {"start": 187.0, "text": "Every great app experience starts with an idea."}, {"start": 190.0, "text": "Maybe you want to build something that feels a little magical,"}, {"start": 193.0, "text": "something that responds intelligently,"}, {"start": 195.0, "text": "or makes a decision that would otherwise require a human or hard coded rules."}, {"start": 200.0, "text": "Machine learning and AI are what make those kinds of experiences possible."}, {"start": 205.0, "text": "Once you have that idea,"}, {"start": 206.0, "text": "the next step is finding or building a model that can power it."}, {"start": 210.0, "text": "Just like your idea itself will evolve over time,"}, {"start": 213.0, "text": "finding the right model is an iterative process."}, {"start": 217.0, "text": "You'll try things, evaluate them against your requirements and refine."}, {"start": 221.0, "text": "Core AI is designed to support that iteration"}, {"start": 224.0, "text": "and make it as fast and frictionless as possible."}, {"start": 227.0, "text": "So to make this concrete, I'll implement a fun game idea I had."}, {"start": 231.0, "text": "It's an app that lets you play a two player snake game"}, {"start": 234.0, "text": "where one snake is powered by an AI model run through Core AI."}, {"start": 238.0, "text": "The app will follow traditional snake rules"}, {"start": 241.0, "text": "where snakes can grow by eating food"}, {"start": 243.0, "text": "and must avoid hitting walls, themselves and the other snake."}, {"start": 247.0, "text": "The last snake standing wins."}, {"start": 250.0, "text": "At each time step, the AI model will see a set of features"}, {"start": 253.0, "text": "describing the current board state, and those features will be accumulated"}, {"start": 257.0, "text": "into the full game history that gets fed to the model."}, {"start": 260.0, "text": "It will then predict the best direction to move."}, {"start": 263.0, "text": "While snake is a simple game,"}, {"start": 265.0, "text": "the tools and APIs used to create this experience are the same foundation"}, {"start": 269.0, "text": "that scale all the way up to the larger, more complex use cases."}, {"start": 274.0, "text": "I was curious to see"}, {"start": 275.0, "text": "what I could put together with PyTorch for this project."}, {"start": 278.0, "text": "With a little help from an AI coding assistant"}, {"start": 280.0, "text": "I was able to sketch out a simple snake action prediction model pretty quickly."}, {"start": 285.0, "text": "To train it, I used a naive simulation to generate training data,"}, {"start": 289.0, "text": "just running the game and recording states and actions."}, {"start": 292.0, "text": "The idea was to start simple and get the model working in my app."}, {"start": 297.0, "text": "So the next step is taking this PyTorch model"}, {"start": 299.0, "text": "and converting it to Core AI."}, {"start": 302.0, "text": "I'll use the new Core AI Torch Python package to easily perform the conversion."}, {"start": 307.0, "text": "First I'll load the trained checkpoint of the SnakeTransformer module,"}, {"start": 311.0, "text": "and prepare a sample input."}, {"start": 313.0, "text": "Then I'll export the torch program using torch.export"}, {"start": 317.0, "text": "and also make sure to use the dynamic_shapes argument"}, {"start": 320.0, "text": "to specify that the sequence length of the features is dynamic,"}, {"start": 324.0, "text": "that way it doesn't get traced with the static sample length of 5."}, {"start": 328.0, "text": "Also I'll run decompositions on the converted program"}, {"start": 331.0, "text": "using Core AI's decomposition table."}, {"start": 334.0, "text": "Next I'll run Core AI's TorchConverter,"}, {"start": 337.0, "text": "specify the names of the inputs and outputs,"}, {"start": 340.0, "text": "and finally save the converted Core AI model to disk."}, {"start": 344.0, "text": "Before leaving the Python environment, one more thing I'll do"}, {"start": 347.0, "text": "is run a test to verify that the converted Core AI model"}, {"start": 350.0, "text": "matches the numerics of my original PyTorch model."}, {"start": 354.0, "text": "This can be done easily with the Core AI framework Python bindings."}, {"start": 358.0, "text": "First I'll load the PyTorch and Core AI models."}, {"start": 361.0, "text": "Then prepare a sample snake game input."}, {"start": 364.0, "text": "Then run that same input through both the PyTorch module"}, {"start": 367.0, "text": "and the Core AI inference function."}, {"start": 370.0, "text": "And finally assert a sufficiently small delta for my use case"}, {"start": 373.0, "text": "between the PyTorch and Core AI outputs."}, {"start": 376.0, "text": "Now that I have the converted AI model,"}, {"start": 379.0, "text": "the next step is to hop into Xcode and integrate the model into my app."}, {"start": 383.0, "text": "First I'll open the AI model file with Xcode,"}, {"start": 386.0, "text": "which shows information about the model."}, {"start": 389.0, "text": "It includes the model size, the distribution of operations"}, {"start": 394.0, "text": "and other helpful metadata."}, {"start": 396.0, "text": "Also in the Functions tab"}, {"start": 398.0, "text": "it shows you the exact function signature of each unique function in the model."}, {"start": 403.0, "text": "In this case the model just has one function,"}, {"start": 406.0, "text": "which takes the features of the game board as an input"}, {"start": 409.0, "text": "and produces logits as an output"}, {"start": 410.0, "text": "which indicate which direction the model thinks would be best to move."}, {"start": 415.0, "text": "Also note that the question mark in the NDArray values"}, {"start": 418.0, "text": "denotes that the dimension has a dynamic shape,"}, {"start": 421.0, "text": "which matches how I converted the model with a dynamic sequence length."}, {"start": 425.0, "text": "Now that I've included the AI model file in my Xcode project"}, {"start": 428.0, "text": "and have examined its structure,"}, {"start": 430.0, "text": "the next step is to use the Core AI framework to run the model."}, {"start": 435.0, "text": "The Core AI framework is a new Swift API surface"}, {"start": 437.0, "text": "for loading and running Core AI models."}, {"start": 440.0, "text": "It offers a progressively disclosing set of APIs,"}, {"start": 443.0, "text": "which makes it simple to get things up and running,"}, {"start": 446.0, "text": "while also having deeper layers of flexibility"}, {"start": 449.0, "text": "for supporting performance critical applications."}, {"start": 452.0, "text": "Also, it uses modern Swift language features like non-escapable types,"}, {"start": 456.0, "text": "to offer memory-safe APIs while not sacrificing performance."}, {"start": 461.0, "text": "Let's begin by discussing the core types within the framework."}, {"start": 465.0, "text": "An AIModel is initialized from a URL to a .aimodel file"}, {"start": 469.0, "text": "and is used primarily to inspect and load one or more inference functions."}, {"start": 474.0, "text": "An InferenceFunction is the runnable object"}, {"start": 477.0, "text": "which represents a single loaded compute graph."}, {"start": 480.0, "text": "In the common case, your AIModel will only have a single main InferenceFunction,"}, {"start": 485.0, "text": "though you can convert a single model with multiple functions."}, {"start": 488.0, "text": "The AIModel and InferenceFunction are typically objects"}, {"start": 492.0, "text": "you'll construct when preparing your app's AI feature."}, {"start": 495.0, "text": "For example this could be on app initialization."}, {"start": 498.0, "text": "NDArray is the type which holds your multi-dimensional input and output data"}, {"start": 503.0, "text": "and you use the run method on an InferenceFunction"}, {"start": 506.0, "text": "to run inference with that data."}, {"start": 508.0, "text": "Finally you can read and process the outputs of the inference."}, {"start": 512.0, "text": "So for implementing the snake game, I'll start by making the ModelPlayer type."}, {"start": 517.0, "text": "At app initialization time,"}, {"start": 519.0, "text": "it'll be initialized with the URL to the AI model file that it should use."}, {"start": 523.0, "text": "Then it will initialize the AIModel,"}, {"start": 526.0, "text": "and load the main inference function from it."}, {"start": 529.0, "text": "Next is the logic for the model player to make decisions."}, {"start": 532.0, "text": "It'll conform to the SnakePlayer protocol that I've defined in my app."}, {"start": 536.0, "text": "The main protocol requirement is the chooseAction function"}, {"start": 540.0, "text": "which is passed in the game's history,"}, {"start": 542.0, "text": "and returns the next action that the snake should take."}, {"start": 545.0, "text": "The first thing to do is create an NDArray to populate with the input features."}, {"start": 550.0, "text": "For this inference function, the expected structure of the NDArray"}, {"start": 554.0, "text": "is 2 dimensional with float32 data,"}, {"start": 557.0, "text": "where the first dimension of the shape is the current sequence length,"}, {"start": 561.0, "text": "and the second is the fixed hidden dimension size."}, {"start": 564.0, "text": "Then it'll write the features into that NDArray"}, {"start": 567.0, "text": "using this writeFeatures helper function"}, {"start": 569.0, "text": "which takes the game and a mutable view of the NDArray."}, {"start": 573.0, "text": "The NDArray.MutableView type is a non-escapable type"}, {"start": 577.0, "text": "which provides safe and efficient access to the backing storage of the NDArray."}, {"start": 582.0, "text": "After preparing the inputs, it'll run inference with them,"}, {"start": 585.0, "text": "and extract the expected output logits ndarray."}, {"start": 589.0, "text": "The last step is to sample the output logits"}, {"start": 592.0, "text": "to pick the next direction that the snake will move,"}, {"start": 594.0, "text": "by passing an ndarray view into the helper function"}, {"start": 598.0, "text": "which will read the values"}, {"start": 599.0, "text": "and choose the direction with the largest corresponding logit."}, {"start": 602.0, "text": "The writeFeatures function is what's populating the input features."}, {"start": 606.0, "text": "Let's briefly go over what these features include."}, {"start": 610.0, "text": "They have the normalized distance of the AI snake's head to all the walls."}, {"start": 615.0, "text": "The normalized relative X and Y distance to the nearest food."}, {"start": 620.0, "text": "Four elements encoding it's current direction."}, {"start": 623.0, "text": "The normalized distance to the other snake."}, {"start": 626.0, "text": "And finally the opponent's direction."}, {"start": 629.0, "text": "Now with this put together I'm going to try a test run"}, {"start": 632.0, "text": "with both snakes powered by the AI model to see how it does."}, {"start": 640.0, "text": "Running it shows that the model is working."}, {"start": 642.0, "text": "However, I see that the game is getting slower as it goes on."}, {"start": 647.0, "text": "Alongside the Core AI framework, there's a new instrument in Xcode"}, {"start": 650.0, "text": "to help you profile the Core AI models running in your app."}, {"start": 654.0, "text": "In this case I've ran the app with Instruments"}, {"start": 656.0, "text": "and I can see the inference intervals getting notably larger over time,"}, {"start": 661.0, "text": "which means the inference calls are increasing in latency."}, {"start": 664.0, "text": "This makes sense because transformer models have quadratic time complexity"}, {"start": 668.0, "text": "with respect to the sequence length."}, {"start": 671.0, "text": "And in our game the sequence length is increasing"}, {"start": 673.0, "text": "with every move the model makes."}, {"start": 675.0, "text": "The next step in this case is to optimize the performance of the model usage."}, {"start": 680.0, "text": "Each time the input sequence is increased,"}, {"start": 682.0, "text": "the transformer model recomputes a set of internal key"}, {"start": 685.0, "text": "and value embeddings for every element in the sequence."}, {"start": 689.0, "text": "A common strategy used to improve the performance"}, {"start": 692.0, "text": "of decoding loops like this when using transformers is to cache keys and values"}, {"start": 697.0, "text": "that are computed for each element in the sequence,"}, {"start": 700.0, "text": "as opposed to re-computing them all from scratch with each inference."}, {"start": 704.0, "text": "This can be achieved through Core AI by using states."}, {"start": 708.0, "text": "States are inputs to the model which are both read,"}, {"start": 710.0, "text": "and updated in-place during inference."}, {"start": 713.0, "text": "By introducing the key and value caches as states on the model,"}, {"start": 717.0, "text": "we both avoid recomputing them on each inference,"}, {"start": 720.0, "text": "and also remove the need to provide the full history of the game as an input"}, {"start": 724.0, "text": "since the data needed from older steps are stored in the states."}, {"start": 729.0, "text": "So after the first input,"}, {"start": 731.0, "text": "each subsequent step uses the cache for history"}, {"start": 734.0, "text": "and only takes the new features of the latest board state."}, {"start": 737.0, "text": "To implement the key/value caching,"}, {"start": 739.0, "text": "I'll go back to the original authoring code"}, {"start": 742.0, "text": "and make a few changes to add in the key and value caches."}, {"start": 745.0, "text": "First I'll update the torch module by adding key and value cache tensors"}, {"start": 750.0, "text": "as buffers within the transformer module, by using the torch register_buffer API."}, {"start": 756.0, "text": "This will later result in these tensors"}, {"start": 758.0, "text": "being mutable buffers in the exported torch program"}, {"start": 761.0, "text": "which Core AI will convert to states."}, {"start": 763.0, "text": "Then in the forward function of the module,"}, {"start": 766.0, "text": "I'll add the logic to actually use the caches."}, {"start": 769.0, "text": "This involves reading previous features keys and values out of the cache."}, {"start": 773.0, "text": "Then writing the computed keys and values for the new features back into the cache."}, {"start": 779.0, "text": "Lastly, I'll rerun the same code from before to re-convert the model,"}, {"start": 783.0, "text": "but now adding in the state_names argument to the convert call"}, {"start": 787.0, "text": "to specify the names of the new state arguments."}, {"start": 790.0, "text": "Now that I've re-converted the model with the new function signature,"}, {"start": 794.0, "text": "I'll update the app code to handle it."}, {"start": 796.0, "text": "To start, I'll update the ModelPlayer to store the key and value cache NDArrays"}, {"start": 801.0, "text": "which will be the state arguments passed to each inference."}, {"start": 805.0, "text": "I'll initialize them with the expected shape for the transformer."}, {"start": 808.0, "text": "In this case I converted the model such that it expects the key and value caches"}, {"start": 814.0, "text": "to always be a fixed size for a maximum possible context length."}, {"start": 818.0, "text": "Then when it's time to run inference,"}, {"start": 820.0, "text": "I'll construct a collection of MutableViews"}, {"start": 823.0, "text": "containing both views of the key and value caches."}, {"start": 827.0, "text": "Then provide those as the states argument of the InferenceFunction.run method."}, {"start": 831.0, "text": "Now the caches will be both read and updated in-place during each inference."}, {"start": 837.0, "text": "Now with the updated model, I'll re-run the app."}, {"start": 840.0, "text": "This time I can see it maintains a steady speed,"}, {"start": 843.0, "text": "no longer slowing down overtime."}, {"start": 846.0, "text": "When tracing the updated app in Instruments,"}, {"start": 848.0, "text": "I can confirm that the inference latency is growing at a much slower rate."}, {"start": 853.0, "text": "Before wrapping up,"}, {"start": 854.0, "text": "I'll show some features that I didn't use while making the snake game,"}, {"start": 857.0, "text": "but that you may find useful when developing your own apps."}, {"start": 861.0, "text": "When converting the snake game models,"}, {"start": 863.0, "text": "I used the coreai-torch package to directly convert the PyTorch module."}, {"start": 868.0, "text": "This flow is simple and works great for many use cases,"}, {"start": 871.0, "text": "but sometimes you may need more control over how your model is authored,"}, {"start": 875.0, "text": "and potentially even how the operations within the model are run."}, {"start": 879.0, "text": "We've only touched the surface of what the Core AI Python package has to offer."}, {"start": 883.0, "text": "It also has support for directly authoring your model with Core AI APIs,"}, {"start": 888.0, "text": "optimizing the model for Apple Silicon,"}, {"start": 890.0, "text": "and defining custom kernel implementations with Metal 4."}, {"start": 894.0, "text": "To learn more about these advanced model authoring flows,"}, {"start": 897.0, "text": "see the talk \"Dive into Core AI model authoring and optimization\"."}, {"start": 901.0, "text": "In addition to debugging performance,"}, {"start": 904.0, "text": "it's also crucial to be able to debug the numerics of your converted model."}, {"start": 908.0, "text": "For this you can use the Core AI Debugger"}, {"start": 911.0, "text": "which allows you to visualize your converted model,"}, {"start": 914.0, "text": "easily inspect intermediate tensor values,"}, {"start": 917.0, "text": "and trace back operations in the converted model"}, {"start": 920.0, "text": "to the Python source code which introduced them."}, {"start": 923.0, "text": "There is also a convenient Core AI debug gauge"}, {"start": 926.0, "text": "which shows you streaming Core AI activity while your app is running in Xcode."}, {"start": 930.0, "text": "This is a great place to spot performance issues"}, {"start": 933.0, "text": "before jumping into instruments."}, {"start": 935.0, "text": "One thing that was glossed over in the snake game implementation"}, {"start": 938.0, "text": "is the process of model specialization."}, {"start": 941.0, "text": "When you ship an AI model with your app,"}, {"start": 943.0, "text": "that is a source representation of the model,"}, {"start": 945.0, "text": "which can be run on any Apple device."}, {"start": 948.0, "text": "However, to actually load and run the model within your app,"}, {"start": 952.0, "text": "it must be specialized for the device that the app is running on."}, {"start": 956.0, "text": "When your model is loaded it is checked to see"}, {"start": 958.0, "text": "if it has already been specialized and cached."}, {"start": 961.0, "text": "The specialization process"}, {"start": 963.0, "text": "can take a significant amount of time for very large models."}, {"start": 967.0, "text": "While future loads are from the cache and fast,"}, {"start": 970.0, "text": "that first time is something you may need to plan for."}, {"start": 973.0, "text": "It is recommended you avoid having model specialization occur"}, {"start": 977.0, "text": "within user interactive flows."}, {"start": 979.0, "text": "Core AI can help you with that."}, {"start": 981.0, "text": "First, Core AI gives you programmatic access"}, {"start": 984.0, "text": "to the default model cache for your app."}, {"start": 987.0, "text": "You can request to load models directly from it."}, {"start": 990.0, "text": "If nil is returned, it is not present and requires specialization."}, {"start": 994.0, "text": "You can use this to gate features or inform the users"}, {"start": 997.0, "text": "that they may need to wait a bit while your app prepares the model."}, {"start": 1002.0, "text": "Second, you can request model specialization explicitly in your app"}, {"start": 1006.0, "text": "independent of it being loaded."}, {"start": 1008.0, "text": "You can do this after downloading assets or when the user opts in to a feature"}, {"start": 1013.0, "text": "so the model is ready to go ahead of time."}, {"start": 1015.0, "text": "And there is a lot more control available."}, {"start": 1018.0, "text": "SpecializationOptions help configure how you want your model"}, {"start": 1021.0, "text": "to be optimized for inference."}, {"start": 1023.0, "text": "With the AIModelCache you can also delete entries you no longer need,"}, {"start": 1027.0, "text": "and control the policy on how long entries persist."}, {"start": 1031.0, "text": "You can even share a cache between multiple apps in the same app group."}, {"start": 1036.0, "text": "Check out the \"Managing model specialization and caching\" article"}, {"start": 1039.0, "text": "on developer.apple.com to learn more."}, {"start": 1044.0, "text": "Independent of when specialization occurs, it still takes time."}, {"start": 1048.0, "text": "Lets take a quick peak inside."}, {"start": 1051.0, "text": "During specialization,"}, {"start": 1053.0, "text": "the model goes through two main transformations."}, {"start": 1056.0, "text": "First, it goes through a core set of compilation steps"}, {"start": 1059.0, "text": "which segment, plan and optimize compute."}, {"start": 1062.0, "text": "Second, executable artifacts are generated for the compute units used."}, {"start": 1067.0, "text": "These artifacts are tied to the device and OS version they were generated on."}, {"start": 1072.0, "text": "Of these two steps, compilation is the one which incurs most of the latency."}, {"start": 1077.0, "text": "The Core AI toolchain can help you reduce that time"}, {"start": 1080.0, "text": "by allowing some compilation to occur ahead of time on your development machine,"}, {"start": 1085.0, "text": "producing a compiled version of the model."}, {"start": 1088.0, "text": "While that compiled model still needs to be specialized for the specific users device,"}, {"start": 1093.0, "text": "there is now much less work to do and finishes significantly faster."}, {"start": 1097.0, "text": "To learn more about this option,"}, {"start": 1099.0, "text": "check out the \"Compiling Core AI models ahead of time\" article"}, {"start": 1103.0, "text": "on developer.apple.com."}, {"start": 1105.0, "text": "Controlling when, where, and how specialization happens"}, {"start": 1109.0, "text": "is one way to help you optimize your users experience."}, {"start": 1113.0, "text": "Another area you may want to optimize"}, {"start": 1115.0, "text": "is removing any overheads in tight inference loops using your model."}, {"start": 1119.0, "text": "The Core AI Framework has several APIs to help you here."}, {"start": 1123.0, "text": "You can dynamically check"}, {"start": 1124.0, "text": "the optimal memory layout of NDArray arguments"}, {"start": 1128.0, "text": "and allocate them with that structure"}, {"start": 1130.0, "text": "to avoid layout conversions at inference time."}, {"start": 1133.0, "text": "You can also pre-allocate output values for the framework to write into,"}, {"start": 1137.0, "text": "to avoid allocating new output values during inference."}, {"start": 1141.0, "text": "And you can also use asynchronous values to efficiently pipeline execution"}, {"start": 1146.0, "text": "of multiple inference functions together."}, {"start": 1148.0, "text": "For most use cases, the higher-level inference APIs"}, {"start": 1152.0, "text": "will get you exactly where you need to be."}, {"start": 1154.0, "text": "But when you're optimizing a tight inference loop"}, {"start": 1157.0, "text": "or integrating a model into a complex compute pipeline,"}, {"start": 1160.0, "text": "these lower-level APIs are there when you need them."}, {"start": 1164.0, "text": "Whether you're just getting started or diving deep,"}, {"start": 1166.0, "text": "the Core AI Models repository is a great place to find what you need."}, {"start": 1171.0, "text": "It has a collection of popular models, each just a single command away"}, {"start": 1175.0, "text": "from being converted and optimized for your app."}, {"start": 1178.0, "text": "AI skills that are experts in Core AI model authoring,"}, {"start": 1182.0, "text": "optimization, and conversion."}, {"start": 1185.0, "text": "And a Swift package with libraries for specific families of models"}, {"start": 1189.0, "text": "that give you higher-level APIs that already have many of those"}, {"start": 1193.0, "text": "low-level inference optimizations built in."}, {"start": 1196.0, "text": "It also provides an API for creating a Core AI Language model,"}, {"start": 1200.0, "text": "which plugs right in to the Foundation Models framework,"}, {"start": 1203.0, "text": "letting you bring your own custom models and token sampling strategies."}, {"start": 1207.0, "text": "To wrap things up: Core AI is available on all Apple Silicon"}, {"start": 1212.0, "text": "to help you build cutting edge AI experiences on all Apple platforms."}, {"start": 1216.0, "text": "It has tight integration"}, {"start": 1218.0, "text": "with the existing Python tools that you're already familiar with,"}, {"start": 1222.0, "text": "a modern Swift framework for running your models efficiently within your app,"}, {"start": 1226.0, "text": "and state of the art debugging tools"}, {"start": 1228.0, "text": "to help you understand how your models are running on Apple devices."}, {"start": 1232.0, "text": "We can't wait to see what sorts of experiences you build."}]}