{"$schema": "https://raw.githubusercontent.com/guitaripod/wwdc-sessions/master/schema/transcript.schema.json", "id": "wwdc2026-282", "language": "eng", "source": "https://developer.apple.com/videos/play/wwdc2026/282", "wordCount": 2346, "segments": [{"start": 7.0, "text": "Hi my name is Quincy Jermyn,"}, {"start": 9.0, "text": "and I'm a software engineer on the visionOS Team."}, {"start": 12.0, "text": "One of the features I use the most on Apple Vision Pro is Mac Virtual Display,"}, {"start": 16.0, "text": "which lets me work on my Mac through a virtual screen provided by Vision Pro."}, {"start": 21.0, "text": "In macOS and visionOS 27,"}, {"start": 24.0, "text": "it's even easier to preview content from Mac to Vision Pro."}, {"start": 28.0, "text": "I'm excited to present Spatial Preview,"}, {"start": 30.0, "text": "a framework that lets people leverage the spatial computing capabilities"}, {"start": 34.0, "text": "of visionOS when working with content on their Mac."}, {"start": 37.0, "text": "You can check out the Spatial Preview framework"}, {"start": 39.0, "text": "in action with the Preview app for macOS,"}, {"start": 42.0, "text": "alongside other features to help people work with spatial content including,"}, {"start": 46.0, "text": "3D content editing, photorealistic rendering,"}, {"start": 50.0, "text": "camera viewpoints, and spatial media output"}, {"start": 53.0, "text": "like Apple Immersive Video frames and spatial photos."}, {"start": 56.0, "text": "Using Mac Virtual Display, people can share content seamlessly to Vision Pro."}, {"start": 61.0, "text": "On visionOS, the Quick Look app receives this content"}, {"start": 64.0, "text": "and now allows them to work immersively"}, {"start": 67.0, "text": "like moving around to the different cameras set up for this living room scene."}, {"start": 71.0, "text": "These are tools that are very useful to get a sense of scale and layout"}, {"start": 74.0, "text": "of a 3D design, like this one!"}, {"start": 78.0, "text": "With Spatial Preview,"}, {"start": 79.0, "text": "people can now extend content from their Mac into the world around them"}, {"start": 83.0, "text": "and these features aren't just available in the Preview app."}, {"start": 86.0, "text": "They're also exposed as an API for you"}, {"start": 89.0, "text": "so you can augment new or existing macOS applications"}, {"start": 92.0, "text": "to take advantage of the power of Vision Pro."}, {"start": 96.0, "text": "These tools enable dynamic workflows with spatial content,"}, {"start": 99.0, "text": "including live synchronization and editing across devices."}, {"start": 103.0, "text": "With real-time synchronization,"}, {"start": 105.0, "text": "apps like Cinema4D and SketchUp are transforming the creative process,"}, {"start": 110.0, "text": "unlocking real-time, collaborative 3D workflows"}, {"start": 113.0, "text": "like iterating on material changes live."}, {"start": 116.0, "text": "And in this session I'm going to cover how you can too!"}, {"start": 120.0, "text": "I'll start with an overview of the framework"}, {"start": 122.0, "text": "designed for developers building content creation tools"}, {"start": 125.0, "text": "or any app that works with spatial content."}, {"start": 128.0, "text": "Next, I'll walk through an example workflow showing how to share and update"}, {"start": 132.0, "text": "documents like Apple Immersive Video frames in place."}, {"start": 136.0, "text": "Lastly, I'll show how you can create apps"}, {"start": 138.0, "text": "that fully immerse people in the 3D content they're working on, live,"}, {"start": 142.0, "text": "by using Universal Scene Description,"}, {"start": 144.0, "text": "which is a format used to describe 3D scenes."}, {"start": 147.0, "text": "By the end of this session,"}, {"start": 149.0, "text": "you'll have everything you need to leverage the Spatial Preview framework"}, {"start": 152.0, "text": "and enable people to work with content from their Macon visionOS."}, {"start": 157.0, "text": "Now lets take a look at the components provided by the framework"}, {"start": 160.0, "text": "in order to get a sense of just how easy it is to set up."}, {"start": 165.0, "text": "The first step is selecting an endpoint that points to a device to share to."}, {"start": 169.0, "text": "If someone's actively using Mac Virtual Display"}, {"start": 172.0, "text": "it's easy to use the device that's already connected."}, {"start": 175.0, "text": "Alternatively, you can add the Device Picker UI to your app"}, {"start": 178.0, "text": "so people can pick from any nearby Vision Pro on the same iCloud account."}, {"start": 184.0, "text": "Next, create a Spatial Preview Session for the content."}, {"start": 187.0, "text": "There are two types of Preview Sessions,"}, {"start": 189.0, "text": "Document Preview Session covers types like spatial photos, videos,"}, {"start": 193.0, "text": "and document types like PDF."}, {"start": 195.0, "text": "USD Preview Sessions handle 3D content."}, {"start": 200.0, "text": "Once the session starts, Quick Look is launched on vision Pro"}, {"start": 203.0, "text": "and the content your app provides to the session appears."}, {"start": 206.0, "text": "No code is required on visionOS to set this up!"}, {"start": 210.0, "text": "Now that you have an overview of how a Spatial Preview Session works,"}, {"start": 214.0, "text": "let's look at an example of how to send and update documents"}, {"start": 217.0, "text": "with Document Preview."}, {"start": 219.0, "text": "In this example I'll take a still from an Apple Immersive Video"}, {"start": 223.0, "text": "which is generated out of the Preview app on Mac."}, {"start": 226.0, "text": "Then, I'll use the Mac Virtual Display endpoint"}, {"start": 229.0, "text": "to start a Document Preview Session,"}, {"start": 231.0, "text": "I'll then provide the image to the session so it appears in Quick Look."}, {"start": 235.0, "text": "Let's look at the code to set this up."}, {"start": 239.0, "text": "First, create a ConnectedSpatialEndpointObserver"}, {"start": 242.0, "text": "to obtain the Spatial Preview Endpoint from Mac Virtual Display."}, {"start": 246.0, "text": "Then create a DocumentPreviewSession"}, {"start": 249.0, "text": "specifying the document type and a name for the session."}, {"start": 254.0, "text": "With the returned endpoint start the session on the selected device"}, {"start": 257.0, "text": "and provide the content URL."}, {"start": 260.0, "text": "Remember, Mac Virtual Display may not always be active"}, {"start": 264.0, "text": "so consider integrating the SpatialPreviewDevicePicker"}, {"start": 267.0, "text": "as a view in your UI to select a different device."}, {"start": 270.0, "text": "Use a sheet in SwiftUI to control when it appears."}, {"start": 274.0, "text": "Then, proceed to create the Document Preview Session just as you saw previously"}, {"start": 278.0, "text": "with the device endpoint selected in the UI."}, {"start": 282.0, "text": "It's this easy to get started with Spatial Preview."}, {"start": 284.0, "text": "Let's see the result."}, {"start": 286.0, "text": "Putting the code I wrote into a button in the UI"}, {"start": 289.0, "text": "launches the content on visionOS when pressed"}, {"start": 292.0, "text": "and I can view it in immersive mode, this is pretty cool!"}, {"start": 295.0, "text": "Let's take this one step further"}, {"start": 297.0, "text": "and build a gallery of immersive architectural renderings."}, {"start": 303.0, "text": "When you call updateContents, you're reusing the same scene"}, {"start": 307.0, "text": "that was launched when you started the session."}, {"start": 309.0, "text": "If you make a new Document Preview Session and call start,"}, {"start": 312.0, "text": "that will launch a new scene."}, {"start": 314.0, "text": "So for the gallery view I'm going to want to leverage"}, {"start": 316.0, "text": "updateContents to ensure I re-use the same scene."}, {"start": 321.0, "text": "In this view, I'll create a row of buttons for all the video frames in the gallery"}, {"start": 325.0, "text": "and make the buttons call updateContents on the session"}, {"start": 328.0, "text": "to swap to a different file when selected."}, {"start": 331.0, "text": "I also set up a task that observes the session state."}, {"start": 335.0, "text": "If the scene is closed on visionOS,"}, {"start": 337.0, "text": "I'll receive a state change here that the session has been invalidated."}, {"start": 341.0, "text": "When you're done with a session, call close to end it"}, {"start": 344.0, "text": "visionOS will automatically dismiss that scene."}, {"start": 348.0, "text": "Now the gallery switches between immersive renderings live within the same scene"}, {"start": 352.0, "text": "using DocumentPreviewSession's updateContents function."}, {"start": 356.0, "text": "This is a great way for people to review design renders"}, {"start": 358.0, "text": "and get a better sense of the content they're working on at scale."}, {"start": 362.0, "text": "Beyond Apple Immersive Video,"}, {"start": 363.0, "text": "many other content types work well with Spatial Preview,"}, {"start": 366.0, "text": "including Spatial Photos, PDFs,"}, {"start": 369.0, "text": "standard images and files and also 3D content!"}, {"start": 373.0, "text": "Now that you've seen how easy it is to send and update documents,"}, {"start": 376.0, "text": "let's look at how to work with 3D content using USDKit with Spatial Preview."}, {"start": 382.0, "text": "If you're new to USD,"}, {"start": 384.0, "text": "I recommend starting with the video \"Understand USD fundamentals\""}, {"start": 387.0, "text": "and to learn how to use the new Swift USDKit framework,"}, {"start": 391.0, "text": "I recommend watching \"Discover USDKit and what's new in OpenUSD\"."}, {"start": 396.0, "text": "In this section I'm going to cover how to use USDKit"}, {"start": 399.0, "text": "to work with 3D content from your macOS app on visionOS."}, {"start": 403.0, "text": "I'll go over how easy it is to set up a USD Preview Session"}, {"start": 407.0, "text": "and start navigating a 3D scene,"}, {"start": 409.0, "text": "looking through cameras and applying material overrides."}, {"start": 413.0, "text": "I'll then cover how to edit content,"}, {"start": 415.0, "text": "and how people can interact and make changes to USD on visionOS"}, {"start": 419.0, "text": "including adding annotations and moving objects around."}, {"start": 423.0, "text": "Last, I'll go over the events and observable properties"}, {"start": 426.0, "text": "of a Spatial Preview Session,"}, {"start": 428.0, "text": "including events for animation playback and session synchronization progress,"}, {"start": 432.0, "text": "and how to customize the features available in a session."}, {"start": 436.0, "text": "Creating a USD Preview Session is similar to a Document Preview Session"}, {"start": 440.0, "text": "except that the content is a USDKit stage."}, {"start": 443.0, "text": "When you call session start a scene opens on visionOS"}, {"start": 446.0, "text": "and the USD content appears in a volumetric view,"}, {"start": 449.0, "text": "people can then choose to go into an Immersive View to see it at full scale."}, {"start": 453.0, "text": "Lets see what the code looks like to set this up."}, {"start": 457.0, "text": "Just like with document sharing,"}, {"start": 458.0, "text": "begin by choosing a target device endpoint."}, {"start": 462.0, "text": "Then use USDKit to load USD content into a stage,"}, {"start": 466.0, "text": "provide that stage to a USDPreviewSession,"}, {"start": 469.0, "text": "and then start the session on the selected device endpoint."}, {"start": 474.0, "text": "The content appears in 3D on visionOS in this bounded view."}, {"start": 479.0, "text": "I can rotate the scene around to inspect it"}, {"start": 482.0, "text": "and then go Immersive to see it at full scale."}, {"start": 485.0, "text": "There are some cameras set up in this scene, as you saw before,"}, {"start": 489.0, "text": "so selecting one here will move me to that viewpoint."}, {"start": 492.0, "text": "I can also look more closely at the geometry"}, {"start": 494.0, "text": "by overriding the materials to be in Wireframe mode."}, {"start": 498.0, "text": "This is all functionality built into visionOS when using Spatial Preview,"}, {"start": 503.0, "text": "no additional setup is required from your Mac app."}, {"start": 506.0, "text": "USD scenes can contain incredibly high fidelity content,"}, {"start": 509.0, "text": "which might be too complex to render on Vision Pro."}, {"start": 513.0, "text": "By default Spatial Preview automatically optimizes the USD content before sharing,"}, {"start": 518.0, "text": "including mesh decimation, texture downsampling,"}, {"start": 522.0, "text": "and potential full scene reconstruction if necessary,"}, {"start": 526.0, "text": "all to ensure that it performs well on Vision Pro."}, {"start": 529.0, "text": "If the scene needs to be reconstructed,"}, {"start": 531.0, "text": "it won't be editable but people can still view it and add annotations."}, {"start": 537.0, "text": "To opt out of this optimization,"}, {"start": 538.0, "text": "pass the .unmodified parameter when creating the session."}, {"start": 542.0, "text": "However, if optimization is disabled, complex scenes may not be shareable to"}, {"start": 547.0, "text": "Vision Pro and an error will be thrown from the start function."}, {"start": 551.0, "text": "For guidance on how to reduce the rendering cost of your content,"}, {"start": 554.0, "text": "check out the Developer Documentation."}, {"start": 557.0, "text": "Now, let see how editing USD content works on macOS and visionOS."}, {"start": 562.0, "text": "In a USD Preview Session, people can make changes from either device!"}, {"start": 566.0, "text": "A live USD stage is used to replicate content"}, {"start": 569.0, "text": "between macOS and visionOS when calling regular USDKit APIs."}, {"start": 574.0, "text": "This is very useful if someone wants to make changes to the content on their Mac"}, {"start": 578.0, "text": "and see them spatially on visionOS,"}, {"start": 581.0, "text": "or capture the edits during a review session on visionOS."}, {"start": 586.0, "text": "Before jumping into the code,"}, {"start": 587.0, "text": "lets go over some details of what a USD stage is and some USD terminology."}, {"start": 593.0, "text": "A stage is composed of one or many layers and layers contains USD prims."}, {"start": 599.0, "text": "A USD prim is an object that can represent many different features"}, {"start": 602.0, "text": "like a 3D transformation, or a mesh."}, {"start": 605.0, "text": "And variant sets can swap in alternative data on a prim."}, {"start": 610.0, "text": "I've set up some USD variants in this living room scene"}, {"start": 612.0, "text": "that change the position and rotation of the furniture using variantSets."}, {"start": 617.0, "text": "When I select one of these variants in USDKit,"}, {"start": 620.0, "text": "the stage is changed and the edits that move the furniture to different locations"}, {"start": 624.0, "text": "are applied and synchronized between macOS and visionOS."}, {"start": 627.0, "text": "The buttons along the toolbar here switch between variants on macOS"}, {"start": 631.0, "text": "and the resulting furniture layout is reflected on visionOS."}, {"start": 635.0, "text": "I can also select a variant in the Quick Look menu and see this change on Mac,"}, {"start": 640.0, "text": "this is a great way to quickly iterate on content from the Mac on visionOS."}, {"start": 645.0, "text": "Now, let's see how to listen for changes coming from visionOS."}, {"start": 649.0, "text": "Changes to this USD stage are automatically synchronized"}, {"start": 652.0, "text": "and you can observe the updates happening in a session"}, {"start": 655.0, "text": "like when someone adds an annotation using standard USD notices."}, {"start": 660.0, "text": "Here I subscribe to ObjectsDidChange notices"}, {"start": 662.0, "text": "that give information about which USD prim paths have changed."}, {"start": 666.0, "text": "I can iterate through those paths and find annotations to update them in the UI."}, {"start": 673.0, "text": "For a text annotation you'll want information like author"}, {"start": 676.0, "text": "and a unique identifier for the annotation in addition to the actual text note."}, {"start": 681.0, "text": "Setting up annotation data in this way allows annotations to show up on visionOS"}, {"start": 687.0, "text": "as long as they are children of a USD prim specified as a documentAnnotationGroup."}, {"start": 693.0, "text": "In order to have USD prims be editable on visionOS"}, {"start": 697.0, "text": "using gestures, they need to have the spatialEditable metadata set on them."}, {"start": 702.0, "text": "People can also easily set this up on their USD asset using Preview on macOS."}, {"start": 708.0, "text": "Let's see this in action."}, {"start": 710.0, "text": "When I apply an annotation on macOS it appears on visionOS,"}, {"start": 713.0, "text": "and if I comment here, 'I like layout A',"}, {"start": 716.0, "text": "that is reflected back in the app on macOS."}, {"start": 719.0, "text": "And since the furniture has spatialEditable metadata, I can move it."}, {"start": 724.0, "text": "Say I want to move this chair out the window,"}, {"start": 727.0, "text": "that would show up on macOS as well."}, {"start": 729.0, "text": "Let's go over the events you can listen for"}, {"start": 732.0, "text": "and the options you can configure within a USD Preview session."}, {"start": 736.0, "text": "You can control what features are available in Quick Look on visionOS"}, {"start": 740.0, "text": "using the option set on USD Preview session start."}, {"start": 744.0, "text": "By default, annotations, object manipulation and USD export are all enabled, by default."}, {"start": 750.0, "text": "You can also subscribe to non-USD events like animation playback time"}, {"start": 754.0, "text": "via Spatial Preview API events."}, {"start": 758.0, "text": "You can monitor the sharing progress of a session using ProgressReporter,"}, {"start": 763.0, "text": "this is useful if you'd like to display a loading bar on macOS"}, {"start": 765.0, "text": "while large data is synchronized over to Vision Pro."}, {"start": 769.0, "text": "For example, here,"}, {"start": 770.0, "text": "I've hooked up the time and playback events provided by USD Preview Session."}, {"start": 774.0, "text": "Now, I can hit the play button"}, {"start": 776.0, "text": "and see this hummingbird animate outside the window in both apps."}, {"start": 780.0, "text": "When you adopt Spatial Preview in your app,"}, {"start": 782.0, "text": "you can unlock so many new workflows for creators."}, {"start": 785.0, "text": "And SharePlay support is built in on visionOS,"}, {"start": 788.0, "text": "so collaborators can join the same live session"}, {"start": 791.0, "text": "to review and edit spatial content simultaneously!"}, {"start": 795.0, "text": "Here, my friend Joy flagged an issue with my room layout,"}, {"start": 798.0, "text": "so I'll adjust it in real time."}, {"start": 800.0, "text": "Every participant's view updates immediately,"}, {"start": 802.0, "text": "eliminating the back-and-forth of traditional asset review."}, {"start": 808.0, "text": "So how can you get started with Spatial Preview today?"}, {"start": 811.0, "text": "We recommend using the USDKit Swift APIs"}, {"start": 814.0, "text": "as Spatial Preview works seamlessly with it."}, {"start": 816.0, "text": "If you already have USD in your Mac app,"}, {"start": 819.0, "text": "you can set up bridging to transfer edits between your USD installation and USDKit."}, {"start": 824.0, "text": "For more information,"}, {"start": 825.0, "text": "check out the Spatial Preview Developer Documentation."}, {"start": 829.0, "text": "Next, explore how you can leverage each type of Preview session;"}, {"start": 833.0, "text": "Document Preview Session and USD Preview Session."}, {"start": 836.0, "text": "They integrate easily into your Mac app"}, {"start": 838.0, "text": "whether you're looking to preview and update documents"}, {"start": 841.0, "text": "or work with 3D content immersively,"}, {"start": 843.0, "text": "it's only a few lines of code to get started."}, {"start": 846.0, "text": "You can go even deeper with 3D using USDKit and Spatial Preview"}, {"start": 850.0, "text": "to support live editing of a USD stage,"}, {"start": 853.0, "text": "enabling real-time collaboration across devices, including over SharePlay."}, {"start": 858.0, "text": "And finally, take advantage of asset review tools like material overrides,"}, {"start": 862.0, "text": "camera viewpoints, object manipulation, variants and annotations."}, {"start": 867.0, "text": "The team and I"}, {"start": 868.0, "text": "can't wait to see how your apps will take advantage of Spatial Preview,"}, {"start": 871.0, "text": "whether you're making a creative app,"}, {"start": 873.0, "text": "content review tool, or something entirely new."}, {"start": 876.0, "text": "Thanks for watching!"}]}