{"$schema": "https://raw.githubusercontent.com/guitaripod/wwdc-sessions/master/schema/transcript.schema.json", "id": "wwdc2026-234", "language": "eng", "source": "https://developer.apple.com/videos/play/wwdc2026/234", "wordCount": 2850, "segments": [{"start": 7.0, "text": "Hello,"}, {"start": 8.0, "text": "my name is Michael Breymann and I'm a human interface designer at Apple,"}, {"start": 11.0, "text": "working on system environments for visionOS."}, {"start": 15.0, "text": "In visionOS, system environments are more than simple backdrops."}, {"start": 18.0, "text": "They are photo-realistic natural landscapes, designed for spatial computing,"}, {"start": 22.0, "text": "that transport you to a different time and place."}, {"start": 25.0, "text": "Here in Haleakala, you get a stunning panoramic view above the clouds."}, {"start": 30.0, "text": "But unlike flat media or a panoramic image,"}, {"start": 33.0, "text": "which does not respond to changes in perspective,"}, {"start": 36.0, "text": "this environment offers true depth and parallax,"}, {"start": 39.0, "text": "giving someone the same experience they would see in the real world."}, {"start": 42.0, "text": "When you combine this with motion and audio, the scene feels alive,"}, {"start": 45.0, "text": "and it immerses the viewer in the space."}, {"start": 48.0, "text": "In this talk,"}, {"start": 49.0, "text": "I'll take you through each part of the design process,"}, {"start": 51.0, "text": "for making your own immersive environments,"}, {"start": 54.0, "text": "pre-production, production, and post-production."}, {"start": 58.0, "text": "I'll share the principles and concepts at each stage"}, {"start": 61.0, "text": "that you'll need to create a compelling immersive experience,"}, {"start": 64.0, "text": "with examples from six of the visionOS system environments:"}, {"start": 67.0, "text": "Mount Hood, the Moon, Jupiter,"}, {"start": 72.0, "text": "Yosemite, Thorsmork, and Bora Bora."}, {"start": 76.0, "text": "It's time to get started with pre-production."}, {"start": 79.0, "text": "When our team kicks off a new environment,"}, {"start": 81.0, "text": "we first spend time identifying some questions about our intent."}, {"start": 84.0, "text": "Why are we building this particular environment?"}, {"start": 87.0, "text": "What are the qualities we want to bring to life?"}, {"start": 89.0, "text": "And how is the viewer going to use this space?"}, {"start": 92.0, "text": "These are all important questions to ask yourself about your project."}, {"start": 96.0, "text": "For example,"}, {"start": 97.0, "text": "when our team started thinking about use cases for system environments,"}, {"start": 100.0, "text": "watching media in large format, was especially exciting to us."}, {"start": 104.0, "text": "We wanted the media experience to feel cinematic,"}, {"start": 107.0, "text": "so we sculpted the terrains to fit a large screen,"}, {"start": 109.0, "text": "and researched the optimal center line for viewing."}, {"start": 113.0, "text": "The Keynote environment was built for a different purpose:"}, {"start": 115.0, "text": "to give people a space to practice public speaking,"}, {"start": 118.0, "text": "as though they are live on stage in the real world."}, {"start": 120.0, "text": "Here, sound is intentionally absent, to keep them focused,"}, {"start": 124.0, "text": "and the lighting is concentrated on the stage,"}, {"start": 127.0, "text": "just as it would be if they were presenting in front of an audience."}, {"start": 130.0, "text": "You might want your environment to be a backdrop for your app's content,"}, {"start": 134.0, "text": "or to showcase an experience from the web."}, {"start": 137.0, "text": "Whatever you're designing,"}, {"start": 138.0, "text": "this ideation phase is such a key part of building environments,"}, {"start": 141.0, "text": "and answering these questions early on,"}, {"start": 143.0, "text": "helps you save time and avoid costly mistakes later in production."}, {"start": 147.0, "text": "If you're designing an environment that exists in the real world, like Mount Hood,"}, {"start": 151.0, "text": "it's incredibly helpful to scout your location in advance."}, {"start": 154.0, "text": "Consider where you want to place the viewer in the scene for their primary viewpoint,"}, {"start": 158.0, "text": "as well as what will be visible to them if they decide to turn around."}, {"start": 162.0, "text": "In visionOS,"}, {"start": 163.0, "text": "viewers will see approximately 81° of the scene in their field of view,"}, {"start": 167.0, "text": "when fully immersed."}, {"start": 169.0, "text": "Scouting is also helpful,"}, {"start": 170.0, "text": "for knowing what features you won't want to showcase in your environment."}, {"start": 173.0, "text": "The panorama in the top view shows our Mount Hood location."}, {"start": 177.0, "text": "Notice the road, and the dense vegetation as the camera pans around to the back side."}, {"start": 181.0, "text": "Our team flagged these for replacement when we got into production."}, {"start": 185.0, "text": "The panorama in the bottom view shows the result after our team made these changes."}, {"start": 190.0, "text": "Of course, sometimes the environment you want to create,"}, {"start": 193.0, "text": "won't be easily accessible in the real world."}, {"start": 196.0, "text": "When we worked on the Moon environment, for example,"}, {"start": 198.0, "text": "we relied on photography from the Apollo missions to help us visualize our scene."}, {"start": 203.0, "text": "Reference materials, like photography and videography,"}, {"start": 205.0, "text": "are essential if you're building an environment like this."}, {"start": 209.0, "text": "You can confirm any assumptions you already had,"}, {"start": 211.0, "text": "and reconcile any discrepancies."}, {"start": 214.0, "text": "You can also use reference material,"}, {"start": 215.0, "text": "to help you better understand a location here on Earth."}, {"start": 218.0, "text": "For example, our team created a lighting study"}, {"start": 221.0, "text": "using Digital Elevation Models of the Yosemite valley, and the Earth's orbit,"}, {"start": 225.0, "text": "to determine the exact days and times we wanted to capture."}, {"start": 230.0, "text": "Whether you're building your environment from scratch or a real location,"}, {"start": 233.0, "text": "here are a few things to keep in mind,"}, {"start": 235.0, "text": "when planning for the next stages of production."}, {"start": 238.0, "text": "Try to visualize your environment in terms of layers,"}, {"start": 240.0, "text": "building from the background to the foreground."}, {"start": 243.0, "text": "Identify which scene elements will need to have motion,"}, {"start": 246.0, "text": "and start thinking about what spatial audio you can attach to them."}, {"start": 250.0, "text": "This will help you get ahead of any complications,"}, {"start": 252.0, "text": "that might disrupt your asset creation."}, {"start": 255.0, "text": "One last thing!"}, {"start": 256.0, "text": "Often, during pre-production,"}, {"start": 258.0, "text": "you will discover things that will change your plan for the better."}, {"start": 261.0, "text": "For the Jupiter environment, for example, we built a scale model of the solar system"}, {"start": 265.0, "text": "so that we could understand how the Sun would light Jupiter's moons."}, {"start": 269.0, "text": "During this process,"}, {"start": 270.0, "text": "we realized that we wanted a system that allows for the passage of time,"}, {"start": 274.0, "text": "and this became a key design requirement for the final environment."}, {"start": 278.0, "text": "To recap, when you begin pre-production for your environment,"}, {"start": 282.0, "text": "make sure you're clear on your intent."}, {"start": 284.0, "text": "Scout your scene by visiting real-world locations"}, {"start": 287.0, "text": "and collecting reference material."}, {"start": 290.0, "text": "Consider the composition and layering of your environment,"}, {"start": 293.0, "text": "and what you might want to remove or change."}, {"start": 295.0, "text": "And use all the resources you get from pre-production,"}, {"start": 298.0, "text": "to refine your ideas through iteration."}, {"start": 301.0, "text": "Because having a solid plan"}, {"start": 303.0, "text": "for how you will approach the build of your environment,"}, {"start": 305.0, "text": "prepares you for the next phase of development: production."}, {"start": 309.0, "text": "It takes time to build complex 3D scenes."}, {"start": 312.0, "text": "But you can make it easier on yourself, and anyone you're working with,"}, {"start": 315.0, "text": "by capturing high-quality photography at the source."}, {"start": 318.0, "text": "The Yosemite environment is a great example."}, {"start": 320.0, "text": "During our team's advance scouting,"}, {"start": 322.0, "text": "we identified the exact viewpoint we wanted."}, {"start": 325.0, "text": "Here, the viewer is in Yosemite Valley, with a clear shot into the mid-distance."}, {"start": 329.0, "text": "This sort of framing is ideal for environments."}, {"start": 332.0, "text": "You can always add detail into the foreground with CG elements,"}, {"start": 336.0, "text": "but it's much harder to remove things that obstruct the view."}, {"start": 339.0, "text": "When we were ready to enter production,"}, {"start": 341.0, "text": "we returned to this spot in the valley with our equipment"}, {"start": 344.0, "text": "and a very detailed shoot schedule."}, {"start": 347.0, "text": "Seasons, weather,"}, {"start": 348.0, "text": "and even the time of day can dramatically change what you're able to capture,"}, {"start": 352.0, "text": "and that was certainly true for the Yosemite environment."}, {"start": 355.0, "text": "Lighting can change quickly, especially around sunrise and sunset."}, {"start": 359.0, "text": "Plan to spend more time in your location, and shoot more photography,"}, {"start": 362.0, "text": "than you think you will need."}, {"start": 364.0, "text": "Every image you take,"}, {"start": 365.0, "text": "can be used to create your environment's 3D assets later on,"}, {"start": 368.0, "text": "so it's important to capture source imagery that is directly usable."}, {"start": 372.0, "text": "Here are a few tips:"}, {"start": 373.0, "text": "use a tripod with your camera leveled 1 meter off the ground,"}, {"start": 377.0, "text": "and ensure a deep depth of field."}, {"start": 379.0, "text": "If possible, set up another camera 2 meters off the ground to trigger at the same time."}, {"start": 383.0, "text": "This alternate view is especially helpful during Post-Production,"}, {"start": 386.0, "text": "to fill in parts of the scene that are not visible from the primary view."}, {"start": 390.0, "text": "And remember, you are creating a 360° environment,"}, {"start": 394.0, "text": "so you will need to use a rig and lens that covers all views,"}, {"start": 397.0, "text": "and ultimately produces a stitched panorama."}, {"start": 400.0, "text": "The dynamic range of an environment can be quite large,"}, {"start": 403.0, "text": "from the Sun to the shadows,"}, {"start": 405.0, "text": "so shoot bracketed exposures to ensure you capture all the details."}, {"start": 409.0, "text": "Know your target output display,"}, {"start": 410.0, "text": "and capture more resolution than you think you will need."}, {"start": 413.0, "text": "For visionOS, an environment will be sharp at 40 pixels per degree,"}, {"start": 417.0, "text": "so 360° panorama of 14,400 by 7,200 pixels is an ideal target."}, {"start": 424.0, "text": "In addition to your primary photography,"}, {"start": 426.0, "text": "it's also a good idea to capture secondary photography on location"}, {"start": 430.0, "text": "for reference and measurements."}, {"start": 432.0, "text": "The point clouds produced by photogrammetry and LiDAR,"}, {"start": 434.0, "text": "can be meshed and used as starting points for 3D assets,"}, {"start": 438.0, "text": "but the data is also useful for distance measurements."}, {"start": 441.0, "text": "You might discover, for example,"}, {"start": 442.0, "text": "that your seemingly flat terrain actually has a slope."}, {"start": 445.0, "text": "And with that knowledge,"}, {"start": 446.0, "text": "you might make different choices about how you construct your 3D assets."}, {"start": 449.0, "text": "Lighting reference is especially helpful"}, {"start": 451.0, "text": "when making CG assets integrate seamlessly into your environment."}, {"start": 455.0, "text": "Macbeth charts, as well as chrome and gray spheres,"}, {"start": 458.0, "text": "should be shot at the same time as your primary photography."}, {"start": 462.0, "text": "Video of scene elements in motion will also provide invaluable reference"}, {"start": 466.0, "text": "when creating believable shader effects."}, {"start": 468.0, "text": "Take note of what sounds, if any, are associated with motion,"}, {"start": 471.0, "text": "so that you can source the appropriate audio."}, {"start": 474.0, "text": "And if it's not possible to capture a panorama through photography,"}, {"start": 477.0, "text": "you will need to create a rendered panorama,"}, {"start": 479.0, "text": "using your favorite digital content creation tool."}, {"start": 482.0, "text": "The advantage of creating a rendered panorama,"}, {"start": 484.0, "text": "is that you have ultimate control over the entire scene at any given time."}, {"start": 488.0, "text": "Production helps set you, and your environment, up for success."}, {"start": 492.0, "text": "Careful preparation, will help you capture the best photography"}, {"start": 495.0, "text": "for the next stage of the process."}, {"start": 497.0, "text": "High-quality imagery gives you, or your technical artists,"}, {"start": 501.0, "text": "valuable data to work with."}, {"start": 503.0, "text": "Secondary photography and reference material"}, {"start": 505.0, "text": "can provide you with extra information about the scene to make your 3D work easier."}, {"start": 510.0, "text": "Lastly, this is not a one-size-fits-all process."}, {"start": 513.0, "text": "If you're a small team with limited resources,"}, {"start": 516.0, "text": "you can adapt your production with these best practices in mind."}, {"start": 520.0, "text": "After the production phase,"}, {"start": 522.0, "text": "you should have enough material to create a high-resolution source panorama."}, {"start": 525.0, "text": "With this and your reference content,"}, {"start": 527.0, "text": "you're now ready to begin designing your 3D asset."}, {"start": 530.0, "text": "The first step, however?"}, {"start": 532.0, "text": "Cleanup."}, {"start": 533.0, "text": "Let's take a look at the principal photography from the Thorsmork environment,"}, {"start": 536.0, "text": "and how our team approached cleaning up this panorama."}, {"start": 539.0, "text": "There are a couple of obvious things here"}, {"start": 541.0, "text": "that you wouldn't want in the final environment."}, {"start": 544.0, "text": "Like the camera rig."}, {"start": 547.0, "text": "As well as footprints and people, which would detract from the landscape."}, {"start": 551.0, "text": "But the cleanup process is also where those notes taken in pre-production come in handy."}, {"start": 556.0, "text": "Let's take a look at a few scene elements,"}, {"start": 558.0, "text": "that are interfering with our ideal composition."}, {"start": 560.0, "text": "Like here,"}, {"start": 561.0, "text": "the vegetation in the mid ground is too busy."}, {"start": 564.0, "text": "And these bushes, are too dominant in frame."}, {"start": 568.0, "text": "Once we identified the key elements we wanted removed,"}, {"start": 570.0, "text": "our team went to work and refined the imagery"}, {"start": 572.0, "text": "through iterations of digital matte painting and rendering CG assets."}, {"start": 576.0, "text": "It's very important to pay attention to color balance and lighting consistency,"}, {"start": 580.0, "text": "throughout this process."}, {"start": 582.0, "text": "Post-production involves making many creative decisions"}, {"start": 584.0, "text": "about the look of your environment,"}, {"start": 586.0, "text": "but there's another critical aspect of this stage."}, {"start": 588.0, "text": "Maintaining visual fidelity as you prepare the final asset."}, {"start": 591.0, "text": "Because a 3D mesh is needed to achieve a sense of parallax and depth,"}, {"start": 595.0, "text": "you'll move your refined panorama, to textures in UV space."}, {"start": 599.0, "text": "The parts of the scene that are not visible in the panorama need to be filled in."}, {"start": 603.0, "text": "This is where secondary photography and CG renders supplement the primary view."}, {"start": 607.0, "text": "During this process,"}, {"start": 608.0, "text": "it's helpful to always do an A/B comparison between your 3D asset and your panorama,"}, {"start": 613.0, "text": "to make sure your texture transfer is retaining the highest quality."}, {"start": 616.0, "text": "Make sure that all scene elements have consistent sharpness,"}, {"start": 619.0, "text": "especially when compared to those nearby, so that they convey the proper scale."}, {"start": 623.0, "text": "Try flopping your scene to look at it in a new way."}, {"start": 626.0, "text": "At this point in post-production,"}, {"start": 628.0, "text": "your eyes may be too accustomed to seeing the same thing,"}, {"start": 631.0, "text": "so you can use this trick to make sure your composition is still working."}, {"start": 634.0, "text": "You can also subject your environment to extreme gamma and gain values."}, {"start": 638.0, "text": "This technique can reveal any color and values inconsistencies,"}, {"start": 641.0, "text": "along with any data loss during texture transfer,"}, {"start": 644.0, "text": "which may become more obvious when viewed on different displays."}, {"start": 647.0, "text": "Many aspects of post-production involve creative choices,"}, {"start": 650.0, "text": "but there's also a lot of technical artistry that supports this phase."}, {"start": 654.0, "text": "To learn more about what is required to render your environments in real-time,"}, {"start": 658.0, "text": "check out \"Optimize your custom environments for visionOS\"."}, {"start": 662.0, "text": "Now that you have a textured mesh,"}, {"start": 664.0, "text": "the visual base of your environment is complete,"}, {"start": 666.0, "text": "but you'll need a couple more elements to complete the sense of immersion."}, {"start": 669.0, "text": "Sound and motion."}, {"start": 671.0, "text": "Sound can be incredibly impactful for creating immersion,"}, {"start": 674.0, "text": "and you can place spatial audio sources to help amplify that effect."}, {"start": 679.0, "text": "For example, in this environment,"}, {"start": 681.0, "text": "an emitter with a rippling water sound attached,"}, {"start": 683.0, "text": "is located in a part of the river where the water flows around large rocks."}, {"start": 688.0, "text": "The design and implementation of spatial audio,"}, {"start": 690.0, "text": "are big topics in themselves."}, {"start": 692.0, "text": "So check out these two excellent talks,"}, {"start": 694.0, "text": "to learn more about how sounds can enhance the sense of immersion,"}, {"start": 697.0, "text": "in your environments."}, {"start": 698.0, "text": "And speaking of the river, it also needs to move!"}, {"start": 701.0, "text": "For any aspect of your environment that needs motion,"}, {"start": 704.0, "text": "custom shaders can help you create the best visuals,"}, {"start": 706.0, "text": "while still being power efficient."}, {"start": 708.0, "text": "Now, Thorsmork is a subarctic landscape,"}, {"start": 712.0, "text": "and there aren't a lot of motion features to design here."}, {"start": 714.0, "text": "So let's travel south, to Bora Bora in French Polynesia,"}, {"start": 717.0, "text": "an environment rich with movement."}, {"start": 720.0, "text": "Landscapes like Bora Bora have lots of complex motion and light interactions."}, {"start": 724.0, "text": "I'll speed up the passage of time, so you can see them more clearly."}, {"start": 727.0, "text": "The clouds, which at first seem static,"}, {"start": 729.0, "text": "are actually evolving, casting shadows throughout the environment."}, {"start": 733.0, "text": "The palm trees sway in the wind,"}, {"start": 735.0, "text": "and the waves break a little differently each time as they hit the shore."}, {"start": 739.0, "text": "All of these details add realism to the environment."}, {"start": 742.0, "text": "But motion and light can be some of the most expensive things to render."}, {"start": 746.0, "text": "For a scene like Bora Bora,"}, {"start": 747.0, "text": "our team had to think creatively about how we could achieve the visual intent,"}, {"start": 750.0, "text": "without breaking our real-time rendering budget."}, {"start": 753.0, "text": "I'm going to share a few of the techniques we used,"}, {"start": 755.0, "text": "and explain how you can take advantage of them in your environment."}, {"start": 758.0, "text": "Let's start with the sky, a huge part of the scene."}, {"start": 762.0, "text": "We wanted it to feel like"}, {"start": 763.0, "text": "the clouds are continuously evolving with a sense of direction."}, {"start": 767.0, "text": "UV flow maps made this possible, at little cost."}, {"start": 770.0, "text": "And by weighting the flow speed in different parts of the sky,"}, {"start": 773.0, "text": "it feels massive, with depth and scale."}, {"start": 776.0, "text": "Cloud shadows can be achieved not through rendered lights, as you might expect,"}, {"start": 780.0, "text": "but rather a scrolling mask that darkens the terrain textures."}, {"start": 784.0, "text": "The direction and speed of the clouds above are matched,"}, {"start": 786.0, "text": "so that they feel connected, even though they're not."}, {"start": 789.0, "text": "Let's go back to real-time speed,"}, {"start": 791.0, "text": "and examine another part of the environment."}, {"start": 794.0, "text": "Here under the palm trees,"}, {"start": 795.0, "text": "the more dense, clearly defined shadows, are pre-rendered into flip book textures"}, {"start": 799.0, "text": "that darken the terrain textures."}, {"start": 801.0, "text": "In this way,"}, {"start": 802.0, "text": "you can trade the expense of rendering soft shadows,"}, {"start": 804.0, "text": "for a believable approximation."}, {"start": 807.0, "text": "Counterintuitively, it's often the case that less is more."}, {"start": 811.0, "text": "Offloading complex shading work to precomputed data textures,"}, {"start": 814.0, "text": "allows you to produce motion"}, {"start": 815.0, "text": "that would otherwise be too expensive to render real-time."}, {"start": 819.0, "text": "These palm fronds, for example, have their mesh complexity greatly reduced."}, {"start": 823.0, "text": "And the wind motion comes from a UV flow map, to produce the final effect."}, {"start": 827.0, "text": "Notice also how the trunks and fronds sway more gently than the leaflets."}, {"start": 831.0, "text": "This is achieved through hierarchical vertex animation and layered sine waves."}, {"start": 835.0, "text": "By stacking low frequency and high frequency motion,"}, {"start": 838.0, "text": "you can create variety that is never repetitive."}, {"start": 841.0, "text": "In fact, lots of expensive rendering can be reduced"}, {"start": 843.0, "text": "through artful layering of textures and compositing operations."}, {"start": 846.0, "text": "Water, for example,"}, {"start": 847.0, "text": "can have its hue saturation and brightness modulated over time,"}, {"start": 851.0, "text": "to simulate subsurface scattering of sunlight."}, {"start": 854.0, "text": "The waves can move through layers of normal maps and scrolling textures."}, {"start": 858.0, "text": "The sum of these effects, is greater than its parts."}, {"start": 861.0, "text": "Making all of these effects work together and feel like they exist in the same world,"}, {"start": 865.0, "text": "requires artistry and creative direction."}, {"start": 868.0, "text": "To make sure you're developing the right features and controls for your shaders,"}, {"start": 871.0, "text": "remember your visual reference can be a great resource"}, {"start": 874.0, "text": "to articulate what you need to build."}, {"start": 876.0, "text": "So let's recap."}, {"start": 878.0, "text": "In post-production,"}, {"start": 879.0, "text": "you take all your planning, reference, and photography,"}, {"start": 882.0, "text": "and bring your environment to life."}, {"start": 884.0, "text": "You'll clean up your panorama,"}, {"start": 885.0, "text": "refine your scene in 3D and make sure your elements feel right,"}, {"start": 889.0, "text": "and add supporting details like sound and motion to complete the sense of immersion."}, {"start": 894.0, "text": "Designing for immersive environments is really about producing convincing results"}, {"start": 897.0, "text": "and presenting an illusion that feels authentic."}, {"start": 900.0, "text": "You're in control of this, so at each stage of production,"}, {"start": 903.0, "text": "you will want to check your creative decisions for their merits."}, {"start": 906.0, "text": "To close, keep these things in mind as you start designing your environment."}, {"start": 910.0, "text": "Be intentional with everything you do."}, {"start": 912.0, "text": "You should be able to explain the reasoning behind everything you add or remove."}, {"start": 916.0, "text": "Create a composition that works."}, {"start": 918.0, "text": "Every scene element should feel like it belongs, wherever it lives."}, {"start": 922.0, "text": "And all aspects of the scene should be tied together with sound and motion."}, {"start": 926.0, "text": "Good design makes your craft and execution stronger,"}, {"start": 929.0, "text": "and applies whether you're an individual, a small team, or a large studio."}, {"start": 934.0, "text": "I'll leave you with a final thought."}, {"start": 936.0, "text": "Often the best way to know your designs are working, is to try to break them."}, {"start": 939.0, "text": "Don't get too attached to any one idea."}, {"start": 942.0, "text": "Do something unexpected, and be receptive to the outcome."}, {"start": 945.0, "text": "Some of your best work will be the result of the surprises"}, {"start": 948.0, "text": "that you embrace along the way."}]}