{"$schema": "https://raw.githubusercontent.com/guitaripod/wwdc-sessions/master/schema/transcript.schema.json", "id": "wwdc2026-322", "language": "eng", "source": "https://developer.apple.com/videos/play/wwdc2026/322", "wordCount": 2095, "segments": [{"start": 7.0, "text": "Hi!"}, {"start": 8.0, "text": "I am Haotian, an engineer on the UI Frameworks team."}, {"start": 13.0, "text": "Since its inception,"}, {"start": 15.0, "text": "SwiftUI has been steadily growing with its capabilities in graphics and layout,"}, {"start": 21.0, "text": "making it the choice for people who want to ship rich and custom experiences"}, {"start": 26.0, "text": "on Apple devices."}, {"start": 28.0, "text": "Apple uses SwiftUI to build advanced effects across its own apps, too."}, {"start": 35.0, "text": "Well, the word advanced can sound intimidating."}, {"start": 39.0, "text": "But here's the thing, even with advanced effects,"}, {"start": 43.0, "text": "SwiftUI apps share the same basic elements."}, {"start": 47.0, "text": "It is like a pipeline."}, {"start": 50.0, "text": "Data flows through a series of standard pipes."}, {"start": 54.0, "text": "It takes something in, transforms it, and passes it along."}, {"start": 61.0, "text": "SwiftUI's progressive disclosure means each pipe already works on its own."}, {"start": 68.0, "text": "But you can connect them, create branches,"}, {"start": 72.0, "text": "or merge the flows. That's when you get creative."}, {"start": 77.0, "text": "The 'advanced' lies in the construction, not the complexity."}, {"start": 82.0, "text": "Here's the roadmap."}, {"start": 84.0, "text": "First, I will take a design and break it apart."}, {"start": 89.0, "text": "Then I will build advanced effects."}, {"start": 92.0, "text": "And finally, I will share how you can incorporate these techniques in your apps,"}, {"start": 98.0, "text": "using a creative pipeline."}, {"start": 100.0, "text": "Here is a design I am building."}, {"start": 103.0, "text": "So, I have been building my own podcast app."}, {"start": 108.0, "text": "This is what it currently looks like, a bare-bones transcript view."}, {"start": 113.0, "text": "And I am going to make it fancy,"}, {"start": 115.0, "text": "like the live lyrics view in Apple Music."}, {"start": 119.0, "text": "With animated cover art and transcripts that scroll in sync with time."}, {"start": 125.0, "text": "How do I even start?"}, {"start": 128.0, "text": "I start with what I already have."}, {"start": 131.0, "text": "My existing user interface already contains all the data I need,"}, {"start": 135.0, "text": "including the cover art,"}, {"start": 139.0, "text": "the playback info, and the transcript text."}, {"start": 145.0, "text": "The question isn't what data I need,"}, {"start": 148.0, "text": "it's how I transform it using the pipeline."}, {"start": 151.0, "text": "Here are a couple of examples."}, {"start": 156.0, "text": "Starting with the cover art,"}, {"start": 158.0, "text": "I need a pipe that converts the image into a visualizer,"}, {"start": 163.0, "text": "and the shader pipe fits here."}, {"start": 167.0, "text": "The visualizer needs to be in motion to reflect our playback state."}, {"start": 173.0, "text": "For dynamic visuals, I connect the time pipe to the pipeline,"}, {"start": 178.0, "text": "that is two pipes merged into one."}, {"start": 183.0, "text": "Well, the time pipe can do more than that."}, {"start": 187.0, "text": "The transcript pipe was transformed to have timestamp overlays,"}, {"start": 192.0, "text": "but it does not know about the current time"}, {"start": 195.0, "text": "and therefore cannot scroll correctly."}, {"start": 199.0, "text": "I can connect the same time pipe to form a pipeline of time-synced scrolling text."}, {"start": 206.0, "text": "And now I have dynamic visuals for the background,"}, {"start": 210.0, "text": "and a scrolling transcript for the foreground."}, {"start": 214.0, "text": "Time to connect those two parallel pipes together."}, {"start": 218.0, "text": "And if you zoom out, you realize that every modifier,"}, {"start": 223.0, "text": "every API, is another stage in the pipeline."}, {"start": 226.0, "text": "It just flows."}, {"start": 229.0, "text": "Just like what was shown,"}, {"start": 231.0, "text": "my podcast app contains advanced layout and graphics."}, {"start": 236.0, "text": "I have a full-screen cover art,"}, {"start": 238.0, "text": "applied with shader effect, and time-driven animation."}, {"start": 243.0, "text": "On the other front, I have a time-synced scrollable transcript view,"}, {"start": 247.0, "text": "refined with floating-view attachment!"}, {"start": 251.0, "text": "I will go through each of those and explain how to achieve them,"}, {"start": 255.0, "text": "starting with the cover art."}, {"start": 259.0, "text": "Here's our raw material."}, {"start": 262.0, "text": "A cover art image."}, {"start": 265.0, "text": "The cover art is beautiful but it's going to sit behind the transcript."}, {"start": 270.0, "text": "I soften it with a .blur modifier so it doesn't compete."}, {"start": 276.0, "text": "Now that my cover art is blurred,"}, {"start": 279.0, "text": "next, I will apply some shader magic."}, {"start": 282.0, "text": "You might ask, what is a shader?"}, {"start": 285.0, "text": "And how different is it from writing SwiftUI code?"}, {"start": 290.0, "text": "Let me explain."}, {"start": 292.0, "text": "This icon here starts as vector,"}, {"start": 296.0, "text": "then it was rasterized by GPU to pixels."}, {"start": 301.0, "text": "At this point, I can run a program on GPU called shader"}, {"start": 305.0, "text": "to decide which color to fill in those pixels."}, {"start": 310.0, "text": "The shader function runs in parallel."}, {"start": 313.0, "text": "Each pixel executes independently, with no awareness of its neighbors."}, {"start": 319.0, "text": "Knowing that, it will make perfect sense how Metal shader"}, {"start": 323.0, "text": "can be called from SwiftUI's shader effect APIs."}, {"start": 327.0, "text": "There are three types of shader effects."}, {"start": 330.0, "text": "Each has different method signature, certain parameters are required,"}, {"start": 336.0, "text": "although you can also append additional ones"}, {"start": 338.0, "text": "for the information you want to forward from SwiftUI to shaders."}, {"start": 344.0, "text": "colorEffect works by transforming each pixel's color to a new color,"}, {"start": 349.0, "text": "where each pixel is provided with the pixel position"}, {"start": 353.0, "text": "and the original view's pixel color at that position."}, {"start": 358.0, "text": "You then return a new color based on that information."}, {"start": 363.0, "text": "This is useful for simple effects"}, {"start": 365.0, "text": "like turning a colored image into a black and white one."}, {"start": 372.0, "text": "distortionEffect works differently."}, {"start": 375.0, "text": "Instead of expecting a color on a certain position,"}, {"start": 379.0, "text": "distortionEffectFunction takes the existing position for a new position"}, {"start": 384.0, "text": "that SwiftUI will sample from the original image."}, {"start": 388.0, "text": "There is no pixel color involved,"}, {"start": 391.0, "text": "you tell SwiftUI 'I want this position's color to follow that position's color'."}, {"start": 397.0, "text": "This is useful for geometric effects like the sheer effect shown here."}, {"start": 404.0, "text": "layerEffect is the most flexible."}, {"start": 408.0, "text": "The layerEffectFunction still works per pixel,"}, {"start": 412.0, "text": "but it provides the layer of the entire view,"}, {"start": 415.0, "text": "which allows you to sample adjacent pixels or the entire region."}, {"start": 420.0, "text": "This is useful for effects like blur"}, {"start": 423.0, "text": "where the output pixel color depends on multiple input pixels."}, {"start": 429.0, "text": "For my use case, distortionEffect works,"}, {"start": 433.0, "text": "but layer effect provides the most flexibility."}, {"start": 437.0, "text": "I'll add a layerEffect modifier,"}, {"start": 441.0, "text": "then I will add a shader function called backgroundWarp."}, {"start": 446.0, "text": "For now it just samples from the original layer at the given position,"}, {"start": 451.0, "text": "which gives me back the same image."}, {"start": 454.0, "text": "But now I have a shader function I can build on."}, {"start": 460.0, "text": "With layerEffect, I can sample anywhere from the original view."}, {"start": 465.0, "text": "For example, I can pass a float2 vector to the shader function,"}, {"start": 470.0, "text": "and use it to offset the sample position in the shader."}, {"start": 476.0, "text": "To match with the function parameters,"}, {"start": 479.0, "text": "I now put a float2 vector from the SwiftUI side."}, {"start": 484.0, "text": "Now as I increase the offset, each pixel runs the shader with this offset value"}, {"start": 491.0, "text": "and so they all uniformly and increasingly sample from a distance."}, {"start": 498.0, "text": "And as I decrease offset to zero, the image goes back."}, {"start": 504.0, "text": "Still, because of the uniform offset,"}, {"start": 507.0, "text": "I only get the shifted pixels in a fixed pattern."}, {"start": 511.0, "text": "I need something more organic, something that varies per pixel."}, {"start": 518.0, "text": "For organic variation, I use a NoiseTexture,"}, {"start": 521.0, "text": "a pre-computed image of smooth, random values."}, {"start": 526.0, "text": "This time, on the SwiftUI side,"}, {"start": 529.0, "text": "I pass the view size alongside the NoiseTexture as an image parameter."}, {"start": 535.0, "text": "And on the Metal side, the image arrives as texture2d."}, {"start": 542.0, "text": "Now I am about to show some really metal Metal code."}, {"start": 548.0, "text": "I first use the current pixel position and the size to get the uv value,"}, {"start": 554.0, "text": "which stands for where I am relative to this image,"}, {"start": 559.0, "text": "it allows me to sample textures without an absolute position."}, {"start": 565.0, "text": "Now I'll unpack the NoiseTexture."}, {"start": 569.0, "text": "It has RGB channels,"}, {"start": 571.0, "text": "the red and green channels are interesting"}, {"start": 574.0, "text": "because each one contains a different noise pattern."}, {"start": 578.0, "text": "If I move my uv, I get different red and green values."}, {"start": 583.0, "text": "This pair of constantly changing values happen to be a good fit"}, {"start": 588.0, "text": "for our organic offset in X and Y since it is different per pixel."}, {"start": 595.0, "text": "Now come back to the Metal shader."}, {"start": 598.0, "text": "I create a sampler with repeat mode so it tiles,"}, {"start": 603.0, "text": "then I sample the noise at each pixel's UV position."}, {"start": 607.0, "text": "The red and green channels give me a two-dimensional offset,"}, {"start": 612.0, "text": "which I scaled and added to the position to sample from the original view."}, {"start": 617.0, "text": "Now, the shader twists the image slightly."}, {"start": 623.0, "text": "That was per-pixel variation, but I want something richer."}, {"start": 628.0, "text": "So I experiment,"}, {"start": 630.0, "text": "what if, instead of one noise sample, I do it twice."}, {"start": 635.0, "text": "The first gives me an initial offset."}, {"start": 639.0, "text": "Then I sample the noise again,"}, {"start": 642.0, "text": "but this time at a position shifted by the initial offset,"}, {"start": 647.0, "text": "and just like that,"}, {"start": 648.0, "text": "I get these organic, flowing blobs."}, {"start": 652.0, "text": "This layered noise approach is a well known technique called domain warping."}, {"start": 658.0, "text": "To explore how I did it, download the sample app,"}, {"start": 661.0, "text": "it even has a preview so you can play with the parameters as you want."}, {"start": 667.0, "text": "Now I have a cool shader effect, but it's still frozen."}, {"start": 671.0, "text": "I need to make it move."}, {"start": 673.0, "text": "That's where time comes in."}, {"start": 677.0, "text": "Different from SwiftUI's transaction-based animation,"}, {"start": 681.0, "text": "shaders are stateless."}, {"start": 683.0, "text": "They have no memory of the previous frame, the output relies only on the parameters."}, {"start": 690.0, "text": "So, if I want animation, I need to pass in a value that changes over time."}, {"start": 698.0, "text": "TimelineView is exactly the pipe that I need to connect."}, {"start": 703.0, "text": "With the animation schedule, it fires every frame with a timestamp."}, {"start": 708.0, "text": "I pass that timestamp into the shader,"}, {"start": 711.0, "text": "add it to the position to sample from noise,"}, {"start": 715.0, "text": "and the pattern starts flowing."}, {"start": 720.0, "text": "That was shader animation, driven by time."}, {"start": 724.0, "text": "For my transcript view, I also need to add time to the mix,"}, {"start": 729.0, "text": "so that the current running transcript line"}, {"start": 731.0, "text": "will be highlighted and centered in the scroll view."}, {"start": 737.0, "text": "Here's my transcript."}, {"start": 740.0, "text": "Text views in a LazyVStack inside a ScrollView."}, {"start": 744.0, "text": "Each line is its own view, familiar SwiftUI."}, {"start": 749.0, "text": "Now I need to make it follow the playback state."}, {"start": 754.0, "text": "I use the playback timestamp to determine which line is current."}, {"start": 759.0, "text": "The current line is bold and clear, the rest fades back."}, {"start": 764.0, "text": "And with the onChange modifier to monitor the current line change,"}, {"start": 768.0, "text": "I scroll to keep the current line centered."}, {"start": 773.0, "text": "I have got my time-synced scroll view working."}, {"start": 776.0, "text": "Now, I want to focus on the small timestamp on the current line."}, {"start": 782.0, "text": "Every line has a timestamp in its overlay,"}, {"start": 786.0, "text": "but only the one for the current line is visible."}, {"start": 790.0, "text": "This way, it doesn't interfere with the layout."}, {"start": 793.0, "text": "It's always there, just waiting to be shown."}, {"start": 798.0, "text": "Let's focus on this one row,"}, {"start": 801.0, "text": "a sub view, attached on the edge of its container."}, {"start": 806.0, "text": "How do I get it there?"}, {"start": 807.0, "text": "The offset modifier cannot do it without knowing the size of both views."}, {"start": 814.0, "text": "First, let's talk about alignment."}, {"start": 817.0, "text": "Every view has alignments."}, {"start": 819.0, "text": "Think of it as the point the layout system uses to position the view,"}, {"start": 825.0, "text": "and it is defined by both axes."}, {"start": 829.0, "text": "When I place the sub view in the overlay container,"}, {"start": 832.0, "text": "the layout system aligns them using the default center alignment."}, {"start": 838.0, "text": "Think of it like a pin punching through both views,"}, {"start": 843.0, "text": "so it holds them together at each view's alignment point."}, {"start": 848.0, "text": "I change the overlay's alignment to .bottomLeading."}, {"start": 853.0, "text": "Now the pin goes through the bottom leading point of each view"}, {"start": 858.0, "text": "and they lock together there."}, {"start": 862.0, "text": "Right now, the layout system asks for the bottom leading alignment,"}, {"start": 867.0, "text": "and so the subview returns its bottom leading point to punch through."}, {"start": 873.0, "text": "If I were to explicitly express this in code,"}, {"start": 876.0, "text": "I would write an alignment guide here to mean bottom is bottom."}, {"start": 882.0, "text": "Now, remember the goal is that"}, {"start": 884.0, "text": "the subview's top edge should touch the bottom edge of the container."}, {"start": 891.0, "text": "What if, I tell the subview that,"}, {"start": 894.0, "text": "when the layout system asks about the bottom alignment,"}, {"start": 898.0, "text": "don't use the default one."}, {"start": 900.0, "text": "Instead, I have a custom override"}, {"start": 903.0, "text": "that moves the bottom alignment to the top edge."}, {"start": 908.0, "text": "And now, when the pin comes to punch through,"}, {"start": 911.0, "text": "it follows that point instead."}, {"start": 915.0, "text": "I get the result by just writing a purely semantic override"}, {"start": 919.0, "text": "without manually offsetting the view."}, {"start": 922.0, "text": "There's more to this API."}, {"start": 924.0, "text": "I can define my own custom alignments,"}, {"start": 927.0, "text": "and the closure gives me ViewDimensions"}, {"start": 930.0, "text": "so I can compute point from the view's actual size."}, {"start": 935.0, "text": "Check out the documentation on \"SwiftUI Alignment\" for the full picture."}, {"start": 943.0, "text": "And here it is."}, {"start": 945.0, "text": "The bare-bones transcript view I started with,"}, {"start": 949.0, "text": "now with an animated background driven by a shader and time,"}, {"start": 954.0, "text": "a transcript that scrolls in sync with playback,"}, {"start": 958.0, "text": "and a floating timestamp positioned with alignment guides."}, {"start": 964.0, "text": "All from the simple pipes,"}, {"start": 966.0, "text": "composed together,"}, {"start": 969.0, "text": "and it works across Apple devices."}, {"start": 974.0, "text": "Let's step back."}, {"start": 976.0, "text": "I took a design, broke it down into layers,"}, {"start": 980.0, "text": "and for each layer I found the right API to turn raw data into views."}, {"start": 986.0, "text": "Each stage's output fed the next stage's input."}, {"start": 991.0, "text": "Connecting stages like this is what I called a creative pipeline."}, {"start": 997.0, "text": "But those were the choices I made for this podcast app."}, {"start": 1001.0, "text": "For your own app, the pipeline can get even more creative."}, {"start": 1005.0, "text": "The inputs could have been gyroscope data instead of audio."}, {"start": 1010.0, "text": "The shader could have been a ripple instead of a twist."}, {"start": 1014.0, "text": "The foreground could have been a freeform canvas instead of a scroll view."}, {"start": 1019.0, "text": "Every combination gives you something different."}, {"start": 1022.0, "text": "That's the creative part, the APIs are the same."}, {"start": 1028.0, "text": "What you feed in and how you connect them, that's yours."}, {"start": 1033.0, "text": "So go make it your thing."}, {"start": 1036.0, "text": "Download the sample project and experiment with the shader,"}, {"start": 1040.0, "text": "change the noise, tweak the speed, try a different image."}, {"start": 1044.0, "text": "Look for opportunities in your own app"}, {"start": 1047.0, "text": "where a small visual effect could make a big difference."}, {"start": 1051.0, "text": "And when you start connecting those pipes together,"}, {"start": 1054.0, "text": "you'll be surprised how quickly something simple becomes something advanced."}, {"start": 1064.0, "text": "Thank you for watching,"}, {"start": 1065.0, "text": "and goodbye!"}]}