{"$schema": "https://raw.githubusercontent.com/guitaripod/wwdc-sessions/master/schema/transcript.schema.json", "id": "wwdc2026-312", "language": "eng", "source": "https://developer.apple.com/videos/play/wwdc2026/312", "wordCount": 1735, "segments": [{"start": 7.0, "text": "Hi, my name is Leo Formaggio,"}, {"start": 9.0, "text": "and I'm an engineer on the Media Frameworks team."}, {"start": 13.0, "text": "Media has such an important presence in our daily lives."}, {"start": 16.0, "text": "It's a podcast on the drive home."}, {"start": 18.0, "text": "A high-energy playlist during a workout."}, {"start": 21.0, "text": "Or watching a movie on a long flight."}, {"start": 24.0, "text": "Whether we are spending time by ourselves, or connecting with people,"}, {"start": 28.0, "text": "media is always around us."}, {"start": 31.0, "text": "On iPhone, it appears right on the lock screen,"}, {"start": 34.0, "text": "Control Center, and in the Dynamic Island."}, {"start": 39.0, "text": "When the iPhone is set down and charging, it's glanceable in StandBy."}, {"start": 44.0, "text": "When getting in the car, it appears front and center in CarPlay."}, {"start": 48.0, "text": "That's the system now-playing experience, and it's available on all Apple platforms."}, {"start": 54.0, "text": "Including Apple Watch, Apple Vision Pro, and Apple TV!"}, {"start": 60.0, "text": "I'll show you how NowPlaying framework"}, {"start": 62.0, "text": "makes it easy to bring media from your app into the system."}, {"start": 67.0, "text": "I'll start with the media sessions API,"}, {"start": 70.0, "text": "showing how to surface your content in the system now-playing experience."}, {"start": 76.0, "text": "Then, I'll explain how to bring content playing on other devices"}, {"start": 80.0, "text": "into the system using remote media sessions."}, {"start": 85.0, "text": "And finally, I'll go over how Media Sharing Extensions"}, {"start": 89.0, "text": "simplify playing media from iPhone to other devices."}, {"start": 94.0, "text": "I'll use the example of an app I developed."}, {"start": 97.0, "text": "It plays ambient sounds to help people focus and relax."}, {"start": 101.0, "text": "From my app, I can select different sounds,"}, {"start": 104.0, "text": "and I can also pause and resume the audio."}, {"start": 108.0, "text": "To surface my content in the system,"}, {"start": 110.0, "text": "I used the media sessions API from NowPlaying."}, {"start": 114.0, "text": "Next, I'll show you how I did it."}, {"start": 116.0, "text": "Here's my PlayerModel –"}, {"start": 118.0, "text": "an @Observable class with a reference to my app's audio engine,"}, {"start": 122.0, "text": "and a property that tracks which sound is currently playing."}, {"start": 126.0, "text": "The MediaSessionRepresentable protocol"}, {"start": 129.0, "text": "is like a contract between my app and the system."}, {"start": 132.0, "text": "When PlayerModel conforms to it, the system will be able to understand"}, {"start": 137.0, "text": "what my app is playing – and how to handle media interactions, like skip or pause."}, {"start": 143.0, "text": "Each session representation needs a unique identifier."}, {"start": 147.0, "text": "The content property is used to describe the sound that's playing."}, {"start": 151.0, "text": "NowPlaying offers content-specific types, like Music, Podcast, and MovieContent,"}, {"start": 158.0, "text": "to describe what kind of media your app is playing."}, {"start": 161.0, "text": "GenericContent is a good fit for my use case, so I used that."}, {"start": 165.0, "text": "Each content is identified by the current sound.id."}, {"start": 170.0, "text": "I used the sound.name as the content title"}, {"start": 172.0, "text": "and a short sound.description as the subtitle."}, {"start": 176.0, "text": "Media type can be either .audio or .video, but my app only plays audio."}, {"start": 182.0, "text": "I set the duration to .continuous,"}, {"start": 185.0, "text": "because my app plays ambient sounds indefinitely."}, {"start": 188.0, "text": "I provide an Artwork with an async closure."}, {"start": 191.0, "text": "The system calls it whenever it needs an image at a specific size."}, {"start": 196.0, "text": "Now, I'll show you what this looks like on the lock screen."}, {"start": 200.0, "text": "The sound name and description appear along with the artwork."}, {"start": 206.0, "text": "The PlaybackSnapshot property is used to display the current playback state."}, {"start": 211.0, "text": "Since ambient sounds are continuous,"}, {"start": 214.0, "text": "I just need to indicate if content isPlaying."}, {"start": 217.0, "text": "For content with a defined duration,"}, {"start": 219.0, "text": "you should also specify an elapsedTime parameter in the snapshot."}, {"start": 223.0, "text": "Through the commands property, I define all the actions my app supports."}, {"start": 228.0, "text": "Each command has a closure"}, {"start": 230.0, "text": "that the system calls when someone performs that action."}, {"start": 233.0, "text": "For example, when I tap the pause button on the lock screen,"}, {"start": 237.0, "text": "the pause command closure is called, and I can pause the player."}, {"start": 242.0, "text": "Notice the button has changed to reflect the paused state."}, {"start": 246.0, "text": "Now, if I tap play on the phone,"}, {"start": 248.0, "text": "the play command closure is called, and I resume the player."}, {"start": 252.0, "text": "Similarly, when I tap the next button on the lock screen,"}, {"start": 256.0, "text": "the next command closure is called."}, {"start": 258.0, "text": "I can skip to the next sound,"}, {"start": 260.0, "text": "and the lock screen updates to reflect the new content."}, {"start": 264.0, "text": "After I adopted MediaSessionRepresentable,"}, {"start": 267.0, "text": "I had to do one more thing to make my content available to the system."}, {"start": 271.0, "text": "MediaSession is what connects the session representation with the system."}, {"start": 276.0, "text": "I initialize it with my PlayerModel,"}, {"start": 278.0, "text": "in the same place where I set up my audio engine."}, {"start": 281.0, "text": "Once that's done, MediaSession starts observing the model,"}, {"start": 285.0, "text": "keeping the now-playing surfaces up to date automatically."}, {"start": 289.0, "text": "That's how I integrated my app's content with the system now-playing experience"}, {"start": 293.0, "text": "using media sessions."}, {"start": 296.0, "text": "For more information, check out the article \"Publishing Media Sessions\""}, {"start": 300.0, "text": "on Apple Developer Documentation."}, {"start": 304.0, "text": "In addition to playing on iPhone,"}, {"start": 306.0, "text": "I made my audio engine available on smart speakers,"}, {"start": 309.0, "text": "which can be controlled by my app."}, {"start": 312.0, "text": "From my app's device picker menu, I choose the speaker I want to control."}, {"start": 317.0, "text": "I tap the Living Room Speaker to start controlling it."}, {"start": 320.0, "text": "And, through a web server,"}, {"start": 322.0, "text": "my app connects to the selected speaker to request the playback state,"}, {"start": 326.0, "text": "and send commands."}, {"start": 328.0, "text": "To surface the content playing on that speaker in the system,"}, {"start": 332.0, "text": "I used the remote media sessions API."}, {"start": 335.0, "text": "This API uses an app extension and push notifications"}, {"start": 339.0, "text": "to receive updates about the speaker."}, {"start": 342.0, "text": "I'll show you what this interaction looks like."}, {"start": 345.0, "text": "When someone interacts with a speaker,"}, {"start": 348.0, "text": "the speaker communicates the state is changed to the server."}, {"start": 351.0, "text": "The server then uses Apple Push Notification service, APNs,"}, {"start": 356.0, "text": "to send a push notification to the iPhone with the updated state."}, {"start": 360.0, "text": "The system launches the app extension with the updated state"}, {"start": 363.0, "text": "from the push notification payload."}, {"start": 366.0, "text": "The app extension then provides the system"}, {"start": 368.0, "text": "with an updated representation of that session."}, {"start": 371.0, "text": "For more information on sending push notifications with APNs,"}, {"start": 376.0, "text": "check out the article \"Setting up a remote notification server\""}, {"start": 380.0, "text": "on developer.apple.com"}, {"start": 383.0, "text": "When the interaction originates from iPhone system UI,"}, {"start": 386.0, "text": "the system calls a command handler in the app extension."}, {"start": 390.0, "text": "The app extension sends the command to the server."}, {"start": 393.0, "text": "And the server notifies the speaker, which reacts to the change."}, {"start": 397.0, "text": "Here is how I adopted remote media sessions in my app."}, {"start": 402.0, "text": "First I created an app extension"}, {"start": 404.0, "text": "conforming to the RemoteMediaSessionExtension protocol."}, {"start": 408.0, "text": "To set it up, I used NowPlaying's RemoteMediaSessionExtensionConfiguration"}, {"start": 414.0, "text": "and the remote-media extensionPoint identifier."}, {"start": 417.0, "text": "The session(:) method is called by the system whenever it needs to interact"}, {"start": 422.0, "text": "with a remote session representation,"}, {"start": 424.0, "text": "for example, to update the user interface, or to handle an interaction."}, {"start": 430.0, "text": "Here, I can use the RemotePlayerState to create my model, and return it."}, {"start": 436.0, "text": "With the app extension configured,"}, {"start": 438.0, "text": "I'll show how I used my model to represent a Remote Media Session."}, {"start": 443.0, "text": "This is my RemotePlayerModel."}, {"start": 445.0, "text": "It's an @Observable class with a reference to ServerClient,"}, {"start": 449.0, "text": "the class I use to communicate with my server."}, {"start": 452.0, "text": "It also keeps track of the server state."}, {"start": 455.0, "text": "I'll use this as a foundation to build my Remote Media Session representation."}, {"start": 461.0, "text": "Each Remote Media Session representation needs a unique identifier."}, {"start": 465.0, "text": "I used the sessionID from my server state."}, {"start": 469.0, "text": "The content property is used to describe the sound playing on the speaker."}, {"start": 473.0, "text": "Once again, I used GenericContent by giving it the sound identifier,"}, {"start": 478.0, "text": "the sound name and the sound description."}, {"start": 482.0, "text": "The media type is .audio and the duration is .continuous."}, {"start": 487.0, "text": "I provided an Artwork object that loads an image for the current sound."}, {"start": 493.0, "text": "The server state indicates if the speaker isPlaying."}, {"start": 496.0, "text": "I can use that to create a PlaybackSnapshot with the corresponding state."}, {"start": 501.0, "text": "Since I'm controlling playback on a remote device,"}, {"start": 505.0, "text": "each command closure sends a request to the server with the corresponding action."}, {"start": 510.0, "text": "For example, if I tap play on iPhone, the play command closure is called."}, {"start": 516.0, "text": "I send a play request to my server, which resumes playback on the speaker."}, {"start": 521.0, "text": "Similarly, when I tap the next button, a request is sent to the server,"}, {"start": 526.0, "text": "and the speaker moves to the next sound."}, {"start": 529.0, "text": "So far, the adoption of RemoteMediaSessionRepresentable"}, {"start": 533.0, "text": "feels very similar to what we saw in media sessions for local playback."}, {"start": 538.0, "text": "Next, I'll cover the remaining properties and methods"}, {"start": 541.0, "text": "that are specific to remote sessions."}, {"start": 544.0, "text": "The devices property tells the system about devices playing in that session."}, {"start": 549.0, "text": "I map my server's device list into MediaDevice values."}, {"start": 554.0, "text": "Each one needs a unique identifier that is stable across different sessions."}, {"start": 559.0, "text": "I provide the name of the device, a device type, like .speaker in my case,"}, {"start": 565.0, "text": "and a list of capabilities, such as the device's volume control type."}, {"start": 570.0, "text": "This is what that looks like in ControlCenter."}, {"start": 573.0, "text": "The device name appears along with the volume level."}, {"start": 578.0, "text": "When I change the volume using the system volume slider,"}, {"start": 581.0, "text": "the volume change closure is called with the updated volume level."}, {"start": 586.0, "text": "Here, I can send a volume change request to my server."}, {"start": 591.0, "text": "The update(:) function is called"}, {"start": 593.0, "text": "when a push notification is received with a new state."}, {"start": 596.0, "text": "For example, when the content changes on the speaker."}, {"start": 599.0, "text": "RemotePlayerState is a struct I defined,"}, {"start": 602.0, "text": "that conforms to RemoteMediaSessionAttributes."}, {"start": 606.0, "text": "It represents my server state and the push notification payload."}, {"start": 610.0, "text": "Here, I update my state variable with the new data."}, {"start": 614.0, "text": "Because my model is observable, NowPlaying detects the change"}, {"start": 618.0, "text": "and updates the system automatically."}, {"start": 621.0, "text": "And that's how I integrated my app's remote media session with the system."}, {"start": 626.0, "text": "For more information,"}, {"start": 627.0, "text": "check out the article \"Publishing remote media sessions\"."}, {"start": 631.0, "text": "I also want to talk about Media Sharing Extensions –"}, {"start": 634.0, "text": "a set of APIs for playing media from iPhone to other speakers and TVs,"}, {"start": 640.0, "text": "all through a unified system interface."}, {"start": 643.0, "text": "Media Sharing Extensions allow you to use the system device picker"}, {"start": 648.0, "text": "for all the media protocols your app supports."}, {"start": 651.0, "text": "This simplifies media device selection in your app,"}, {"start": 655.0, "text": "and the selection is reflected on system surfaces, like Control Center."}, {"start": 660.0, "text": "Traditionally, supporting a media protocol"}, {"start": 663.0, "text": "meant embedding its SDK into your app bundle."}, {"start": 667.0, "text": "With Media Sharing Extensions,"}, {"start": 669.0, "text": "the protocol implementations live outside your app"}, {"start": 672.0, "text": "and are managed by the system."}, {"start": 674.0, "text": "Your app can focus on the media content rather than the playback technology."}, {"start": 680.0, "text": "As more protocols become available,"}, {"start": 682.0, "text": "apps built with Media Sharing Extensions can use them without adopting another SDK."}, {"start": 689.0, "text": "That covered how to bring local and remote media sessions"}, {"start": 693.0, "text": "into the system now-playing experience using NowPlaying framework,"}, {"start": 698.0, "text": "and how Media Sharing Extensions can simplify sending media to other devices."}, {"start": 704.0, "text": "For apps that play media locally or control playback on remote devices –"}, {"start": 708.0, "text": "adopt NowPlaying to bring your content into the Lock Screen,"}, {"start": 712.0, "text": "Control Center, and beyond."}, {"start": 715.0, "text": "It's a straightforward integration that gives people control over media"}, {"start": 719.0, "text": "even outside your app."}, {"start": 721.0, "text": "Learn more about Media Sharing Extensions."}, {"start": 724.0, "text": "They let your app use the system media device picker"}, {"start": 728.0, "text": "and expand your reach when playing media to other devices."}, {"start": 732.0, "text": "For more information about Media Sharing Extensions,"}, {"start": 735.0, "text": "check out the article \"Routing media to third-party devices\"."}, {"start": 740.0, "text": "I can't wait to see your app extended to the system now-playing experience."}, {"start": 745.0, "text": "Thank you for watching, and blue skies!"}]}