{"$schema": "https://raw.githubusercontent.com/guitaripod/wwdc-sessions/master/schema/transcript.schema.json", "id": "wwdc2026-341", "language": "eng", "source": "https://developer.apple.com/videos/play/wwdc2026/341", "wordCount": 1938, "segments": [{"start": 7.0, "text": "Hi, I’m Tracy!"}, {"start": 9.0, "text": "I’m an engineer on the Camera Software team."}, {"start": 12.0, "text": "I’m excited to show you how to support"}, {"start": 15.0, "text": "the Center Stage front camera in your iOS app."}, {"start": 19.0, "text": "I love taking selfies!"}, {"start": 21.0, "text": "Whether I’m traveling solo or with a group of friends."}, {"start": 26.0, "text": "Getting the right framing isn’t always easy."}, {"start": 29.0, "text": "If you’re anything like me, you’ve probably reached for a selfie stick"}, {"start": 34.0, "text": "or passed your phone to your friend with the longest arms."}, {"start": 38.0, "text": "The Center Stage front camera on iPhone 17, iPhone Air, and iPhone 17 Pro"}, {"start": 45.0, "text": "takes care of this."}, {"start": 47.0, "text": "It gives you greater flexibility in framing your shots."}, {"start": 51.0, "text": "I'll start with an overview of the Center Stage front camera."}, {"start": 56.0, "text": "Then, I'll cover APIs to support Center Stage for photos."}, {"start": 61.0, "text": "I'll wrap up with how to support Center Stage for video recordings"}, {"start": 65.0, "text": "and video calls."}, {"start": 68.0, "text": "Traditionally, smartphone front camera sensors have a 4x3 aspect ratio,"}, {"start": 75.0, "text": "which limits your framing based on the phone orientation."}, {"start": 79.0, "text": "However, the Center Stage front camera has a square image sensor."}, {"start": 85.0, "text": "This square shape lets you choose any aspect ratio."}, {"start": 90.0, "text": "You can take a portrait selfie, or a landscape one,"}, {"start": 94.0, "text": "without having to rotate your iPhone."}, {"start": 97.0, "text": "This gives you a more secure, one-handed grip."}, {"start": 100.0, "text": "And since the camera is centered, your eye contact feels more natural."}, {"start": 106.0, "text": "This square sensor is also paired with a lens that has a 95 degree field of view,"}, {"start": 113.0, "text": "the widest on any iPhone front camera."}, {"start": 116.0, "text": "This wide field of view helps you better frame group selfies,"}, {"start": 120.0, "text": "stabilize videos, and keep you centered during video calls."}, {"start": 125.0, "text": "Now that you know what the Center Stage front camera can do,"}, {"start": 129.0, "text": "I’ll show you how to bring perfect photo framing experience into your app."}, {"start": 135.0, "text": "The key feature here is Auto Zoom and Auto Rotate."}, {"start": 140.0, "text": "By combining the square sensor, wide field of view,"}, {"start": 144.0, "text": "and automatic face and gaze detection,"}, {"start": 147.0, "text": "the Center Stage front camera smartly adapts its orientation"}, {"start": 152.0, "text": "and adjusts between narrow and wide views."}, {"start": 156.0, "text": "Let me show you how that works."}, {"start": 158.0, "text": "Here, I’m taking a selfie outside."}, {"start": 161.0, "text": "As my friend Karen joins me, the frame zooms out."}, {"start": 166.0, "text": "As more friends get into the shot, the frame rotates to include everyone."}, {"start": 172.0, "text": "To build this experience in your app,"}, {"start": 174.0, "text": "I'll first review the capture session setup,"}, {"start": 177.0, "text": "and then dive into two APIs:"}, {"start": 180.0, "text": "dynamic aspect ratio, and smart framing monitor."}, {"start": 184.0, "text": "Lastly, I’ll cover sensor orientation compensation."}, {"start": 189.0, "text": "Starting with a typical photo capture setup."}, {"start": 193.0, "text": "First, create an AVCaptureSession."}, {"start": 196.0, "text": "Find the Center Stage front camera, represented in the API as AVCaptureDevice"}, {"start": 203.0, "text": "with the .builtInUltraWideCamera device type."}, {"start": 207.0, "text": "Create an AVCaptureDeviceInput for that camera."}, {"start": 212.0, "text": "To get a camera preview, add an AVCaptureVideoPreviewLayer"}, {"start": 217.0, "text": "to the capture session."}, {"start": 220.0, "text": "Also, add an AVCapturePhotoOutput to receive photos."}, {"start": 225.0, "text": "As you add inputs and outputs, the capture session implicitly forms"}, {"start": 230.0, "text": "AVCaptureConnections between those with compatible media types."}, {"start": 236.0, "text": "With that setup in place, I'll move to dynamic aspect ratio,"}, {"start": 241.0, "text": "the building block for both manual and automatic framing."}, {"start": 245.0, "text": "Starting in iOS 26, AVCaptureDevice includes a dynamicAspectRatio property."}, {"start": 253.0, "text": "When you set this property,"}, {"start": 255.0, "text": "the capture device crops your chosen aspect ratio"}, {"start": 259.0, "text": "out of the square image sensor as shown earlier,"}, {"start": 263.0, "text": "without rebuilding the capture session or interrupting preview."}, {"start": 268.0, "text": "The switch is seamless and quick."}, {"start": 271.0, "text": "Here is a table that lists example formats"}, {"start": 274.0, "text": "that support dynamic aspect ratio."}, {"start": 278.0, "text": "It requires the front-facing .builtInUltraWideCamera"}, {"start": 282.0, "text": "as mentioned before."}, {"start": 284.0, "text": "It's only supported on the square formats,"}, {"start": 286.0, "text": "with resolutions ranging from 1280 up to 4032."}, {"start": 293.0, "text": "There are five aspect ratios available:"}, {"start": 296.0, "text": "3x4, 4x3,"}, {"start": 298.0, "text": "9x16, 16x9, and 1x1."}, {"start": 303.0, "text": "Be aware that the 4032 photo format only supports 3x4 and 4x3"}, {"start": 311.0, "text": "because these two aspect ratios give you the highest resolution for photos."}, {"start": 317.0, "text": "With those format requirements in mind,"}, {"start": 320.0, "text": "let me show you how to implement a Tap to Rotate button"}, {"start": 324.0, "text": "using dynamic aspect ratio to switch between orientations."}, {"start": 329.0, "text": "First, select the Center Stage front camera"}, {"start": 333.0, "text": "by creating an AVCaptureDevice.DiscoverySession."}, {"start": 337.0, "text": "Specify .builtInUltraWideCamera as the only device type you’re interested in,"}, {"start": 344.0, "text": "and .front as a position."}, {"start": 347.0, "text": "Since you’ve only asked for a single device,"}, {"start": 350.0, "text": "you can just get the first element in the discovery session’s device array."}, {"start": 356.0, "text": "Find a format that supports your desired aspect ratio."}, {"start": 360.0, "text": "Here, I’ll use 4x3 as an example."}, {"start": 364.0, "text": "Check each format's supportedDynamicAspectRatios property."}, {"start": 369.0, "text": "This returns an array of aspect ratios the format supports."}, {"start": 374.0, "text": "For simplicity, I'm picking the first match."}, {"start": 378.0, "text": "After locking the device for configuration,"}, {"start": 381.0, "text": "set your chosen format as the activeFormat."}, {"start": 385.0, "text": "Next, set the dynamic aspect ratio to 4x3."}, {"start": 391.0, "text": "This call returns the timestamp of the first buffer"}, {"start": 395.0, "text": "when the change takes effect."}, {"start": 398.0, "text": "With that code in place, your app now supports Tap to Rotate,"}, {"start": 403.0, "text": "letting you change orientation with a single tap."}, {"start": 407.0, "text": "Next up is the smart framing monitor API."}, {"start": 410.0, "text": "It works alongside dynamic aspect ratio"}, {"start": 414.0, "text": "to automatically adjust framing as people move in and out of the scene."}, {"start": 419.0, "text": "Starting in iOS 26, AVCaptureDevice includes"}, {"start": 424.0, "text": "an AVCaptureSmartFramingMonitor object."}, {"start": 428.0, "text": "This monitor gives periodic framing recommendations"}, {"start": 432.0, "text": "based on automatic face and gaze detection."}, {"start": 436.0, "text": "Each recommendation contains an aspect ratio and a zoom factor"}, {"start": 441.0, "text": "that your app can apply or ignore."}, {"start": 445.0, "text": "Since this monitor is designed for photo capture,"}, {"start": 448.0, "text": "it only provides recommendations when you use the 4032 photo format."}, {"start": 454.0, "text": "Now I'll walk through how to set up the monitor in code."}, {"start": 459.0, "text": "With the ultra-wide front camera already selected"}, {"start": 463.0, "text": "from the dynamic aspect ratio setup,"}, {"start": 466.0, "text": "you can go ahead and find a format that supports"}, {"start": 470.0, "text": "both your desired aspect ratios and smart framing."}, {"start": 475.0, "text": "Once you've found the format, lock the device for configuration"}, {"start": 479.0, "text": "and set the activeFormat."}, {"start": 483.0, "text": "Next, get the smartFramingMonitor from the camera."}, {"start": 487.0, "text": "By default, the monitor provides no recommendations."}, {"start": 492.0, "text": "Here, I’m setting enabledFramings to all of the supportedFramings."}, {"start": 498.0, "text": "But you can limit this to a subset of supported framings,"}, {"start": 502.0, "text": "such as just the 4x3 aspect ratio with narrow and wide zoom factors."}, {"start": 509.0, "text": "After you configure your monitor,"}, {"start": 511.0, "text": "key-value observe the monitor’s recommendedFraming property."}, {"start": 516.0, "text": "As the monitor recommends a new framing,"}, {"start": 519.0, "text": "apply the recommendation by setting the camera’s dynamic aspect ratio"}, {"start": 525.0, "text": "and video zoom factor."}, {"start": 527.0, "text": "For a smooth preview transition, set the aspectRatio first,"}, {"start": 532.0, "text": "then the zoomFactor."}, {"start": 535.0, "text": "Now you can start the monitor at any time,"}, {"start": 539.0, "text": "including while your AVCaptureSession is running."}, {"start": 543.0, "text": "If your UI allows people to turn off automatic framing,"}, {"start": 548.0, "text": "unregister your key-value observation and stopMonitoring."}, {"start": 554.0, "text": "Your app now supports Auto Zoom and Auto Rotate,"}, {"start": 558.0, "text": "giving people the best framing recommendations for group selfies."}, {"start": 564.0, "text": "One last topic for photo capture, sensor orientation compensation."}, {"start": 569.0, "text": "Since the earliest iPhone front cameras,"}, {"start": 572.0, "text": "the sensor has been mounted in Landscape Left orientation."}, {"start": 577.0, "text": "If you take a selfie while holding the phone in Portrait orientation,"}, {"start": 582.0, "text": "the image buffer is delivered to the photo output"}, {"start": 585.0, "text": "in the native sensor orientation."}, {"start": 589.0, "text": "The buffer carries an EXIF orientation metadata tag,"}, {"start": 594.0, "text": "indicating it should be rotated 270 degrees on playback."}, {"start": 600.0, "text": "However, on iPhone 17, iPhone Air, and iPhone 17 Pro,"}, {"start": 605.0, "text": "the Center Stage front camera sensor is mounted in Portrait orientation."}, {"start": 612.0, "text": "If your app relies on rotation values that worked before,"}, {"start": 616.0, "text": "photos may appear sideways or upside down."}, {"start": 621.0, "text": "To handle this, AVCapturePhotoOutput"}, {"start": 625.0, "text": "automatically applies sensor orientation compensation by default."}, {"start": 630.0, "text": "It physically rotates photos"}, {"start": 633.0, "text": "and updates EXIF metadata before delivering them to your app."}, {"start": 638.0, "text": "The resulting photos are in landscape left orientation,"}, {"start": 642.0, "text": "just like on previous iPhone front cameras."}, {"start": 647.0, "text": "You can keep using the same rotation values you used before."}, {"start": 652.0, "text": "Keep in mind that this compensation is only applied to HEIC, JPEG,"}, {"start": 658.0, "text": "and uncompressed processed photos."}, {"start": 661.0, "text": "It is never applied to Bayer RAW or Apple ProRAW captures."}, {"start": 666.0, "text": "Starting in iOS 26, you can control this behavior"}, {"start": 671.0, "text": "with the cameraSensorOrientation- CompensationEnabled property."}, {"start": 676.0, "text": "If you use AVCapturePhotoOutput,"}, {"start": 679.0, "text": "test with compensation off for best performance,"}, {"start": 684.0, "text": "and make sure photo orientation remains correct."}, {"start": 688.0, "text": "To learn more about image rotation handling with AVCaptureRotationCoordinator,"}, {"start": 695.0, "text": "check out the \"Support external cameras in your iPadOS app”"}, {"start": 700.0, "text": "from WWDC 2023."}, {"start": 703.0, "text": "Now, on to Center Stage for videos."}, {"start": 707.0, "text": "I’ll cover video recordings, and video calls."}, {"start": 713.0, "text": "The dynamic aspect ratio API also works great for recordings."}, {"start": 718.0, "text": "You can Tap to Rotate for a wider view."}, {"start": 722.0, "text": "However, QuickTime movie tracks require all samples to have the same dimensions."}, {"start": 728.0, "text": "Video recording will need to stop"}, {"start": 730.0, "text": "if you change the dynamic aspect ratio during capture."}, {"start": 735.0, "text": "Here's the photo capture setup I walked through before."}, {"start": 739.0, "text": "To modify it for video recording,"}, {"start": 741.0, "text": "you can use AVCaptureMovieFileOutput instead of AVCapturePhotoOutput."}, {"start": 748.0, "text": "In such a setup, recording will stop automatically"}, {"start": 752.0, "text": "when the aspect ratio is changed."}, {"start": 755.0, "text": "You can also use AVCaptureVideoDataOutput with AVAssetWriter to record videos."}, {"start": 763.0, "text": "The setDynamicAspectRatio completion timestamp"}, {"start": 767.0, "text": "allows you to end the current recording and start a new one"}, {"start": 772.0, "text": "with the updated aspect ratio."}, {"start": 775.0, "text": "Video recordings also benefit from cinematic stabilization modes:"}, {"start": 781.0, "text": "cinematicExtended and cinematicExtendedEnhanced."}, {"start": 786.0, "text": "Both are face-aware on the Center Stage front camera."}, {"start": 791.0, "text": "They prioritize keeping the subject stable over the background."}, {"start": 796.0, "text": "Next, I'll talk about Center Stage for video calls."}, {"start": 800.0, "text": "Here, I’m on a FaceTime call."}, {"start": 803.0, "text": "When a friend joins me on the trail,"}, {"start": 805.0, "text": "the camera automatically widens to include both of us."}, {"start": 810.0, "text": "Center Stage is already supported if your video conferencing app"}, {"start": 815.0, "text": "uses the Voice over IP background mode"}, {"start": 818.0, "text": "to keep calls connected when the phone is locked."}, {"start": 822.0, "text": "People can directly turn on Center Stage from Control Center's Video Effects menu."}, {"start": 829.0, "text": "If your app is not using Voice over IP background mode,"}, {"start": 833.0, "text": "you can still adopt the Center Stage API."}, {"start": 837.0, "text": "This is available on the iPhone front camera,"}, {"start": 840.0, "text": "starting with iPhone 17, iPhone Air, and iPhone 17 Pro."}, {"start": 846.0, "text": "Like all system-wide video effects,"}, {"start": 848.0, "text": "such as Portrait, Studio Light, and Gestures,"}, {"start": 853.0, "text": "Center Stage is enabled per process."}, {"start": 856.0, "text": "Once active, it applies to any supported camera in your app."}, {"start": 862.0, "text": "Here's the photo capture setup again."}, {"start": 865.0, "text": "For video calls, the output is different."}, {"start": 869.0, "text": "Video conferencing apps typically use a video data output"}, {"start": 873.0, "text": "to receive video buffers as a stream."}, {"start": 877.0, "text": "Display, encoding, and transmission are all handled at the app layer."}, {"start": 883.0, "text": "With this setup in mind, here's how to enable Center Stage."}, {"start": 888.0, "text": "First, find a supported format."}, {"start": 891.0, "text": "Set it as the active one after locking the device."}, {"start": 895.0, "text": "By default, people toggle Center Stage through Control Center, not your app."}, {"start": 901.0, "text": "Before turning it on, set a control mode: cooperative or app."}, {"start": 906.0, "text": "Cooperative mode lets people also control it from a button in your app."}, {"start": 912.0, "text": "Then, set isCenterStageEnabled to true."}, {"start": 916.0, "text": "With that, Center Stage is active."}, {"start": 920.0, "text": "Framing now automatically adjusts to keep all people centered."}, {"start": 926.0, "text": "To learn more about the Center Stage API when it was first added for iPad,"}, {"start": 931.0, "text": "check out \"What’s New in camera capture\" from WWDC 2021."}, {"start": 938.0, "text": "Beyond Center Stage, there's one more way to improve your video call experience."}, {"start": 944.0, "text": "The Center Stage front camera supports"}, {"start": 947.0, "text": "a real-time, low-latency stabilization mode starting in iOS 26."}, {"start": 953.0, "text": "It's off by default."}, {"start": 954.0, "text": "To turn it on, set the preferredVideoStabilizationMode"}, {"start": 959.0, "text": "on the AVCaptureConnection to lowLatency."}, {"start": 962.0, "text": "Here’s a side-by-side comparison of the same video call"}, {"start": 967.0, "text": "with low latency stabilization on and off."}, {"start": 971.0, "text": "In the left video, stabilization is off."}, {"start": 974.0, "text": "There’s noticeable shake as I walk."}, {"start": 978.0, "text": "In the right video, stabilization is on."}, {"start": 981.0, "text": "The footage is much more stable."}, {"start": 985.0, "text": "With that, you have everything you need"}, {"start": 987.0, "text": "to support the Center Stage front camera in your iOS app."}, {"start": 992.0, "text": "Here are some next steps to further improve your app’s framing experience."}, {"start": 998.0, "text": "First, incorporate framing controls into your app"}, {"start": 1002.0, "text": "to support orientation switching, toggles for Auto Zoom and Auto Rotate,"}, {"start": 1009.0, "text": "and Center Stage activation."}, {"start": 1012.0, "text": "Second, optimize front camera performance."}, {"start": 1016.0, "text": "Test with sensor orientation compensation turned off."}, {"start": 1021.0, "text": "Make sure your app handles photo rotation correctly."}, {"start": 1025.0, "text": "Also consider supporting 18-megapixel photo capture"}, {"start": 1030.0, "text": "for exceptional resolution and details."}, {"start": 1033.0, "text": "For best practices, check out \"Implement high resolution photo capture”"}, {"start": 1039.0, "text": "from WWDC 2026."}, {"start": 1043.0, "text": "The Center Stage front camera has completely changed the way I take selfies"}, {"start": 1048.0, "text": "and make video calls on my iPhone."}, {"start": 1051.0, "text": "I look forward to seeing how it transforms your app."}, {"start": 1055.0, "text": "Thank you for watching."}]}