# YouTube migration MediaCMS can pull video in from YouTube: the file, the title, the description, the tags, and an English subtitle if the video has one. Give it video urls, a playlist or a channel and it fetches them. Only superusers can see or use it. The entry point is **Migrations** in the top right menu. This is not like the Kaltura migration and does not try to be. There is no account to sign in to, no catalogue of users and categories to walk, and nothing to recreate. A YouTube migration is a list of things to fetch and one MediaCMS user to give them to. ## What it needs installed yt-dlp, which does all the work of resolving urls and fetching files: ```bash pip install -r requirements.txt ``` It is pinned like everything else. **Expect to upgrade it.** YouTube changes how its pages work every few weeks, and when it does, yt-dlp stops being able to read them until a new release lands. A migration that suddenly fails with an extraction error is almost always that, not a bug here: ```bash pip install -U yt-dlp ``` ffmpeg is already required by MediaCMS for everything else it does, and yt-dlp uses the same binary. ## Setting one up 1. Go to `/migrations`, press **New migration** and pick YouTube. 2. Paste what you want, one per line, into **Videos, playlists or channels**: ``` https://www.youtube.com/watch?v=aqz-KE-bpKQ aqz-KE-bpKQ https://www.youtube.com/playlist?list=PL6B3937A5D230E335 https://www.youtube.com/@BlenderOfficial/videos ``` A bare video id works. A playlist or a channel is expanded to the videos in it, in the order it lists them, and a video named twice is fetched once. 3. Press **Test connection**. It resolves each line and reports how many videos it found, without downloading anything. 4. Set the owner and the options. 5. **Save and start migration**, or schedule it for later on the **Migration scheduling** tab. ## When YouTube will not answer From a laptop, public video needs no credentials. From a server, expect to need cookies: **YouTube answers a request from a datacentre address with a bot check** whether the video is public or not, and no amount of retrying changes it. **Cookies file contents** is the only field that speaks to this. It is also needed for private, members-only or age-gated video. It is encrypted at rest and the API never returns it. How the jar is exported decides whether it works: 1. A **separate browser profile**, or a private window with the extension allowed in it. 2. Sign in there with an account you can afford to hand over: a jar is a live session. 3. Export with a cookies.txt extension, in Netscape format. 4. Paste it in, save, **Test connection**. 5. Then leave that profile alone. Do not browse YouTube in it, and do not log out. Step 5 decides it. `__Secure-1PSIDTS` and `__Secure-3PSIDTS` rotate every few hours and stay bound to the session that exported them, so a jar from a browser that keeps running is superseded within minutes. YouTube then answers **"The page needs to be reloaded"** — which is why a jar with all 22 cookies and expiry dates months away can still break requests that would have succeeded without it. A migration tries anonymously first for that reason. When neither works, the address is the problem and the portal cannot route around it. Reproduce it over ssh with the call the migration makes: ```bash yt-dlp --flat-playlist --skip-download --print id "https://www.youtube.com/watch?v=VIDEO_ID" ``` A bot check there, with no cookies, is YouTube refusing that machine. What is left is to fetch from somewhere it is not challenged: another host, a proxy in front of yt-dlp, or `--cookies` outside the app. yt-dlp moves fast on this, so update it before concluding anything. ## Options | Option | Default | Effect | | --- | --- | --- | | Owner | `admin` | Every imported video is given to this existing MediaCMS user. | | Import captions | on | The English subtitle, if the video has one. | | Preserve views | **off** | Copy the YouTube view count onto the imported video. | | Attempt to skip transcoding | on | Fetch one rendition per encoding profile and file them as they are, instead of re-encoding. See [Renditions](#renditions). | **Uploader names do not become accounts.** A channel is not a person on your portal, and inventing one user per YouTube uploader would leave you with accounts nobody can sign in to. So everything a migration brings in belongs to the one owner you pick. **Captions are English only.** YouTube offers automatically generated captions in over 150 languages, and taking them would attach 150 subtitle rows to every video while saying nothing the video does not already say. Only a subtitle the uploader actually provided is taken, and only `en`. **Nothing else is copied.** No thumbnail, because MediaCMS makes its own, along with the sprites and the hover preview. No upload date, no licence, no category, no channel name. ## Where the file comes from **YouTube never serves the uploader's original.** It serves its own transcodes and nothing else, so a video imported from YouTube is a re-encode by definition. What lands in MediaCMS as the original file is the best rendition YouTube would give us, not the master. It also does not serve a file with video and audio in it. Every stream is one or the other, so a playable file is always two joined together. yt-dlp does that with ffmpeg, which is a container change rather than a re-encode, so it costs bandwidth and almost no processing. ## Renditions With **Attempt to skip transcoding** on, one rendition is fetched for each encoding profile your portal has. YouTube already holds an h264 stream at those sizes, so fetching one is cheaper than encoding it, and nothing is transcoded at all. A portal with the usual six profiles gets something like: ``` 720 filed as h264-720 360 filed as h264-360 144 filed as h264-144 480 filed as h264-480 240 filed as h264-240 ``` The lower renditions are small, so a whole ladder costs roughly a third more bandwidth than the top one on its own. Which streams get picked matters. YouTube offers its best quality in VP9 and AV1, which this portal's profiles and its player are not built around, so the migration asks for **h264 in mp4**. The cost is a ceiling: above roughly 1080p only the newer codecs are offered, so a 4K source arrives at its best h264 size and no higher. Turn the option off to fetch only the best stream and have MediaCMS encode its own ladder from it. You get the same profiles, at the cost of the encoding time. ## How a rendition is labelled Sizes are read as the **short side**, so `1280x720` and `720x1280` are both 720. A vertical video ladders exactly as a landscape one does, and reading its height instead would file a 720 as a 1080 and then skip its best stream for being too tall. Each fetched file is filed under the nearest profile **at or below** its size, never above. An 818p file is labelled 720p. That is deliberate. A file is allowed to be better than its label and never worse, and the HLS quality menu snaps the same way, so the download list and the player agree on one number. Odd sizes like 818p come from sources that are not 16:9; an ordinary upload matches a profile exactly and the question does not arise. Two streams that round to the same profile produce one rendition, not two. If nothing is at or below your smallest profile there is nothing to file it under, and MediaCMS encodes the video instead, with a line in the migration log saying so. ## The hover preview Skipping transcoding skips the step that would normally produce the small preview that plays when you hover a video, because MediaCMS makes that as one of its encoding profiles. So it is asked for on its own after an import, and an imported video gets a preview like any other. Sprites and HLS were never affected. ## Adding encoding profiles later Pairing happens when a video is imported, against whatever profiles exist at that moment. Add a profile and then run a migration and it is used automatically. Add one afterwards and nothing changes for video already imported, which is how MediaCMS behaves for ordinary uploads too. ## Re-running A video already imported is skipped, recognised by its YouTube id. So a migration can be run again to pick up whatever was added to a playlist or a channel since, without re-fetching anything. That holds across migrations as well: all YouTube migrations share one source identity, so two of them pointed at overlapping playlists will not import the same video twice. ## One thing to be aware of Downloading from YouTube is at odds with their terms of service unless the video is yours or its licence allows it. Moving your own channel onto your own portal is the case this was built for. What you point it at is your call to make.