HyperDL

License Root Architecture UI Release

On-device media downloader for rooted Android. No cloud, no accounts, no Termux needed at runtime.
Maintained by @noticesa

--- ## Table of Contents - [Features](#features) - [Requirements](#requirements) - [Installation](#installation) - [Usage](#usage) - [Cookies (Optional)](#cookies-optional) - [Audio Format](#audio-format) - [Private Vault](#private-vault) - [Clipboard Auto-Download](#clipboard-auto-download) - [Platform Support](#platform-support) - [Architecture](#architecture) - [Directory Layout](#directory-layout) - [Build System](#build-system) - [Troubleshooting](#troubleshooting) - [FAQ](#faq) - [Community & Support](#community--support) - [License](#license) --- ## Features - **Fully on-device** — links resolve and download locally; nothing passes through third-party proxies. - **Zero runtime dependencies** — ships its own Python 3 stack, `ffmpeg`, and media pipeline. Termux is only used to build, never to run. - **Material 3 monochrome WebUI** — single-file Vue app served straight from your root manager. - **12+ platforms** — TikTok, YouTube, Instagram, Facebook, Pinterest, Reddit, X, Bluesky, Threads, Bilibili, Streamable, plus direct file and HLS links. - **Resolution picker** — YouTube video from 144p up to 4K, audio up to FLAC. - **Smart resume** — interrupted downloads continue where they stopped; stale resume state that triggers HTTP 416 is detected and restarted from scratch automatically. - **Cookie-aware extraction** — optional Netscape cookies unlock age-gated and login-walled media, with clear errors that tell expired sessions apart from missing ones. - **Clipboard watcher** — optional background daemon grabs supported links the moment they are copied. - **Private vault** — matching links land in a hidden, gallery-excluded folder. > [!TIP] > Everything below runs as root on the device itself. There is no server component and no account to create. --- ## Requirements | Requirement | Details | | :--- | :--- | | Device | Android 10+, ARM64 | | Root | KernelSU, APatch, or Magisk (plus a manager that can open module WebUI, e.g. MMRL) | | Network | Active internet connection while downloading | | Build only | Termux or ARM64 Linux with `clang`, `zip`, `node` + `npm`, `python3` | > [!NOTE] > Root is required because HyperDL installs as a system module and reads/writes outside the app sandbox. --- ## Installation ### Method 1: Flash the release zip (recommended) 1. Download `HyperDL-v1.3.56.zip` from the [Releases page](https://github.com/itswill00/HyperDL/releases). 2. Open your root manager (**KernelSU**, **APatch**, or **Magisk**). 3. Go to **Modules** > **Install from storage** and pick the zip. 4. Open the module WebUI from your root manager — no reboot needed. ### Method 2: Live local deploy (developers) ```sh ./build.sh --deploy ``` This rebuilds the native bridge, rebundles the extractor and WebUI, syncs the runtime, and updates `/data/adb/modules/hyperdl` in place. --- ## Usage 1. Paste a supported link into the **Downloader** tab. 2. Pick a format (video resolution or audio). 3. Hit download and watch live progress in the console. 4. Finished files land in `/storage/emulated/0/Download/HyperDL/`, grouped by platform and uploader. | WebUI Tab | What it does | | :--- | :--- | | **Downloader** | Paste links, pick formats, queue links, track progress, manage the download list | | **Cookies** | View and manage the optional cookie session used for gated media | | **Terminal** | Run commands and inspect live output on-device | > [!TIP] > Copying a supported link while the clipboard daemon is active fills the input for you — no manual paste needed. ### Queue The **Queue** button next to Download lines a link up instead of starting it. Queued links download one by one on their own, in order. Remove or clear entries any time before they start. ### Failed downloads A failed download stays on screen with its error message. **Retry** reuses the same format and options; **Dismiss** clears it. The list survives restarts, so an interrupted session never loses track of what failed. ### Playlists Flip the **Playlist** switch on a series or playlist link to fetch every item as its own file. Set an optional from/to range to grab only part of a long list. ### Subtitles Video downloads offer subtitle tracks where the platform provides them: None, Indonesian, English, or both. The picker shows which languages a video actually has, and tracks save next to the video as separate subtitle files. --- ## Cookies (Optional) Cookies are **never required** for public media. They only matter for age-gated posts, private content, or platforms throttling anonymous access. - Cookie file: `/data/adb/hyperdl/cookies.txt` (Netscape HTTP format). - Manage it from the **Cookies** tab — no file manager round-trips. - Errors distinguish a **missing** session ("try without cookies") from an **expired** one ("refresh your cookies") per platform. ```sh # Example: export from a desktop browser profile, then place at: /data/adb/hyperdl/cookies.txt ``` > [!WARNING] > A cookie file is a live login session. Never share it publicly or commit it to git. --- ## Audio Format Default audio output is MP3. To change it, write one word to the config file: | Value | Output | | :--- | :--- | | `mp3` | MP3 (default) | | `flac` | Lossless FLAC | | `m4a` | M4A | | `opus` | Opus | ```sh echo "flac" > /data/adb/hyperdl/audio_format.conf ``` --- ## Private Vault The vault keeps selected downloads out of the gallery. When enabled, matching links are routed to a hidden folder carrying a `.nomedia` marker: ```text Download/HyperDL/.vault/Stream/ ``` - Toggle: WebUI action backed by `/data/adb/hyperdl/vault.enabled`. - Domain list: `/data/adb/hyperdl/vault_domains.conf` (safe to extend with your own entries — updates only append missing lines, never overwrite yours). - Editor: the vault tab shows the watched list with a count. Saving validates every line and skips bad ones instead of breaking routing.
How the routing works The extractor checks every URL against `vault_domains.conf`. On a match, the output directory switches to `.vault/Stream` and a `.nomedia` file is ensured at `.vault/` so gallery apps skip the whole tree. The clipboard watcher and DNS bypass read the same list, so behavior stays consistent everywhere.
--- ## Clipboard Auto-Download A lightweight daemon watches the system clipboard for supported links: ```sh # Start / stop (flag file controls auto-start on boot) touch /data/adb/hyperdl/autodl.enabled # enable rm /data/adb/hyperdl/autodl.enabled # disable sh /data/adb/modules/hyperdl/bin/hyperdl_daemon start sh /data/adb/modules/hyperdl/bin/hyperdl_daemon stop ``` --- ## Platform Support | Platform | Format Options | Extraction Strategy | Fallback Provider | | :--- | :--- | :--- | :--- | | **TikTok** | Video (no watermark), Audio (FLAC HD), Photos | TikWM API | SSR HTML scrape / yt-dlp | | **YouTube** | Video (resolution picker 4K–144p), Audio (FLAC HD) | Android-client yt-dlp | Embedded stream resolver | | **Instagram** | Reels, Posts, Carousel media | GraphQL API / JSON embed | yt-dlp (cookie-aware) | | **Facebook** | Reels, Public videos, Watch clips | Direct progressive stream scrape | yt-dlp extractor | | **Pinterest** | Videos, Pins, Story media, Original images | PinResource unauth JSON API | yt-dlp extractor | | **Reddit** | Videos (MP4), Galleries, Single posts | Reddit JSON API / old Reddit | yt-dlp extractor | | **X (Twitter)** | Videos, Photos, Animated GIFs | GraphQL API / FxTwitter / VxTwitter | yt-dlp extractor | | **Bluesky** | Videos (MP4), Photos, Galleries | AT Protocol public XRPC | yt-dlp extractor | | **Threads** | Videos (MP4), Photos | OpenGraph embedded scrape | yt-dlp extractor | | **Bilibili** | Videos, Audio | Web API / scraper | yt-dlp DASH remuxing | | **Streamable** | Videos (MP4) | Streamable JSON API | yt-dlp extractor | | **Direct URLs** | MP4, WEBM, MP3, M3U8 streams | Chunked direct streamer | Python urllib pipeline | --- ## Architecture ```text +-------------------------------+ | Root Manager / WebUI | | (KernelSU / APatch / MMRL) | +---------------+---------------+ | ksu.exec() / IPC | v +---------------+---------------+ | Native C Bridge | | (bin/libhyperdl.so, C99 -O3) | +-------+---------------+-------+ | | Direct SQLite DL | | Dynamic Spawning MediaStore Index | | v v +---------------+---+ +---------------+---------------+ | /system/lib64/ | | Standalone Python 3 Runtime | | libsqlite.so | | (runtime/bin/python3) | +-------------------+ +---------------+---------------+ | +--------------+--------------+ | | v v +---------------+-------+ +---------------+-------+ | Core Extractor | | Standalone yt-dlp | | (hyperdl.bundle .pyc) | | (.pyc zipapp archive) | +-----------------------+ +-----------------------+ | | +--------------+--------------+ | Streaming / Chunked I/O | v +-------------------------------+ | /storage/emulated/0/ | | Download/HyperDL/ | +-------------------------------+ ``` ### Core Components 1. **Native C bridge (`bin/libhyperdl.so`)** - C99 ELF binary compiled with full optimization (`-O3`). - Queries Android MediaStore via `/system/lib64/libsqlite.so` in under 1ms — replacing legacy `content query` JVM calls that take up to 1.4s. - Ships a self-healing embedded fallback (compressed Base64 extractor payload in `.rodata`) for recovery when bundles are missing or corrupted. - Non-blocking background session manager with POSIX detachment and atomic PID tracking. 2. **Standalone ARM64 Python runtime (`runtime/`)** - Self-contained Python 3 stack targeting Android Bionic libc. - Standard library packed as stripped bytecode in `runtime/lib/python314.zip` for near-instant imports. - Bundled shared libraries (`libpython3.14.so`, `libcrypto.so.3`, `libssl.so.3`, `libandroid-support.so`) plus a local CA bundle. 3. **Core extractor package (`bin/hyperdl.bundle`)** - Zipapp of precompiled `.pyc` modules — zero cold-start compile latency. - Chunked HTTP streaming with resume support and transient-network recovery. 4. **Standalone yt-dlp (`bin/yt-dlp`)** - Fallback and primary YouTube pipeline as a bytecode-optimized archive (cold start cut from ~11s to ~1.5s on ARM64). - Android client emulation to avoid throttling and client-validation failures. 5. **Material 3 monochrome WebUI (`webroot/index.html`)** - Vue 3 + Vite app bundled into one standalone HTML file. - Platform badges, Netscape cookie manager, live console, and native player intents. 6. **Clipboard daemon (`bin/hyperdl_daemon`)** - Watches Android clipboard events, validates links by regex, and hands them to the extractor hands-free. --- ## Directory Layout ```text HyperDL/ ├── build.sh # Build, package, deploy, and release script ├── module.prop # Magisk / KernelSU module metadata ├── customize.sh # On-device installation script ├── service.sh # Late-start service initialization ├── uninstall.sh # Module removal and cache purge ├── update.json # Release-feed metadata (served from this repo) ├── AGENTS.md # Contributor invariants — read before changing code ├── bin/ │ ├── libhyperdl.so # Native 64-bit ELF bridge (built from src/main.c) │ ├── hyperdl.bundle # Compiled bytecode extractor (built from engine/) │ ├── hyperdl_daemon # Clipboard listener daemon │ ├── clip.jar # Clipboard helper │ └── yt-dlp # Bytecode-optimized standalone yt-dlp ├── engine/ │ ├── downloader.py # Public entry point (re-exports _impl) │ └── _impl.py # Extraction logic and stream pipelines ├── runtime/ # Standalone ARM64 Python 3 + ffmpeg (git-ignored build output) ├── scripts/ │ ├── bundle_engine.py # Compiles the extractor into .pyc bundle and C header │ ├── bundle_runtime.py # Assembles the standalone ARM64 Python runtime │ ├── optimize_ytdlp.py # Converts yt-dlp into a pure bytecode archive │ └── bump_version.py # Version bump helper used by build.sh --bump ├── src/ │ ├── main.c # Native C bridge source │ └── sitecustomize.py # On-device Python bootstrap ├── release_notes/ │ └── vX.Y.Z.md # GitHub Release description per version ├── tests/ │ ├── test_engine.py # Extractor unit tests │ └── test_bump_version.py # Version helper argument tests ├── webroot/ │ └── index.html # Inlined single-file WebUI (built from webui/) ├── webui/ │ ├── src/ │ │ ├── App.vue # Multi-tab view (Downloader, Cookies, Terminal) │ │ ├── assets/main.css # Material 3 monochrome theme │ │ ├── components/ # Vector icons and platform chips │ │ └── helpers/shell.js # KernelSU / APatch execution bridge │ ├── package.json │ └── vite.config.js # Single-file build configuration └── releases/ # Release zips (internal build output) ``` --- ## Build System ### Command Line Options | Option | Action | | :--- | :--- | | `-b, --bump [type]` | Bump version (`patch` \| `minor` \| `major`, default: `patch`) | | `-d, --deploy` | Deploy to `/data/adb/modules/hyperdl` for live testing | | `-c, --clean` | Remove build artifacts | | `-r, --release` | Tag and publish the Full Zip to this repo | | `-h, --help` | Show help | ### Typical Workflows ```sh ./build.sh --deploy # rebuild + push live to the device ./build.sh --clean # wipe dist, binaries, and caches ./build.sh # package releases/HyperDL-vX.Y.Z.zip ./build.sh --release # package, tag, and publish to this repo ``` ### Version Bumping `scripts/bump_version.py` is the only supported way to move the version, and `./build.sh --bump` is a thin wrapper around it: ```sh python3 scripts/bump_version.py --help # usage, touches nothing python3 scripts/bump_version.py patch # v1.3.55 -> v1.3.56 python3 scripts/bump_version.py minor # v1.3.55 -> v1.4.0 python3 scripts/bump_version.py v1.3.60 # jump to an explicit version # --notes also rewrites the update.json notes field python3 scripts/bump_version.py patch --notes "Scraper recovery paths repaired" ``` The helper writes to disk immediately and has no dry-run, so every argument is validated before the first file is opened: an unknown flag, a stray word, or a bump type combined with an explicit version aborts with exit code `2` and changes nothing. ### Build Targets Packaging outputs a single Full Zip to internal `releases/`: - `HyperDL-v1.3.56.zip` — canonical release package with the full embedded runtime. > [!IMPORTANT] > Every release is a **Full Zip**. There is no OTA/delta mechanism — never create or expect `HyperDL-OTA-*.zip` files. ### Running Tests ```sh python3 -m unittest discover -s tests -v cd webui && npm test ``` The suite covers the extractor, the version helper, the release metadata standard, and the standalone isolation rules. The isolation tests read `bin/libhyperdl.so` and `runtime/lib/python314.zip` and skip when those artifacts have not been built yet, so a fresh clone still passes. ### Rebuilding the Bundled Runtime `scripts/bundle_runtime.py` assembles the ARM64 Python 3 and FFmpeg closure and needs a working Termux `ffmpeg`/`ffprobe` plus `patchelf` on the build machine. It is not part of the normal loop: `./build.sh` only re-runs it when you invoke it directly. A plain `./build.sh` reuses the committed `runtime/` payload and repackages it. --- ## Troubleshooting
Download fails with HTTP Error 416 Fixed in v1.3.45: YouTube downloads no longer force resume against short-lived signed URLs, and any range error is automatically retried from the start after purging stale partials. Update the module and try again.
403 Forbidden or "login required" Public links should work without cookies. For gated content, add a Netscape cookie file via the **Cookies** tab. If cookies are already set, the session likely expired — re-export them from your browser.
First YouTube download is slow to start Normal. The bundled yt-dlp pays a one-time cold-start cost (~1.5s) while resolving the stream. Subsequent downloads in the same session start faster.
WebUI opens blank Open it through your root manager's module WebUI action (MMRL recommended). Deploy fresh with `./build.sh --deploy` if `webroot/index.html` is missing or outdated.
Clipboard links are not captured Make sure the daemon flag exists and the daemon is running: ```sh touch /data/adb/hyperdl/autodl.enabled sh /data/adb/modules/hyperdl/bin/hyperdl_daemon start ```
--- ## FAQ
Do I need Termux installed to use HyperDL? No. Termux is only the build environment. The module carries its own Python runtime and tools.
Where do my downloads go? `/storage/emulated/0/Download/HyperDL/`, organized by platform and uploader. Vault matches go to the hidden `.vault/Stream/` subfolder instead.
Does HyperDL upload my links anywhere? No. Resolution and downloading happen on the device. The only network traffic is the direct request to the platform hosting the media.
How do updates work? Full-Zip releases ship from this repository's [Releases page](https://github.com/itswill00/HyperDL/releases) feed (`update.json`). Flash the new zip over the old one — no data wipe needed. Older versions (up to v1.3.42) stay archived in the former HyperDL-Release repo. The console tab also has a HyperDL update row: it shows the latest version with its notes, so you can see what changed before flashing.
--- ## Community & Support - **Telegram**: [@noticesa](https://t.me/noticesa) for updates, direct assistance, and discussions. - **Support development**: if HyperDL saves you time, consider supporting via [SociaBuzz Tribe](https://sociabuzz.com/noticesa/tribe). --- ## License This project is licensed under the [GNU General Public License v3.0](LICENSE). Developed and maintained by [@noticesa](https://t.me/noticesa).