```mediaByFilename: { attachments: '*.zip' }```
Internally, pattern matching is done by [minimatch](https://github.com/isaacs/minimatch), which supports glob patterns. By default, patterns are case-sensitive. To ignore case, start the pattern with `!`.Supported actions:
When you download content, a directory is created for the campaign that hosts the content. Content directories, which stores the downloaded content, are then placed under the campaign directory.
> If campaign info could not be obtained from content, then content directory
> will be created directly under outDir.
Content can be a post or product. A directory is created for each piece of content. Downloaded items for the content are placed under this directory.
A format must contain at least one of the following unique identifier fields: - `content.id`: ID of content - `content.slug`: last segment of the content URL In addition, a format can contain the following fields: - `content.name`: post title or product name - `content.type`: type of content ('product' or 'post') - `content.publishDate`: publish date (ISO UTC format) Characters enclosed in square brackets followed by a question mark denote conditional separators. If the value of a field could not be obtained or is empty, the conditional separator immediately adjacent to it will be omitted from the name. Default: '{content.id}[ - ]?{content.name}' Fallback: '{content.type}-{content.id}' #### Media filename format Filename format of a downloaded item. A format is a string pattern consisting of fields enclosed in curly braces. A format must contain at least one of the following fields: - `media.id`: ID of the item downloaded (assigned by Patreon) - `media.filename`: can be one of the following, in order of availability: - original filename included in the item's API data; or - filename derived from the header of the response to the HTTP download request. In addition, a format can contain the following fields: - `media.type`: type of item (e.g. 'image' or 'video') - `media.variant`: where applicable, the variant of the item (e.g. 'original', 'thumbnailSmall'...for images) - `src.type`: the type of the items's source: 'post', 'product', 'campaign' or 'collection' - `src.id`: the ID of the items's source - `src.title`: title of the item's source - `src.date`: the publish / creation date of the item's source > If `media.variant` is not included in the format, it will be appended to it if `allMediaVariants` is `true`. Sometimes `media.filename` could not be obtained, in which case it will be replaced with `media.id`, unless it is already present in the format. Characters enclosed in square brackets followed by a question mark denote conditional separators. If the value of a field could not be obtained or is empty, the conditional separator immediately adjacent to it will be omitted from the name. Default: '{media.filename}' Fallback: '{media.type}-{media.id}' #### Filtering posts by tier To download posts belonging to specific tier(s), set the `include.postsInTier` option. Values can be: - `any`: any tier (i.e. no filter) - Array of tier IDs (`string[]`) To obtain the IDs of tiers for a particular creator, first get the campaign through `PatreonDownloader.getCampaign()`, then inspect the `rewards` property: ``` const signal = new AbortSignal(); // optional const logger = new MyLogger(); // optional - see Logger section const campaign = await PatreonDownloader.getCampaign('johndoe', signal, logger); // Sometimes a creator is identified only by user ID, in which case you would do this: // const campaign = await PatreonDownloader.getCampaign({ userId: '80821958' }, signal, logger); const tiers = campaign.rewards; tiers.forEach((tier) => { console.log(`${tier.id} - ${tier.title}`); }); ``` See [Campaign](./api/interfaces/Campaign.md), [Reward](./api/interfaces/Reward.md). #### External downloaders You can specify external downloaders for embedded videos / links. Each entry in the `embedDownloaders` option is an object with the following properties: | Proprety | Description | |----------|-------------| | `provider` | Name of the provider of embedded content. E.g. `youtube`, `vimeo` (case-insensitive) | | `exec` | The command to run to download the embedded content | `exec` can contain fields enclosed in curly braces. They will be replaced with actual values at runtime: | Field | Description | |-----------------------|-------------| | `post.id` | ID of the post containing the embedded video | | `embed.provider` | Name of the provider | | `embed.provider.url` | Link to the provider's site | | `embed.url` | Link to the video page supplied by the provider | | `embed.subject` | Subject of the video | | `embed.html` | The HTML code that embeds the video player on the Patreon page | | `dest.dir` | The directory where the video should be saved | For example usage of `exec`, see [example-embed.conf](./example-embed.conf). > External downloaders are not subject to `request.maxRetries` and `fileExistsAction` settings. This is because `patreon-dl` has no control over the downloading process nor knowledge about the outcome of it (including where and under what name the file was saved). ### Configuring YouTube connection In its simplest form, the process of connecting `patreon-dl` to a YouTube account is as follows: 1. Obtain credentials by having the user visit a Google page that links his or her account to a 'device' (which in this case is actually `patreon-dl`). 2. Save the credentials, as a JSON string, to a file. 3. Pass the path of the file to `PatreonDownloader.getInstance()` To obtain credentials, you can use the `YouTubeCredentialsCapturer` class: ``` import { YouTubeCredentialsCapturer } from 'patreon-dl'; // Note: you should wrap the following logic inside an async // process, and resolve when the credentials have been saved. const capturer = new YouTubeCredentialsCapturer(); /** * 'pending' event emitted when verification data is ready and waiting * for user to carry out the verification process. */ capturer.on('pending', (data) => { // `data` is an object: { verificationURL:`info`, `debug`, `warn` or `error`. Default: `info`
Output messages up to the specified severity level.
| | `include` | What to include in log messages: (object)The pattern to format data-time strings, when `include.dateTime` is `true`.
Date-time formatting is provided by [dateformat](https://github.com/felixge/node-dateformat) library. Refer to the README of that project for pattern rules.
Default: 'mmm dd HH:MM:ss'
| - `FileLogger` Like `ConsoleLogger`, but writes messages to file. ``` import { FileLogger } from 'patreon-dl'; const myLogger = new FileLogger(options); const downloader = await PatreonDownloader.getInstance(url, { ... logger: myLogger }); ``` `options`: all `ConsoleLogger` options plus the following: | Option | Description | |----------------|-----------------------------------------------| | `init` |Values that determine the name of the log file (object):
You might want to provide this if you are creating multiple `FileLogger` instances and filenames are to be formatted with the date, otherwise the date-time part of the filenames might have different values.
Path to directory of the log file.
The path can be a string pattern consisting of the following fields enclosed in curly braces:
Name of the log file.
The path can be a string pattern consisting of the following fields enclosed in curly braces:
Default: '{datetime.yyyymmdd}-{log.level}.log'
| | `fileExistsAction` |What to do if log file already exists? One of the following values:
Default: `append`
| - `ChainLogger` Combines multiple loggers into one single logger. ``` import { ConsoleLogger, FileLogger, ChainLogger } from 'patreon-dl'; const consoleLogger = new ConsoleLogger(...); const fileLogger = new FileLogger(...); const chainLogger = new ChainLogger([ consoleLogger, fileLogger ]); const downloader = await PatreonDownloader.getInstance(url, { ... logger: chainLogger }); ``` ### Aborting To prematurely end a download process, use `AbortController` to send an abort signal to the downloader instance. ``` const downloader = await PatreonDownloader.getInstance(...); const abortController = new AbortController(); downloader.start({ signal: abortController.signal }); ... abortController.abort(); // Downloader aborts current and pending tasks, then ends. ``` ### Workflow and Events #### Workflow 1. Downloader analyzes given URL and determines what targets to fetch. 2. Downloader begins fetching data from Patreon servers. Emits `fetchBegin` event. 2. Downloader obtains the target(s) from the fetched data for downloading. 3. For each target (which can be a campaign, product or post): 1. Downloader emits `targetBegin` event. 2. Downloader determines whether the target needs to be downloaded, based on downloader configuration and target info such as accessibility. - If target is to be skipped, downloader emits `targetEnd` event with `isSkipped: true`. It then proceeds to the next target, if any. 3. If target is to be downloaded, downloader saves target info (subject to downloader configuration), and emits `phaseBegin` event with `phase: saveInfo`. When done, downloader emits `phaseEnd` event. 3. Downloader begins saving media belonging to target (again, subject to downloader configuration). Emits `phaseBegin` event with `phase: saveMedia`. 1. Downloader saves files that do not need to be downloaded, e.g. embedded video / link info. 2. Downloader proceeds to download files (images, videos, audio, attachments, etc.) belonging to the target in batches. For each batch, downloader emits `phaseBegin` event with `phase: batchDownload`. When done, downloader emits `phaseEnd` event with `phase: batchDownload`. - In this `phaseBegin` event, you can attach listeners to the download batch to monitor events for each download. See [Download Task Batch](#download-task-batch). 4. Downloader emits `phaseEnd` event with `phase: saveMedia`. 5. Downloader emits `targetEnd` event with `isSkipped: false`, and proceeds to the next target. 4. When there are no more targets to be processed, or a fatal error occurred, downloader ends with `end` event. #### Events ``` const downloader = await PatreonDownloader.getInstance(...); downloader.on('fetchBegin', (payload) => { ... }); downloader.start(); ``` Each event emitted by a `PatreonDownloader` instance has a payload, which is an object with properties containing information about the event. | Event | Description | |---------------|-----------------------------------------------| | `fetchBegin` |Emitted when downloader begins fetching data about target(s).
Payload properties:
Emitted when downloader begins processing a target.
Payload properties:
Emitted when downloader is done processing a target.
Payload properties:
If `isSkipped` is `true`, the following additional properties are available:
Emitted when downloader begins a phase in the processing of a target.
Payload properties:
If `phase` is `batchDownload`, the following additional property is available:
Emitted when a phase ends for a target.
Payload properties:
Emitted when downloader ends.
Payload properties:
Emitted when a download starts.
Payload properties:
Emitted when a download progress is updated.
Payload properties:
Note: sometimes `length` is `undefined`, in which case `percent` will also be `undefined`.
Emitted when a download is complete.
Payload properties:
Emitted when a download error occurs.
Payload properties:
Emitted when a download is aborted.
Payload properties:
Emitted when a download is skipped.
Payload properties:
If `reason.name` is `destFileExists`, `reason` will also contain the following property:
Emitted when a download task is spawned from another task.
Payload properties:
Emitted when the batch is complete and there are no more downloads pending.
Payload properties: *none*
| ## Web server `patreon-dl` comes with a web server for serving downloaded content. You can utilize the web server as follows: ``` import { WebServer } from 'patreon-dl'; const server = new WebServer(options); await server.start(); console.log(`Web server listening on port: `, server.getConfig().port); ... await server.stop(); ``` `options` is an object with the following properties (all *optional*): | Option | Description | |-----------------|-------------------------------------------------------------| | `dataDir` | Path to directory containing downloaded content. This mirrors the `outDir` downloader option. Default: current working directory. | | `port` | Port number to listen on. Default: `3000` or a random port number if `3000` is already in use. | `logger` | See [Logger](#logger), but note that creation of `FileLogger` is different in the context of web server logging (see below). | #### Web server file logging To create a file logger for the web server: ``` const fileLogger = new FileLogger({ logFilePath: 'path/to/log/file', fileExistsAction: 'append' // or 'overwrite' }); const server = new WebServer({ ... logger: fileLogger }); ```