`, `

`, etc.), up to the specified heading depth. A number between `1` and `6` is expected. Using `--toc-level` with a value greater than `1` implies `--toc`. #### `--hyphenate` Hyphenation is enabled by default for `pdf`, and disabled for `epub`, `html`, and `md`. You can opt into hyphenation with the `--hyphenate` flag, or disable it with the `--no-hyphenate` flag. See also the [Hyphenation and justification](#hyphenation-and-justification) recipe. #### `--inline` Embed images inline with the document. Images are fetched and converted to Base64-encoded `data` URLs. This option is particularly useful for `html` to produce self-contained HTML files. #### `--md.=` Pass options to the underlying Markdown stringifier, [`mdast-util-to-markdown`](https://github.com/syntax-tree/mdast-util-to-markdown#options). These are the default Markdown options: ```js const DEFAULT_MARKDOWN_OPTIONS = { fences: true, emphasis: '_', strong: '_', resourceLink: true, rule: '-' }; ``` #### `--unsafe` Disables some [JSDOM validations](https://github.com/jsdom/jsdom/blob/main/lib/jsdom/living/helpers/validate-names.js) that may throw an error when parsing invalid HTML pages (See [#177](https://github.com/danburzo/percollate/issues/177)). ## Recipes ### Basic bundling To turn a single web page into a PDF: ```bash percollate pdf --output=some.pdf https://example.com ``` To bundle _several_ web pages into a single PDF, specify them as separate arguments to the command: ```bash percollate pdf --output=some.pdf https://example.com/page1 https://example.com/page2 ``` You can use common Unix commands and keep the list of URLs in a newline-delimited text file: ```bash cat urls.txt | xargs percollate pdf --output=some.pdf ``` To transform several web pages into individual PDF files at once, use the `--individual` flag: ```bash percollate pdf --individual https://example.com/page1 https://example.com/page2 ``` If you'd like to fetch the HTML with an external command, you can use `-` as an operand, which stands for `stdin` (the standard input): ```bash curl https://example.com/page1 | percollate pdf --url=https://example.com/page1 - ``` Notice we're using the `url` option to tell percollate the source of our (now-anonymous) HTML it gets on stdin, so that relative URLs on links and images resolve correctly. ### Web feeds Percollate has basic support for processing XML web feeds in [Atom](https://datatracker.ietf.org/doc/html/rfc4287) or [RSS](https://www.rssboard.org/rss-specification) format. When processing a web feed, every entry in the feed becomes its own article, as if percollate received all the entry URLs as operands. The command below produces an EPUB book from the feed contents: ```bash percollate epub https://example.com/posts.xml ``` To produce individual output files for the feed entries, use the `--individual` flag: ```bash percollate epub --individual https://example.com/posts.xml ``` The content of the articles is read from the feed file rather than fetched anew. The content is passed through the DOM enhancements and sanitized as usual, but it’s not processed with Readability.

To fetch the HTML pages for entries in Atom and RSS feeds

If instead you’d like to fetch and process the original HTML pages corresponding to the entries in the Atom/RSS feed, use [`hred`](https://github.com/danburzo/hred/) to extract the URLs and feed them to percollate with `xargs`. Below is an example `hred` query for extracting URLs from Atom feeds, explained in more detail on the [hred recipes](https://danburzo.ro/toolbox/hred/) page. ```bash curl https://example.com/posts.xml | \ hred -xcr 'entry > link:is([rel=alternate],:not([rel]))@href' | \ xargs percollate epub ```

### The `--css` option The `--css` option lets you pass a small snippet of CSS to percollate. Here are some common use-cases: #### Custom page size / margins The default page size is A5 (portrait). You can use the `--css` option to override it using [any supported CSS `size`](https://www.w3.org/TR/css3-page/#page-size): ```bash percollate pdf --css "@page { size: A3 landscape }" http://example.com ``` Similarly, you can define: - custom margins, e.g. `@page { margin: 0 }` - the base font size: `html { font-size: 10pt }` #### Changing the font stacks The default stylesheet includes CSS variables for the fonts used in the PDF: ```css :root { --main-font: Palatino, 'Palatino Linotype', 'Times New Roman', 'Droid Serif', Times, 'Source Serif Pro', serif, 'Apple Color Emoji', 'Segoe UI Emoji', 'Segoe UI Symbol'; --alt-font: 'helvetica neue', ubuntu, roboto, noto, 'segoe ui', arial, sans-serif, 'Apple Color Emoji', 'Segoe UI Emoji', 'Segoe UI Symbol'; --code-font: Menlo, Consolas, monospace; } ``` | CSS variable | What it does | | ------------- | ------------------------------------- | | `--main-font` | The font stack used for body text | | `--alt-font` | Used in headings, captions, et cetera | | `--code-font` | Used for code snippets | To override them, use the `--css` option: ```bash percollate pdf --css ":root { --main-font: 'PT Serif'; --alt-font: Roboto; }" http://example.com ``` > 💡 To work correctly, you must have the fonts installed on your machine. Custom web fonts currently require you to use a custom CSS stylesheet / HTML template. #### Remove the appended `href`s from hyperlinks The idea with percollate is to make PDFs that can be printed without losing where the hyperlinks point to. However, for some link-heavy pages, the appended `href`s can become bothersome. You can remove them using: ```bash percollate pdf --css "a:after { display: none }" http://example.com ``` #### Hyphenation and justification Hyphenation is only enabled by default for PDFs, but you can opt in or out of it for any output format with [a flag](#--hyphenate). When hyphenation is enabled, paragraphs will be justified: ```css .article__content p { text-align: justify; } ``` If you prefer left-aligned text: ```bash percollate pdf --css ".article__content p { text-align: left }" http://example.com ``` ### The `--style` option The `--style` option lets you use your own CSS stylesheet instead of the default one. Here are some common use-cases for this option: > ⚠️ TODO add examples here ### The `--template` option The `--template` option lets you use a custom HTML template for the PDF. > 💡 The HTML template is parsed with [nunjucks](https://mozilla.github.io/nunjucks/), which is a close JavaScript relative of Twig for PHP, Jinja2 for Python and L for Ruby. Here are some common use-cases: #### Customizing the page header / footer Puppeteer can print some basic information about the page in the PDF. The following CSS class names are available for the header / footer, into which the appropriate content will be injected: - `date` — The formatted print date - `title` — The document title - `url` — document location (**Note:** this will print the path of the _temporary html_, not the original web page URL) - `pageNumber` — the current page number - `totalPages` — total pages in the document > 👉 See the [Chromium source code](https://cs.chromium.org/chromium/src/components/printing/resources/print_header_footer_template_page.html) for details. You place your header / footer template in a `template` element in your HTML: ```html ``` See the [default HTML](./templates/default.html) for example usage. You can add CSS styles to the header / footer with either the `--css` option or a separate CSS stylesheet (the `--style` option). > 💡 The header / footer template [do not inherit their styles](https://github.com/puppeteer/puppeteer/issues/1853) from the rest of the page (i.e. they are not part of the cascade), so you'll have to write the full CSS you want to apply to them. An example from the [default stylesheet](./templates/default.css): ```css .footer-template { font-size: 10pt; font-weight: bold; } ``` ## Updating To keep the tool up-to-date, you can run: ```bash npm install -g percollate ``` Occasionally, an upgrade might not go according to plan; in this case, you can uninstall and re-install `percollate`: ```bash npm uninstall -g percollate && npm install -g percollate ``` ## How it works All export formats follow a common pipeline: 1. Fetch the page(s) using [`node-fetch`](https://github.com/node-fetch/node-fetch) 2. If an AMP version of the page exists, use that instead (disable with `--no-amp` flag) 3. [Enhance](./src/enhancements.js) the DOM using [`jsdom`](https://github.com/jsdom/jsdom) 4. Pass the DOM through [`@mozilla/readability`](https://github.com/mozilla/readability) to strip unnecessary elements 5. Apply the [HTML template](./templates/default.html) and the [stylesheet](./templates/default.css) to the resulting HTML Different formats then use different tools to produce the final file. PDFs are rendered with [`puppeteer`](https://github.com/puppeteer/puppeteer). EPUBs have external images fetched and bundled together with the HTML of each article. When the `--inline` option is used, images are instead converted to `data` URLs and embedded into the HTML. HTMLs are saved without any further changes. When the `--inline` option is used, images are converted to `data` URLs and embedded into the HTML. External images are not otherwise fetched. Markdown files are produced the same way as HTMLs, then processed with a series of utilities from the [unified.js](https://unifiedjs.com/) umbrella. ## Limitations Percollate inherits the limitations of two of its main components, Readability and Puppeteer (headless Chrome). The imperative approach Readability takes will not be perfect in each case, especially on HTML pages with atypical markup; you may occasionally notice that it either leaves in superfluous content, or that it strips out parts of the content. You can confirm the problem against [Firefox's Reader View](https://blog.mozilla.org/firefox/reader-view/). In this case, consider [filing an issue on `@mozilla/readability`](https://github.com/mozilla/readability/issues). Using a browser to generate the PDF is a double-edged sword. On the one hand, you get excellent support for web platform features. On the other hand, [print CSS](https://www.smashingmagazine.com/2018/05/print-stylesheets-in-2018/) as defined by W3C specifications is only partially implemented, and it seems unlikely that support will be improved any time soon. However, even with modest print support, I think Chrome is the best (free) tool for the job. ## Troubleshooting On some Linux machines you'll need to [install a few more Chrome dependencies](https://github.com/puppeteer/puppeteer/blob/master/docs/troubleshooting.md#chrome-headless-doesnt-launch) before `percollate` works correctly. (_Thanks to @ptica for [sorting it out](https://github.com/danburzo/percollate/issues/19#issuecomment-428496041)_) The `percollate pdf` command supports the `--no-sandbox` Puppeteer flag, but make sure you're [aware of the implications](https://github.com/puppeteer/puppeteer/blob/master/docs/troubleshooting.md#chrome-headless-fails-due-to-sandbox-issues) before disabling the sandbox. ## Using Firefox to render PDFs > This feature is experimental. Please log an issue if you notice anything wrong. Starting with `percollate` 3.x, it's possible to use Firefox Nightly as an alternative browser with which to render PDFs. To make Firefox available to Percollate, use the following install command: ```bash PUPPETEER_PRODUCT=firefox npm install percollate ``` After installation, `percollate pdf` commands can be run with the `--browser=firefox` option. ### Limitations of Firefox PDF rendering At the moment, rendering PDFs with Firefox has the following limitations: - The pages can't have headers and footers, so there are no page numbers. ## Contributing Contributions of all kinds are welcome! See [CONTRIBUTING.md](./CONTRIBUTING.md) for details. ## See also Here are some other projects to check out if you're interested in building books using the browser: - [bindery.js](https://github.com/evnbr/bindery) ([website](https://bindery.info/)) - [Foliojs](https://github.com/foliojs) - [Ketty](https://gitlab.coko.foundation/coko-org/products/ketty/ketty) ([website](https://ketty.community/)) - [Magicbook](https://github.com/magicbookproject/magicbook) - [monolith](https://github.com/Y2Z/monolith) - [paged.js](https://github.com/pagedjs/pagedjs/) ([website](https://pagedjs.org/)) - [Postlight Parser](https://github.com/postlight/parser) - [SingleFileZ](https://github.com/gildas-lormeau/SingleFileZ) - [weasyprint](https://github.com/Kozea/WeasyPrint) ([website](https://weasyprint.org/))