#+title: Org-transclusion User Manual
#+author: Noboru Ota <me@nobiot.com>
#+macro: version 1.3.x
#+macro: modified 05 April 2024

#+language: en
#+export_file_name: org-transclusion.texi
#+texinfo_dir_category: Emacs
#+texinfo_dir_title: Org-transclusion: (org-transclusion)
#+texinfo_dir_desc: Transclusion in Org mode
#+texinfo: @paragraphindent asis

#+options: toc:nil ':t

#+html: <a href="https://www.gnu.org/software/emacs/"><img alt="GNU Emacs" src="https://img.shields.io/static/v1?logo=gnuemacs&logoColor=fafafa&label=Made%20for&message=GNU%20Emacs&color=7F5AB6&style=flat"/></a>
#+html: <a href="http://elpa.gnu.org/packages/org-transclusion.html"><img alt="GNU ELPA" src="https://elpa.gnu.org/packages/org-transclusion.svg"/></a>
#+html: <a href="http://elpa.gnu.org/devel/org-transclusion.html"><img alt="GNU-devel ELPA" src="https://elpa.gnu.org/devel/org-transclusion.svg"/></a>
#+html: <img alt="GPLv3" src="https://img.shields.io/badge/License-GPLv3-blue.svg">

This manual is for Org-transclusion version {{{version}}}.

Last updated: {{{modified}}}.

#+transclude: [[../README.org::whatis]]

#+texinfo: @insertcopying

* COPYING
:PROPERTIES:
:COPYING: t
:END:

Copyright (C) 2021  Free Software Foundation, Inc.

#+begin_quote
Permission is granted to copy, distribute and/or modify this document
under the terms of the GNU Free Documentation License, Version 1.3 or
any later version published by the Free Software Foundation; with no
Invariant Sections, with the Front-Cover Texts being “A GNU Manual,” and
with the Back-Cover Texts as in (a) below.  A copy of the license is
included in the section entitled “GNU Free Documentation License.”

(a) The FSF’s Back-Cover Text is: “You have the freedom to copy and
modify this GNU manual.”
#+end_quote

* Example Use Cases & Main Features

#+transclude: [[../README.org::#use-cases]] :only-contents

* Installation

#+transclude: [[../README.org::*Installation]] :only-contents

* Getting Started
:PROPERTIES:
:DESCRIPT: To get running in 5 minutes
:CUSTOM_ID: getting-started
:END:

#+findex: org-transclusion-add
#+findex: org-transclusion-add-all
#+findex: org-transclusion-make-from-link
#+findex: org-transclusion-open-source
#+findex: org-transclusion-move-to-source
#+findex: org-transclusion-refresh
#+vindex: org-transclusion-map

The basic idea of Org-transclusion is simple: insert a copy of text content via a file link or ID link within an Org file. This is an Org Mode version of transclusion.

To transclude content via a link, use one of the following commands:

- ~org-transclusion-add~

- ~org-transclusion-add-all~

- ~org-transclusion-make-from-link~

For example, if you have an ID link in your Org file like this:

#+begin_example
[[id:20210501T171427.051019][Bertrand Russell]]
#+end_example

Simply type to add =#+transclude:= in front of the link like the example below.

#+begin_example
#+transclude: [[id:20210501T171427.051019][Bertrand Russell]]
#+end_example

Put your cursor somewhere on this keyword line and type ~M-x org-transclusion-add RET~, and you will see the text content that the ID points to replace the whole line. If you have multiple links with a transclude keyword, you can type ~M-x org-transclusion-add-all RET~ to add all transclusions in the current buffer.

Alternatively, you can also put cursor somewhere on the link and type  ~M-x org-transclusion-make-from-link RET~. That will insert another line with ~#+transclusion:~ keyword added in front of a copy of the original link in the next empty line.

The transcluded paragraphs will be visually marked with a vertical line "| " in the fringe (on graphical display) or in the beginning of line (on a text-only terminal) by default. The source (original) of the transcluded paragraphs will be also visually marked with an overlay. The appearance of these visual elements can be customized (refer to section [[#faces][Faces & fringe bitmap]]).

The transcluded text is *read-only* but you can copy it and export it as normal text. Org-transclusion remembers where it has transcluded the text from (its source buffer).

You can call a number of useful commands with a single letter (by default). For example, you can press =o= to open the source buffer of the transclusion at point, or =O= (capital "o") to open and move to it. Press =g= to refresh the transclusion. Press =e= to start live-sync edit. For more detail, inspect the documentation of each command. Press =d= to remove the transcluded content, putting the original =#+transclude: [[id:id-of-the-content]]=.

This single-letter-context-menu is defined in ~org-transclusion-map~. The default keybindings are shown below. Adapt them to your liking, especially if you use Vim keybindings with Evil Mode, etc.

#+name: org-transclusion-map
#+caption: Default org-transclusion-map
#+begin_example
key             binding
---             -------

C-c             Prefix Command
TAB             org-cycle
D               org-transclusion-demote-subtree
O               org-transclusion-move-to-source
P               org-transclusion-promote-subtree
d               org-transclusion-remove
C-d             org-transclusion-detach
e               org-transclusion-live-sync-start
g               org-transclusion-refresh
o               org-transclusion-open-source

    † Also inherits ‘org-mode-map’ as of development version 1.1.1.0.20220106.203639
      and planned to be part of 1.2.0

#+end_example

This should get you started with Org-transclusion. There are more options and customizing options available for you to fine-tune the text content you transclude. Explore the rest of the user manual and play with Org-transclusion to get familiar with it.

* Usage
:PROPERTIES:
:DESCRIPTION: Features in detail
:CUSTOM_ID: usage
:END:
** Org-transclusion mode, activate, and deactivate

#+cindex: Activate / Deactivate
#+findex: org-transclusion-mode
#+findex: org-transclusion-activate
#+findex: org-transclusion-deactivate
#+cindex: Properties
#+cindex: Property - :disable-auto
#+vindex: org-transclusion-add-all-on-activate

Org-transclusion is a local minor mode; however, you do not need to explicitly call ~org-transclusion-mode~. The minor mode is intended to be just a convenient wrapper to let you easily toggle between =activate= and =deactivate=.

As you saw in the [[#getting-started][Getting Started section]] above, calling ~org-transclusion-add~ or ~org-transclusion-add-all~ is enough to add transclusions in your current buffer.

The minor mode is automatically turned on locally for your current buffer through one of these commands. All it does is to call ~org-transclusion-activate~  to activate hooks and some other variables. Their main purpose is to keep files in the filesystem clear of the transcluded content.

Turn off the minor mode or use ~org-transclusion-deactivate~; you will remove all the transclusions in the current buffer and clear the hooks and other setup variables.

If you prefer, you can use ~org-transclusion-mode~ as your entry command for transclusion. When customizable variable ~org-transclusion-add-all-on-activate~ is non-nil (it is =t= by default), turning on the minor mode calls the ~org-transclusion-add-all~ command to attempt to add all transclusions automatically in the current buffer.

You can control whether or not transclusions are to be added automatically per transclude keyword. By default, ~org-transclusion-add-all~ (it is also used by ~org-transclusion-mode~) will work on every transclude keyword in the buffer. Add =:disable-auto= property to a keyword as shown in the example below; =add-all= skips transclude keywords with it.

#+begin_example
#+transclude: [[file:path/to/file.org]] :disable-auto
#+end_example

You can override the =:disable-auto= property by manually calling ~org-transclusion-add~ at point. There are various properties like =:disable-auto= to control each transclusion (refer to [[#cindex][Index - Features]] for a list).

** Org links supported
:PROPERTIES:
:CUSTOM_ID: org-links-supported
:END:

#+cindex: Org Links Supported
#+cindex: Property - :only-contents

Transclusion has been tested to work for the following types of links:

- File link for an entire org file/buffer; e.g. =[[file:~/org/file.org][My Org Notes]]=
- File link with =::*heading=
- File link with =::#custom-id=
- File link with =::name= for blocks (e.g. blocked quotations), tables, and links
- File link with =::dedicated-target=; this is intended for linking to a paragraph. See below.
- ID link =id:uuid=
- File link for non-org files (tested with =.txt= and =.md=); for these, the whole buffer gets transcluded

#+ATTR_TEXINFO: :tag Note
#+begin_quote
Search-options =::/regex/= and =::number= do not work as intended.
#+end_quote


For transcluding a specific paragraph, there are two main ways: Org Mode's [[https://orgmode.org/manual/Internal-Links.html#Internal-Links][dedicated-target]] and =:only-contents= property.

For dedicated targets, the target paragraph must be identifiable by a dedicated target with a =<<paragraph-id>>=:

#+begin_example
Lorem ipsum dolor sit amet, consectetur adipiscing elit.
Suspendisse ac velit fermentum, sodales nunc in,
tincidunt quam. <<paragraph-id>>
#+end_example

It is generally assumed that the =paragraph-id= is placed after its content, but it is not an absolute requirement; it can be in the beginning (before the content) or in the middle of it.

For the =:only-contents= property, refer to sub-section [[#filtering-org-elements-per-transclusion][Filtering Org elements per transclusion]].

** Transclude over a network with ~hyper://~, ~http://~, etc.
:PROPERTIES:
:CUSTOM_ID: network-transclusion
:END:
#+cindex: Links with hyper://, http://, and other protocols

Version 1.4 includes changes which make it possible for asynchronously
transclude content, e.g., over a network.  To transclude content via
=http://=, first install [[https://git.sr.ht/~ushin/org-transclusion-http][org-transclusion-http]], then run ~M-x
org-transclusion-add~ on the following link to transclude the HTML
element with the "#emacs" id:

#+begin_example
#+transclude: [[https://ushin.org/software.html#emacs]]
#+end_example

To transclude content via =hyper://=, first install [[https://ushin.org/hyperdrive/hyperdrive-manual.org][hyperdrive.el]], then
run ~M-x org-transclusion-add~ on the following link to transclude the
the Org heading with the property ~CUSTOM_ID: emacs~ inside the
~/software.org~ file inside the USHIN hyperdrive:

#+begin_example
#+transclude: [[hyper://aaj45d88g4eenu76rpmwzjiabsof1w8u6fufq6oogyhjk1ubygxy/software.org#%3A%3A%23emacs]]
#+end_example

** Control levels of headlines per transclusion

#+cindex: Property - :level
#+findex: org-transclusion-demote-subtree
#+findex: org-transclusion-promote-subtree
#+findex: org-transclusion-make-from-link

When you transclude Org contents, you can specify a different headline level than those of the source Org file.

Use the =:level= property with a value of single digit number from 1 to 9 like this example below.

#+begin_example
#+transclude: [[file:path/to/file.org::*Headline]] :level 2
#+end_example

The top level of the transcluded headline will be set to the value of =:level= property -- in this example, level 2 regardless of that in the source. When the headline contains sub-headlines, they will be all automatically promoted or demoted to align according to how many levels the top of the subtree will move.

When you transclude an entire Org file, it may contain multiple subtrees. In such cases, the top-most level among the subtrees will be set according to the =:level= property; the rest of headlines in the buffer will align accordingly.

Other ways to control include the following.

- ~org-transclusion-make-from-link~ ::
  Make a transclusion keyword from a link at point. If you pass a positive number 1-9 with =digit-argument= (e.g. prefix ~M-x~ with ~C-2~ to pass "2"), this function automatically puts the =:level= property to the resultant transclusion keyword.

- ~org-transclusion-promote-subtree~ ::
  Promote transcluded subtree at point. Mapped to "P" (capital "p") by default in ~org-transclusion-map~

- ~org-transclusion-demote-subtree~ ::
  Demote transcluded subtree at point. Mapped to "D" (capital "d") by default in ~org-transclusion-map~

** Filter Org elements per transclusion
:PROPERTIES:
:CUSTOM_ID: filtering-org-elements-per-transclusion
:END:

#+cindex: Filters
#+vindex: org-transclusion-exclude-elements
#+vindex: org-transclusion-include-first-section
#+cindex: Property - :only-contents

You can control what elements to include in many different ways with using various filters. The filters work in two layers: customizable variable and properties per transclude keyword.

The following two customizable variables are applicable to all transclusions globally. You can think of them as the global default.

- =org-transclusion-exclude-elements= ::

  This customizable variable globally defines the exclusion filter for elements. It is a list of symbols; the acceptable values can be seen by inspecting =org-element-all-elements=. The default is to exclude =property-drawer=.

  Refer also to the [[#customizable-filter-to-exclude-certain-org-elements][sub-section on this user option]].

- =org-transclusion-include-first-section= ::

  This customizing variable globally defines whether or not to include the first section of the source Org file. The first section is the part before the first headline -- that's the section that typically contains =#+title=, =#+author=, and so on. Many people also write notes in it without adding any headlines. Note that this user option's default is now =t= (changed from =nil= as users seem to spend time to "correct" this issue). Turn it to =t= if you wish to transclude the content from the first section of your Org files. If you wish to exclude the "meta data" defined by =#+title= and others, exclude =keyword= as described in this section -- these meta data are defined with using the =keyword= element of Org Mode.

  Refer also to the [[#include-the-section-before-the-first-headline-org-file-only][sub-section on this user option]].

In addition to the global user options above, you can fine-tune the default exclusion filter per transclusion. Add following properties to transclusions you wish to apply additional filters.

- =:only-contents= ::
  This property lets you exclude titles of headlines when you transclude a subtree (headline); you transclude only the contents. When the subtree contains sub-headlines, all the contents will be transcluded.

  Add =:only-contents= without any value like this example:

#+begin_example
#+transclude: [[file:path/to/file.org]] :only-contents
#+end_example

- =:exclude-elements= ::
  This property lets you *add* elements to exclude per transclusion on top of the variable ~org-transclusion-exclude-elements~ defines. You cannot *remove* the ones defined by it; thus, it is intended that you use the customizable variable as your global default and fine-tune it by the property per transclusion.

  Add =:exclude-elements= with a list of elements (each one as defined by =org-element-all-elements=) separated by a space inside double quotation marks like this example:

#+begin_example
#+transclude: [[file:path/to/file.org]] :exclude-elements "drawer keyword"
#+end_example

You can combine =:only-contents= and =:exclude-elements= to control how you transclude a subtree. With these properties, you can really have great control over what to include and exclude. It might be a little overwhelming at a time but the changes via properties are easy to change -- simply press =d= to remove the transclusion, change the properties, and transclude again to see a new result.

- =:expand-links= ::
  This property lets you expand the file names in links to absolute file names.

You can use the property in the following way.

#+begin_example
#+transclude: [[file:path/to/file.org]] :expand-links
#+end_example

As an exanmple if you have the content of this source file like this:

#+begin_example
This is a sample file link to [[file:test2.org]].
#+end_example

For this source file, you will get the following transclusion with the link expanded [fn:1]

#+begin_example
This is a sample file link to [[file:/home/user/org/test2.org]].
#+end_example

[fn:1] "Expanding" in Emacs means converting a relative file name to an absolute one. Info node `(elisp) File-Name-Expansion'

*** Notes on excluding the headline element

If you add =headline= as a list of elements to exclude, you exclude sub-headlines within your subtrees and you will still transclude the content of the top-most headline of the subtrees.

If you are transcluding only one subtree, this should be intuitive. If you transclude a whole buffer, you might be transcluding multiple subtrees. In some cases, this can be a little anti-intuitive. In the following examples, you will be transcluding three subtrees -- even though the first headline levels are lower than the third one, the first two are still the top-most level of their own respective subtrees.

#+begin_example
  ** Headline 1
     Content of Headline 1
  ** Headline 2
     Content of Headline 2
  * Headline 3
    Content of Headline
#+end_example

** Detach transclusion and copy source buffer
:PROPERTIES:
:CUSTOM_ID: detach
:END:

#+cindex: Detach transclusion
#+findex: org-transclusion-detach

Detach is added to the current development version and is planned to be part of release version 1.4.0, and thus available only in GNU-devel ELPA. If you are using GNU ELPA, it will be available with the release of 1.4.0.

    - New command ~org-transclusion-detach~ can be used on the transclusion
      at point. It turns it into a normal, edtitable text content.

      You can undo this operation.

      In addition, you can press ~C-d~ directly on the transclusion at point
      to detach it.  This is because the command is bound to ~C-d~ by
      default in 'org-transclusion-map'.

    - ~org-transclusion-refresh~ now accepts universal argument such as
      ~C-u M-x org-transclusion-refresh~ and detaches the transclusion at
      point.

      You can undo this operation.

    - ~org-transclusion-add~ now accepts universal argument such as 'C-u M-x
      org-transclusion-add' and copies the source content rather than
      transclude it.

      You can undo this operation.

** Live-sync edit
:PROPERTIES:
:CUSTOM_ID: live-sync-edit
:END:

#+cindex: Live-sync edit
#+findex: org-transclusion-live-sync-start
#+findex: org-transclusion-live-sync-exit
#+findex: org-transclusion-live-sync-paste
#+vindex: org-transclusion-live-sync-map

*Experimental.* You can start live-sync edit by pressing =e= (by default) on a text element you want to edit. This will call ~org-transclusion-live-sync-start~ and put a colored overlay on top of the region being live-synced and brings up another buffer that visits the source file of the transclusion. The source buffer will also have a corresponding overlay to the region being edited and live-synced.

If you have other windows open, they will be temporarily hidden -- Org-transclusion will remembers your current window layout and attempts to recover it when you exit live-sync edit.

In the live-sync edit region, you can freely type to edit the transclusion or source regions; they will sync simultaneously.

Once done with editing, press =C-c C-c= to exit live-sync edit. The key is bound to ~org-transclusion-live-sync-exit~. It will turn off the live sync edit but keep the transclusion on.

In the live-sync edit region, the normal ~yank~ command (~C-y~) is replaced with a special command ~org-transclusion-live-sync-paste~. This command lets the pasted text inherit the text-properties of the transcluded region correctly; the normal yank does not have this feature and thus causes some inconvenience in live-sync edit. If you use vim keybindings (e.g. ~evil-mode~), it is advised that you review the default keybindings. You can customize the local keybindings for the live-sync region by ~org-transclusion-live-sync-map~.

*Note*: During live-sync edit, file's content gets saved to the file system as is -- i.e. the transcluded text will be saved instead of the =#+transclude:= keyword. If you kill buffer or quit Emacs, other hooks will still remove the transclusion to keep the file clear of the transcluded copy, leaving only the keyword in the file system.

#+begin_src elisp :exports no
  (substitute-command-keys "\\{org-transclusion-live-sync-map}")
#+end_src

#+name: org-transclusion-live-sync-map
#+caption: Default org-transclusion-live-sync-map
#+begin_example
 key                   binding
 ---                   -------

 C-c                   Prefix Command
 C-y                   org-transclusion-live-sync-paste

 C-c C-c               org-transclusion-live-sync-exit

    † Also inherits ‘org-mode-map’
#+end_example

** Transclude source file into src-block
:PROPERTIES:
:CUSTOM_ID: transclude-source-file-into-src-block
:END:

#+cindex: Transclude into Org's src-block
#+cindex: Property - :src
#+cindex: Property - :rest

This feature is provided as an [[#extensions][extension]] (default on).

You can transclude a source file into an Org's src block. Use the =:src= property and specify the language you would like to use like this:

#+begin_example
#+transclude: [[file:../../test/python-1.py]] :src python
#+end_example

The content you specify in the link gets wrapped into a src-block with the language like this:

#+begin_example
,#+begin_src python
[... content of python-1.py]
,#+end_src
#+end_example

Use =:rest= property to define additional properties you would like to add for the src-block. The double quotation marks are mandatory for the =:rest= property.

#+begin_example
#+transclude: [[file:../../test/python-3.py]]  :src python :rest ":session :results value"
#+end_example

The source block will have the additional properties:
#+begin_example
,#+begin_src python :session :results value
#+end_example

With since 2023-03-28 update, you can also use live-sync the source code
transcluded into src-block. It is planned to be added to the next stable
release.

** Transclude range of lines for text and source files
:PROPERTIES:
:CUSTOM_ID: transclude-range-of-lines-for-text-and-source-files
:END:

#+cindex: Transclude range of lines

This feature is provided as an [[#extensions][extension]] (default on).

When you transclude text files other than Org files, you can use following properties to specify a range of lines to transclude.

*** =:lines= property to specify a range of lines

#+cindex: Property - :lines

You can specify a range of lines to transclude from a source and text file. Use the =:lines= property like this.

#+begin_example
#+transclude: [[file:../../test/test.txt]] :lines 3-5
#+end_example

The rage is specified by the number "3-5"; in this case, lines from 3 to 5, both lines inclusive.

To transclude a single line, have the the same number in both places (e.g. 10-10, meaning line 10 only).

One of the numbers can be omitted. When the first number is omitted (e.g. -10), it means from the beginning of the file to line 10. Likewise, when the second number is omitted (e.g. 10-), it means from line 10 to the end of file.

You can combine =:lines= with =:src= to transclude only a certain range of source files (Example 1 below).

For Org's file links, you can use [[https://orgmode.org/manual/Search-Options.html][search options]] specified by the "::" (two colons) notation. When a search finds a line that includes the string, the Org-transclude counts it as the starting line 1 for the  =:lines= property.

Example 1: This transcludes the four lines of the source file from the line that contains string "id-1234" (including that line counted as line 1).
#+begin_example
#+transclude: [[file:../../test/python-1.py::id-1234]] :lines 1-4 :src python
#+end_example

Example 2: This transcludes only the single line that contains the line found by the search option for text string "Transcendental Ontology"
#+begin_example
#+transclude: [[file:../../test/test.txt::Transcendental Ontology]] :lines 1-1
#+end_example

#+ATTR_TEXINFO: :tag Note
#+begin_quote
Search-options =::/regex/= and =::number= do not work as intended.
#+end_quote

*** =:end= property to specify a search term to dynamically look for the end of a range

#+cindex: Property - :end

You can add =:end= property and specify the search term as its value. Surround the search term with double quotation marks (mandatory).

See Example 3 below. This transclusion will look for =id-1234= as the beginning line of the range as specified by the search option =::id-1234= in the link. With the =:end= property, the search string =id-1234 end here= defines the end of the range. The search looks for =id-123 end here= in the body text. When found, the line one before the search string is used as the ending line (thus, the transcluded range will not contain string =id-1234 end here=).

You can also combine =:lines= with =:end=.  It will only displace the beginning, and the end part of the range (the second number after the hyphen "-") is ignored. In the same example, the beginning of the range is the next line after "id-1234" (line 2).  Instead of transcluding all the text until the end of the buffer, the end is defined by the =:end= property (one line before the string =id-1234 end here= is found).

Example 3:
#+begin_example
#+transclude: [[file:../../test/python-1.py::id-1234]] :lines 2- :src python :end "id-1234 end here"
#+end_example

*** =:thing-at-point= or =:thingatpt= property to specify a "thing" to transclude from the source.

#+cindex: Property - :thing-at-point
#+cindex: Property - :thingatpt

You can add =:thing-at-point= or =:thingatpt= (they are synonyms with each other and both work) property to transclude a certain "thing" at point to be transcluded. The following text elements have been tested to work:

- sentence
- paragraph
- defun
- sexp

#+ATTR_TEXINFO: :tag Note
#+begin_quote
As it stands now, you cannot use =:thing-at-point= or =:thingatpt= to specify the precise beginning of the thing within a line -- it is always the beginning of the line.
#+end_quote

#+ATTR_TEXINFO: :tag Note
#+begin_quote
The definition of sentence is controlled by various variables in Emacs. It is suggested to consult the documentation of function ~sentence-end~. For those  who do not use double-space after a period (in the language this is relevant), it is probably more intuitive to turn off customizing  ~sentence-end-double-space~. The default behavior is this: "Non-nil means a single space does not end a sentence."
#+end_quote

See Example 4. When you have a text file for prose, you can use paragraph or sentence as a "thing" to transclude. In this example, the beginning of the source of transclude is specified by the search option with "::Once upon a time".  The two colons "::" immediately following the file name denote the search option, which is a built-in Org syntax. Then the paragraph or sentence beginning "Once upon a time" is transcluded. When the source file contains multiple occurrences of "Once upon a time", the first one is taken.

Example 4:
#+begin_example
#+transclude: [[./things-at-point-dir/story.txt::Once upon a time][story]]  :thing-at-point paragraph

#+transclude: [[./things-at-point-dir/story.txt::Once upon a time][story]]  :thingatpt sentence
#+end_example

You can combine =:thing-at=point= (or =:thingatpt=) with =:src= property so that the transcluded text is wrapped in Org's source block. Example 5 shows a way to do so. id​:​1234567890 is assumed to be present as comment in the source file.

Example 5:
#+begin_example
#+transclude: [[./things-at-point-dir/baz.el::id:1234567890][barz-baz-fuzz]]  :src elisp

#+transclude: [[./things-at-point-dir/baz.el::foo][barz-baz-fuzz]]  :src elisp :thing-at-point sexp

#+transclude: [[./things-at-point-dir/baz.el::id:1234567890][barz-baz-fuzz]]  :src elisp :thingatpt defun
#+end_example

*** Transclude HTML content with Pandoc
#+cindex: Transclude HTML content with Pandoc

If you have Pandoc installed, you can transclude local HTML files
as Org documents.

This feature is provided as an [[#extensions][extension]] (default off).

#+begin_example
#+transclude: [[file:../test/source-html-no-ext]]
#+end_example

Since it's not currently possible to add anchor links to local =file:=
Org links, HTML transclusions always render the entire document as
Org.  However, packages which transclude HTML documents over a
network, such as [[https://git.sr.ht/~ushin/hyperdrive.el][hyperdrive.el]] and [[https://git.sr.ht/~ushin/org-transclusion-http][org-transclusion-http]], are able to
render specific HTML elements by tag.

** Extensions
:PROPERTIES:
:CUSTOM_ID: extensions
:END:

#+cindex: Extensions
#+vindex: org-transclusion-extensions
#+cindex: Extension - org-transclusion-indent-mode
#+cindex: Extension - org-transclusion-src-lines
#+cindex: Extension - org-transclusion-font-lock
#+cindex: Extension - org-transclusion-html

Org-transclusion provides a simple extension framework, where you can use ~customize~ to selectively add new features.

If you use ~customize~, the features are loaded automatically. Note that it does not "unload" the feature until you relaunch Emacs.

If you do not use ~customize~ (e.g. Doom), you may need to explicitly require an extension. For example, to activate ~org-transclusion-indent-mode~, you might need to add something like this in your configuration file.

 #+BEGIN_SRC emacs-lisp
 ;; Ensure that load-path to org-transclusion is already added
 ;; If you installed it with the built-in package.el, this should be already done.
 ;; (add-to-list  'load-path "path/to/org-transclusion/")
 (add-to-list 'org-transclusion-extensions 'org-transclusion-indent-mode)
 (require 'org-transclusion-indent-mode)
 #+END_SRC

Currently, the following extensions are available.

- (off by default) ~org-transclusion-indent-mode~ ::

  Support org-indent-mode. Known limitation: when transcluded, the source will temporarily lose the indentation. When the transclusion is removed, the source will regain the indentation.

- (on by default) ~org-transclusion-src-lines~ ::
  Add features for =:src= and =:lines= properties to =#+transclude=. It is meant for non-Org files such as program source and text files

- (on by default) ~org-transclusion-font-lock~ ::
  Add font-lock for =#+transclude=. Org mode's standard syntax treats the combination of a =#+transclude:= keyword and a link used by Org-transclusion as a keyword. This means it applies the ~org-meta-line~ face and the link part cannot be toggled as a normal link. This extension adds ~org-transclusion-keyword~ face to the keyword part and lets the link part to be treated as a normal link for ~org-toggle-link-display~.

- (off by default) ~org-transclusion-html~ ::
  Transclude local HTML files by converting them to Org with Pandoc.

* Customizing

#+vindex: org-transclusion-extensions
#+vindex: org-transclusion-add-all-on-activate
#+vindex: org-transclusion-mode-lighter
#+vindex: org-transclusion-open-source-display-action-list
#+vindex: org-transclusion-after-add-functions
#+cindex: Hooks - org-transclusion-after-add-functions

You can customize settings in the customization group `org-transclusion'.

- ~org-transclusion-extensions~ :: Defines extensions to be loaded with
  org-transclusion.el. If you use ~customize~, the extensions are loaded by it.
  If you don't, you likely need to explicitly use ~require~ to load them. See [[#extensions][sub-section]]

- ~org-transclusion-add-all-on-activate~ :: Defines whether or not all the
  active transclusions (with =t=) get automatically transcluded on minor mode
  activation (~org-transclusion-mode~). This does not affect the manual
  activation when you directly call ~org-transclusion-activate~

- ~org-transclusion-exclude-elements~ :: See [[#customizable-filter-to-exclude-certain-org-elements][sub-section]]

- ~org-transclusion-include-first-section~ :: See [[#include-the-section-before-the-first-headline-org-file-only][sub-section]]

- ~org-transclusion-open-source-display-action-list~ :: You can customize the
  way the ~org-transclusion-open-source~ function displays the source buffer for
  the transclusion. You specify the "action" in the way defined by the built-in
  ~display-buffer~ function. Refer to its in-system documentation (with ~C-h f~)
  for the accepted values. ~M-x customize~ can also guide you on what types of
  values are accepted.

- ~org-transclusion-mode-lighter~ :: Define the lighter for Org-transclusion
  minor mode. The default is " OT".

- ~org-transclusion-after-add-functions~ :: Functions to be called after a
  transclusion content has been added. The hook runs after the content and the
  read-only text property have been added so it is not supposed to manipulate
  the content but to add further text properties.  For example, it is used by
  the `org-transclusion-indent-mode' extension to support `org-indent-mode'.
  The functions are called with arguments beg and end, pointing to the
  beginning and end of the transcluded content.

The hook can be also used to activate ~org-latex-preview~ for transclusions:

#+begin_src elisp
  (add-hook 'org-transclusion-after-add-functions
            #'(lambda (_beg _end) (org-latex-preview '(16))))
#+end_src

** Customizable filter to exclude certain Org elements
:PROPERTIES:
:CUSTOM_ID: customizable-filter-to-exclude-certain-org-elements
:END:

#+vindex: org-transclusion-exclude-elements

Set customizable variable ~org-transclusion-exclude-elements~ to define which elements to be *excluded* in the transclusion.

The filter works for all supported types of links within an Org file when transcluding an entire Org file, and parts of it (headlines, custom ID, etc.). There is no filter for non-Org files.

It is a list of symbols, and the default is =(property-drawer)=. The accepted values are the ones defined by ~org-element-all-elements~ (Org's standard set of elements; refer to its documentation for an exhaustive list).

You can also fine-tune the exclusion filter per transclusion. Refer to the sub-section on [[#filtering-org-elements-per-transclusion][filtering Org elements per transclusion]].

** Include the section before the first headline (Org file only)
:PROPERTIES:
:CUSTOM_ID: include-the-section-before-the-first-headline-org-file-only
:END:

#+vindex: org-transclusion-include-first-section

You can include the first section (section before the first headline) of an Org file. It is toggled via customizable variable ~org-transclusion-include-first-section~. Its default value is =t=. Set it to =t= (or non-nil) to transclude the first section. It also works when the first section is followed by headlines.

** Faces & fringe bitmap
:PROPERTIES:
:CUSTOM_ID: faces
:END:

#+vindex: org-transclusion-keyword
#+vindex: org-transclusion-source-fringe
#+vindex: org-transclusion-fringe
#+vindex: org-transclusion-source
#+vindex: org-transclusion-source-edit
#+vindex: org-transclusion
#+vindex: org-transclusion-edit
#+vindex: org-transclusion-fringe-bitmap

*** Face for the =#+transclude= keyword

This feature is provided as an [[#extensions][extension]] (default on).

- ~org-transclusion-keyword~ ::

  You can set your own face to the =#+transclude= keyword with using the ~org-transclusion-keyword~ face.

*** Faces for the fringes next to transcluded region and source region

If the fringes that indicate transcluding and source regions are not visible in your system (e.g. Doom), try adding background and/or foreground colors to these custom faces.

- ~org-transclusion-source-fringe~
- ~org-transclusion-fringe~

To customize a face, it's probably the easiest to use ~M-x customize-face~. If you want to use Elisp for some reason (e.g. on Doom), something like this below should set faces. Experiment with the colors of your choice. By default, the faces above have no values.

#+BEGIN_SRC elisp
(set-face-attribute
 'org-transclusion-fringe nil
 :foreground "green"
 :background "green")
#+END_SRC

For colors, where "green" is,  you can also use something like "#62c86a" (Emacs calls it "RGB triple"; you can refer to in-system manual Emacs > Colors). You might also like to refer to a list of currently defined faces in your Emacs by ~list-faces-display~.

Other faces:

The default for faces ~org-transclusion~ and ~org-transclusion-source~ is no color or any other specification (transparent). An intended use of these faces is when you use packages such as ~org-indent-mode~ or ~org-modern~, which might override the fringe ~org-transclusion~ places. To make the translusion and its source visible, you might like to put some background color to source and transclusion.

- ~org-transclusion-source~ ::
  The default is no color specification (transparent).
- ~org-transclusion-source-edit~
- ~org-transclusion~ ::
  The default is no color specification (transparent).
- ~org-transclusion-edit~
- ~org-transclusion-fringe-bitmap~ ::
  It is used for the fringe that indicates the transcluded region. It works only in on a graphical display (not on a text-only terminal).

** Keybindings
#+vindex: org-transclusion-map
#+vindex: org-transclusion-live-sync-map

- ~org-transclusion-map~
#+transclude: [[./org-transclusion-manual.org::org-transclusion-map]]

- ~org-transclusion-live-sync-map~
#+transclude: [[./org-transclusion-manual.org::org-transclusion-live-sync-map]]

* Known Limitations

Note this section is still incomplete, not exhaustive for "known" limitations.

- Org link's search-options =::/regex/= and =::number= do not work as intended.

- ~org-transclusion-live-sync-start~ does not support all Org elements ::
  For transclusions of Org elements or buffers, live-sync works only on the following elements:
  =center-block=, =drawer=, =dynamic-block=, =latex-environment=, =paragraph=, =plain-list=, =quote-block=, =special-block=, =table=, and =verse-block=.

  It is known that live-sync does not work for the other elements; namely:
  =comment-block=, =export-block=, =example-block=, =fixed-width=, =keyword=, =src-block=, and =property-drawer=.

  More technical reason for this limitation is documented in the docstring of function ~org-transclusion-live-sync-enclosing-element~.

  Work is in progress to lift this limitation but I'm still experimenting different ideas.

- ~org-indent-mode~ may not work well with Org-transclusion ::
  A new extension has been added to support ~org-indent-mode~ Refer to [[#extensions][this section]].

- Doom's customization may interfere with Org-transclusion ::
  Refer to issue [[https://github.com/nobiot/org-transclusion/issues/52][#52]]. The symptom is that in Doom you get an error message that includes this: "progn: ‘recenter’ing a window that does not display current-buffer." Adding this in your configuration has been reported to fix the issue:

  ~(advice-remove 'org-link-search '+org--recenter-after-follow-link-a)~

  It is probably rather drastic a measure. I will appreciate it if you find a less drastic way that works. Thank you.

- Org refile does not work "properly" on the transcluded headlines ::
  Refer to issue [[https://github.com/nobiot/org-transclusion/issues/20][#20]]. I don't intend to support this -- refile the source, not the transcluded copy.

- Undo detach does not add the overlay back on the source ::
  This should not break any feature. You can safely refresh the transclusion and recover the source overlay. You can also safely open or moved to the source while the source overlay is not present.

- For =:thing-at-point= or =:thingatpt=, you cannot use them to specify the precise beginning of the thing within a line -- it is always the beginning of the line.

* Credits
** Original idea by John Kitchin
:PROPERTIES:
:CUSTOM_ID: original-idea-by-john-kitchin
:END:

https://github.com/alphapapa/transclusion-in-emacs#org-mode

#+begin_quote
{O} transcluding some org-elements in multiple places
[2016-12-09 Fri] John Kitchin asks:

I have an idea for how I could transclude “copies” or links to org-elements in multiple places and keep them up to date. A prototypical example of this is I have a set of org-contacts in one place, and I want to create a new list of people for a committee in a new place made of “copies” of the contact headlines. But I do not really want to duplicate the headlines, and if I modify one, I want it reflected in the other places. I do not want just links to those contacts, because then I can not do things with org-map-entries, and other org-machinery which needs the actual headlines/properties present. Another example might be I want a table in two places, but the contents of them should stay synchronized, ditto for a code block.

This idea was inspired by https://github.com/gregdetre/emacs-freex.

The idea starts with creating (wait for it…) a new link ;) In a document where I want to transclude a headline, I would enter something like:

transclude:some-file.org::*headline title

Then, I would rely on the font-lock system to replace that link with the headline and its contents (via the :activate-func link property), and to put an overlay on it with a bunch of useful properties, including modification hooks that would update the source if I change the the element in this document, and some visual indication that it is transcluded (e.g. light gray background/tooltip).

I would create a kill-buffer hook function that would replace that transcluded content with the original link. A focus-in hook function would make sure the transcluded content is updated when you enter the frame. So when the file is not open, there is just a transclude link indicating what should be put there, and when it is open, the overlay modification hooks and focus hook should ensure everything stays synchronized (as long as external processes are not modifying the contents).

It seems like this could work well for headlines, and named tables, src blocks, and probably any other element that can be addressed by a name/ID.
#+end_quote

** Text-Clone
=text-clone.el= is an extension of text-clone functions written as part of GNU Emacs in =subr.el=.  The first adaption to extend text-clone functions to work across buffers was published in StackExchange by the user named Tobias in March 2020. It can be found at https://emacs.stackexchange.com/questions/56201/is-there-an-emacs-package-which-can-mirror-a-region/56202#56202. The text-clone library takes this line of work further.

* Contributing

#+transclude: [[../README.org::*Contributing]] :only-contents

* Index - Features
:PROPERTIES:
:CUSTOM_ID: cindex
:APPENDIX: t
:INDEX:    cp
:DESCRIPTION: Key concepts & features
:END:

* Index - Commands
:PROPERTIES:
:APPENDIX: t
:INDEX:    fn
:DESCRIPTION: Interactive functions
:END:

* Index - User Options
:PROPERTIES:
:APPENDIX: t
:INDEX:    vr
:DESCRIPTION: Customizable variables & faces
:END:

* GNU Free Documentation License
:PROPERTIES:
:appendix: t
:END:

#+texinfo: @include fdl.texi

#  LocalWords:  href img src devel GPLv texinfo insertcopying toc RET findex
#  LocalWords:  vindex cindex dir

# Local Variables:
# time-stamp-start: "modified +\\\\?"
# End: