MKSQUASHFS - a tool to create Squashfs filesystems
As Squashfs is a read-only filesystem, the Mksquashfs program must be used to
create populated squashfs filesystems.
SYNTAX:mksquashfs source1 source2 ... FILESYSTEM [OPTIONS] [-e list of
exclude dirs/files]
Filesystem compression options:
-b <block_size> set data block to <block_size>. Default 128 Kbytes.
Optionally a suffix of K or M can be given to specify
Kbytes or Mbytes respectively
-comp <comp> select <comp> compression
Compressors available:
gzip (default)
lzo
lz4
xz
zstd
-noI do not compress inode table
-noId do not compress the uid/gid table (implied by -noI)
-noD do not compress data blocks
-noF do not compress fragment blocks
-noX do not compress extended attributes
-no-compression do not compress any of the data or metadata. This is
equivalent to specifying -noI -noD -noF and -noX
Filesystem build options:
-tar read uncompressed tar file from standard in (stdin)
-no-strip act like tar, and do not strip leading directories
from source files
-tarstyle alternative name for -no-strip
-cpiostyle act like cpio, and read file pathnames from standard in
(stdin)
-cpiostyle0 like -cpiostyle, but filenames are null terminated. Can
be used with find -print0 action
-reproducible build filesystems that are reproducible (default)
-not-reproducible build filesystems that are not reproducible
-mkfs-time <time> set filesystem creation timestamp to <time>. <time> can
be an unsigned 32-bit int indicating seconds since the
epoch (1970-01-01) or a string value which is passed to
the "date" command to parse. Any string value which the
date command recognises can be used such as "now",
"last week", or "Wed Feb 15 21:02:39 GMT 2023"
-all-time <time> set all file timestamps to <time>. <time> can be an
unsigned 32-bit int indicating seconds since the epoch
(1970-01-01) or a string value which is passed to the
"date" command to parse. Any string value which the date
command recognises can be used such as "now", "last
week", or "Wed Feb 15 21:02:39 GMT 2023"
-root-time <time> set root directory time to <time>. <time> can be an
unsigned 32-bit int indicating seconds since the epoch
(1970-01-01) or a string value which is passed to the
"date" command to parse. Any string value which the date
command recognises can be used such as "now", "last
week", or "Wed Feb 15 21:02:39 GMT 2023"
-root-mode <mode> set root directory permissions to octal <mode>
-root-uid <value> set root directory owner to specified <value>,
<value> can be either an integer uid or user name
-root-gid <value> set root directory group to specified <value>,
<value> can be either an integer gid or group name
-all-root make all files owned by root
-force-uid <value> set all file uids to specified <value>, <value> can be
either an integer uid or user name
-force-gid <value> set all file gids to specified <value>, <value> can be
either an integer gid or group name
-pseudo-override make pseudo file uids and gids override -all-root,
-force-uid and -force-gid options
-no-exports do not make filesystem exportable via NFS (-tar default)
-exports make filesystem exportable via NFS (default)
-no-sparse do not detect sparse files
-no-tailends do not pack tail ends into fragments (default)
-tailends pack tail ends into fragments
-no-fragments do not use fragments
-no-duplicates do not perform duplicate checking
-no-hardlinks do not hardlink files, instead store duplicates
-keep-as-directory if one source directory is specified, create a root
directory containing that directory, rather than the
contents of the directory
Filesystem filter options:
-p <pseudo-definition> add pseudo file definition. The definition should
be quoted
-pf <pseudo-file> add list of pseudo file definitions from <pseudo-file>,
use - for stdin. Pseudo file definitions should not be
quoted
-sort <sort_file> sort files according to priorities in <sort_file>. One
file or dir with priority per line. Priority -32768 to
32767, default priority 0
-ef <exclude_file> list of exclude dirs/files. One per line
-wildcards allow extended shell wildcards (globbing) to be used in
exclude dirs/files
-regex allow POSIX regular expressions to be used in exclude
dirs/files
-max-depth <levels> descend at most <levels> of directories when scanning
filesystem
-one-file-system do not cross filesystem boundaries. If a directory
crosses the boundary, create an empty directory for
each mount point. If a file crosses the boundary
ignore it
-one-file-system-x do not cross filesystem boundaries. Like
-one-file-system option except directories are also
ignored if they cross the boundary
Filesystem extended attribute (xattrs) options:
-no-xattrs do not store extended attributes
-xattrs store extended attributes (default)
-xattrs-exclude <regex> exclude any xattr names matching <regex>. <regex> is a
POSIX regular expression, e.g. -xattrs-exclude '^user.'
excludes xattrs from the user namespace
-xattrs-include <regex> include any xattr names matching <regex>. <regex> is a
POSIX regular expression, e.g. -xattrs-include '^user.'
includes xattrs from the user namespace
-xattrs-add <name=val> add the xattr <name> with <val> to files. If an
user xattr it will be added to regular files and
directories (see man 7 xattr). Otherwise it will be
added to all files. <val> by default will be treated as
binary (i.e. an uninterpreted byte sequence), but it can
be prefixed with 0s, where it will be treated as base64
encoded, or prefixed with 0x, where val will be treated
as hexidecimal. Additionally it can be prefixed with
0t where this encoding is similar to binary encoding,
except backslashes are specially treated, and a
backslash followed by 3 octal digits can be used to
encode any ASCII character, which obviously can be used
to encode control codes. The option can be repeated
multiple times to add multiple xattrs
Mksquashfs runtime options:
-version print version, licence and copyright message
-exit-on-error treat normally ignored errors as fatal
-quiet no verbose output
-info print files written to filesystem
-no-progress do not display the progress bar
-progress display progress bar when using the -info option
-percentage display a percentage rather than the full progress bar.
Can be used with dialog --gauge etc.
-throttle <percentage> throttle the I/O input rate by the given percentage.
This can be used to reduce the I/O and CPU consumption
of Mksquashfs
-limit <percentage> limit the I/O input rate to the given percentage.
This can be used to reduce the I/O and CPU consumption
of Mksquashfs (alternative to -throttle)
-processors <number> use <number> processors. By default will use number of
processors available
-mem <size> use <size> physical memory for caches. Use K, M or G to
specify Kbytes, Mbytes or Gbytes respectively
-mem-percent <percent> use <percent> physical memory for caches. Default 25%
-mem-default print default memory usage in Mbytes
Filesystem append options:
-noappend do not append to existing filesystem
-root-becomes <name> when appending source files/directories, make the
original root become a subdirectory in the new root
called <name>, rather than adding the new source items
to the original root
-no-recovery do not generate a recovery file
-recovery-path <name> use <name> as the directory to store the recovery file
-recover <name> recover filesystem data using recovery file <name>
Filesystem actions options:
-action <action@expr> evaluate <expr> on every file, and execute <action>
if it returns TRUE
-log-action <act@expr> as above, but log expression evaluation results and
actions performed
-true-action <act@expr> as above, but only log expressions which return TRUE
-false-action <act@exp> as above, but only log expressions which return FALSE
-action-file <file> as action, but read actions from <file>
-log-action-file <file> as -log-action, but read actions from <file>
-true-action-file <f> as -true-action, but read actions from <f>
-false-action-file <f> as -false-action, but read actions from <f>
Tar file only options:
-default-mode <mode> tar files often do not store permissions for
intermediate directories. This option sets the default
directory permissions to octal <mode>, rather than 0755.
This also sets the root inode mode
-default-uid <uid> tar files often do not store uids for intermediate
directories. This option sets the default directory
owner to <uid>, rather than the user running Mksquashfs.
This also sets the root inode uid
-default-gid <gid> tar files often do not store gids for intermediate
directories. This option sets the default directory
group to <gid>, rather than the group of the user
running Mksquashfs. This also sets the root inode gid
-ignore-zeros allow tar files to be concatenated together and fed to
Mksquashfs. Normally a tarfile has two consecutive 512
byte blocks filled with zeros which means EOF and
Mksquashfs will stop reading after the first tar file on
encountering them. This option makes Mksquashfs ignore
the zero filled blocks
Expert options (these may make the filesystem unmountable):
-nopad do not pad filesystem to a multiple of 4K
-offset <offset> skip <offset> bytes at the beginning of FILESYSTEM.
Optionally a suffix of K, M or G can be given to specify
Kbytes, Mbytes or Gbytes respectively.
Default 0 bytes
-o <offset> synonym for -offset
Miscellaneous options:
-fstime <time> alternative name for -mkfs-time
-always-use-fragments alternative name for -tailends
-root-owned alternative name for -all-root
-noInodeCompression alternative name for -noI
-noIdTableCompression alternative name for -noId
-noDataCompression alternative name for -noD
-noFragmentCompression alternative name for -noF
-noXattrCompression alternative name for -noX
-help output this options text to stdout
-h output this options text to stdout
-Xhelp print compressor options for selected compressor
Pseudo file definition format:
"filename d mode uid gid" create a directory
"filename m mode uid gid" modify filename
"filename b mode uid gid major minor" create a block device
"filename c mode uid gid major minor" create a character device
"filename f mode uid gid command" create file from stdout of command
"filename s mode uid gid symlink" create a symbolic link
"filename i mode uid gid [s|f]" create a socket (s) or FIFO (f)
"filename x name=val" create an extended attribute
"filename l linkname" create a hard-link to linkname
"filename L pseudo_filename" same, but link to pseudo file
"filename D time mode uid gid" create a directory with timestamp time
"filename M time mode uid gid" modify a file with timestamp time
"filename B time mode uid gid major minor"
create block device with timestamp time
"filename C time mode uid gid major minor"
create char device with timestamp time
"filename F time mode uid gid command" create file with timestamp time
"filename S time mode uid gid symlink" create symlink with timestamp time
"filename I time mode uid gid [s|f]" create socket/fifo with timestamp time
Source1 source2 ... are the source directories/files containing the
files/directories that will form the squashfs filesystem. If a single
directory is specified (i.e. mksquashfs source image.sqfs) the squashfs
filesystem will consist of that directory, with the top-level root
directory containing the contents.
If multiple source directories or files are specified, Mksquashfs will merge
the specified sources into a single filesystem, with the root directory
containing each of the source files/directories. The name of each directory
entry will be the basename of the source path. If more than one source
entry maps to the same name, the conflicts are named xxx_1, xxx_2, etc. where
xxx is the original name.
To make this clear, take two example directories. Source directory
"/home/phillip/test" contains "file1", "file2" and "dir1".
Source directory "goodies" contains "goodies1", "goodies2" and "goodies3".
usage example 1:
% mksquashfs /home/phillip/test image.sqfs
This will generate a squashfs filesystem with root entries
"file1", "file2" and "dir1".
example 2:
% mksquashfs /home/phillip/test goodies image.sqfs
This will create a squashfs filesystem with the root containing
entries "test" and "goodies" corresponding to the source
directories "/home/phillip/test" and "goodies".
example 3:
% mksquashfs /home/phillip/test goodies test image.sqfs
This is the same as the previous example, except a third
source directory "test" has been specified. This conflicts
with the first directory named "test" and will be renamed "test_1".
Multiple sources allow filesystems to be generated without needing to
copy all source files into a common directory. This simplifies creating
filesystems.
The -keep-as-directory option can be used when only one source directory
is specified, and you wish the root to contain that directory, rather than
the contents of the directory. For example:
example 4:
% mksquashfs /home/phillip/test image.sqfs -keep-as-directory
This is the same as example 1, except for -keep-as-directory.
This will generate a root directory containing directory "test",
rather than the "test" directory contents "file1", "file2" and "dir1".
If you want the full path to retained (like TAR behaviour), you can specify the
-no-strip option.
example 5:
% mksquashfs /home/phillip/test image.sqfs -no-strip
This will make a filesystem with "home", "home/phillip" and "home/phillip/test"
directories, see section 4 for more details.
The Dest argument is the destination where the squashfs filesystem will be
written. This can either be a conventional file or a block device. If the file
doesn't exist it will be created, if it does exist and a squashfs
filesystem exists on it, Mksquashfs will append. The -noappend option will
write a new filesystem irrespective of whether an existing filesystem is
present. See section 12 for more details about appending.
1. CHANGING COMPRESSION ALGORITHM AND COMPRESSION SPECIFIC OPTIONS
------------------------------------------------------------------
By default Mksquashfs will compress using the GZIP compression algorithm. This
algorithm offers a good trade-off between compression ratio, and memory and time
taken to decompress.
Squashfs also supports LZ4, LZO, XZ and ZSTD compression. LZO offers worse
compression ratio than GZIP, but is faster to decompress. XZ offers better
compression ratio than GZIP, but at the expense of greater memory and time
to decompress (and significantly more time to compress). LZ4 is similar
to LZO. ZSTD has been developed by Facebook, and aims to compress well and
be fast to decompress. You should experiment with the compressors to
see which is best for you.
If you're not building the squashfs-tools and kernel from source, then the tools
and kernel may or may not have been built with support for LZ4, LZO, XZ or ZSTD
compression. The compression algorithms supported by the build of Mksquashfs
can be found by typing mksquashfs -help.
The full list of compressors available and their compression specific options
are:
Compressors available and compressor specific options:
gzip (default)
-Xcompression-level <compression-level>
<compression-level> should be 1 .. 9 (default 9)
-Xwindow-size <window-size>
<window-size> should be 8 .. 15 (default 15)
-Xstrategy strategy1,strategy2,...,strategyN
Compress using strategy1,strategy2,...,strategyN in turn
and choose the best compression.
Available strategies: default, filtered, huffman_only,
run_length_encoded and fixed
lzo
-Xalgorithm <algorithm>
Where <algorithm> is one of:
lzo1x_1
lzo1x_1_11
lzo1x_1_12
lzo1x_1_15
lzo1x_999 (default)
-Xcompression-level <compression-level>
<compression-level> should be 1 .. 9 (default 8)
Only applies to lzo1x_999 algorithm
lz4
-Xhc
Compress using LZ4 High Compression
xz
-Xbcj filter1,filter2,...,filterN
Compress using filter1,filter2,...,filterN in turn
(in addition to no filter), and choose the best compression.
Available filters: x86, arm, armthumb, powerpc, sparc, ia64
-Xdict-size <dict-size>
Use <dict-size> as the XZ dictionary size. The dictionary size
can be specified as a percentage of the block size, or as an
absolute value. The dictionary size must be less than or equal
to the block size and 8192 bytes or larger. It must also be
storable in the xz header as either 2^n or as 2^n+2^(n+1).
Example dict-sizes are 75%, 50%, 37.5%, 25%, or 32K, 16K, 8K
etc.
zstd
-Xcompression-level <compression-level>
<compression-level> should be 1 .. 22 (default 15)
If the compressor offers compression specific options (all the compressors now
have compression specific options except the deprecated lzma1 compressor)
then these options are also displayed (.i.e. in the above XZ is shown with two
compression specific options). The compression specific options are, obviously,
specific to the compressor in question, and the compressor documentation and
web sites should be consulted to understand their behaviour. In general
the Mksquashfs compression defaults for each compressor are optimised to
give the best performance for each compressor, where what constitutes
best depends on the compressor. For GZIP/XZ best means highest compression,
for LZO/LZ4 best means a tradeoff between compression and (de)-compression
overhead (LZO/LZ4 by definition are intended for weaker processors).
2. REDUCING CPU AND I/O USAGE
-----------------------------
By default Mksquashfs will use all the CPUs available to compress and create the
filesystem, and will read from the source files and write to the output
filesystem as fast as possible. This maximises both CPU usage and I/O.
Sometimes you don't want Mksquashfs to use all CPU and I/O bandwidth. For those
cases Mksquashfs supports two complementary options, -processors and -throttle.
The -processors option can be used to reduce the number of parallel compression
threads used by Mksquashfs. Reducing this to 1 will create the minimum number
of threads, and this will reduce CPU usage, and that in turn will reduce I/O
(because CPUs are normally the bottleneck).
The -throttle option reduces the speed Mksquashfs reads from the source files.
The value is a percentage (obviously from 1 - 100), and 50 will reduce the
read rate by half (the read thread will spend half its time idling), and 75
by three quarters. Reducing the speed of I/O will also reduce the CPU
usage as there is insufficient data rate to use all cores.
Which option should you use? Both will effectively reduce CPU and I/O in
normal cases where intensive use is being made of both I/O and CPUs. But
in edge cases there can be an imbalance where reducing one has no effect, or
it can't be reduced any further. For instance when there is only 1 or 2 cores
available, setting -processors to the minimum of 1 may still use too much
CPU. Additionally if your input source is slow Mksquashfs may still max it out
with -processors set to the minimum of 1. In this case you can use -throttle
in addition to -processors or on its own.
3. CHANGING GLOBAL COMPRESSION DEFAULTS USED IN MKSQUASHFS
----------------------------------------------------------
There are a large number of options that can be used to control the
compression in Mksquashfs. By and large the defaults are the most
optimum settings and should rarely need to be changed.
Note, this does not apply to the block size, increasing the block size from the
default of 128 Kbytes will increase compression (especially for the XZ and ZSTD
compressors) and should increase I/O performance too. However, a block size of
greater than 128 Kbytes may increase latency in certain cases (where the
filesystem contains lots of fragments, and no locality of reference is
observed). For this reason the block size default is configured to the less
optimal 128 Kbytes. Users should experiment with 256 Kbyte sizes or above.
The -b option allows the block size to be selected, both "K" and "M" postfixes
are supported, this can be either 4K, 8K, 16K, 32K, 64K, 128K, 256K, 512K or
1M bytes.
The -noI, -noD, -noF and -noX options can be used to force Mksquashfs to not
compress inodes/directories, data, fragments and extended attributes
respectively. Giving all options generates an uncompressed filesystem.
The -no-fragments option tells Mksquashfs to not generate fragment blocks. A
fragment block consists of multiple small files (all smaller than the block
size) packed and compressed together. This produces much better compression
than storing and compressing these files separately. It also typically
improves I/O as multiple files in the same directory are read at the same time.
You don't want to enable this option unless you fully understand the effects.
The -tailends option tells Mksquashfs to always generate fragments for files
irrespective of the file length. By default only small files less than the data
block size are packed into fragment blocks. The tail ends of files which do not
fit fully into a block, are NOT by default packed into fragments. This is a
legacy setting when hard disks were mechanical, and had slow seek times. In
general setting this option will gain a little more compression, without
affecting I/O performance.
The -no-duplicates option tells Mksquashfs to not check the files being added to
the filesystem for duplicates. This can result in quicker filesystem
generation although obviously compression will suffer badly if there is a lot
of duplicates.
4. TAR STYLE HANDLING OF SOURCE PATHNAMES IN MKSQUASHFS
-------------------------------------------------------
Mksquashfs has always stripped the leading directories of any source pathnames
given on the command line.
For example given the command line
% mksquashfs dir-a/dir-b/dir-c/file1 dir-A/dir-B/file2 sqfs
Mksquashfs will strip the leading directories, and put file1 and file2 into
the same root directory. If file1 and file2 are directories it will place the
directories into the same root directory, but, if only one directory is
specified, it will put the contents of that directory into the root directory.
Obviously, for a lot of the time this is what you want. But, if it isn't what
you want it can be quite difficult to get Mksquashfs to do something different.
A lot of people don't like this, and would prefer Mksquashfs acted like "tar",
which does not strip leading directories. This allows you to create a directory
hierarchy from the pathnames of the supplied files. In the above example, the
tar archive would contain the pathnames "dir-a/dir-b/dir-c/file1" and
"dir-A/dir-B/file2".
Mksquashfs will act like tar if given the option -no-strip, or -tarstyle.
5. CPIO STYLE HANDLING OF SOURCE PATHNAMES IN MKSQUASHFS
--------------------------------------------------------
Mksquashfs allows you to pipe the set of files to be added to the filesystem to
standard in (stdin), if the -cpiostyle option is given.
As with cpio, directories are not recursively scanned and their contents added.
Evey file to be added to the filesystem must be explicitly specified.
Typically the list of files to be added will be produced via find, or a similar
utility.
For example
% find /home/phillip/squashfs-tools | mksquashfs - img.sqfs -cpiostyle
Will create an image containing everything in squashfs-tools and its
sub-directories. Note, "-" is given as the source pathname in Mksquashfs, and
indicates no commmand line sources.
The following will add just the files ending in .c, .h and .o.
% find /home/phillip/squashfs-tools -name "*.[cho]" | mksquashfs - img.sqfs -cpiostyle
6. SPECIFYING THE UIDs/GIDs USED IN THE FILESYSTEM
--------------------------------------------------
By default files in the generated filesystem inherit the UID and GID ownership
of the original file. However, Mksquashfs provides a number of options which
can be used to override the ownership.
The -all-root option forces all file uids/gids in the generated Squashfs
filesystem to be root. This allows root owned filesystems to be built without
root access on the host machine.
The "-force-uid uid" option forces all files in the generated Squashfs
filesystem to be owned by the specified uid. The uid can be specified either by
name (i.e. "root") or by number.
The "-force-gid gid" option forces all files in the generated Squashfs
filesystem to be group owned by the specified gid. The gid can be specified
either by name (i.e. "root") or by number.
7. EXCLUDING FILES FROM THE FILESYSTEM
--------------------------------------
The -e and -ef options allow files/directories to be specified which are
excluded from the output filesystem. The -e option takes the exclude
files/directories from the command line, the -ef option takes the
exlude files/directories from the specified exclude file, one file/directory
per line.
Two styles of exclude file matching are supported: basic exclude matching, and
extended wildcard matching. Basic exclude matching is a legacy feature
retained for backwards compatibility with earlier versions of Mksquashfs.
Extended wildcard matching should be used in preference.
7.1 BASIC EXCLUDE MATCHING
--------------------------
Each exclude file is treated as an exact match of a file/directory in
the source directories. If an exclude file/directory is absolute (i.e.
prefixed with /, ../, or ./) the entry is treated as absolute, however, if an
exclude file/directory is relative, it is treated as being relative to each of
the sources in turn, i.e.
% mksquashfs /tmp/source1 source2 output_fs -e ex1 /tmp/source1/ex2 out/ex3
Will generate exclude files /tmp/source1/ex2, /tmp/source1/ex1, source2/ex1,
/tmp/source1/out/ex3 and source2/out/ex3.
7.2 EXTENDED EXCLUDE FILE HANDLING
----------------------------------
Extended exclude file matching treats each exclude file as a wildcard or
regex expression. To enable wildcard matching specify the -wildcards
option, and to enable regex matching specify the -regex option. In most
cases the -wildcards option should be used rather than -regex because wildcard
matching behaviour is significantly easier to understand!
In addition to wildcards/regex expressions, exclude files can be "anchored" or
"non-anchored". An anchored exclude is one which matches from the root of the
directory and nowhere else, a non-anchored exclude matches anywhere. For
example given the directory hierarchy "a/b/c/a/b", the anchored exclude
"a/b" will match "a/b" at the root of the directory hierarchy, but
it will not match the "/a/b" sub-directory within directory "c", whereas a
non-anchored exclude would.
A couple of examples should make this clearer.
Anchored excludes
1. mksquashfs example image.sqsh -wildcards -e 'test/*.gz'
Exclude all files matching "*.gz" in the top level directory "test".
2. mksquashfs example image.sqsh -wildcards -e '*/[Tt]est/example*'
Exclude all files beginning with "example" inside directories called
"Test" or "test", that occur inside any top level directory.
Using extended wildcards, negative matching is also possible.
3. mksquashfs example image.sqsh -wildcards -e 'test/!(*data*).gz'
Exclude all files matching "*.gz" in top level directory "test",
except those with "data" in the name.
Non-anchored excludes
By default excludes match from the top level directory, but it is
often useful to exclude a file matching anywhere in the source directories.
For this non-anchored excludes can be used, specified by pre-fixing the
exclude with "...".
Examples:
1. mksquashfs example image.sqsh -wildcards -e '... *.gz'
Exclude files matching "*.gz" anywhere in the source directories.
For example this will match "example.gz", "test/example.gz", and
"test/test/example.gz".
2. mksquashfs example image.sqsh -wildcards -e '... [Tt]est/*.gz'
Exclude files matching "*.gz" inside directories called "Test" or
"test" that occur anywhere in the source directories.
Again, using extended wildcards, negative matching is also possible.
3. mksquashfs example image.sqsh -wildcards -e '... !(*data*).gz'
Exclude all files matching "*.gz" anywhere in the source directories,
except those with "data" in the name.
8. FILTERING AND ADDING EXTENDED ATTRIBUTES (XATTRs)
----------------------------------------------------
Mksquashfs has a number of options which allow extended attributes (xattrs) to
be filtered from the source files or added to the created Squashfs filesystem.
The -no-xattrs option removes any extended attributes which may exist in the
source files, and creates a filesystem without any extended attributes.
The -xattrs-exclude option specifies a regular expression (regex), which
removes any extended attribute that matches the regular expression from all
files. For example the regex '^user.' will remove all User extended attributes.
The -xattrs-include option instead specifies a regular expression (regex)
which includes any extended attribute that matches, and removes anything
that does't match. For example the regex '^user.' will only keep User
extended attributes and anything else will be removed.
Mksquashfs also allows you to add extended attributes to files in the Squashfs
filesystem using the -xattrs-add option. This option takes an xattr name and
value pair separated by the '=' character.
The extended attribute name can be any valid name and can be in the namespaces
security, system, trusted, or user. User extended attributes are added to files
and directories (see man 7 xattr for explanation), and the others are added to
all files.
The extended attribute value by default will be treated as binary (i.e. an
uninterpreted byte sequence), but it can be prefixed with 0s, where it will be
treated as base64 encoded, or prefixed with 0x, where it will be treated as
hexidecimal.
Obviously using base64 or hexidecimal allows values to be used that cannot be
entered on the command line such as non-printable characters etc. But it
renders the string non-human readable. To keep readability and to allow
non-printable characters to be entered, the 0t prefix is supported. This
encoding is similar to binary encoding, except backslashes are specially
treated, and a backslash followed by three octal digits can be used to encode
any ASCII character, which obviously can be used to encode non-printable values.
The following four command lines are equivalent
mksquashfs dir image.sqfs -xattrs-add "user.comment=hello world"
mksquashfs dir image.sqfs -xattrs-add "user.comment=0saGVsbG8gd29ybGQ="
mksquashfs dir image.sqfs -xattrs-add "user.comment=0x68656c6c6f20776f726c64"
mksquashfs dir image.sqfs -xattrs-add "user.comment=0thello world"
Obviously in the above example there are no non-printable characters and so
the 0t prefixed string is identical to the first line. The following three
command lines are identical, but where the space has been replaced by the
non-printable NUL '\0' (null character).
mksquashfs dir image.sqfs -xattrs-add "user.comment=0thello\000world"
mksquashfs dir image.sqfs -xattrs-add "user.comment=0saGVsbG8Ad29ybGQ="
mksquashfs dir image.sqfs -xattrs-add "user.comment=0x68656c6c6f00776f726c64"
9. PSEUDO FILE SUPPORT
----------------------
Mksquashfs supports pseudo files, these allow files, directories, character
devices, block devices, fifos, symbolic links, hard links and extended
attributes to be specified and added to the Squashfs filesystem being built,
rather than requiring them to be present in the source files. This, for
example, allows device nodes to be added to the filesystem without requiring
root access.
Pseudo files also support "dynamic pseudo files" and a modify operation.
Dynamic pseudo files allow files to be dynamically created when Mksquashfs
is run, their contents being the result of running a command or piece of
shell script. The modify operation allows the mode/uid/gid of an existing
file in the source filesystem to be modified.
Two Mksquashfs options are supported, -p allows one pseudo file to be specified
on the command line, and -pf allows a pseudo file to be specified containing a
list of pseduo definitions, one per line.
9.1 CREATING A DYNAMIC FILE
---------------------------
Pseudo definition
Filename f mode uid gid command
mode is the octal mode specifier, similar to that expected by chmod.
uid and gid can be either specified as a decimal number, or by name.
command can be an executable or a piece of shell script, and it is executed
by running "/bin/sh -c command". The stdout becomes the contents of
"Filename".
Examples:
Running a basic command
-----------------------
/somedir/dmesg f 444 root root dmesg
creates a file "/somedir/dmesg" containing the output from dmesg.
Executing shell script
----------------------
RELEASE f 444 root root \
if [ ! -e /tmp/ver ]; then \
echo 0 > /tmp/ver; \
fi; \
ver=`cat /tmp/ver`; \
ver=$((ver +1)); \
echo $ver > /tmp/ver; \
echo -n `cat /tmp/release`; \
echo "-dev #"$ver `date` "Build host" `hostname`
Creates a file RELEASE containing the release name, date, build host, and
an incrementing version number. The incrementing version is a side-effect
of executing the shell script, and ensures every time Sqfstar is run a
new version number is used without requiring any other shell scripting.
The above example also shows that commands can be split across multiple lines
using "\". Obviously as the script will be presented to the shell as a single
line, a semicolon is need to separate individual shell commands within the
shell script.
Reading from a device (or fifo/named socket)
--------------------------------------------
input f 444 root root dd if=/dev/sda1 bs=1024 count=10
Copies 10K from the device /dev/sda1 into the file input. Ordinarily Sqfstar
given a device, fifo, or named socket will place that special file within the
Squashfs filesystem, the above allows input from these special files to be
captured and placed in the Squashfs filesystem.
9.2 CREATING A BLOCK OR CHARACTER DEVICE
----------------------------------------
Pseudo definition
Filename type mode uid gid major minor
Where type is either
b - for block devices, and
c - for character devices
mode is the octal mode specifier, similar to that expected by chmod.
uid and gid can be either specified as a decimal number, or by name.
For example:
/dev/chr_dev c 666 root root 100 1
/dev/blk_dev b 666 0 0 200 200
creates a character device "/dev/chr_dev" with major:minor 100:1 and
a block device "/dev/blk_dev" with major:minor 200:200, both with root
uid/gid and a mode of rw-rw-rw.
9.3 CREATING A DIRECTORY
-------------------------
Pseudo definition
Filename d mode uid gid
mode is the octal mode specifier, similar to that expected by chmod.
uid and gid can be either specified as a decimal number, or by name.
For example:
/pseudo_dir d 666 root root
creates a directory "/pseudo_dir" with root uid/gid and mode of rw-rw-rw.
9.4 CREATING A SYMBOLIC LINK
-----------------------------
Pseudo definition
Filename s mode uid gid symlink
uid and gid can be either specified as a decimal number, or by name.
Note mode is ignored, as symlinks always have "rwxrwxrwx" permissions.
For example:
symlink s 0 root root example
8reates a symlink "symlink" to file "example" with root uid/gid.
9.5 CREATING HARD LINKS (FILE REFERENCES)
-----------------------------------------
The "f" Pseudo definition allows a regular file to be created from the output of
a command (or shell). Often this is used to reference a file outside the source
directories by executing "cat", e.g.
README f 0555 0 0 cat /home/phillip/latest-version/README
Because this is a quite frequent use of the definition, an alternative faster
"File reference" or Hard Link Pseudo definition exists:
README l /home/phillip/latest-version/README
Will create a reference to "/home/phillip/latest-version/README",
and obviously the timestamp/mode and owership will be used.
The definition also be used to create additional references to
files within the source directories. For instance if "phillip/latest/README"
was a file being added to the filesystem, then
README l phillip/latest/README
Will create a Hard Link (and increment the nlink count on the inode).
In both cases, the path to the file being referenced is the system
filesystem path, and can be absolute (prefixed with /), or relative
to the current working directory.
There is an additional 'L' Pseudo definition, which closes a loophole in
the above 'l' definition. The 'l' Pseudo definition cannot create references
or Hard Links to files created by Pseudo definitions, because by
definition they do not exist in the system filesystem.
with 'L' the referenced file is expected to be a Pseudo file, and in this case
the path is taken to be from the root of the Squashfs filesystem being created,
e.g.
char-dev c 0555 0 0 1 2
link L char-dev
Will create a Hard Link named "link" to the character device called "char-dev"
created by the previous Pseudo definition.
9.6 CREATING SOCKETS/FIFOS
---------------------------
Pseudo definition
filename i mode uid gid [s|f]
To create a Unix domain socket, 's' should be used, i.e.
filename i 0777 root root s
and to create a FIFO, 'f' should be used, i.e.
filename i 0777 root root f
9.7 ADDING EXTENDED ATTRIBUTES TO FILES
---------------------------------------
Pseudo definition
filename x name=val
Will add the extended attribute <name> to <filename> with <val> contents. See
Section 7 for a description of the <val> formats supported.
9.8 MODIFYING ATTRIBUTES OF AN EXISTING FILE
--------------------------------------------
Pseudo definition
Filename m mode uid gid
mode is the octal mode specifier, similar to that expected by chmod.
uid and gid can be either specified as a decimal number, or by name.
For example:
dmesg m 666 root root
Changes the attributes of the file "dmesg" in the filesystem to have
root uid/gid and a mode of rw-rw-rw, overriding the attributes obtained
from the source filesystem.
10. EXTENDED PSEUDO FILE DEFINITIONS WITH TIMESTAMPS
----------------------------------------------------
The Pseudo file definitions described above do not allow the timestamp
of the created file to be specified, and the files will be timestamped
with the current time.
Extended versions of the Pseudo file definitions are supported which
take a <time> timestamp. These are distinquished from the previous
definitions by using an upper-case type character. For example the "D"
definition is identical to the "d" definition, but it takes a <time>
timestamp.
The list of extended definitions are:
filename F time mode uid gid command
filename D time mode uid gid
filename B time mode uid gid major minor
filename C time mode uid gid major minor
filename S time mode uid gid symlink
filename I time mode uid gid [s|f]
filename M time mode uid gid
<time> can be either an unsigned decimal integer (which represents the
seconds since the epoch of 1970-01-01 00:00 UTC), or a "date string"
which is parsed and converted into an integer since the epoch, by calling
the "date" command.
Because most date strings have spaces, they will need to be quoted, and if
entered on the command line, these quotes will need to be protected from the
shell by backslashes, i.e.
% mksquashfs dir image.sqsh -p "file D \"1 jan 1980\" 0777 phillip phillip"
Obviously anything "date" accepts as a valid string can be used, such as
"yesterday", "last week" etc.
11. APPENDING TO SQUASHFS FILESYSTEMS
-------------------------------------
Running Mksquashfs with the destination directory containing an existing
filesystem will add the source items to the existing filesystem. By default,
the source items are added to the existing root directory.
To make this clear... An existing filesystem "image" contains root entries
"old1", and "old2". Source directory "/home/phillip/test" contains "file1",
"file2" and "dir1".
example 1:
% mksquashfs /home/phillip/test image
Will create a new "image" with root entries "old1", "old2", "file1", "file2" and
"dir1"
example 2:
% mksquashfs /home/phillip/test image -keep-as-directory
Will create a new "image" with root entries "old1", "old2", and "test".
As shown in the previous section, for single source directories
'-keep-as-directory' adds the source directory rather than the
contents of the directory.
example 3:
% mksquashfs /home/phillip/test image -keep-as-directory -root-becomes
original-root
Will create a new "image" with root entries "original-root", and "test". The
'-root-becomes' option specifies that the original root becomes a subdirectory
in the new root, with the specified name.
The append option with file duplicate detection, means squashfs can be
used as a simple versioning archiving filesystem. A squashfs filesystem can
be created with for example the linux-2.4.19 source. Appending the linux-2.4.20
source will create a filesystem with the two source trees, but only the
changed files will take extra room, the unchanged files will be detected as
duplicates.
12. APPENDING RECOVERY FILE FEATURE
-----------------------------------
Recovery files are created when appending to existing Squashfs
filesystems. This allows the original filesystem to be recovered
if Mksquashfs aborts unexpectedly (i.e. power failure).
The recovery files are called squashfs_recovery_xxx_yyy, where
"xxx" is the name of the filesystem being appended to, and "yyy" is a
number to guarantee filename uniqueness (the PID of the parent Mksquashfs
process).
Normally if Mksquashfs exits correctly the recovery file is deleted to
avoid cluttering the filesystem. If Mksquashfs aborts, the "-recover"
option can be used to recover the filesystem, giving the previously
created recovery file as a parameter, i.e.
mksquashfs dummy image.sqsh -recover squashfs_recovery_image.sqsh_1234
The writing of the recovery file can be disabled by specifying the
"-no-recovery" option.
13. MKSQUASHFS ACTIONS INTRODUCTION
-----------------------------------
The Mksquashfs Actions code allows an "action" to be executed on a file if one
or more "tests" succeed. If you're familiar with the "find" command, then an
action is similar to "-print", and a test is similar to say "-name" or "-type".
To illustrate this it is useful to give two concrete examples.
example 1: the fragment action
% mksquashfs /home/phillip/github github.sqsh -action "fragment(cfiles) @ name(*.[ch])" -action "fragment(ofiles) @ name(*.o)"
This example defines two "fragment actions" which control the packing of files
within fragments. Specifically, it creates a specialised fragment called
"cfiles" which packs files matching the wildcard name "*.[ch]".
It also creates another specialised fragment called "ofiles" which packs files
matching the wilcard name "*.o".
Producing specialised fragments which only pack files which match a range of
tests, can produce better compression and/or I/O performance as it can optimise
similarity or access patterns. But it can also produce worse compression, and
so you should always test the effect.
Additionally, you should be able to see that an action definition is split into
an action function before the "@", and one or more test functions after the @.
Quoting is necessary here to protect it from interpretation by the shell. Also
the spacing before and after the "@" isn't necessary and is used here for added
readability.
example 2: the uncompressed action
% mksquashfs /home/phillip backup.sqsh -action "uncompressed @ ( name(*.jpg) || name(*.mpg) ) || ( name(*.img) && filesize(+1G) )"
This is a more complex example. It tells Mksquashfs to not try and compress any
file which matches the wildcard names "*.jpg" and "*.mpg". But it also tells
Mksquashfs not to try and compress files which match the wildcard name "*.img"
and are also 1 Gigabyte in size or larger.
This example introduces the fact that tests can be combined using the logical
operators && (and), || (or) and ! (not), and can be bracketed.
Please see the ACTIONS-README file for syntax and extra information.
14. MISCELLANEOUS OPTIONS
------------------------
The -info option displays the files/directories as they are compressed and
added to the filesystem. The original uncompressed size of each file
is printed, along with DUPLICATE if the file is a duplicate of a
file in the filesystem.
The -nopad option informs Mksquashfs to not pad the filesystem to a 4K multiple.
This is performed by default to enable the output filesystem file to be mounted
by loopback, which requires files to be a 4K multiple. If the filesystem is
being written to a block device, or is to be stored in a bootimage, the extra
pad bytes are not needed.