[[storage_cephfs]]
Ceph Filesystem (CephFS)
------------------------
ifdef::wiki[]
:pve-toplevel:
:title: Storage: CephFS
endif::wiki[]

Storage pool type: `cephfs`

CephFS implements a POSIX-compliant filesystem, using a https://ceph.com[Ceph]
storage cluster to store its data. As CephFS builds upon Ceph, it shares most of
its properties. This includes redundancy, scalability, self-healing, and high
availability.

TIP: {pve} can xref:chapter_pveceph[manage Ceph setups], which makes
configuring a CephFS storage easier. As modern hardware offers a lot of
processing power and RAM, running storage services and VMs on same node is
possible without a significant performance impact.

To use the CephFS storage plugin, you must replace the stock Debian Ceph client,
by adding our xref:sysadmin_package_repositories_ceph[Ceph repository].
Once added, run `apt update`, followed by `apt dist-upgrade`, in order to get
the newest packages.

WARNING: Please ensure that there are no other Ceph repositories configured.
Otherwise the installation will fail or there will be mixed package versions on
the node, leading to unexpected behavior.

[[storage_cephfs_config]]
Configuration
~~~~~~~~~~~~~

This backend supports the common storage properties `nodes`,
`disable`, `content`, as well as the following `cephfs` specific properties:

fs-name::

Name of the Ceph FS.

monhost::

List of monitor daemon addresses. Optional, only needed if Ceph is not running
on the {pve} cluster. With `monhost` set, {pve} treats the cluster as external
and keeps a separate key copy for this storage, even if the monitors belong to
the Ceph cluster of this {pve} cluster.

path::

The local mount point. Optional, defaults to `/mnt/pve/<STORAGE_ID>/`.

username::

Ceph client ID. Optional, defaults to `admin`. Use only the ID without the
`client.` prefix.

subdir::

CephFS subdirectory to mount. Optional, defaults to `/`.

fuse::

Access CephFS through FUSE, instead of the kernel client. Optional, defaults
to `0`.

.Configuration example for an external Ceph cluster (`/etc/pve/storage.cfg`)
----
cephfs: cephfs-external
        monhost 10.1.1.20 10.1.1.21 10.1.1.22
        path /mnt/pve/cephfs-external
        content backup
        username pve-cephfs
        fs-name cephfs
----
NOTE: Don't forget to set up the client's secret key file, if cephx was not
disabled.

[[storage_cephfs_auth]]
Authentication
~~~~~~~~~~~~~~

NOTE: If Ceph is installed locally on the {pve} cluster, the following is done
automatically when adding the storage.

If you use `cephx` authentication, which is enabled by default, provide the
secret of a dedicated identity with only the capabilities required for this
storage. Obtain or create the identity as described in
Ceph User Management.footnoteref:[cephusermgmt]
The example below uses `client.pve-cephfs`; do not use `client.admin` for a
storage client.

Export the bare key through SSH to one {pve} node:

----
# ssh root@<external-ceph-server> \
    'ceph auth get-key client.pve-cephfs' > /root/cephfs.secret
----

Then pass that file and the identity without its `client.` prefix to `pvesm`:

----
# pvesm add cephfs <name> --monhost "10.1.1.20 10.1.1.21 10.1.1.22" \
    --fs-name cephfs --content backup --username pve-cephfs \
    --keyring /root/cephfs.secret
----

After `pvesm` succeeds, remove `/root/cephfs.secret` because it contains the key
in plain text.

When configuring an external CephFS storage in the web interface, paste the
secret into the appropriate field. CephFS uses the bare key, not the RBD keyring
format with a `[client.<USER>]` section. {pve} stores the secret at
`/etc/pve/priv/ceph/<STORAGE_ID>.secret`.

If the external cluster rotates this identity, replace the stored secret before
remounting the storage. The local cephx migration helper never rotates keys of
external clusters.

Storage Features
~~~~~~~~~~~~~~~~

The `cephfs` backend is a POSIX-compliant filesystem, on top of a Ceph cluster.

.Storage features for backend `cephfs`
[width="100%",cols="m,m,3*d",options="header"]
|==============================================================================
|Content types              |Image formats  |Shared |Snapshots |Clones
|vztmpl iso backup snippets |none           |yes    |yes^[1]^  |no
|==============================================================================
^[1]^ While no known bugs exist, snapshots are not yet guaranteed to be stable,
as they lack sufficient testing.

ifdef::wiki[]

See Also
~~~~~~~~

* link:/wiki/Storage[Storage]

endif::wiki[]