Name: @PACKAGE@ Version: @VERSION@ Release: @PHOBOS_RELEASE@%{?dist} Summary: Parallel Heterogeneous OBject-Oriented Storage Group: Applications/System License: LGPLv2.1 URL: http://www-hpc.cea.fr Source0: %{name}-%{version}.tar.gz %define phobosdocdir %{_docdir}/%{name} # build options || .rpmmacros options || change to default action # ============== ==================== =========================== # --with rados || %%_with_rados 1 || build the rados libraries %bcond_with rados %define __python /usr/bin/python3 %if 0%{?rhel} >= 8 %global python_prefix python3 %global postgres_prefix postgresql %else %global python_prefix python36 %global postgres_prefix postgresql94 %endif BuildRequires: %{postgres_prefix}-devel # g_ptr_array_find needs glib2 2.54 BuildRequires: glib2-devel >= 2.54 BuildRequires: %{python_prefix}-devel BuildRequires: jansson-devel >= 2.5 BuildRequires: libattr-devel BuildRequires: libini_config-devel BuildRequires: openssl-devel BuildRequires: sg3_utils-devel BuildRequires: protobuf-c-devel BuildRequires: systemd BuildRequires: autoconf BuildRequires: automake BuildRequires: libtool BuildRequires: make BuildRequires: openssl-devel >= 0.9.7 BuildRequires: xxhash-devel BuildRequires: libcmocka-devel BuildRequires: libuuid-devel BuildRequires: python3-argcomplete BuildRequires: python3-sphinx Requires: %{postgres_prefix}-server Requires: %{postgres_prefix}-contrib Requires: glib2 >= 2.54 Requires: jansson Requires: libini_config Requires: openssl Requires: attr Requires: libattr Requires: %{python_prefix} %if 0%{?rhel} < 8 Requires: python-argparse %endif Requires: %{python_prefix}-PyYAML Requires: %{python_prefix}-clustershell Requires: %{python_prefix}-psycopg2 Requires: %{python_prefix}-tabulate %if 0%{?rhel} > 8 Requires: %{python_prefix}-setuptools %endif Requires: protobuf Requires: protobuf-c Requires: openssl >= 0.9.7 Requires: xxhash-libs Requires: libuuid %description Phobos aims to implement high performance distributed object storage on a wide variety of media including tapes, POSIX file systems, other object stores... This initial version is however limited to a single host. It can manage tapes in a SCSI library and local directories. %package devel Summary: C API for Phobos Object Store. Requires: phobos Requires: glib2-devel >= 2.54 Requires: jansson-devel >= 2.5 Requires: libattr-devel %description devel This package contains header files and required libraries to access Phobos object store from a C program. %if %{with rados} %package rados-adapters Summary: RADOS adapters for Phobos BuildRequires: librados2-devel Requires: phobos Requires: librados2 %description rados-adapters This package contains required libraries to use Ceph/RADOS as a Phobos storage backend. Verify that the Phobos configuration file has been updated to support RADOS functionalities. %endif %prep %setup -q %build CONFIGURE_OPTIONS="" %if %{with rados} CONFIGURE_OPTIONS="$CONFIGURE_OPTIONS --enable-rados" %endif %if 0%{?rhel} < 8 export PKG_CONFIG_PATH=/usr/pgsql-9.4/lib/pkgconfig %endif %configure $CONFIGURE_OPTIONS make %{?_smp_mflags} %install %make_install install -D -p -m 0640 doc/cfg/template.conf %{buildroot}/%{_sysconfdir}/phobos.conf # create another copy of configuration in documentation directory install -D -p -m 0640 doc/cfg/template.conf %{buildroot}/%{phobosdocdir}/template.conf install -p -m 0640 README.md %{buildroot}/%{phobosdocdir}/README.md rm -f $RPM_BUILD_ROOT/%{_libdir}/phobos/libpho_*.a rm -f $RPM_BUILD_ROOT/%{_libdir}/phobos/libpho_*.la mkdir -p %{buildroot}/%{_sysconfdir}/ld.so.conf.d/ echo "%{_libdir}/phobos" > %{buildroot}/%{_sysconfdir}/ld.so.conf.d/phobos.conf mkdir %{buildroot}/%{_sysconfdir}/bash_completion.d register-python-argcomplete phobos > %{buildroot}/%{_sysconfdir}/bash_completion.d/phobos-argcomplete.sh mkdir -p %{buildroot}/%{_sharedstatedir}/phobos %files %{_libdir}/libphobos_store.so* %{_libdir}/libphobos_admin.so* %{_libdir}/phobos/libpho_core.so* %{_libdir}/phobos/libpho_*_dir.so* %{_libdir}/phobos/libpho_*_tape.so* %{_libdir}/phobos/libpho_*_ltfs.so* %{_libdir}/phobos/libpho_*_posix.so* %{_libdir}/phobos/libpho_*_raid1.so* %{_libdir}/phobos/libpho_*_raid4.so* %{_libdir}/phobos/libpho_*_dummy.so* %{_libdir}/phobos/libpho_*_scsi.so* %{_sbindir}/pho_*_helper %{_sbindir}/phobos_db %{python_sitearch}/phobos/* %{python_sitearch}/phobos-*-py?.?.egg-info %{_bindir}/phobos %{_sbindir}/phobosd %{_sbindir}/phobos_tape_library_test %{_sbindir}/phobos_tape_library_test.bash %{_sbindir}/phobos_tape_library_mtx_test.bash %{_sbindir}/phobos_tlc %{_sbindir}/phobos_hsm_sync_dir %{_sbindir}/phobos_hsm_release_dir %{_mandir}/man1/* %config(noreplace) %{_unitdir}/phobosd.service %config(noreplace) %{_unitdir}/phobos_tlc.service %config(noreplace) %{_sysconfdir}/phobos.conf %config(noreplace) %{_sysconfdir}/ld.so.conf.d/phobos.conf %config %{_sysconfdir}/bash_completion.d/phobos-argcomplete.sh %dir %{phobosdocdir} %doc %{phobosdocdir}/README.md %doc %{phobosdocdir}/template.conf %dir %{_sharedstatedir}/phobos %files devel %{_includedir}/pho_attrs.h %{_includedir}/pho_cfg.h %{_includedir}/pho_comm.h %{_includedir}/pho_common.h %{_includedir}/pho_dss.h %{_includedir}/pho_dss_wrapper.h %{_includedir}/pho_ldm.h %{_includedir}/pho_types.h %{_includedir}/phobos_store.h %{_includedir}/phobos_admin.h %{_libdir}/libphobos_store.la %{_libdir}/libphobos_admin.la %{_libdir}/pkgconfig/phobos.pc %post ldconfig %if %{with rados} %files rados-adapters %{_libdir}/phobos/libpho_*rados*.so* %post rados-adapters ldconfig %endif %changelog * Fri Oct 09 2026 - Patrice Lucas - 3.4.1-1 - Grouped read: queue preallocation has been improved to guarantee the use of media already present and available. - "phobos locate" is now compatible with incomplete copies missing extents on some of their media. - the fs_status of a medium is now set to FULL when a ENOSPC, EROFS or EDQUOT errors occurs instead of FAILED. - the last_load for each media is now recorded in its stats as a timestamp. - the `nb_errors` stat for each media is now updated each time an error occurs on that media. - Add the "phobos stats" command to retrieve statistics usage for each media. - Add "--extent-idx" to "phobos copy rebuild" to rebuild a list of specifics extents - Parallel IO: the medium synchronization is now performed concurrently with the ongoing IOs of a device when nb_max_parallel_io is not 1 for its family. It no longer waits for their end, and the device stays selectable while a synchronization is pending. - HSM release: add specific fill thresholds on the source copy extents, used only when dir_release_specific_higher_threshold is strictly lower than dir_release_higher_threshold. - All user-provided strings are now escaped in the SQL requests, so that quotes in them (e.g. an apostrophe in a full path) no longer cause errors. - Escape metadata UTF8 invalid bytes - fix object_size EA truncation and buffer overflow * Mon Aug 31 2026 - Patrice Lucas - 3.4.0-1 - WARNING: the stat dev.umount_error is changed to dev.umount_errors . - WARNING: When calling the pho_xfer_desc_clean function after a `phobos_get` call will now free the node_name parameter. - WARNING: phobos_getmd function signature changes. Removal of the xfer completion callback and user-profile data parameter. - WARNING: phobos_getmd now only populates the version and the attributes (User defined metadata) fields. objid and uuid fields are no longer populated. - WARNING: phobos rpms installation and upgrade no longer overwrites existing sysconf and service files. - Add a thread global context. This context will be used as a transparent mechanism to share a dss handle through multiple Phobos API calls. - Parallel IO per device are now enabled. The maximum number of parallel IO per device and for each family can now be configured to more than 1. 0 means unlimited. - Add the "phobos copy rebuild" command to rebuild still readable copy - Add retry loop with exponential backoff for DB requests - Phobos_setmd is added to the API to update into the DSS the user metadata of any existing object. - Add the "phobos copy delete_incomplete" command to delete incomplete copies and the related objects - Manage partial number of extent per split - The layout DSS select scalability is improved. Remove extent uuid and object uuid/version select in dss_insert_layout. - Adding a list of media to avoid to use is added to the write allocation of the copy and rebuild operation. For copy, this means the destination copy will necessarily be written on media different from the source copy. For rebuild, the rebuilt extents of a split will be written on media different from the other extents of that split. - Allow copy list with UUID and without OID - Decrease "not found obj" to debug - Set the status of a media to used after a delete. After a delete --hard or copy delete, set the status of a media that was full to used. - Add --library option media update and set-access. Also, add a warning message if a dss_update call updates nothing. - Add documentation on how to handle unavailable tapes - Update getmd to use uuid and version of an object - Order copy to get by the readable quality of their status - Add pkg-config build infrastructure - Update the documentation of phobos_store.h - The pho_ldm_helper format_ltfs function returns the mkltfs error code instead of 1 in case of error. - Apply DSS filter clause to deprecated_object. When selecting objects from both tables, the filter clause was enforced to the objects but overlooked for the deprecated objects. - Fix dev_picker if lock_expirancy is non-null - Add '-L' for '--library' option - The TLC's statistics functionality has been enhanced. Add stats about request duration. Distinguish success/error for TLC requests. Add doc for TLC stats. - Raid4 writes layout index in physical extent xattr - Grouped_read: staled queues have been fixed. - Grouped read: SIGSEGV crashes on drive mount failure have been fixed. * Thu Mar 26 2026 - Patrice Lucas - 3.3.1-1 - WARNING: the `--orphan` from `phobos extent list` changed to `--state`. It allows the filtering of the extents with their states. - Add `phobos_admin_extent_list` in the store API to list only the extents - Add a "--force" option to "phobos dir delete" to delete dir even if the local path does not exist. - Add size column to deprecated object list query - HSM improvements: split configuration per copies couple, log errors into a file, stdout only mode with grouping and metadata with base64 - Profile of the default copy name is now taken into account - Cleanup put param filling - CLI: allow object list with UUID and without OID * Fri Feb 13 2026 - Patrice Lucas - 3.3.0-1 - WARNING: the db schema moves from 3.2 to 3.3 and a migration is needed - Metrology is now available - TLC can be secured by the use of multiple devices - Add a way to mark a tape as lost - LRS is strengthened against client error. LRS releases the resources on client disconnection. LRS is resistant to client crash when sending message. - Store clients support input file size errors - Add grouping_on_dir configuration parameter to disengage dir from grouping - Add sort option to dss_extent_get, and delete dss_get_extents_order_by_ctime function - Lock update management - hsm sync by grouping option - hsm sync does not update the sync time on error - Fix locate with grouping by adding grouping copy to media info duplication * Thu Nov 13 2025 - Patrice Lucas - 3.2.0-1 - WARNING: the db schema moves from 3.0 to 3.2 and a migration is needed - WARNING: change the `phobos ping [phobosd|tlc]` command to `phobos phobosd ping` and `phobos tlc ping` - New phobos_hsm_sync_dir command to copy objects from local dir to other media - ctime ordered grouped read - ctime ordered repack - We manage a last locate timestamp and the corresponding delay in the LRS to prevent after locate resource starvation. - Added object size to object table in DSS. Same field is added to deprecated objects. - adding creation_time to extent table into DSS. This ctime is inherited from the corresponding existing copy creation_time. - store: existing object grouping is now taken into account when creating a copy - fix management of extent and object registration on partial release failure - prevent negative physical space free update - scale orphan extent detection at repack - lrs: make some fields of the lrs_dev and pho_resource structures atomic to prevent data races between a scheduler thread and a device thread using these structures. - sched gets new medium health only on the chosen medium - add 'adm_status' to phobos drive status output - make global 'phobos_context' thread-safe - make 'libpho_core' a shared library - Added macros to get the version Phobos from the API (major, minor and patch) and macros to check the current version is higher than a given one * Mon Jun 23 2025 - Patrice Lucas - 3.1.0-1 - limited number of write per grouping: the new "fifo_max_write_per_grouping" configuration parameter allows to limit the number of concurrent writes per grouping on one phobosd instance. The default 0 value reproduces the previous unlimited behaviour. - fix large sends (e.g. during a 'lib scan" with over 1500 slots) - handle external command errors better by using the glib2 spawn_check_wait_status function - use the io_block_size parameter to set the write size of new objects on media * Tue May 27 2025 - Patrice Lucas - 3.0.0-1 - WARNING: the db schema moves from 2.2 to 3.0 and a migration is needed - WARNING: TLC section names in the configuration file are changed from [tlc_libname] to [tlc "libname"] - WARNING: phobos_locate has a new argument: the copy_name. The API compatibility is broken. (The lhsm_phobos copytool must be also updated.) - object copy create/list/delete is available, with "--copy-name" at locate, put and get, "get_preferred_order" configuration option taken into account at get - config man page is now available - autocompletion is added to CLI (check the README.md for more information on how to activate it) - the CLI arguments are homogenized between commands operating on the same target - fix client crash on "del --hard" - cli,api: default library for lib scan/refresh and ldm open is available - lrs: fix fifo find_write_device for unloaded tape without grouping - raid: check the real size of an extent against the expected size - locate: avoid locking media not compatible - fix miscellaneous memory leaks fix - bugfix: incomplete set of orphan extents when deleting with several replicas - lrs: bugfix when requeuing a format request - lrs: do not decrement the health counter without TLC * Tue Jan 14 2025 - Patrice Lucas - 2.2.0-1 - WARNING: the db schema moves from 2.1 to 2.2 and a migration is needed - WARNING: XFER structure is modified to old N objects and the API compatibility is broken. (For example, the lhsm_phobos copytool must be also updated to version 1.3 or greater.) - WARNING: the `alias` feature is renamed `profile`. In that extent, all `-a`/`--alias` options are renamed `-p`/`--profile`. - WARNING: the `-p` option, standing for `--lyt-params`, is uppercased, thus changed to `-P`. - WARNING: the mput is replaced by `put --file` - WARNING: tlc daemon is renamed phobos_tlc - grouping: this new feature is available to group objects on similar media. - no-split: we add a new `no-split` mode to put a list of object on the same medium. - `--new-library`: this new admin option is available to change the library of a drive, a dir, a tape or a rados pool. - `library` parameter is added to alias. - `distutils` dependency is changed to `setuptools`. - `io_block_size` parameter is extended by family. - `dir_full_threshold` and `rados_pool_full_threshold` options complete the `tape_full_threshold` option. - `listen_interface` setting is added for tlc. - `--library` option is added to phobos extent list. - `--errors` option is added to clear logs. - `--status` option is added to `phobos media list` and `phobos drive list`. * Mon Oct 14 2024 - Patrice Lucas - 2.1.0-1 - WARNING: the db schema moves from 2.0 to 2.1 and a migration is needed - Multilibrary: Resource (tape or drive for example) can now belong to different libraries. Default libraries per family must be set in phobos.conf. - Rename: we can change the object id of an existing object. - Repack: we can copy non deprecated object of one tape to a new one and format the old tape to remove deprecated objects. - RAID4: a new 2+1 layout is available. Object is split on two media and a third one contains XOR. - --sort/--rsort: phobos extent/media/drive/object lists can be sorted. - Empty dir/tape/rados_pool deletion: medium without any extent can be deleted. - LTFS release: a phobos cli command is available to release the ltfs lock of one drive. - Hard deletion: --hard option is added to the delete command. - Object creation and access times are managed by phobos. - path normalization: dir path normalization (trailing "/" or relative path for example) are managed by phobos CLI and API. * Fri May 17 2024 - Patrice Lucas - 2.0.0-1 - WARNING: the db schema moves from 1.95 to 2.0 and a migration is needed - Device, medium: errors are now logged into the DSS and managed through a health counter - TLC: Library is now accessed through the Tape Library Controler - Extent migration between tapes is now ready and paves the road for the future tape repack feature. - Tape import: we can now import into a Phobos system a tape from an other Phobos system (recovery or transfer). * Wed Nov 29 2023 - Guillaume Courrier - 1.95.1-2 - Add missing pho_ldm.h in installed headers * Fri Nov 24 2023 - Patrice Lucas - 1.95.1-1 - ltfs: improve ltfs log with serial number - ltfs: return an error if labels are different at mount operation - store, admin, cli: fix store and admin global context share - LRS : fix unloading a tape from a source which is not a slot - compilation : fix gcc compilation warning on RHEL 9 based system - doc : add documentation for the currently_dedicated_to option * Thu Oct 19 2023 - Patrice Lucas - 1.95-2 - Phobos-devel requires libattr-devel * Thu Sep 21 2023 - Yoann Valeri - 1.95-1 - Release 1.95 (Tethys) - WARNING: the db schema moves from 1.93 to 1.95 and a migration is needed. - Added early locking when locating an object - Improved locate to check whether an object can be read with the current resources (unlocked driver/tapes, driver/tape generation compatibility) - Creation of I/O schedulers for efficient request handling: FIFO, Group Read, Fair Share. - Added the "sched fair_share" command to configure the Fair Share algorithm - Created library tests to certify Phobos can work on the target library - Calculate MD5 and/or XXH128 (if available) as extent checksum when putting objects - Retrieve MD5 and XXH128 checksums at extent listing - Added SCSI logs into DSS * Mon Nov 7 2022 - Yoann Valeri - 1.94.1-1 - Added RADOS as data storage backend - Added module-loader to manage io, fs, dev, lib and layout adapters dynamically - Modified drive lock API and command - Added drive migrate command - Improved command documentation - Internal LRS parallelization (read, write and format managed by device threads) - Added locate modification design * Tue Apr 19 2022 - Yoann Valeri - 1.94-1 - Release 1.94 (Rhea) - Improved documentation on setup, admin commands, object management, listing commands - Added thresholds for when to synchronize a medium (time, number of requests and written size). Synchronizations are now done all at once, not one by one - LRS is now composed of the duo of threads: * communication and scheduler threads, for communicating with clients and managing a specific family of device/media; * device threads, to manage a device and basic operations regarding a medium. Multiple device/medium can now operate in parallel for operations like mount/umount/load/unload and formats, which do not block the execution of read and write requests anymore - Media don't unload when terminating the LRS - LRS will wait for every I/O completion before terminating - LRS won't stop when a client is disconnected - New commands: * drive delete * drive status * lock clean - Format manage multiple formats at the same time, and added a parameter to limit the number of concurrent formats - Added max-width and no-truncate arguments to object list * Tue Nov 30 2021 - Patrice Lucas - 1.93-1 - LRS lock refactoring - Added layout parameters to migration - Removed layout simple and added simple alias - Limited sync to write operations * Fri Jul 2 2021 - Yoann Valeri - 1.92-1 - Locate command - Uuid and version options to get command * Mon May 17 2021 - Yoann Valeri - 1.91.2-1 - Modified concurrency lock management in DSS - Modified DB schema - Pattern option to extent and object list - Ping command - Configurable block size when writing data - Put overwrite - Added phobos locate command's design documentation * Wed Mar 3 2021 - Sebastien Gougeaud - 1.91.1-1 - Delete and undelete commands for objects - Extent path generation now considers uuid and version of objects * Wed Jan 13 2021 - Sebastien Gougeaud - 1.91.0-1 - Release 1.91 (Japet) - Python3 port - New DB version towards the future deletion mechanism - Alias feature to specify storage class of service - Media set-access command to control permitted medium operations - Daemon management optimization (ressource locks, etc.) * Thu Sep 10 2020 - Thomas Leibovici - 1.90.0-1 - Release 1.90 (Hi'iaka): 1.9x is the release cycle towards 2.0 - Phobos now runs a daemon (phobosd) on each IO node to manage devices and schedule IOs - Object and extent listing and filtering commands - Layout and family selection from (m)put command line - Split too large extents on multiple media - Port to RHEL/CentOS8 - Various fixes and refactoring * Fri Jan 10 2020 - Thomas Leibovici - 1.3.1-1 - Pass 'scsi generic' devices to LTFS instead of 'scsi tape'. * Tue Nov 26 2019 - Thomas Leibovici - 1.3.0-1 - Release 1.3 (Vala) - Now uses LTFS 2.4. No longer requires lin_tape. - Media list filter by tag, e.g. "phobos tape list -T foo,bar" - Drive list filter by model, e.g. "phobos drive list -m ULTRIUM-TD6" - Fix various memory leaks and wrong behaviors * Thu Jul 18 2019 - Thomas Leibovici - 1.2.0-1 - Final 1.2 release (Echo) - New features: * Dual logging (stderr and syslog) * phobos lib scan (improved 'mtx status') * phobos_db tool (DB schema management, DB migration) * retry when the number of concurrent PUT/GET > nb drives * media tag management * manage drive-tape compatibility rules (WIP) * Mon Jan 15 2018 - Thomas Leibovici - 1.1-4 - Minor refresh with latest patches * Fri Sep 08 2017 - Henri Doreau - 1.1-3 - Propery set LGPL headers to the project files - Reimplement CLI using ctypes instead of SWIG - Fix a bug where errors on flush would trigger an assertion failure - Misc packaging improvements to meet ocean quality standards * Fri Aug 04 2017 - Thomas Leibovici - 1.1-2 - Change DB conversion script to properly convert JSON integers - Make DSS resilient to missing stats fields in JSONs - Fix default commands for ltfs mount/umount/format - Don't leave a drive busy when its filesystem is full - Fix invalid write intent size of 0 - Set rpath to load layout modules - SCSI library: check target slot is empty before move - Make tape move resilient to invalid slot * Wed Jun 21 2017 - Thomas Leibovici - 1.1-1 - Generic Layout management + 2 implementations: 'simple' and 'raid1'. - Configuration changes. See installed template (see /etc/phobos.conf.rpmnew). - Set filesystem label at format time, and check it at mount time. - Improved management of full devices (set R/O, tag full...) - SCSI library: when needed, search free slot to unload a tape to. - Phobos 1.0 to phobos 1.1 DB conversion script. - New DSS filter API (no change in commands or user interface). - Miscellaneous bug fixes