Release note for Gfarm 2.7.16
=============================

[NOT-RELEASED-YET]

New feature
* etc/gfmd.failover.conf: configuration file which will be read at failover.
  please create 0-byte file as this name at least, otherwise "cannot open"
  error will be logged at failover.

Release note for Gfarm 2.7.16
=============================

[2019.11.30]

Updated feature
* reduce logs when files on a readonly gfsd are removed

Release note for Gfarm 2.7.15
=============================

[2019.10.24]

New configuration directive
* rdma_gid_index - specify the gid index to use GRH

Updated feature
* gfarmbb - status command to check process, network and directory
  status for Gfarm/BB

Bug fix
* Remote write may fail due to completion error when using InfiniBand

Release note for Gfarm 2.7.14
=============================

[2019.9.10]

New command
* gfarmbb - start and stop scripts for Gfarm/BB node-local on-demand
  file system

New configuration directive
* log_file - specify a log file
* quota_check_retry_interval, directory_quota_check_retry_interval -
  specifies the interval to re-calculate the quota.  This is
  introduced to make replica check progress.  Default is 60 seconds.
* schedule_rpc_timeout - specifies the timeout for RPC.

Updated feature
* gfmd, gfsd - -F option to specify a log file

Bug fix
* gfquotacheck - may fail due to timeout
* libgfarm - too high CPU load may cause gfsd protocol mismatch
* gfsd - fix race condition to terminate that may fail to record the
  write verify information

Release note for Gfarm 2.7.13
=============================

[2019.3.22]

New configuration directive
* replication_at_write_open - if this directive is enabled, even when
  all file replicas of a file are stored at read-only filesystem
  nodes, it can be updated by replicating it to a read-write
  filesystem node before opening in the write mode.  Default is
  enable.

Updated feature
* read-only mode - filesystem node can be changed to read only by
  gfhost -m -f 1 hostname
* gfdf - -R option to know real disk usage

Bug fix
* gfmd failover does not work, which was introduced by the version
  2.7.12
* compilation error with InfiniBand, which was introduced by the
  version 2.7.12
* gfquota -d does not work against a writable directory

Release note for Gfarm 2.7.12
=============================

[2019.2.20]

Documentation
* failover and disaster recovery documents

Updated feature
* gfmd - improve replica check process especially using replicainfo
  that means file system group based replica location control feature
* gfstatus - -l and -L options to list configuration names
* support 8 Exbibyte file system node

Bug fix
* gfdf - fix output with -a option

Release note for Gfarm 2.7.11
=============================

[2018.6.25]

Updated feature
* change default of gfsd_connection_cache from 16 to 256.
* adjust log level

Release note for Gfarm 2.7.10
=============================

[2018.2.22]

Updated feature
* gfmd - improve ticket lock performance, which is introduced since
  the version 2.7.9

Bug fix
* directory quota may be incorrect when moving a file during the
  replica check
* slave gfmd may crash when it is stopped during busy metadata updates
* gfmd crashes when it is stopped at early boot stage

Release note for Gfarm 2.7.9
============================

[2018.1.16]

Updated feature
* gfmd - improve response time during replica check when enabling
  replica info.  By default, use a ticket lock for FIFO thread
  scheduling instead of a stock mutex lock.

Bug fix
* replica check does not honor directory quota 
* fix deadlock or segfault when reading files using InfiniBand RDMA if
  reconnection happens

Release note for Gfarm 2.7.8
============================

[2017.12.03]

Updated feature
* show the number of ongoing replications by SIGUSR2

Bug fix
* spool_check_level disable makes gfsd unstable, which has been
  introduced since the version 2.7.6
* fix data race when switching gfsd or gfmd connections, that may make
  segfault
* fix data race about gfsd status

Release note for Gfarm 2.7.7
============================

[2017.11.26]

New configuration directive
* metadb_server_remove_scan_log_interval - interval to log statistical
  information about removed file scan
* metadb_server_remove_scan_interval_factor - interval between removed
  file scans

Documentation
* manual pages - gfrepcheck(1)

Updated feature
* gfmd - improve response time even under too many gfmd-initiated
  replications, file removals and updates
* gfdf - -i option to print number of inodes
* gfrepcheck - status command to obtain the current status

Release note for Gfarm 2.7.6
============================

[2017.10.1]

New configuration directive
* replication_busy_host - specify whether destinations of
  gfmd-initiated replication include busy hosts or not.  Default is
  enable.  Note that file replica creation to busy file system nodes
  is disabled since 2.7.2.  To keep this behavior, specify
  replication_busy_host disable
* replica_check_remove_grace_used_space_ratio,
  replica_check_remove_grace_time - specify grace space ratio and
  grace time for replica check remove
* spool_check_parallel, spool_check_parallel_max,
  spool_check_parallel_per_capacity, spool_check_parallel_step -
  specifies the number of parallelism and granularity of spool check

Updated feature
* gfsd - parallel spool check for faster spool check processing
* gfmd - gfmd-initiated replication selects a file replica to be
  replicated randomly to avoid concentration
* gfmd - more rigit disk available check to avoid no space error
* gfrepcheck - gfrepcheck stop stops replica check processing
  soon.

Release note for Gfarm 2.7.5
============================

[2017.8.9]

New configuration directive
* max_directory_depth - maximum directory depth.  Default is 100.

Release note for Gfarm 2.7.4
============================

[2017.7.14]

New configuration directive
* directory_quota_check_start_delay - delay to start directory quota
  check to aggregate multiple checks

Updated feature
* gfrm - do not try to remove symlinks with the -h option
* gfmd - more log in peer_set_host
* postgresql backend - accept IPv6 connection, which is required in
  CentOS 7.3.1611

Bug fix
* gfmd - invalid replica cannot be moved to lost+found, which is
  introduced by the version 2.7.0
* recursive option does not return error properly
* compilation error on FreeBSD-10.2/amd64
* gfsd - invalid file remains when fhclose fails
* gfmd does not recover from the warning of abandon invalid journal
* gfmd - gfmd may fail to restart when using the none backend

Release note for Gfarm 2.7.3
============================

[2017.3.26]

Updated feature
* gfprep, gfpcopy - a new scheduling to mitigate destination conflict
  under concurrent gfprep and gfpcopy executions
* start replica check when a busy host becomes idle

New configuration directive
* metadb_server_remover_queue_length - queue length for remove
  requests to make it a low priority not to prevent from replication
  requests

Bug fix
* gfmd may not respond when sending journal logs to async slaves
* write only files may not be moved to lost+found
* Gfarm RPM packages cannot be installed on RHEL7 or CentOS7

Release note for Gfarm 2.7.2
============================

[2017.2.4]

New configuration directive
* replicainfo - disables file system group based replica location
  control feature.  Only number of file replicas is maintained.  This
  directive is available for gfmd

Updated feature
* gfmd - support none backend database for temporal Gfarm file system
  without any backend database.  Journaling keeps metadata change log
  up to the journal file size
* gfmd - do not create file replicas in busy file system nodes

Release note for Gfarm 2.7.1
============================

[2017.1.14]

Updated feature
* gfpcopy, gfprep - do not copy to busy hosts by default.  This will
  be disabled by -B option
* gfpcopy, gfprep - -I option to specify interval to collect host
  information.  Default is 300 seconds.
* gfsd - adjust log level for non-critical replication failure

Bug fix
* incorrect metadb_server_list causes assertion failure

Release note for Gfarm 2.7.0
============================

[2016.12.8]

New feature
* InfiniBand RDMA support
* Directory quota (see doc/html/en/user/dirquota.html for details)

New command
* gfdirquota - manage directory set for directory quota

New configuration directive
* ib_rdma - enables or disables InfiniBand RDMA data transfer.  When
  configuring --with-infiniband, default is enable.
* rdma_device_name, rdma_port - optional directive to specify device
  name and port number, respectively, for RDMA data transfer
* rdma_min_size - minimum size for RDMA data transfer.  Default is
  65536 bytes
* rdma_mr_reg_mode - specifies memory registration mode.  Default is
  static.
* rdma_mr_reg_static_min_size, rdma_mr_reg_static_max_size - minimum
  and maximum size for static RDMA buffer for each gfsd connection.
  Default is 1MiB and 16MiB, respectively
* directory_quota_count_per_user_limit - maximum number of directory
  sets for each user.  Default is 100

Updated feature
* gfedquota, gfquota - -D and -d options for directory quota

Release note for Gfarm 2.6.15
=============================

[2016.12.3]

Updated feature
* OpenSSL 1.1.0 support
* gfprep - -mm option to keep an original copy in migration mode
* gfsd - log message of write_verify process includes the hostname

Bug fix
* gfprep -m may fail due to file busy even though all operations succeed

Release note for Gfarm 2.6.14
=============================

[2016.10.26]

New command
* gflsof - list current opened Gfarm files

Updated feature
* increase default timeout to connect to gfmd to 35 seconds to cope
  with fault of primary and secondary DNS servers

Release note for Gfarm 2.6.13
=============================

[2016.9.27]

New configuration directive
* max_open_files - specify the maximum number of open files.  Default
  is 1024.

Updated feature
* gfmd - keep minimum disk space to reduce no space risk

Release note for Gfarm 2.6.12
=============================

[2016.8.16]

New feature
* symbolic link support for master-slave redundant metadata servers -
  metadb_server_list directive can be used several times to specify
  several metadata server groups

Updated feature
* iostat - new data format that aligns the cache line to avoid
  unnecessary false sharing.  This requires ganglia plugin update by
  config-{gfsd,gfmd}-iostat -f

Bug fix
* fix compilation error on Linux/alpha.  This patch is contributed by
  Michael Cree via Dmitry Smirnov [https://bugs.debian.org/827021]
* gfpcopy - plug memory leak, which was introduced in Gfarm 2.6.6
* gfmd - quota is not effective just after gfmd start-up or failover
* gfmd - incorrect grace period for soft quota after gfmd restart or
  failover
* gfsd - may give up to reconnect to gfmd
* gfsd - incorrect period for write_verify is reported at gfsd crash
  on Linux 2.6.22 or before

Release note for Gfarm 2.6.11
=============================

[2016.6.2]

Bug fix
* gfpcopy does not tolerate file migration or file replica removal
* gfs_pio_close() returns GFARM_ERR_BAD_FILE_DESCRIPTOR when it is
  called against a never accessed 0-byte file
* gfmd may deadlock in GSI when using Globus Toolkit 6.0 or later
* multi spool directories - directories having the same prefix cannot
  be specified

Release note for Gfarm 2.6.10
=============================

[2016.4.25]

New configuration directive
* metadb_server_nfs_root_squash_support - disables nfs root squash
  support to improve the metadata performance if it is not necessary
* write_verify_log_interval - specifies the interval to record
  statistics of write verify 

Updated feature
* profile and progress logging support for write verify

Bug fix
* when gfmd is down, gfsd may not log failed close operation at exit
* multiple spool directories may cause inconsistent file creation.  If
  this happens, alert and move it to the spool top directory.

Release note for Gfarm 2.6.9
============================

[2016.4.3]

New configuration directive
* spool_server_back_channel_rcvbuf_limit and
  metadb_server_back_channel_sndbuf_limit - limits outstanding
  requests between gfsd and gfmd to reduce risk of the disconnection.
  Default is 4096 bytes.

Updated feature
* gfsd - support multiple spool directories
* most available disk space is selected when creating files
* gfwhere - -m option to select files having the specified number or
  less number of replicas
* gfwhere - -h option to select files that stored on the specified
  host
* gfstatus - -m option to change the configuration parameter in gfmd
  with -M option.  Currently, 'digest', 'write_verify',
  'write_verify_interval', 'write_verify_retry_interval' can be
  changed.  Note that write_verify is enabled only after gfsd is
  restarted.
* write_verify process of gfsd terminates and dumps file information
  to be verified by TERM signal
* gfspoolpath - -I option to specify inode number and generation
  number instead of a filename

Bug fix
* updated files during the connection between gfsd and gfmd is
  disconnected cannot be accessed
* master gfmd may crash when connecting from an expired slave gfmd
  [sf.net ticket #947]
* delete metadata when a copy cannot be created due to no space error
* mtime of gfs_pio_stat() and gfs_stat() is inconsistent, which may
  cause failure to extract symbolic links from tar files

Release note for Gfarm 2.6.8
============================

[2016.1.16]

New feature
* write verify - detect silent data corruption by re-reading files

New configuration directive
* write_verify - enables the write verify feature
* write_verify_interval - specifies the wait time in seconds of the
  write verify after the file creation.  Default is 21600 seconds (6
  hours).
* write_verify_retry_interval - specifies the wait time in seconds to
  retry write verify when the file is opened for writing at the write
  verify time.  Default is 600 seconds (10 minutes).

Updated feature
* gfstatus - -M option to ask configuration, e.g. write_verify or
  digest, in gfmd
* gfmd reports file names opened for writing when gfsd exits without
  closing them
* quota check for file replication, chown, and chgrp

Bug fix
* gfmd crashes when quota prevents file creation [sf.net ticket #936]
* gfsd - plug memory leak at gfmd-initiated replication, which was
  introduced by Gfarm 2.6.0

Release note for Gfarm 2.6.7
============================

[2015.11.28]

Updated feature
* make gfarm2fs return EIO instead of EINVAL, when no replica is
  available [sf.net ticket #924]

Bug fix
* gfrep and gfprep may create incomplete replica when network error
  happens, which is introduced by Gfarm 2.6.0 [sf.net ticket #926]
* local host is not selected when reading a file at first operation,
  which was introduced by Gfarm 2.6.0
* When a file system node crashes during receiving a file replica, the
  metadata remains [sf.net ticket #913]
* client_digest_check does not return an error even when checksum
  mismatch happens [sf.net ticket #915, #916]
* gfarm clients may access uninitialized data, and may dump core, if
  $HOME/.gfarm2rc does not exist [sf.net ticket #912]
* changing gfmd port number by "gfmdhost -m -p" makes gfmd crash
  [sf.net ticket #918]
* When trying to modify a file stored at a file server with no space
  or in read-only mode, return no space immediately instead of
  retrying for a while

Release note for Gfarm 2.6.6
============================

[2015.8.29]

New configuration directive
* client_parallel_max - the maximum number of parallel operations that
  can be specified by gfpcopy and gfprep

Documentation
* manual pages - gfs_pio_sendfile(3), gfs_pio_recvfile(3)
* manual pages - gfarm_environ(7) that describes environment variables
  for Gfarm including ciphers in GSI communication
* user manual - cipher-comparison.html that describes the performance
  of each cipher in typical architectures

Updated feature
* gfpcopy, gfprep - -k option not to copy symlinks [sf.net ticket
  #904]
* gfpcopy, gfprep - -X option to exclude files or directories [sf.net
  ticket #901]
* gfpcopy, gfprep - retry if copy or replication fails [sf.net ticket
  #903]
* gfmv - more like mv(1) not just calling gfs_rename(2) [sf.net ticket
  #897]
* automatic quota check - gfquotacheck is not needed to be called
  explicitly [sf.net ticket #902]
* gfcksum - -l option to show a long path for each file [sf.net ticket
  #879]
* adjust log level [sf.net ticket #876, #877]
* gfjournal - performance improvement [sf.net ticket #872]
* automatic repair at replication when the file is corrupted [sf.net
  ticket #836]
* gfjournaldump - support ruby 1.9 or later

Bug fix
* gfjournal - fix incorrect output for offset [sf.net ticket #878]
* gfjournaldump - fix incorrect output [sf.net ticket #870]
* gfmd may crash when gfedquota is executed after gfmd failover
  [sf.net ticket #906]

Release note for Gfarm 2.6.5.1
==============================

[2015.7.5]

Bug fix
* initialization for thread-safe OpenSSL is not called at the
  beginning
* when specifying the same entry for rename, the entry is unlinked
  [sf.net ticket #450]
* gfrmdir / returns 'is a directory' [sf.net ticket #451]
* file system node is not removed in the backend database when it has
  no file replica, which was introduced by Gfarm 2.6.5 [sf.net ticket
  #868]
* replica check does not start when the file system node group is
  changed [sf.net ticket #866]
* replica check does not start when gfarm.replicainfo is removed
  [sf.net ticket #867]

Release note for Gfarm 2.6.5
============================

[2015.6.25]

New feature
* automatic removal of excessive number of file replicas
* gfrepcheck - remove and reduce_log commands to control automatic
  replica removal and log summary output features
* gfpcopy - HPSS support
* gfprep, gfpcopy - -z and -Z options to specify the minimum and
  maximum file size to copy, respectively

New configuration directive
* replica_check_remove - disables automatic replica removal feature
* replica_check_reduced_log - disables log summary output feature in
  replica check
* metadb_replica_remover_by_host_sleep_time
* metadb_replica_remover_by_host_inode_step

Updated feature
* file replicas can be removed only when available number of replicas
  is greater than required [sf.net ticket #857]
* gfexport - accept multiple input files [sf.net ticket #853]
* gfspooldigest - show statistics at the end
* libgfarm - do not connect to servers when client certificate is
  expired
* gfhost -d removes file replicas stored on the deleted host [sf.net
  ticket #860]
* gfjournal -m - performance improvement
* gfmd - SIGHUP is not needed to reload CA certificate and CRL
* slave gfmd - do not exit when journal receive queue overflows

Bug fix
* gfmd may crash in GSI [sf.net ticket #854]
* gfmd - replica check does not create required replicas when gfsd is
  up [sf.net ticket #856]
* client may block when GSI authentication fails
* client may crash when connection problem happens [sf.net ticket
  #852]
* slave gfmd may exit when connection problem happens [sf.net ticket
  #851]
* gfmkdir - fix invalid memory reference in error case

Release note for Gfarm 2.6.4.1
==============================

[2015.4.30]

Updated feature
* gfspooldigest - -M option to select files modified older than the
  specified days ago
* make progress of replica check

Bug fix
* gfspooldigest - segfault when gfmd failover happens [sf.net ticket
  #848]

Release note for Gfarm 2.6.4
============================

[2015.4.26]

Updated feature
* gfpcopy - overwrite files when having no file replica
* gfspooldigest (aka gfspoolmd5) - -G option to read files in a spool
  directory in Gfarm API, which is useful to move corrupted files to
  lost+found
* gffilepath - -l and -P options to specify the metadata directory and
  the backend database path, respectively
* master gfmd stops after all journal transfer for safe failover to an
  asynchronous slave server [sf.net ticket #845]
* adjust log level

Bug fix
* cannot build on Mac OS X and NetBSD since Gfarm 2.6.1 [sf.net ticket
  #846]
* gffilepath - it does not support hardlinked files

Release note for Gfarm 2.6.3
============================

[2015.4.13]

Bug fix
* gfs_pio_sendfile/recvfile may generate incorrect digest
* when a slave gfmd is changed from asynchronous to synchronous, it
  stops to work [sf.net ticket #829]

Release note for Gfarm 2.6.2
============================

[2015.3.27]

Updated feature
* unconfig scripts are generated by config-gfarm and config-gfsd to
  undo the configuration process [sf.net ticket #819]

Bug fix
* mtime becomes Jan 1, 1970 when client_digest_check is enabled, which
  was introduced by Gfarm 2.6.1 [sf.net ticket #832]
* missing checksum check during automatic replication when a file is
  created [sf.net ticket #835]
* fix lost update problem that happens concurrent read and write cases
* gfreg - leaves an empty file incorrectly when a diskfull host is
  specified by -h option [sf.net ticket #842]
* config-gfarm, config-gfsd - fix permission of configuration file of
  systemd [sf.net ticket #830]
* fix infinite loop of replica check due to file size or checksum
  mismatch

Release note for Gfarm 2.6.1
============================

[2015.3.2]

New feature
* move corrupted replica to lost+found when checksum error happens
  [sf.net ticket #802]

Documentation
* SETUP - update firewall setting [sf.net ticket #816]
* SETUP - user registration setting required on metadata servers and
  file system nodes [sf.net ticket #803]
* manual pages - gfrm(1), config-gfarm(8), config-gfarm-update(8),
  config-gfsd(8) and gfdump.postgresql(8) [sf.net ticket #799]
* Gfarm-FAQ - gfsd -dv may help to find a reason of authentication
  error

Updated feature
* gfprep, gfpcopy - keep minimum disk space to reduce no space risk
* config-gfarm, config-gfarm-update - -d digest_type option to enable
  digest check
* config-gfarm - -E option to enable data checksum support in a
  backend database.  PostgreSQL 9.3 or later is required to use this
  option [sf.net ticket #804]
* gfmd - performance improvement of initialization process [sf.net
  ticket #808]
* gfmd - multi-master check at the startup process [sf.net ticket
  #807]
* configure - support GNU hurd for host_os_name [sf.net ticket #806]

Bug fix
* gfpcopy - count skipped files correctly [sf.net ticket #810]
* gfs_pio_sendfile() may set incorrect check sum when it is opened in
  GFARM_FILE_APPEND [sf.net ticket #812]
* gfsd - failover notification may delay [sf.net ticket #792]

Release note for Gfarm 2.6.0
============================

[2014.12.17]

New feature
* end-to-end data integrity for silent data corruption [sf.net ticket
  #774]
* checksum during replication [sf.net tickets #769 and #771]
* replica locations specified by file system node groups [sf.net
  tickets #494 and #711]
* gfhostgroup - manage file system node group
* fast failover by notifying to gfsd [sf.net ticket #721]
* client transparent failover [sf.net ticket #527]
* support sticky bit of directories [sf.net ticket #762]

New configuration directive
* client_digest_check - enables end-to-end data integrity

New API
* gfs_pio_sendfile, gfs_pio_recvfile - bulk data transfer

Updated feature
* gfncopy - -S option to specify replica locations by node groups
  [sf.net ticket #712]
* gfpcopy, gfexport, gfreg - performance improvement [sf.net ticket
  #728]
* support sha256 digest type [sf.net ticket #757]
* gfmdhost - introduce a new state under synchronization [sf.net
  ticket #752]

Bug fix
* fix race condition when updating files during a failover [sf.net
  ticket #341]
* file metadata may not be updated when using direct access feature on
  a file system node [sf.net ticket #461]

Release note for Gfarm 2.5.8.13
===============================

[2014.12.16]

Updated feature
* config-gfarm - CentOS 7 support [sf.net ticket #787]
* gfprep - suppress insufficient number of replicas error since it is
  norm when -x option is specified 
* gfmdhost - -N and -1 options for health check purpose [sf.net ticket
  #779]

Bug fix
* GFARM_FILE_APPEND does not work on Mac OS X [sf.net ticket #789]

Release note for Gfarm 2.5.8.12
===============================

[2014.10.24]

Updated feature
* gfmd - do not start master mode when another master gfmd running to
  avoid unexpected misconfiguration [sf.net ticket #777]

Release note for Gfarm 2.5.8.11
===============================

[2014.10.2]

Updated feature
* show local access time, remote access time and API counts when
  profiling feature enabled

Bug fix
* memory leak in slave gfmd, when a symbolic link is created [sf.net
  ticket #776]

Release note for Gfarm 2.5.8.10
===============================

[2014.8.21]

Updated feature
* return EIO instead of EINVAL when checksum mismatch happens
* gfdf, gfsched, gfhost - exit with non-zero code when no filesystem
  node is available

Bug fix
* gfstat - -r option may cause file migrated or no filesystem node
  when filesystem nodes are full

Release note for Gfarm 2.5.8.9
==============================

[2014.7.27]

Updated feature
* checksum calculation support for ftruncate

Bug fix
* gfprep - migration may fail when the destination domain is specified

Release note for Gfarm 2.5.8.8
==============================

[2014.6.29]

Updated feature
* gfpcopy - do not overwrite new files when the size is different
* gfls - print setuid/setgid/sticky bits [sf.net ticket #761]
* gfstat - -r and -h option to obtain file size of file data not
  metadata
* gfhost - can be executed by _gfarmfs user
* config-gfarm - support PostgreSQL 9.3 or later [sf.net ticket #764]
* gfs_chown and gfs_lchown - return success, if the owner is not
  actually changed [sf.net ticket #758]

Bug fix
* fix lost replica when disconnection happens during close
* gfs_chmod and gfs_lchmod - lacks some posix semantics [sf.net ticket
  #760]
* gfstatus - -d option does not work

Release note for Gfarm 2.5.8.7
==============================

[2014.4.27]

New configuration directive
* direct_local_access - disables direct local access when accessing
  the local files
* gfsd_connection_timeout - specify timeout to connect gfsd.  The
  default is 30 seconds.

Updated feature
* gfspoolmd5 - -n option to add checksum for files whose checksum is
  not calculated yet.
* set checksum when the gfmd failover happens
* gfs_fstat_cksum_set() - only gfarmroot can set the checksum

Bug fix
* gfspoolmd5 - skip incomplete replicas
* authentication faild error may be incorrectly displayed when using
  two and more authentication method

Release note for Gfarm 2.5.8.6
==============================

[2014.3.24]

Bug fix
* sharedsecret configuration allows unauthenticated access in a
  certain case, which is introduced by Gfarm 2.5.8.5 [sf.net ticket
  #754]

Release note for Gfarm 2.5.8.5
==============================

[2014.3.23]

New command
* gfcksum - print and calculate checksum
* gfrepcheck - start or suspend the replica check thread
* gfspoolinum, gfspoolgen - obtain inode number or generation number
  from the spool path or lost+found filename
* gfdirpath - return path name in Gfarm from directory inode number
* gffilepath - return path name in Gfarm from inode number
* gfspoolmd5 - check md5 checksum of all files in spool directory

New configuration directive
* digest - enables data integrity check by checksum calculation and
  specify the digest type

New API
* gfs_stat_cksum, gfs_fstat_cksum and gfs_stat_cksum_free - obtain
  checksum
* gfs_pio_cksum - calculate checksum
* gfs_fstat_cksum_set - set checksum from client
* gfs_pio_fhopen, gfs_pio_fhopendir - open a file or a directory from
  inode number and generation number
* gfs_fgetdirpath - return path name from GFS_Dir

Updated feature
* gfusage - optput total
* return checksum mismatch error when reading entire file or closed
* print the progress of replica_check when gfmd catches SIGUSR2
* gfxattr - -I option to manipulate the extended attribute by inode
  number
* grid-proxy-agent - -bits option to specify the number of bits in key
* adjust log level

Bug fix
* gfmd may crash when SIGHUP is caught
* gfpcopy - invalid files may remain when close fails
* clients may not retry to connect to gfmd in shared secret
  authentication [sf.net ticket #740]
* gfs_pio_stat() does not obtain file info in a file system node when
  it is called just after gfs_pio_open()

Release note for Gfarm 2.5.8.4
==============================

[2014.1.18]

New command
* gfsudo - execute a command in gfarmroot privilege

Updated feature
* support GFARM_FILE_EXCLUSIVE
* gfrm - -D option to specify a domain name of file replicas to be
  deleted
* gfls - -h option to display in human readable format
* gfusage, gfquota - -h and -H options to display in human readable format
* gfpcopy - -e option to enable multiple gfpcopy executions for the
  same target directory
* gfncopy - -w option to wait until replicas are created
* gfgroup - -a and -r options to add and remove group members
* replica check also checks files whose gfarm.ncopy is not set [sf.net
  trac #733]
* increase error level when all replicas are lost and when all hosts
  are down [sf.net trac #732]
* gfsd - tolerate read-only file system error and no space left on
  device error

Bug fix
* gfmd crash when gfmd is stopped during the startup process [sf.net
  trac #736, #737]
* gfmd may crash when a write error happens in the startup process
  [sf.net trac #734]

Release note for Gfarm 2.5.8.3
==============================

[2013.10.2]

Updated feature
* gfprep, gfpcopy - -J option to specify the number of processes to
  read directories [sf.net trac #726]

Bug fix
* PhysicalSpace and PhysicalNum in quota are incorrect when removing
  files, which is introduced by Gfarm 2.5.8.2 [sf.net trac #730]

Release note for Gfarm 2.5.8.2
==============================

[2013.7.29]

New configuration directive
* spool_base_load - additional CPU load for gfsd to pretend to be busy
* shared_key_file - specify a pathname of the shared key file

Updated feature
* Japanese documents including manual pages and HTML documents are
  encoded in UTF-8 [sf.net trac #698]
* all entries are included in FileNum in quota [sf.net trac #717]
* gfprep, gfpcopy - -M option to limit the total file size to be
  copied [sf.net trac #707]

Bug fix
* slave gfmd fails to restart after gfhost -d [sf.net trac #697]
* gfprep -m cannot migrate replicas when two or more source nodes are
  specified [sf.net trac #709]
* gfprep -m removes a new valid replica when the source replica cannot
  be removed [sf.net trac #708]
* recursive operations for gfarm:// and gfarm://host:port do not work
* gfdump -r and config-gfarm-update do not work in private mode in
  debian
* plug memory leak when file trace enabled [sf.net trac #703]
* 0-byte file can be created when using gfarm2fs 1.2.8 or later
  [sf.net trac #702]

Release note for Gfarm 2.5.8.1
==============================

[2013.5.22]

Updated feature
* gftest - support relative path and multiple Gfarm file systems when
  mounted by gfarm2fs

Bug fix
* gfmd - fix assertion error when the sequence number of operations is
  larger than 4G
* gfprep - -x option may cause "file busy" [sf.net trac #689]
* invalid replica may remain when a replication fails [sf.net trac
  #680]
* gfdump.postgresql - fix bashism

Release note for Gfarm 2.5.8
============================

[2013.4.22]

New command
* gftest - check entry type

New feature
* gfprep, gfpcopy - honor write_target_domain directive

Updated feature
* gfmd init script - kill to immediately stop the gfmd
* log level adjustment to make ZABBIX alerts usable - step 3 [sf.net
  trac #679]
* gfpcopy - preserve the modification time of directories, and honor
  umask [sf.net trac #662 and #663]

Bug fix
* no filesystem node happens when updating a file not stored in
  write_target_domain
* operation not permitted happens when creating the same sequence of
  files at the same time [sf.net trac #643]
* private mode in ubuntu does not work since gfarm-2.5.8-rc3 [sf.net
  trac #661]
* replica_check_host_down_thresh does not work [sf.net trac #655]
* "protocol error" is incorrectly reported even when it is a network
  error [sf.net trac #658]
* postgres server for gfarm may not correctly start up on Ubuntu 12.04
  [sf.net trac #652]
* gfpcopy cannot copy read-only directories [sf.net trac #660]
* gfrm -h can remove a replica that may be used for replication
  [sf.net trac #676]
* write-mode open may fail due to inappropriate replica scheduling
  [sf.net trac #682]

Release note for Gfarm 2.5.8-rc3
================================

[2013.3.27]

New feature
* support GFARM_FILE_APPEND
* gfpcopy - copy missing or obsolete files when the target directory
  exists [sf.net trac #632]
* gfstress.rb - -T and --timeout option to specify the time for the
  stress test in seconds.  Default is infinite.

Updated feature
* gfreg - performance improvement [sf.net trac #484]

Bug fix
* gfpcopy-test.sh does not report throughput [sf.net trac #628]
* libgfarm - gfs_realpath() may crash [sf.net trac #591]
* gfsd - terminates even when there are still opening files in write
  mode if two or more files are opened in write mode
* gfsd fails to start when the connection to gfmd is disconnected
  during the spool check [sf.net trac #587]
* gfsd - die when unexpected error happens in gfs_server_status since
  gfmd disconnects the connection [sf.net trac #595]
* number of replicas may increase when opened in the truncate mode,
  which is introduced by Gfarm 2.5.7.1  [sf.net trac #583]
* gfmd crashes when gfmd-initiated replication is invoked against a
  filesystem node which is down, which is introduced by Gfarm
  2.5.8-rc1 [sf.net trac #601]
* potential deadlock in gfmd [sf.net trac #614]
* gfmd may crash when gfsd is down [sf.net trac #589, #616]
* fix gfmd slavestart in SuSE and default [sf.net trac #627]
* failover does not work after accessed by a Gfarm URL [sf.net trac
  #625]
* gfmd executed by slavestart (force_slave) crashes when promoting to
  the master [sf.net trac #635]
* invalid free() in libgfarm at network interface down, which is
  introduced by Gfarm 2.5.8-rc2 [sf.net trac #640]
* replica check may fail by device busy or by already in progress
  [sf.net trac #648]
* gfmdhost command can make gfmd crash [sf.net trac #594]

Release note for Gfarm 2.5.8-rc2
================================

[2013.1.8]

Bug fix
* libgfarm - gfm_host_is_local() may crash when gfmd connection is
  disconnected [sf.net trac #580]
* gfmd - segfault when executing gfwhere against a file including
  unnecessary copies [sf.net trac #575]
* gfmd - fix invalid strdup_ck() calls [sf.net trac #460]
* search from all hosts when there is no candidate in the target
  domain specified by "write_target_domain"
* ls fails once after restart or failover of gfmd [sf.net trac #578]
* master should log a message when a slave is expired [sf.net trac
  #566]
* any authenticated user can change metadata server configuration
  [sf.net trac #572]
* gfmdhost shows "?" (unknown) when an async slave reconnects [sf.net
  trac #568]
* gfprep stops when there there are less available destination hosts
  than desired [sf.net trac #574]

Release note for Gfarm 2.5.8-rc1
================================

[2012.12.3]

New feature
* replica_check - automatic replica creation in case of file system
  node failure, changing gfarm.ncopy and moving a directory 
* ganglia plugin to monitor performance
* gfncopy - set and display NCOPY of file [sf.net trac #497]
* gfchmod, gfchown, gfchgrp - -R option to operate recursively
* libgfarm - select near filesystem nodes more reliably [sf.net trac
  #535]
* gfspoolpath - show a physical pathname in a spool directory on a
  filesystem node [st.net trac #493]
* gfservice - control Gfarm servers from a remote host
* config-gfmd-iostat, config-gfsd-iostat - set up performance
  monitoring environment

New configuration directive
* atime - specify access time update type.  default is relative.
  [sf.net trac #523]
* write_local_priority - enables or disables local host affinity
  scheduling [sf.net trac #125]
* write_target_domain - specify a target domain for new files to
  stored
* spool_check_level - specify spool check level for gfsd [sf.net trac
  #502]
* replica_check_minimum_interval - specify the minimum interval for
  replica_check [sf.net trac #453]
* schedule_rtt_thresh_diff, schedule_rtt_thresh_ratio - scheduling
  parameter to decide local networks [sf.net trac #535]

Updated feature
* support nanosecond for atime, mtime and ctime [sf.net trac #529]
* gfarm clients - support relative path and multiple Gfarm file
  systems when mounted by gfarm2fs
* gfls, gfdf, gfstatus, gfsched - -V option to show Gfarm version
* gfsd - spool directory consistency check at start-up
* gfstat - print inum+gen in hex that will help to find files in
  lost+found
* gfrm -h, gfrep -x, gfprep -x - prevent to remove file replicas when
  the total number is less than specified by gfarm.ncopy
* log level adjustment to make ZABBIX alerts usable - steps 1 and 2
  [sf.net trac #511, #542]
* gfmd init script - slavestart to start in slave mode
* support 1024 or greater number of filesystem nodes [sf.net trac
  #477]

Bug fix
* invalid XML value causes gfmd crash [sf.net trac #549]
* gfmd aborts in journal_file_read_serialized() to send journal to
  asynchronous slave [sf.net trac #513]
* cannot set an empty default ACL [sf.net trac #517]
* incomplete replica is left [sf.net trac #408]
* obsolete replicas remain existing [sf.net trac #406]
* local replica may not be selected if there are replicas in the same
  network [sf.net trac #141]
* master gfmd incorrectly diagnoses "expired" state about slave gfmd
  [sf.net trac #374]
* async slave gfmd loses a connection with master gfmd [sf.net trac
  #550]
* unnecessary automatic replication when a file is opened for writing
  multiple times simultaneously [sf.net trac #438]
* gfsd stops to work with a message "<err> ![1000551] cannot set
  canonical hostname of this node", when gfmd is overloaded [sf.net
  trac #504] 
* gfls - fix space padding in multicolumn format
* overflow when parsing int value in the configuration file [sf.net
  trac #546]
* installation errors on Mac OS X [sf.net trac #552]

Release note for Gfarm 2.5.7.2
==============================

[2012.11.1]

New configuration directive
* spool_server_listen_backlog, metadb_server_listen_backlog -
  specifies the listen backlog [sf.net trac #444]

Updated feature
* gfprep, gfpcopy - -P option to report bandwidth when copying each
  file
* gfperf-web - print stack trace in error case

Bug fix
* gfprep -p may report minus seconds [sf.net trac #498]
* client-initiated replica creation fails, if the file is too large
  [sf.net trac #488]
* config-gfarm-update -r cannot enable metadb replication [sf.net
  trac #301]
* gfsd tries to re-connect only once at the close time [sf.net trac
  #318]

Release note for Gfarm 2.5.7.1
==============================

[2012.10.11]

Bug fix
* gfpcopy, gfprep - may work slowly [sf.net trac #436]
* gfchown - segfault in an error case [sf.net trac #473]
* gfdump.postgresql - remove unnecessary journal file when starting
  the slave gfmd after restoring the database
* gfmd - gfmd crashes, if an error occurs while reading a journal file
  in db_journal_fetch() [sf.net trac #468]
* gfmd - fix possible lost all replicas when multiple clients open the
  same file with GFARM_FILE_TRUNC simultaneously [sf.net trac #441]
* gfmd, gfsd, libgfarm - lost all replicas may happen due to a race
  condition between creating and reading the same file [sf.net trac
  #455]
* gfmd, gfsd - gfsd -ccc (or -c, -cc) cannot fix invalid files having
  inconsistent file size on the spool, introduced in 2.5.6 [sf.net
  trac #439]
* gfmd - potential gfmd memory corruption when scheduling file system
  nodes [sf.net trac #449]
* libgfarm - log original errno if it is not known [sf.net trac #469]

Release note for Gfarm 2.5.7
============================

[2012.9.3]

Updated feature
* gfprep, gfpcopy - improve destination host scheduling not to choose
  the same destination at the same time
* libgfarm, gfsd - support nanosecond resolution for file
  creation/modification/access time if it is supported by the file
  system node
* gfmd - suppress too many "accept: Too many open files" messages when
  thousands of clients access [sf.net trac #412]

Bug fix
* gfmd - file replicas may be inconsistent if the file is updated
  within a second but it has the same size.

Release note for Gfarm 2.5.6
============================

[2012.8.24]

New feature
* gfsd - -ccc option to check consistency of metadata.  When a
  referenced physical file does not exist, the corresponding metadata
  is deleted.  Unreferenced physical files are moved to lost+found/.

New command
* gfjournaladmin - manipulates a journal file.
* grid-proxy-agent - a utility to re-new the proxy cert automatically.

New configuration directive
* metadb_server_max_descriptors - specifies the limit of descriptors
  for gfmd. [sf.net trac #411]
* schedule_concurrency, schedule_concurrency_per_net,
  schedule_candidates_ratio, schedule_rtt_thresh - parameters for
  client scheduling.
* fatal_action - specifies an action when fatal error happens. [sf.net
  trac #410]

Bug fix
* gfrep - gfrep fails if a symlink to a directory is included.
* gfrm - with -h option, directories should not be removed.
* libgfarm - segfault when resolving symlink in the top
  directory. [sf.net trac #393]
* libgfarm - URL without a port number cannot be parsed. [sf.net trac
  #377]
* libgfarm - global username should be obtained by gfmd in
  GSI. [sf.net trac #394]
* config-gfarm - filename including non utf-8 multibyte characters
  crashes the gfmd. [sf.net trac #387]
* gfsd, gfmd - automatic replication may be unintentionally
  failed. [sf.net trac #184]
* gfmd - physical file removal may be delayed. [sf.net trac #76]
* gfmd - metadata consistency check is broken, and may crash the
  gfmd, which was introduced in 2.5.0. [sf.net trac #379]
* gfmd - segfault by specifying the same user multiple times when
  creating or modifying a group. [sf.net trac #397]
* gfmd - back channel to gfsd may be unintentionally
  disconnected. [sf.net trac #79]
* gfmd - crash by too large extended attribute. [sf.net trac #399]
* gfmd - crash when multiple clients updates the same file
  simultaneously. [sf.net trac #407]
* gfmd - user information change is not effective online in
  GSI. [sf.net trac #392 and #423]
* gfmd - when all replica of a nonzero-size file cannot be accessed,
  return error, instead of unintentionally creating a zero-size file.
* gfmd - possible crash when the metadata server information is
  changed. [sf.net trac #391]
* gfmd - too long extended or XML extended attribute may crash the
  gfmd. [sf.net trac #427]
* gfmd - crash when non utf-8 XML extended attribute is
  stored. [sf.net trac #389]
* gfmd - frozen by receiving SIGUSR1 during initialization. [sf.net
  trac #405]
* gfmd - race condition about peer_free() that may crash the
  gfmd. [sf.net trac #426]

Release note for Gfarm 2.5.5
============================

[2012.4.3]

New Command
* gfprep - parallel file replication command
* gfpcopy - parallel file copy command
* gfpcopy-test.sh - test script to investigate effective number of
  parallel file copy creations

New configuration directive
* client_parallel_copy - specify the number of parallel operations for
  gfpcopy and gfprep.

Bug fix
* gfmd - fix assertion failure when multiple clients open the
  same file, which was introduced in 2.4.1. [sf.net trac #375]
* gfmd - fix segfault during replication [sf.net trac #200, #256, and
  #372]
* gfmd - journal records are not completely applied at start up
  [sf.net trac #371]
* gfmd - directory inode is not freed, when unlink() against the
  directory is called during the directory is opened [sf.net trac
  #358]
* gfmdhost does not output expired state, which was introduced in
  2.5.4 [sf.net trac #369]
* Gfarm-FAQ - correct the description about how to dump core
* gfmd - fix a segfault when multiple clients, each has a valid
  certificate, try to authenticate but fail. [sf.net trac #348]
  This fix requires to explicitly send SIGHUP to gfmd and gfsd to
  reload CA certificate and CRL.

Release note for Gfarm 2.5.4.1
==============================

[2012.2.25]

Updated feature
* network_receive_timeout directive - default is changed to 60 seconds
  from 20 seconds since it is too short when gfmd is overloaded.

Bug fix
* config-gfarm-update - does not work [sf.net trac #345]
* gfmdhost - -c option causes gfmd segfault [sf.net trac #346]

Release note for Gfarm 2.5.4
============================

[2012.2.22]

New Command
* gfperf.rb - performance measurement tool
* gfstress.rb - heavy load generator
* gfiops - iops measurement tool

New API
* gfs_seekdir and gfs_telldir

New configuration directive
* debug_command - specify the command to be executed when SIGQUIT,
  SIGILL, SIGTRAP, SIGFPE, SIGBUS, and SIGSEGV are caught

Documentation
* Gfarm-FAQ - how to dump core

Updated feature
* link a thread-safe globus flavor
* support private mode in debian
* gfstatus - -d option for debug messages
* gfjournal - -m option to display the latest sequence number [sf.net
  trac #308]
* gfdump.postgresql - -n option to suppress stopping and starting the
  metadata server
* gfsd - terminates only if there is no file opened in write mode
* gfmd - gfmd hostname can be specified by the configuration file
  [sf.net trac #309]
* gfmd - slave gfmd exits when the database is out of date [sf.net
  trac #328]
* display backtrace when gfmd/gfsd aborts [sf.net trac #304]

Bug fix
* fix connection error in case of GSI and local host
* gfjournal - deadlock with -lv option [sf.net trac #335]
* gfdump.postgresql - always exit 1 in the dump mode
* gfmd - fix potential segfault [sf.net trac #337]
* gfmd - plug memory leak when updating metadata host at start up
  [sf.net trac #333]
* gfmd - plug memory leak when changing a slave gfmd to a master gfmd
  [sf.net trac #332]
* gfmd - fix segfault when disconnecting a master-slave gfmd channel
  [sf.net trac #330]
* gfmd - fix assertion failure when a slave gfmd reconnects to the
  master gfmd after the database is out of date [sf.net trac #322]
* gfmd - journal file cannot be correctly opened when it is
  overwritten once [sf.net trac #321]
* gfmd - plug file descriptor leak when the journal records are
  expired [sf.net trac #320]
* gfmd - fix internal error during sending journal records [sf.net
  trac #314]

Release note for Gfarm 2.5.3
============================

[2011.12.31]

Performance improvement
* improve communication latency in GSI authentication

Bug fix
* slave gfmd may become readonly after restart of the backend
  database. [sf.net trac #281]
* gfmd - fix segfault or assertion failure. [sf.net trac #288 and
  #291]

Release note for Gfarm 2.5.2
============================

[2011.12.19]

New configuration directive
* file_trace for audit log of file creation, modification, and removal.
  [sf.net trac #282]

Documentation
* INSTALL.RPM - includes how to create binary packages.

Bug fix
* journal file may not be correctly read when the gfmd stops. [sf.net
  trac #284]
* a client hostname is not correctly recorded in the log at
  authentication failure. [sf.net trac #283]
* a client hostname is not correctly recorded in the log when the
  client is not a file system node. [sf.net trac #262]
* gfmd may stop when the backend database is restarted.
* gfmd may stop without any message. [sf.net trac #278]
* journal file is not applied when gfmd restarts.
* Gfarm file system may become read-only when metadata replication is
  enabled. [sf.net trac #271]
* incomplete file replica is left when gfrep fails. [sf.net trac #260]
* when gfmd restarts, it fails to start due to the duplicate
  application of journal log. [sf.net trac #267]
* config-gfarm fails when the username includes '-'. [sf.net trac
  #261]

Release note for Gfarm 2.5.1
============================

[2011.9.14]

New configuration directive
* metadb_server_force_slave to execute the old master metadata server
  as a slave server without restoring the backend database.  This
  directive is used when re-joining the master metadata server as a
  slave server. [sf.net trac #240]
* metadb_journal_recvq_size to specify the size of journal receive
  queue.  The default is changed from 1,000 to 100,000.

Updated feature
* gfmdhost - introduce new status characters: 'x' means the slave
  metadata server is too old to synchronize. 'e' means an error
  happens when connecting to the master metadata server.
* gfdump.postgresql - dump and restore between PostgreSQL
  installation with different admin account
* gfstat - -M option to show the corresponding metadata server
  information

Bug fix
* fix segfault when removing and adding metadata servers
* gfmdhost incorrectly reports slave server status
* gfmd cannot be executed on FreeBSD 8.1 reported by
  gfarm-announce:101727
* fix race condition about mdhost hash table and journal file
* fix lost wake-up bug among reader and writer threads for the journal
  file [sf.net trac #239]

Release note for Gfarm 2.5.0
============================

[2011.8.5]

New Feature
* synchronous and asynchronous metadata replication support

New command
* gfmdhost - manages replicated metadata servers
* gfjournal - displays metadata journal data

New configuration directive
* metadb_server_list to specify replicated metadata servers
* metadb_replication to enable the replicated metadata feature
* synchronous_replication_timeout to speciy the timeout for
  synchronous metadata replication 
* synchronous_journaling to disable to flush the journal data in each
  metadata transaction
* metadb_journal_dir and metadb_journal_max_size to specify the
  journal directory and the maximum file size, respectively
* network_receive_timeout to specify the timeout of packet reception.
  This timeout is for failover of reading a file replica against the
  gfsd failure
* client_file_bufsize to specify the file buffer size in Gfarm client
  library.  Default is 1048568 bytes.  It should be less than 1048568
  not for deterioration of remote access performance for version 2.5.0
  or earlier.
* log_auth_verbose to display verbose authentication log in GSI.

Documentation
* metadata replication tutorial
* manual page - gfmdhost(1)
* install document for debian

Updated feature
* failover during reading files against the gfsd failure
* config-gfarm, config-gfarm-update - -r option to support replicated
  metadata servers, and -j option to specify a journal directory
* upgrading to a master gfmd by sending SIGUSR1 signal to a slave gfmd
* initialize GSI environment by sending SIGHUP signal to a gfmd
* Globus debian package support

Bug fix
* GSI authorization fails when CA or CRL expires [sf.net trac #235]

Release note for Gfarm 2.4.2
============================

[2011.4.22]

New Feature
* extended Access Control List (ACL) based on POSIX 1003.1e DRAFT 17
* gfarm_root.{user,group} extended attributes to specify root
  privileged users
* support of a symbolic link in Gfarm URL format to federate Gfarm
  file systems

New Command
* gfsetfacl, gfgetfacl - managing extended ACLs

New API
* Gfarm ACL APIs - gfs_acl_*
* gfs_lgetxattr, gfs_lsetxattr, gfs_lremovexattr, gfs_llistxattr,
  gfs_lgetxmlattr, gfs_lsetxmlattr, gfs_lremovexmlattr, gfs_llistxmlattr
* gfs_lchown, gfs_lchmod, gfs_lutimes - operations that do not follow
  a symlink

Documentation
* manual pages - gfexport(1), gfreg(1)
  gfquota(1), gfedquota(1), gfquotacheck(1), gfusage(1) - managing quota
  gfgetfacl(1), gfsetfacl(1) - managing extended ACLs
  gfarm_attr(5) - extended attribute
* SETUP - new basic functionality section

Performance improvement
* latency improvement in a recursive operation of gfarm commands

Updated feature
* gfchmod, gfchown - -h option not to follow a symlink, and wildcard
  expansion
* config-gfarm - support Ubuntu 10.10
* namespace of extended attribute - see gfarm_attr(5)
* gfrm - -r and -n options

Bug fix
* libgfarm APIs do not follow symlinks [sf.net trac #185]
* gfsd does not work on NetBSD-5 [sf.net trac #208]
* error detection problem of gfmkdir -p [sf.net trac #218]
* missing sanity check when moving a directory to its descendant
  [sf.net trac #198]
* when a directory is moved, its ".." link is not maintained
  correctly [sf.net trac #182]
* struct gfs_stat:st_nlink for directories is not maintained
  correctly [sf.net trac #199] 
* minimum disk space is not correctly handled by gfrep [sf.net trac
  #195]
* signal handlers don't work on CentOS 5.5 [sf.net trac #192]

Release note for Gfarm 2.4.1
============================

[2010.12.30]

New Feature
* automatic replication in gfmd [sf.net trac #129]
  extended attribute 'gfarm.ncopy' specifies the number of file
  replicas to be created automatically.
  % echo -n 2 | gfxattr -s / gfarm.ncopy

New configuration in gfmd.conf
* no_file_system_node_timeout and gfmd_reconnection_timeout directives
  to specify the timeout to try to find a file system node and to
  reconnect to the gfmd, respectively.  Default is 30 seconds.
* known_network directive to specify a network for file system nodes.
  File system nodes not specified by the known_network directive are
  assumed to be in an IPv4 class C network as same as before.

Performance improvement
* gfmd-initiated file replication [sf.net trac #123]
* extended attribute cache in gfmd and libgfarm [sf.net trac #128]

Updated feature
* gfmd - check and repair nlinks at start-up [sf.net trac #57]
  unreferenced inodes moved to /lost+found
* gfsd - if the input/output error occurs, kill oneself to cope with
  the hardware failure [sf.net trac #85]
* select a file system node having enough capacity regardless of
  existent file replicas when a file is overwritten [sf.net #68]
* avoid scheduling conflict of file system nodes
* try another file replica when the specified physical file does not
  exist
* gfdf - -P option to specify the metadata server
* gfdf - -S and -r options to sort output by available capacity and in
  reverse order, respectively [sf.net trac #131]
* gfdf - -h and -H options to display sizes in human readable format
  [sf.net trac #126]
* gfmkdir - -p option to create parent directories if needed, and not
  to return error even if the target directory exists
* gfsched - -c option to schedule file system nodes for a file
  creation [sf.net trac #169]

Bug fix
* fix error handling of protocols in terms of extended attributes in
  gfmd [sf.net trac #124]
* gfmd - plug memory leak in case of authentication error [sf.net trac
  #166]
* gfmd - a file system node temporarily cannot be used if replication
  error happens due to the connection error to the source target
  [sf.net trac #162]
* Won't compile on FreeBSD 7.3 [sf.net trac #138]
* gfrep - may fail to create two or more file replicas [sf.net trac
  #114]
* gfrep - gfrep may cause no space left on device [sf.net trac #152]

Release note for Gfarm 2.4.0
============================

[2010.7.22]

New Feature
* update-based replica consistency protocol
  that keeps number of file replicas at file modification

New configuration in gfmd.conf
* simultaneous_replication_receivers directive to specify the maximum
  number of pending replication requests to each gfsd

Documentation
* how to access Gfarm via samba from Windows clients

Updated feature
* gfwhere - several options to obtain in progress or obsolete replica
  information

Bug fix
* fix possibility to lost files in a rare case when updating and
  replicating at the same time
* gfhost -R does not work [sf.net trac #120]
* retry another file system node in GFARM_ERR_FILE_MIGRATED case
  [sf.net trac #117]
* fix errors found by static analysis including unchecked return value
  and missing check against null
* compilation failure with old OpenLDAP library since the version
  2.3.0 [sf.net trac #121]
* compilation failure on Linux 2.4 since the version 2.3.0
  [sf.net trac #122]

Not supported
* no_file_system_node_timeout and gfmd_reconnection_timeout directives
  to specify the timeout to try to find a file system node and to
  reconnect to the gfmd, respectively.  Default is 30 seconds.
* gfmd - check and repair nlinks at start-up
* gfsd - if the input/output error occurs, kill oneself to cope with
  the hardware failure
* select a file system node having enough capacity regardless of
  existent file replicas when a file is overwritten

Release note for Gfarm 2.3.2
============================

[2010.7.1]

New Command
* gfsched - schedule and display available file system nodes

New API
* new scheduling APIs - gfarm_schedule_hosts_domain_all,
  gfarm_host_sched_info_free, gfarm_schedule_hosts{,_acyclic}{,_to_write}

New configuration in gfarm2.conf
* no_file_system_node_timeout and gfmd_reconnection_timeout directives
  to specify the timeout to try to find a file system node and to
  reconnect to the gfmd, respectively.  Default is 30 seconds.

Documentation
* manual pages - gfsched(1),
  gfarm(3), gfarm_initialize(3), gfarm_terminate(3), 
  gfs_pio_create(3), gfs_pio_open(3), gfs_pio_close(3), gfs_pio_write(3),
  gfs_pio_read(3)

Updated feature
* support OpenSSL 1.0.0
* support kfreebsd-gnu and linux-gnuabi
* gfmd - check and repair nlinks at start-up
* gfsd - if the input/output error occurs, kill oneself to cope with
  the hardware failure
* gfrm - -f option to force to remove

Bug fix
* gfrep - -x option does not remove excessive number of file replicas
  in case some file replica creations fail
* gfhost - -c/-m/-d without a hostname doesn't cause an error [sf.net
  trac #93]
* gfs_pio_fstat() - may not return correct file size [sf.net trac
  #111]
* gfsd - fix the back channel process stop when the connection error
  happens
* gfsd failed to report an error that its hostname is not registered
  in gfmd (i.e. "gfhost -M")
* file close operation is missing in gfsd when a client crashed
  [sf.net trac #2]
* fix data race to calculate total amount of disk usage when a file
  system node is up and down
* fix missing metadata update when GFARM_FILE_TRUNC is specified
  [sf.net trac #103]
* fix missing permission check when GFARM_FILE_RDONLY|GFARM_FILE_TRUNC
  is specified [sf.net trac #107]
* try the next auth method in case of permission denied
* the test program "fsx" causes an assertion failure [sf.net trac
  #102]
* fix compilation errors on FreeBSD 8.0
* fix bashism reported by checkbashisms
* UNIX sockets and their parent directories are not removed when gfsd
  is stopped [sf.net trac #94]

Release note for Gfarm 2.3.1
============================

[2010.3.29]

New Feature
* quota (see doc/quota.en for details)
* Gfarm URL, gfarm://metaserver:port/path/name
* partial support of multiple metadata servers
* message catalog framework for I18N
* group map file

New Command
* gfquota, gfedquota, gfquotacheck - manipulate disk quota
* gfusage - show file system usage

New configuration in gfmd.conf
* local_group_map directive to specify a group map file that maps
  global group names to local group names for legacy clients
* log_message_verbose_level directive to specify how verbose the log
  message is
* schedule_idle_load_thresh and schedule_busy_load_thresh directives
  to specify threshold of CPU load average for client scheduling
* schedule_virtual_load directive to specify virtual load to add when
  a host is scheduled

Updated feature
* gfrep - -x option to remove excess file replicas
* gfrep - enable capacity check
* gfexport - improve performance
* gfhost, gfgroup, gfuser - support multiple metadata servers by the
  -P option 
* gfstat - -c option to display the number of file replicas
* gfwhere - display error if there is no file replica
* gfuser, gfgroup - display specified users or groups if any
* gfvoms-sync - search certificates in the same order as Globus
* change the maximum number of open files from 256 to 1024
* select a file system node having enough capacity regardless of
  existent file replicas when a file is overwritten
* read only mode for gfsd
* support re-registration of users, groups and hosts without gfmd
  restart
* fix inconsistency if the corresponding file data is missing
* gfmd reconnection process only starts if the connection error
  happens
* wait for a target gfsd to reconnect to the gfmd if there is no file
  system node
* do not allow to remove gfarmadm group
* change log format

Bug fix
* fix rename problem such that a directory cannot be deleted or orphan
  inodes remain after rebooting gfmd
* fix missing automatic reconnection to gfmd
* fix file replication error during the file update
* fix file update problem during replication
* fix GSI authentication error handling
* fix duplicate registration of the same GSI DN
* fix file migration issue described in the tracker
  https://sourceforge.net/tracker/?func=detail&aid=1976260&group_id=172718&atid=862742
* fix access control for chmod
* gfmd - fix segfault when the postgresql database is stopped

Release note for Gfarm 2.3.0
============================

[2009.8.23]

New Feature
* automatic reconnection to gfmd 
* extended attribute
* XML extended attribute that is searchable by XPath
* group management by the Virtual Organization Membership Service
  (VOMS)
* gfarmroot group - super users' group in Gfarm
* multiple spools support in GSI
  [contribution from Nikola Garafolic]

New Command
* gfvoms-sync - synchronizes the group information with the VOMS
* gfxattr - set, get, remove, and list extended attributes
* gffindxmlattr - search XML extended attributes by XPath

New API
* gfs_{set,get,list,remove}xattr, gfs_f{set,get,remove}xattr -
  extended attribute support
* gfs_{set,get,list,remove}xmlattr - XML extended attribute support
* gfs_findxmlattr, gfs_getxmlent, gfs_closexmlattr - XML extended
  attribute search

New configuration in gfmd.conf
* metadb_server_dbq_size directive to specify a queue length for
  backend database

Updated feature
* gfdf - add the -n option to display hostnames of available file
  system nodes
* config-gfarm-update - the --update-symlink option is replaced with
  the --update option, which updates the metadata schema of backend
  database for not only symbolic links but also extended attributes
* config-gfarm-update - support ldap backend
* config-gfarm - add -X option to enable XML extended attribute.
* gfmd - use epoll if available that improves the response time in
  Linux

Bug fix
* gfmd - fix a critical problem such that there is a case to remain
  invalid old replicas even when it is updated.
* gfmd - fix a problem such that an application hangs when opening a
  file in write mode that is opened by another application.
* config-gfarm - fix unbound variable error when setting the DEBUG
  environment variable.
* update st_atime and st_mtime in backend database

Release note for Gfarm 2.2.0
============================

[2009.3.26]

New Feature
* symbolic link support
* hundreds of clients support by thread pool
* directory listing speedup by attribute cache

New Command
* gfstatus - display Gfarm configuration status
* gfln - create a hard link or a symbolic link

New Admin Command
* config-gfarm-update - update the backend database.
  'config-gfarm-update --update-symlink' updates the backend database
  to manage symbolic links.  Note that the postgresql backend is only
  supported for now.

New API
* gfs_symlink, gfs_readlink - symbolic link support
* gfs_lstat

New configuration in gfmd.conf
* metadb_server_stack_size directive to specify a stack size of
  threads
* metadb_server_thread_pool_size directive to specify the maximum
  number of threads in a thread pool
* metadb_server_job_queue_length directive to specify the length of a
  job request queue
* metadb_server_heartbeat_interval directive to specify the interval
  of heartbeat to each gfsd

Documentation
* manual pages -  gfchgrp(1), gfchmod(1), gfchown(1), gfgroup(1),
  gfhost(1), gfln(1), gfls(1), gfmv(1), gfstatus(1), gfuser(1).
* manual pages in HTML

Updated feature
* gfls - accomodate attribute caching and support display regarding
  symlinks and hardlinks
* gfchmod - display more user friendly error message
* gfmd - support hundreds of clients by introducing a thread pool
* gfmd - authorize peers concurrently by making network related
  functions MT safe
* gfmd - schedule the local host only when it has enough disk space
* gfsd - support gfmd reconnection at the start up
* (GSI) allow to delegate a limited proxy
* (GSI) disable GSS_C_NT_USER_NAME (spool_server_cred_type user) by
  default since it is not supported by Globus GSI

Bug fix
* libgfarm - fix missing intialization of a pointer
* gfmd - fix race condition
* gfsd - fix -c and -cc options to check invalid files not only
  invalid file replicas
* gfsd - terminate the master process when the back channel process dies
* (GSI) use distinguished name of delegated credential

Release note for Gfarm 2.1.1
============================

[2008.9.27]

New Feature
* on-demand replication

New Command
* gfdump.postgresql - gfdump for postgresql backend

New API
* gfs_link - hardlink support
  [contribution from Rainer Rohmfeld]

Documentation
* manual pages - gfrep(1)

Updated feature
* gfgroup - group membership modification support
* C++ support
* support Globus 4.2.0

Bug fix
* segfault when listing a directory that includes a removed user
  [sf.net tracker bugs #2009543]
* race condition during authentication
* LP64 portability problem when using GSI authentication
* "too many open files" problem when replicating many files

Release note for Gfarm 2.1.0
============================

[2008.5.27]

New Feature
* Grid Security Infrastructure (GSI) support

New Command
* gfgroup - manage groups
* gfdf - report usage of Gfarm file system

New API
* gfs_statfs(3)
* gfs_replicate_to(3), gfs_migrate_to(3), gfs_migrate_from_to(3)

Documentation
* manual pages - gfdf(1), gfmkdir(1), gfrmdir(1), gfwhere(1), gfwhoami(1),
  gfkey(1), gfarm2.conf(5), gfsd(8), and gfmd(8).
* SETUP, Gfarm-FAQ

Performance enhancement
* enlarge the block size of remote I/O from 256KB to 1MB to improve
  remote read/write performance from a distant location

Updated feature
* gfrep - full feature support
* gfrm - support deletion of a file replica
* gfsd - support a feature to check a spool directory when starting up
* config-gfarm - support of a private mode and GSI

Bug fix
* gfhost - port number cannot be modified
* gfrep - recover the state when replica creation fails
  [sf.net tracker bugs #1929840]
* a security hole of GFS_PROTO_REPLICA_RECV, pointed out by
  [gfarm-discuss-ja:03295]
* possible segfault when deleting a user or a host
* default available capacity is set accidentally to 100GB
* fix race condition when obtaining all host information

Release note for Gfarm 2.0.0
============================

The Gfarm file system is a distributed file system consisting of the
local storage of commodity PCs.  Many PCs in a local area network,
compute nodes in a single cluster, multiple clusters in wide area,
comprise a large-scale, high-performance shared network file system.

Gfarm 2.0.0 (aka Gfarm v2) is a major new release to solve several
security issues found in Gfarm v1 and to improve performance for small
files.

[2007.11.28]

Features

 * Gfarm file system metadata daemon (gfmd)

   Gfmd manages file system metadata including locations of file
   replicas, and monitors file system nodes.

 * Gfarm file system daemon (gfsd)

   Gfsd is an I/O daemon running on every file system node for remote
   file operations and a file replication.

 * Client commands

   file management - gfls, gfreg, gfrm, gfmv, gfexport, gfstat
   file replica management - gfwhere, gfrep
   directory management - gfmkdir, gfrmdir
   host management - gfhost
   security - gfkey
   user/group management - gfuser, gfchmod, gfchown
   configuration - config-gfarm, config-gfsd

 * GfarmFS-FUSE (gfarm2fs)

   GfarmFS-FUSE enables to mount a Gfarm file system in userspace.

Unsupported features

 * GSI authentication

$Id$