# 1.0.0 (2024-04-06)
## π₯ Breaking Changes
- Merging maint to make the first major release. [PR #7577](https://github.com/datalad/datalad/pull/7577) (by [@yarikoptic](https://github.com/yarikoptic))
## π Enhancements and New Features
- Increase minimal Git version to 2.25. Fixes [#7389](https://github.com/datalad/datalad/issues/7389) via [PR #7431](https://github.com/datalad/datalad/pull/7431) (by [@adswa](https://github.com/adswa))
# 0.19.6 (2024-02-02)
## π Enhancements and New Features
- Add the "http_token" authentication mechanism which provides 'Authentication: Token {TOKEN}' header. [PR #7551](https://github.com/datalad/datalad/pull/7551) (by [@yarikoptic](https://github.com/yarikoptic))
## π Internal
- Update `pytest_ignore_collect()` for pytest 8.0. [PR #7546](https://github.com/datalad/datalad/pull/7546) (by [@jwodder](https://github.com/jwodder))
- Add manual triggering support/documentation for release workflow. [PR #7553](https://github.com/datalad/datalad/pull/7553) (by [@yarikoptic](https://github.com/yarikoptic))
# 0.19.5 (2023-12-28)
## π§ͺ Tests
- Fix text to account for a recent change in git-annex dropping sub-second clock precision.
As a result we might not report push of git-annex branch since there would be none.
[PR #7544](https://github.com/datalad/datalad/pull/7544) (by [@yarikoptic](https://github.com/yarikoptic))
# 0.19.4 (2023-12-13)
## π Bug Fixes
- Update target detection for adjusted mode datasets has been improved.
Fixes [#7507](https://github.com/datalad/datalad/issues/7507) via
[PR #7522](https://github.com/datalad/datalad/pull/7522)
(by [@mih](https://github.com/mih))
- Fix typos found by new codespell 2.2.6 and also add checking/fixing "hidden files". [PR #7530](https://github.com/datalad/datalad/pull/7530) (by [@yarikoptic](https://github.com/yarikoptic))
## π Documentation
- Improve threaded-runner documentation. Fixes [#7498](https://github.com/datalad/datalad/issues/7498) via [PR #7500](https://github.com/datalad/datalad/pull/7500) (by [@christian-monch](https://github.com/christian-monch))
## π Internal
- add RRID to package metadata. [PR #7495](https://github.com/datalad/datalad/pull/7495) (by [@adswa](https://github.com/adswa))
- Fix time_diff* and time_remove benchmarks to account for long RFed interfaces. [PR #7502](https://github.com/datalad/datalad/pull/7502) (by [@yarikoptic](https://github.com/yarikoptic))
## π§ͺ Tests
- Cache value of the has_symlink_capability to spare some cycles. [PR #7471](https://github.com/datalad/datalad/pull/7471) (by [@yarikoptic](https://github.com/yarikoptic))
- RF(TST): use setup_method and teardown_method in TestAddArchiveOptions. [PR #7488](https://github.com/datalad/datalad/pull/7488) (by [@yarikoptic](https://github.com/yarikoptic))
- Announce test_clone_datasets_root xfail on github osx. [PR #7489](https://github.com/datalad/datalad/pull/7489) (by [@yarikoptic](https://github.com/yarikoptic))
- Inform asv that there should be no warmup runs for time_remove benchmark. [PR #7505](https://github.com/datalad/datalad/pull/7505) (by [@yarikoptic](https://github.com/yarikoptic))
- BF(TST): Relax matching of git-annex error message about unsafe drop, which was changed in 10.20231129-18-gfd0b510573. [PR #7541](https://github.com/datalad/datalad/pull/7541) (by [@yarikoptic](https://github.com/yarikoptic))
# 0.19.3 (2023-08-10)
## π Bug Fixes
- Type annotate get_status_dict and note that we can pass Exception or CapturedException which is not subclass. [PR #7403](https://github.com/datalad/datalad/pull/7403) (by [@yarikoptic](https://github.com/yarikoptic))
- BF: create-sibling-gitlab used to raise a TypeError when attempting a recursive operation in a dataset with uninstalled subdatasets. It now raises an impossible result instead. [PR #7430](https://github.com/datalad/datalad/pull/7430) (by [@adswa](https://github.com/adswa))
- Pass branch option into recursive call within Install - for the cases whenever install is invoked with URL(s). Fixes [#7461](https://github.com/datalad/datalad/issues/7461) via [PR #7463](https://github.com/datalad/datalad/pull/7463) (by [@yarikoptic](https://github.com/yarikoptic))
- Allow for reckless=ephemeral clone using relative path for the original location. Fixes [#7469](https://github.com/datalad/datalad/issues/7469) via [PR #7472](https://github.com/datalad/datalad/pull/7472) (by [@yarikoptic](https://github.com/yarikoptic))
## π Documentation
- Fix a property name and default costs described in "getting subdatasets" section of `get` documentation.
Fixes [#7458](https://github.com/datalad/datalad/issues/7458) via
[PR #7460](https://github.com/datalad/datalad/pull/7460)
(by [@mslw](https://github.com/mslw))
## π Internal
- Copy an adjusted environment only if requested to do so.
[PR #7399](https://github.com/datalad/datalad/pull/7399)
(by [@christian-monch](https://github.com/christian-monch))
- Eliminate uses of `pkg_resources`. Fixes [#7435](https://github.com/datalad/datalad/issues/7435) via [PR #7439](https://github.com/datalad/datalad/pull/7439) (by [@jwodder](https://github.com/jwodder))
## π§ͺ Tests
- Disable some S3 tests of their VCR taping where they fail for known issues. [PR #7467](https://github.com/datalad/datalad/pull/7467) (by [@yarikoptic](https://github.com/yarikoptic))
# 0.19.2 (2023-07-03)
## π Bug Fixes
- Remove surrounding quotes in output filenames even for newer version of annex. Fixes [#7440](https://github.com/datalad/datalad/issues/7440) via [PR #7443](https://github.com/datalad/datalad/pull/7443) (by [@yarikoptic](https://github.com/yarikoptic))
## π Documentation
- DOC: clarify description of the "install" interface to reflect its convoluted behavior. [PR #7445](https://github.com/datalad/datalad/pull/7445) (by [@yarikoptic](https://github.com/yarikoptic))
# 0.19.1 (2023-06-26)
## π Internal
- Make compatible with upcoming release of git-annex (next after 10.20230407) and pass explicit core.quotepath=false to all git calls. Also added `tools/find-hanged-tests` helper.
[PR #7372](https://github.com/datalad/datalad/pull/7372)
(by [@yarikoptic](https://github.com/yarikoptic))
## π§ͺ Tests
- Adjust tests for upcoming release of git-annex (next after 10.20230407) and ignore DeprecationWarning for pkg_resources for now.
[PR #7372](https://github.com/datalad/datalad/pull/7372)
(by [@yarikoptic](https://github.com/yarikoptic))
# 0.19.0 (2023-06-14)
## π Enhancements and New Features
- Address gitlab API special character restrictions. [PR #7407](https://github.com/datalad/datalad/pull/7407) (by [@jsheunis](https://github.com/jsheunis))
- BF: The default layout of create-sibling-gitlab is now ``collection``. The previous default, ``hierarchy`` has been removed as it failed in --recursive mode in different edgecases. For single-level datasets, the outcome of ``collection`` and ``hierarchy`` is identical. [PR #7410](https://github.com/datalad/datalad/pull/7410) (by [@jsheunis](https://github.com/jsheunis) and [@adswa](https://github.com/adswa))
## π Bug Fixes
- WTF - bring back and extend information on metadata extractors etc, and allow for sections to have subsections and be selected at both levels [PR #7309](https://github.com/datalad/datalad/pull/7309) (by [@yarikoptic](https://github.com/yarikoptic))
- BF: Run an actual git invocation with interactive commit config. [PR #7398](https://github.com/datalad/datalad/pull/7398) (by [@adswa](https://github.com/adswa))
## π© Dependencies
- Raise minimal version of tqdm (progress bars) to v.4.32.0
[PR #7330](https://github.com/datalad/datalad/pull/7330)
(by [@mslw](https://github.com/mslw))
## π Documentation
- DOC: Add a "User messaging" design doc. [PR #7310](https://github.com/datalad/datalad/pull/7310) (by [@jsheunis](https://github.com/jsheunis))
## π§ͺ Tests
- Remove nose-based testing utils and possibility to test extensions using nose. [PR #7261](https://github.com/datalad/datalad/pull/7261) (by [@yarikoptic](https://github.com/yarikoptic))
# 0.18.5 (2023-06-13)
## π Bug Fixes
- More correct summary reporting for relaxed (no size) --annex. [PR #7050](https://github.com/datalad/datalad/pull/7050) (by [@yarikoptic](https://github.com/yarikoptic))
- ENH: minor tune up of addurls to be more tolerant and "informative". [PR #7388](https://github.com/datalad/datalad/pull/7388) (by [@yarikoptic](https://github.com/yarikoptic))
- Ensure that data generated by timeout handlers in the asynchronous
runner are accessible via the result generator, even if no other
other events occur.
[PR #7390](https://github.com/datalad/datalad/pull/7390)
(by [@christian-monch](https://github.com/christian-monch))
- Do not map (leave as is) trailing / or \ in github URLs. [PR #7418](https://github.com/datalad/datalad/pull/7418) (by [@yarikoptic](https://github.com/yarikoptic))
## π Documentation
- Use `sphinx_autodoc_typehints`. Fixes [#7404](https://github.com/datalad/datalad/issues/7404) via [PR #7412](https://github.com/datalad/datalad/pull/7412) (by [@jwodder](https://github.com/jwodder))
## π Internal
- Discontinue ConfigManager abuse for Git identity warning. [PR #7378](https://github.com/datalad/datalad/pull/7378) (by [@mih](https://github.com/mih)) and [PR #7392](https://github.com/datalad/datalad/pull/7392) (by [@yarikoptic](https://github.com/yarikoptic))
## π§ͺ Tests
- Boost python to 3.8 during extensions testing. [PR #7413](https://github.com/datalad/datalad/pull/7413) (by [@yarikoptic](https://github.com/yarikoptic))
- Skip test_system_ssh_version if no ssh found + split parsing into separate test. [PR #7422](https://github.com/datalad/datalad/pull/7422) (by [@yarikoptic](https://github.com/yarikoptic))
# 0.18.4 (2023-05-16)
## π Bug Fixes
- Provider config files were ignored, when CWD changed between different datasets during runtime.
Fixes [#7347](https://github.com/datalad/datalad/issues/7347) via
[PR #7357](https://github.com/datalad/datalad/pull/7357)
(by [@bpoldrack](https://github.com/bpoldrack))
## π Documentation
- Added a workaround for an issue with documentation theme (search
function not working on Read the Docs).
Fixes [#7374](https://github.com/datalad/datalad/issues/7374) via
[PR #7385](https://github.com/datalad/datalad/pull/7385)
(by [@mslw](https://github.com/mslw))
## π Internal
- Type-annotate `datalad/support/gitrepo.py`. [PR #7341](https://github.com/datalad/datalad/pull/7341) (by [@jwodder](https://github.com/jwodder))
## π§ͺ Tests
- Fix failing testing on CI
[PR #7379](https://github.com/datalad/datalad/pull/7379) (by [@yarikoptic](https://github.com/yarikoptic))
- use sample S3 url DANDI archive,
- use our copy of old .deb from datasets.datalad.org instead of snapshots.d.o
- use specific miniconda installer for py 3.7.
# 0.18.3 (2023-03-25)
## π Bug Fixes
- Fixed that the `get` command would fail, when subdataset source-candidate-templates where using the `path` property from `.gitmodules`.
Also enhance the respective documentation for the `get` command.
Fixes [#7274](https://github.com/datalad/datalad/issues/7274) via
[PR #7280](https://github.com/datalad/datalad/pull/7280)
(by [@bpoldrack](https://github.com/bpoldrack))
- Improve up-to-dateness of config reports across manager instances. Fixes [#7299](https://github.com/datalad/datalad/issues/7299) via [PR #7301](https://github.com/datalad/datalad/pull/7301) (by [@mih](https://github.com/mih))
- BF: GitRepo.merge do not allow merging unrelated unconditionally. [PR #7312](https://github.com/datalad/datalad/pull/7312) (by [@yarikoptic](https://github.com/yarikoptic))
- Do not render (empty) WTF report on other records. [PR #7322](https://github.com/datalad/datalad/pull/7322) (by [@yarikoptic](https://github.com/yarikoptic))
- Fixed a bug where changing DataLad's log level could lead to failing git-annex calls.
Fixes [#7328](https://github.com/datalad/datalad/issues/7328) via
[PR #7329](https://github.com/datalad/datalad/pull/7329)
(by [@bpoldrack](https://github.com/bpoldrack))
- Fix an issue with uninformative error reporting by the datalad special remote.
Fixes [#7332](https://github.com/datalad/datalad/issues/7332) via
[PR #7333](https://github.com/datalad/datalad/pull/7333)
(by [@bpoldrack](https://github.com/bpoldrack))
- Fix save to not force committing into git if reference dataset is pure git (not git-annex). Fixes [#7351](https://github.com/datalad/datalad/issues/7351) via [PR #7355](https://github.com/datalad/datalad/pull/7355) (by [@yarikoptic](https://github.com/yarikoptic))
## π Documentation
- Include a few previously missing commands in html API docs.
Fixes [#7288](https://github.com/datalad/datalad/issues/7288) via
[PR #7289](https://github.com/datalad/datalad/pull/7289)
(by [@mslw](https://github.com/mslw))
## π Internal
- Type-annotate almost all of `datalad/utils.py`; add `datalad/typing.py`. [PR #7317](https://github.com/datalad/datalad/pull/7317) (by [@jwodder](https://github.com/jwodder))
- Type-annotate and fix `datalad/support/strings.py`. [PR #7318](https://github.com/datalad/datalad/pull/7318) (by [@jwodder](https://github.com/jwodder))
- Type-annotate `datalad/support/globbedpaths.py`. [PR #7327](https://github.com/datalad/datalad/pull/7327) (by [@jwodder](https://github.com/jwodder))
- Extend type-annotations for `datalad/support/path.py`. [PR #7336](https://github.com/datalad/datalad/pull/7336) (by [@jwodder](https://github.com/jwodder))
- Type-annotate various things in `datalad/runner/`. [PR #7337](https://github.com/datalad/datalad/pull/7337) (by [@jwodder](https://github.com/jwodder))
- Type-annotate some more files in `datalad/support/`. [PR #7339](https://github.com/datalad/datalad/pull/7339) (by [@jwodder](https://github.com/jwodder))
## π§ͺ Tests
- Skip or xfail some currently failing or stalling tests. [PR #7331](https://github.com/datalad/datalad/pull/7331) (by [@yarikoptic](https://github.com/yarikoptic))
- Skip with_sameas_remote when rsync and annex are incompatible. Fixes [#7320](https://github.com/datalad/datalad/issues/7320) via [PR #7342](https://github.com/datalad/datalad/pull/7342) (by [@bpoldrack](https://github.com/bpoldrack))
- Fix testing assumption - do create pure GitRepo superdataset and test against it. [PR #7353](https://github.com/datalad/datalad/pull/7353) (by [@yarikoptic](https://github.com/yarikoptic))
# 0.18.2 (2023-02-27)
## π Bug Fixes
- Fix `create-sibling` for non-English SSH remotes by providing `LC_ALL=C` for the `ls` call. [PR #7265](https://github.com/datalad/datalad/pull/7265) (by [@nobodyinperson](https://github.com/nobodyinperson))
- Fix EnsureListOf() and EnsureTupleOf() for string inputs. [PR #7267](https://github.com/datalad/datalad/pull/7267) (by [@nobodyinperson](https://github.com/nobodyinperson))
- create-sibling: Use C.UTF-8 locale instead of C on the remote end. [PR #7273](https://github.com/datalad/datalad/pull/7273) (by [@nobodyinperson](https://github.com/nobodyinperson))
- Address compatibility with most recent git-annex where info would exit with non-0. [PR #7292](https://github.com/datalad/datalad/pull/7292) (by [@yarikoptic](https://github.com/yarikoptic))
## π© Dependencies
- Revert "Revert "Remove chardet version upper limit"". [PR #7263](https://github.com/datalad/datalad/pull/7263) (by [@yarikoptic](https://github.com/yarikoptic))
## π Internal
- Codespell more (CHANGELOGs etc) and remove custom CLI options from tox.ini. [PR #7271](https://github.com/datalad/datalad/pull/7271) (by [@yarikoptic](https://github.com/yarikoptic))
## π§ͺ Tests
- Use older python 3.8 in testing nose utils in github-action test-nose. Fixes [#7259](https://github.com/datalad/datalad/issues/7259) via [PR #7260](https://github.com/datalad/datalad/pull/7260) (by [@yarikoptic](https://github.com/yarikoptic))
# 0.18.1 (2023-01-16)
## π Bug Fixes
- Fixes crashes on windows where DataLad was mistaking git-annex 10.20221212 for
a not yet released git-annex version and trying to use a new feature.
Fixes [#7248](https://github.com/datalad/datalad/issues/7248) via
[PR #7249](https://github.com/datalad/datalad/pull/7249)
(by [@bpoldrack](https://github.com/bpoldrack))
## π Documentation
- DOC: fix EnsureCallable docstring. [PR #7245](https://github.com/datalad/datalad/pull/7245) (by [@matrss](https://github.com/matrss))
## π Performance
- Integrate buffer size optimization from datalad-next, leading to significant
performance improvement for status and diff.
Fixes [#7190](https://github.com/datalad/datalad/issues/7190) via
[PR #7250](https://github.com/datalad/datalad/pull/7250)
(by [@bpoldrack](https://github.com/bpoldrack))
# 0.18.0 (2022-12-31)
## π₯ Breaking Changes
- Move all old-style metadata commands `aggregate_metadata`, `search`, `metadata` and `extract-metadata`, as well as the `cfg_metadatatypes` procedure and the old metadata extractors into the datalad-deprecated extension.
Now recommended way of handling metadata is to install the datalad-metalad extension instead.
Fixes [#7012](https://github.com/datalad/datalad/issues/7012) via
[PR #7014](https://github.com/datalad/datalad/pull/7014)
- Automatic reconfiguration of the ORA special remote when cloning from RIA
stores now only applies locally rather than being committed.
[PR #7235](https://github.com/datalad/datalad/pull/7235)
(by [@bpoldrack](https://github.com/bpoldrack))
## π Enhancements and New Features
- A repository description can be specified with a new `--description`
option when creating siblings using `create-sibling-[gin|gitea|github|gogs]`.
Fixes [#6816](https://github.com/datalad/datalad/issues/6816)
via [PR #7109](https://github.com/datalad/datalad/pull/7109)
(by [@mslw](https://github.com/mslw))
- Make validation failure of alternative constraints more informative.
Fixes [#7092](https://github.com/datalad/datalad/issues/7092) via
[PR #7132](https://github.com/datalad/datalad/pull/7132)
(by [@bpoldrack](https://github.com/bpoldrack))
- Saving removed dataset content was sped-up, and reporting of types of removed
content now accurately states `dataset` for added and removed subdatasets,
instead of `file`. Moreover, saving previously staged deletions is now also
reported.
[PR #6784](https://github.com/datalad/datalad/pull/6784) (by [@mih](https://github.com/mih))
- `foreach-dataset` command got a new possible value for the --output-streamns|--o-s
option 'relpath' to capture and pass-through prefixing with path to subds. Very
handy for e.g. running `git grep` command across subdatasets.
[PR #7071](https://github.com/datalad/datalad/pull/7071)
(by [@yarikoptic](https://github.com/yarikoptic))
- New config `datalad.create-sibling-ghlike.extra-remote-settings.NETLOC.KEY=VALUE` allows to add and/or overwrite local configuration for the created sibling by the commands `create-sibling-`. [PR #7213](https://github.com/datalad/datalad/pull/7213) (by [@matrss](https://github.com/matrss))
- The `siblings` command does not concern the user with messages about
inconsequential failure to annex-enable a remote anymore.
[PR #7217](https://github.com/datalad/datalad/pull/7217)
(by [@bpoldrack](https://github.com/bpoldrack))
- ORA special remote now allows to override its configuration locally.
[PR #7235](https://github.com/datalad/datalad/pull/7235)
(by [@bpoldrack](https://github.com/bpoldrack))
- Added a 'ria' special remote to provide backwards compatibility with datasets
that were set up with the deprecated [ria-remote](https://github.com/datalad/git-annex-ria-remote).
[PR #7235](https://github.com/datalad/datalad/pull/7235)
(by [@bpoldrack](https://github.com/bpoldrack))
## π Bug Fixes
- When ``create-sibling-ria`` was invoked with a sibling name of a pre-existing sibling, a duplicate key in the result record caused a crashed.
Fixes [#6950](https://github.com/datalad/datalad/issues/6950) via
[PR #6952](https://github.com/datalad/datalad/pull/6952) (by [@adswa](https://api.github.com/users/adswa))
## π Documentation
- create-sibling-ria's docstring now defines the schema of RIA URLs and clarifies internal layout of a RIA store.
[PR #6861](https://github.com/datalad/datalad/pull/6861) (by [@adswa](https://api.github.com/users/adswa))
- Move maintenance team info from issue to CONTRIBUTING.
[PR #6904](https://github.com/datalad/datalad/pull/6904) (by [@adswa](https://api.github.com/users/adswa))
- Describe specifications for a DataLad GitHub Action.
[PR #6931](https://github.com/datalad/datalad/pull/6931) (by [@thewtex](https://api.github.com/users/thewtex))
- Fix capitalization of some service names.
[PR #6936](https://github.com/datalad/datalad/pull/6936) (by [@aqw](https://api.github.com/users/aqw))
- Command categories in help text are more consistently named.
[PR #7027](https://github.com/datalad/datalad/pull/7027) (by [@aqw](https://api.github.com/users/aqw))
- DOC: Add design document on Tests and CI. [PR #7195](https://github.com/datalad/datalad/pull/7195) (by [@adswa](https://github.com/adswa))
- CONTRIBUTING.md was extended with up-to-date information on CI logging, changelog and release procedures. [PR #7204](https://github.com/datalad/datalad/pull/7204) (by [@yarikoptic](https://github.com/yarikoptic))
## π Internal
- Allow EnsureDataset constraint to handle Path instances.
Fixes [#7069](https://github.com/datalad/datalad/issues/7069) via
[PR #7133](https://github.com/datalad/datalad/pull/7133)
(by [@bpoldrack](https://github.com/bpoldrack))
- Use `looseversion.LooseVersion` as drop-in replacement for `distutils.version.LooseVersion`
Fixes [#6307](https://github.com/datalad/datalad/issues/6307) via
[PR #6839](https://github.com/datalad/datalad/pull/6839)
(by [@effigies](https://api.github.com/users/effigies))
- Use --pathspec-from-file where possible instead of passing long lists of paths to git/git-annex calls.
Fixes [#6922](https://github.com/datalad/datalad/issues/6922) via
[PR #6932](https://github.com/datalad/datalad/pull/6932) (by [@yarikoptic](https://api.github.com/users/yarikoptic))
- Make clone_dataset() better patchable ny extensions and less monolithic.
[PR #7017](https://github.com/datalad/datalad/pull/7017) (by [@mih](https://api.github.com/users/mih))
- Remove `simplejson` in favor of using `json`.
Fixes [#7034](https://github.com/datalad/datalad/issues/7034) via
[PR #7035](https://github.com/datalad/datalad/pull/7035) (by [@christian-monch](https://api.github.com/users/christian-monch))
- Fix an error in the command group names-test.
[PR #7044](https://github.com/datalad/datalad/pull/7044) (by [@christian-monch](https://api.github.com/users/christian-monch))
- Move eval_results() into interface.base to simplify imports for command implementations. Deprecate use from interface.utils accordingly. Fixes [#6694](https://github.com/datalad/datalad/issues/6694) via [PR #7170](https://github.com/datalad/datalad/pull/7170) (by [@adswa](https://github.com/adswa))
## π Performance
- Use regular dicts instead of OrderedDicts for speedier operations. Fixes [#6566](https://github.com/datalad/datalad/issues/6566) via [PR #7174](https://github.com/datalad/datalad/pull/7174) (by [@adswa](https://github.com/adswa))
- Reimplement `get_submodules_()` without `get_content_info()` for substantial performance boosts especially for large datasets with few subdatasets. Originally proposed in [PR #6942](https://github.com/datalad/datalad/pull/6942) by [@mih](https://github.com/mih), fixing [#6940](https://github.com/datalad/datalad/issues/6940). [PR #7189](https://github.com/datalad/datalad/pull/7189) (by [@adswa](https://github.com/adswa)). Complemented with [PR #7220](https://github.com/datalad/datalad/pull/7220) (by [@yarikoptic](https://github.com/yarikoptic)) to avoid `O(N^2)` (instead of `O(N*log(N))` performance in some cases.
- Use --include=* or --anything instead of --copies 0 to speed up get_content_annexinfo. [PR #7230](https://github.com/datalad/datalad/pull/7230) (by [@yarikoptic](https://github.com/yarikoptic))
## π§ͺ Tests
- Re-enable two now-passing core test on Windows CI.
[PR #7152](https://github.com/datalad/datalad/pull/7152) (by [@adswa](https://api.github.com/users/adswa))
- Remove the `with_testrepos` decorator and associated tests for it
Fixes [#6752](https://github.com/datalad/datalad/issues/6752) via
[PR #7176](https://github.com/datalad/datalad/pull/7176) (by [@adswa](https://api.github.com/users/adswa))
# 0.17.10 (2022-12-14)
## π Enhancements and New Features
- Enhance concurrent invocation behavior of `ThreadedRunner.run()`. If possible invocations are serialized instead of raising *re-enter* runtime errors. Deadlock situations are detected and runtime errors are raised instead of deadlocking.
Fixes [#7138](https://github.com/datalad/datalad/issues/7138) via
[PR #7201](https://github.com/datalad/datalad/pull/7201)
(by [@christian-monch](https://github.com/christian-monch))
- Exceptions bubbling up through CLI are now reported on including their chain
of __cause__.
Fixes [#7163](https://github.com/datalad/datalad/issues/7163) via
[PR #7210](https://github.com/datalad/datalad/pull/7210)
(by [@bpoldrack](https://github.com/bpoldrack))
## π Bug Fixes
- BF: read RIA config from stdin instead of temporary file. Fixes [#6514](https://github.com/datalad/datalad/issues/6514) via [PR #7147](https://github.com/datalad/datalad/pull/7147) (by [@adswa](https://github.com/adswa))
- Prevent doomed annex calls on files we already know are untracked. Fixes [#7032](https://github.com/datalad/datalad/issues/7032) via [PR #7166](https://github.com/datalad/datalad/pull/7166) (by [@adswa](https://github.com/adswa))
- Comply to Posix-like clone URL formats on Windows. Fixes [#7180](https://github.com/datalad/datalad/issues/7180) via [PR #7181](https://github.com/datalad/datalad/pull/7181) (by [@adswa](https://github.com/adswa))
- Ensure that paths used in the datalad-url field of .gitmodules are posix. Fixes [#7182](https://github.com/datalad/datalad/issues/7182) via [PR #7183](https://github.com/datalad/datalad/pull/7183) (by [@adswa](https://github.com/adswa))
- Bandaids for export-to-figshare to restore functionality. [PR #7188](https://github.com/datalad/datalad/pull/7188) (by [@adswa](https://github.com/adswa))
- Fixes hanging threads when `close()` or `del` where called in `BatchedCommand` instances. That could lead to hanging tests if the tests used the `@serve_path_via_http()`-decorator
Fixes [#6804](https://github.com/datalad/datalad/issues/6804) via
[PR #7201](https://github.com/datalad/datalad/pull/7201)
(by [@christian-monch](https://github.com/christian-monch))
- Interpret file-URL path components according to the local operating system as described in RFC 8089. With this fix, `datalad.network.RI('file:...').localpath` returns a correct local path on Windows if the RI is constructed with a file-URL.
Fixes [#7186](https://github.com/datalad/datalad/issues/7186) via
[PR #7206](https://github.com/datalad/datalad/pull/7206)
(by [@christian-monch](https://github.com/christian-monch))
- Fix a bug when retrieving several files from a RIA store via SSH, when the annex key does not contain size information. Fixes [#7214](https://github.com/datalad/datalad/issues/7214) via [PR #7215](https://github.com/datalad/datalad/pull/7215) (by [@mslw](https://github.com/mslw))
- Interface-specific (python vs CLI) doc generation for commands and their parameters was broken when brackets were used within the interface markups.
Fixes [#7225](https://github.com/datalad/datalad/issues/7225) via
[PR #7226](https://github.com/datalad/datalad/pull/7226)
(by [@bpoldrack](https://github.com/bpoldrack))
## π Documentation
- Fix documentation of `Runner.run()` to not accept strings. Instead, encoding
must be ensured by the caller.
Fixes [#7145](https://github.com/datalad/datalad/issues/7145) via
[PR #7155](https://github.com/datalad/datalad/pull/7155)
(by [@bpoldrack](https://github.com/bpoldrack))
## π Internal
- Fix import of the `ls` command from datalad-deprecated for benchmarks.
Fixes [#7149](https://github.com/datalad/datalad/issues/7149) via
[PR #7154](https://github.com/datalad/datalad/pull/7154)
(by [@bpoldrack](https://github.com/bpoldrack))
- Unify definition of parameter choices with `datalad clean`.
Fixes [#7026](https://github.com/datalad/datalad/issues/7026) via
[PR #7161](https://github.com/datalad/datalad/pull/7161)
(by [@bpoldrack](https://github.com/bpoldrack))
## π§ͺ Tests
- Fix test failure with old annex. Fixes [#7157](https://github.com/datalad/datalad/issues/7157) via [PR #7159](https://github.com/datalad/datalad/pull/7159) (by [@bpoldrack](https://github.com/bpoldrack))
- Re-enable now passing test_path_diff test on Windows. Fixes [#3725](https://github.com/datalad/datalad/issues/3725) via [PR #7194](https://github.com/datalad/datalad/pull/7194) (by [@yarikoptic](https://github.com/yarikoptic))
- Use Plaintext keyring backend in tests to avoid the need for (interactive)
authentication to unlock the keyring during (CI-) test runs.
Fixes [#6623](https://github.com/datalad/datalad/issues/6623) via
[PR #7209](https://github.com/datalad/datalad/pull/7209)
(by [@bpoldrack](https://github.com/bpoldrack))
# 0.17.9 (2022-11-07)
## π Bug Fixes
- Various small fixups ran after looking post-release and trying to build Debian package. [PR #7112](https://github.com/datalad/datalad/pull/7112) (by [@yarikoptic](https://github.com/yarikoptic))
- BF: Fix add-archive-contents try-finally statement by defining variable earlier. [PR #7117](https://github.com/datalad/datalad/pull/7117) (by [@adswa](https://github.com/adswa))
- Fix RIA file URL reporting in exception handling. [PR #7123](https://github.com/datalad/datalad/pull/7123) (by [@adswa](https://github.com/adswa))
- HTTP download treated '429 - too many requests' as an authentication issue and
was consequently trying to obtain credentials.
Fixes [#7129](https://github.com/datalad/datalad/issues/7129) via
[PR #7129](https://github.com/datalad/datalad/pull/7129)
(by [@bpoldrack](https://github.com/bpoldrack))
## π© Dependencies
- Unrestrict pytest and pytest-cov versions. [PR #7125](https://github.com/datalad/datalad/pull/7125) (by [@jwodder](https://github.com/jwodder))
- Remove remaining references to `nose` and the implied requirement for building the documentation
Fixes [#7100](https://github.com/datalad/datalad/issues/7100) via
[PR #7136](https://github.com/datalad/datalad/pull/7136)
(by [@bpoldrack](https://github.com/bpoldrack))
## π Internal
- Use datalad/release-action. Fixes [#7110](https://github.com/datalad/datalad/issues/7110). [PR #7111](https://github.com/datalad/datalad/pull/7111) (by [@jwodder](https://github.com/jwodder))
- Fix all logging to use %-interpolation and not .format, sort imports in touched files, add pylint-ing for % formatting in log messages to `tox -e lint`. [PR #7118](https://github.com/datalad/datalad/pull/7118) (by [@yarikoptic](https://github.com/yarikoptic))
## π§ͺ Tests
- Increase the upper time limit after which we assume that a process is stalling.
That should reduce false positives from `datalad.support.tests.test_parallel.py::test_stalling`,
without impacting the runtime of passing tests.
[PR #7119](https://github.com/datalad/datalad/pull/7119)
(by [@christian-monch](https://github.com/christian-monch))
- XFAIL a check on length of results in test_gracefull_death. [PR #7126](https://github.com/datalad/datalad/pull/7126) (by [@yarikoptic](https://github.com/yarikoptic))
- Configure Git to allow for "file" protocol in tests. [PR #7130](https://github.com/datalad/datalad/pull/7130) (by [@yarikoptic](https://github.com/yarikoptic))
# 0.17.8 (2022-10-24)
## Bug Fixes
- Prevent adding duplicate entries to .gitmodules. [PR #7088](https://github.com/datalad/datalad/pull/7088) (by [@yarikoptic](https://github.com/yarikoptic))
- [BF] Prevent double yielding of impossible get result
Fixes [#5537](https://github.com/datalad/datalad/issues/5537).
[PR #7093](https://github.com/datalad/datalad/pull/7093) (by
[@jsheunis](https://github.com/jsheunis))
- Stop rendering the output of internal `subdatset()` call in the
results of `run_procedure()`.
Fixes [#7091](https://github.com/datalad/datalad/issues/7091) via
[PR #7094](https://github.com/datalad/datalad/pull/7094)
(by [@mslw](https://github.com/mslw) & [@mih](https://github.com/mih))
- Improve handling of `--existing reconfigure` in
`create-sibling-ria`: previously, the command would not make the
underlying `git init` call for existing local repositories, leading
to some configuration updates not being applied. Partially addresses
https://github.com/datalad/datalad/issues/6967 via
https://github.com/datalad/datalad/pull/7095 (by @mslw)
- Ensure subprocess environments have a valid path in `os.environ['PWD']`,
even if a Path-like object was given to the runner on subprocess creation
or invocation.
Fixes [#7040](https://github.com/datalad/datalad/issues/7040) via
[PR #7107](https://github.com/datalad/datalad/pull/7107)
(by [@christian-monch](https://github.com/christian-monch))
- Improved reporting when using `dry-run` with github-like
`create-sibling*` commands (`-gin`, `-gitea`, `-github`,
`-gogs`). The result messages will now display names of the
repositories which would be created (useful for recursive
operations).
[PR #7103](https://github.com/datalad/datalad/pull/7103)
(by [@mslw](https://github.com/mslw))
# 0.17.7 (2022-10-14)
## Bug Fixes
- Let `EnsureChoice` report the value is failed validating.
[PR #7067](https://github.com/datalad/datalad/pull/7067) (by
[@mih](https://github.com/mih))
- Avoid writing to stdout/stderr from within datalad sshrun. This could lead to
broken pipe errors when cloning via SSH and was superfluous to begin with.
Fixes https://github.com/datalad/datalad/issues/6599 via
https://github.com/datalad/datalad/pull/7072 (by @bpoldrack)
- BF: lock across threads check/instantiation of Flyweight instances. Fixes [#6598](https://github.com/datalad/datalad/issues/6598) via [PR #7075](https://github.com/datalad/datalad/pull/7075) (by [@yarikoptic](https://github.com/yarikoptic))
## Internal
- Do not use `gen4`-metadata methods in `datalad metadata`-command.
[PR #7001](https://github.com/datalad/datalad/pull/7001) (by
[@christian-monch](https://github.com/christian-monch))
- Revert "Remove chardet version upper limit" (introduced in 0.17.6~11^2) to bring back upper limit <= 5.0.0 on chardet. Otherwise we can get some deprecation warnings from requests [PR #7057](https://github.com/datalad/datalad/pull/7057) (by [@yarikoptic](https://github.com/yarikoptic))
- Ensure that `BatchedCommandError` is raised if the subprocesses of `BatchedCommand` fails or raises a `CommandError`. [PR #7068](https://github.com/datalad/datalad/pull/7068) (by [@christian-monch](https://github.com/christian-monch))
- RF: remove unused code str-ing PurePath. [PR #7073](https://github.com/datalad/datalad/pull/7073) (by
[@yarikoptic](https://github.com/yarikoptic))
- Update GitHub Actions action versions.
[PR #7082](https://github.com/datalad/datalad/pull/7082) (by
[@jwodder](https://github.com/jwodder))
## Tests
- Fix broken test helpers for result record testing that would falsely pass.
[PR #7002](https://github.com/datalad/datalad/pull/7002) (by [@bpoldrack](https://github.com/bpoldrack))
# 0.17.6 (2022-09-21)
## Bug Fixes
- UX: push - provide specific error with details if push failed due to
permission issue. [PR #7011](https://github.com/datalad/datalad/pull/7011)
(by [@yarikoptic](https://github.com/yarikoptic))
- Fix datalad --help to not have *Global options* empty with python 3.10 and
list options in "options:" section.
[PR #7028](https://github.com/datalad/datalad/pull/7028)
(by [@yarikoptic](https://github.com/yarikoptic))
- Let `create` touch the dataset root, if not saving in parent dataset.
[PR #7036](https://github.com/datalad/datalad/pull/7036) (by
[@mih](https://github.com/mih))
- Let `get_status_dict()` use exception message if none is passed.
[PR #7037](https://github.com/datalad/datalad/pull/7037) (by
[@mih](https://github.com/mih))
- Make choices for `status|diff --annex` and `status|diff --untracked` visible.
[PR #7039](https://github.com/datalad/datalad/pull/7039) (by
[@mih](https://github.com/mih))
- push: Assume 0 bytes pushed if git-annex does not provide bytesize.
[PR #7049](https://github.com/datalad/datalad/pull/7049) (by
[@yarikoptic](https://github.com/yarikoptic))
## Internal
- Use scriv for CHANGELOG generation in release workflow.
[PR #7009](https://github.com/datalad/datalad/pull/7009) (by
[@jwodder](https://github.com/jwodder))
- Stop using auto.
[PR #7024](https://github.com/datalad/datalad/pull/7024)
(by [@jwodder](https://github.com/jwodder))
## Tests
- Allow for any 2 from first 3 to be consumed in test_gracefull_death.
[PR #7041](https://github.com/datalad/datalad/pull/7041) (by
[@yarikoptic](https://github.com/yarikoptic))
---
# 0.17.5 (Fri Sep 02 2022)
#### π Bug Fix
- BF: blacklist 23.9.0 of keyring as introduces regression [#7003](https://github.com/datalad/datalad/pull/7003) ([@yarikoptic](https://github.com/yarikoptic))
- Make the manpages build reproducible via datalad.source.epoch (to be used in Debian packaging) [#6997](https://github.com/datalad/datalad/pull/6997) ([@lamby](https://github.com/lamby) bot@datalad.org [@yarikoptic](https://github.com/yarikoptic))
- BF: backquote path/drive in Changelog [#6997](https://github.com/datalad/datalad/pull/6997) ([@yarikoptic](https://github.com/yarikoptic))
#### Authors: 3
- Chris Lamb ([@lamby](https://github.com/lamby))
- DataLad Bot (bot@datalad.org)
- Yaroslav Halchenko ([@yarikoptic](https://github.com/yarikoptic))
---
# 0.17.4 (Tue Aug 30 2022)
#### π Bug Fix
- BF: make logic more consistent for files=[] argument (which is False but not None) [#6976](https://github.com/datalad/datalad/pull/6976) ([@yarikoptic](https://github.com/yarikoptic))
- Run pytests in parallel (-n 2) on appveyor [#6987](https://github.com/datalad/datalad/pull/6987) ([@yarikoptic](https://github.com/yarikoptic))
- Add workflow for autogenerating changelog snippets [#6981](https://github.com/datalad/datalad/pull/6981) ([@jwodder](https://github.com/jwodder))
- Provide `/dev/null` (`b:\nul` on πΎ Windows) instead of empty string as a git-repo to avoid reading local repo configuration [#6986](https://github.com/datalad/datalad/pull/6986) ([@yarikoptic](https://github.com/yarikoptic))
- RF: call_from_parser - move code into "else" to simplify reading etc [#6982](https://github.com/datalad/datalad/pull/6982) ([@yarikoptic](https://github.com/yarikoptic))
- BF: if early attempt to parse resulted in error, setup subparsers [#6980](https://github.com/datalad/datalad/pull/6980) ([@yarikoptic](https://github.com/yarikoptic))
- Run pytests in parallel (-n 2) on Travis [#6915](https://github.com/datalad/datalad/pull/6915) ([@yarikoptic](https://github.com/yarikoptic))
- Send one character (no newline) to stdout in protocol test to guarantee a single "message" and thus a single custom value [#6978](https://github.com/datalad/datalad/pull/6978) ([@christian-monch](https://github.com/christian-monch))
#### π§ͺ Tests
- TST: test_stalling -- wait x10 not just x5 time [#6995](https://github.com/datalad/datalad/pull/6995) ([@yarikoptic](https://github.com/yarikoptic))
#### Authors: 3
- Christian MΓΆnch ([@christian-monch](https://github.com/christian-monch))
- John T. Wodder II ([@jwodder](https://github.com/jwodder))
- Yaroslav Halchenko ([@yarikoptic](https://github.com/yarikoptic))
---
# 0.17.3 (Tue Aug 23 2022)
#### π Bug Fix
- BF: git_ignore_check do not overload possible value of stdout/err if present [#6937](https://github.com/datalad/datalad/pull/6937) ([@yarikoptic](https://github.com/yarikoptic))
- DOCfix: fix docstring GeneratorStdOutErrCapture to say that treats both stdout and stderr identically [#6930](https://github.com/datalad/datalad/pull/6930) ([@yarikoptic](https://github.com/yarikoptic))
- Explain purpose of create-sibling-ria's --post-update-hook [#6958](https://github.com/datalad/datalad/pull/6958) ([@mih](https://github.com/mih))
- ENH+BF: get_parent_paths - make / into sep option and consistently use "/" as path separator [#6963](https://github.com/datalad/datalad/pull/6963) ([@yarikoptic](https://github.com/yarikoptic))
- BF(TEMP): use git-annex from neurodebian -devel to gain fix for bug detected with datalad-crawler [#6965](https://github.com/datalad/datalad/pull/6965) ([@yarikoptic](https://github.com/yarikoptic))
- BF(TST): make tests use _path_ helper for Windows "friendliness" of the tests [#6955](https://github.com/datalad/datalad/pull/6955) ([@yarikoptic](https://github.com/yarikoptic))
- BF(TST): prevent auto-upgrade of "remote" test sibling, do not use local path for URL [#6957](https://github.com/datalad/datalad/pull/6957) ([@yarikoptic](https://github.com/yarikoptic))
- Forbid drop operation from symlink'ed annex (e.g. due to being cloned with --reckless=ephemeral) to prevent data-loss [#6959](https://github.com/datalad/datalad/pull/6959) ([@mih](https://github.com/mih))
- Acknowledge git-config comment chars [#6944](https://github.com/datalad/datalad/pull/6944) ([@mih](https://github.com/mih) [@yarikoptic](https://github.com/yarikoptic))
- Minor tuneups to please updated codespell [#6956](https://github.com/datalad/datalad/pull/6956) ([@yarikoptic](https://github.com/yarikoptic))
- TST: Add a testcase for #6950 [#6957](https://github.com/datalad/datalad/pull/6957) ([@adswa](https://github.com/adswa))
- BF+ENH(TST): fix typo in code of wtf filesystems reports [#6920](https://github.com/datalad/datalad/pull/6920) ([@yarikoptic](https://github.com/yarikoptic))
- DOC: Datalad -> DataLad [#6937](https://github.com/datalad/datalad/pull/6937) ([@aqw](https://github.com/aqw))
- BF: fix typo which prevented silently to not show details of filesystems [#6930](https://github.com/datalad/datalad/pull/6930) ([@yarikoptic](https://github.com/yarikoptic))
- BF(TST): allow for a annex repo version to upgrade if running in adjusted branches [#6927](https://github.com/datalad/datalad/pull/6927) ([@yarikoptic](https://github.com/yarikoptic))
- RF extensions github action to centralize configuration for extensions etc, use pytest for crawler [#6914](https://github.com/datalad/datalad/pull/6914) ([@yarikoptic](https://github.com/yarikoptic))
- BF: travis - mark our directory as safe to interact with as root [#6919](https://github.com/datalad/datalad/pull/6919) ([@yarikoptic](https://github.com/yarikoptic))
- BF: do not pretend we know what repo version git-annex would upgrade to [#6902](https://github.com/datalad/datalad/pull/6902) ([@yarikoptic](https://github.com/yarikoptic))
- BF(TST): do not expect log message for guessing Path to be possibly a URL on windows [#6911](https://github.com/datalad/datalad/pull/6911) ([@yarikoptic](https://github.com/yarikoptic))
- ENH(TST): Disable coverage reporting on travis while running pytest [#6898](https://github.com/datalad/datalad/pull/6898) ([@yarikoptic](https://github.com/yarikoptic))
- RF: just rename internal variable from unclear "op" to "io" [#6907](https://github.com/datalad/datalad/pull/6907) ([@yarikoptic](https://github.com/yarikoptic))
- DX: Demote loglevel of message on url parameters to DEBUG while guessing RI [#6891](https://github.com/datalad/datalad/pull/6891) ([@adswa](https://github.com/adswa) [@yarikoptic](https://github.com/yarikoptic))
- Fix and expand datalad.runner type annotations [#6893](https://github.com/datalad/datalad/pull/6893) ([@christian-monch](https://github.com/christian-monch) [@yarikoptic](https://github.com/yarikoptic))
- Use pytest to test datalad-metalad in test_extensions-workflow [#6892](https://github.com/datalad/datalad/pull/6892) ([@christian-monch](https://github.com/christian-monch))
- Let push honor multiple publication dependencies declared via siblings [#6869](https://github.com/datalad/datalad/pull/6869) ([@mih](https://github.com/mih) [@yarikoptic](https://github.com/yarikoptic))
- ENH: upgrade versioneer from versioneer-0.20.dev0 to versioneer-0.23.dev0 [#6888](https://github.com/datalad/datalad/pull/6888) ([@yarikoptic](https://github.com/yarikoptic))
- ENH: introduce typing checking and GitHub workflow [#6885](https://github.com/datalad/datalad/pull/6885) ([@yarikoptic](https://github.com/yarikoptic))
- RF,ENH(TST): future proof testing of git annex version upgrade + test annex init on all supported versions [#6880](https://github.com/datalad/datalad/pull/6880) ([@yarikoptic](https://github.com/yarikoptic))
- ENH(TST): test against supported git annex repo version 10 + make it a full sweep over tests [#6881](https://github.com/datalad/datalad/pull/6881) ([@yarikoptic](https://github.com/yarikoptic))
- BF: RF f-string uses in logger to %-interpolations [#6886](https://github.com/datalad/datalad/pull/6886) ([@yarikoptic](https://github.com/yarikoptic))
- Merge branch 'bf-sphinx-5.1.0' into maint [#6883](https://github.com/datalad/datalad/pull/6883) ([@yarikoptic](https://github.com/yarikoptic))
- BF(DOC): workaround for #10701 of sphinx in 5.1.0 [#6883](https://github.com/datalad/datalad/pull/6883) ([@yarikoptic](https://github.com/yarikoptic))
- Clarify confusing INFO log message from get() on dataset installation [#6871](https://github.com/datalad/datalad/pull/6871) ([@mih](https://github.com/mih))
- Protect again failing to load a command interface from an extension [#6879](https://github.com/datalad/datalad/pull/6879) ([@mih](https://github.com/mih))
- Support unsetting config via `datalad -c :` [#6864](https://github.com/datalad/datalad/pull/6864) ([@mih](https://github.com/mih))
- Fix DOC string typo in the path within AnnexRepo.annexstatus, and replace with proper sphinx reference [#6858](https://github.com/datalad/datalad/pull/6858) ([@christian-monch](https://github.com/christian-monch))
- Improved support for saving typechanges [#6793](https://github.com/datalad/datalad/pull/6793) ([@mih](https://github.com/mih))
#### β οΈ Pushed to `maint`
- BF: Remove duplicate ds key from result record ([@adswa](https://github.com/adswa))
- DOC: fix capitalization of service names ([@aqw](https://github.com/aqw))
#### π§ͺ Tests
- BF(TST,workaround): just xfail failing archives test on NFS [#6912](https://github.com/datalad/datalad/pull/6912) ([@yarikoptic](https://github.com/yarikoptic))
#### Authors: 5
- Adina Wagner ([@adswa](https://github.com/adswa))
- Alex Waite ([@aqw](https://github.com/aqw))
- Christian MΓΆnch ([@christian-monch](https://github.com/christian-monch))
- Michael Hanke ([@mih](https://github.com/mih))
- Yaroslav Halchenko ([@yarikoptic](https://github.com/yarikoptic))
---
# 0.17.2 (Sat Jul 16 2022)
#### π Bug Fix
- BF(TST): do proceed to proper test for error being caught for recent git-annex on windows with symlinks [#6850](https://github.com/datalad/datalad/pull/6850) ([@yarikoptic](https://github.com/yarikoptic))
- Addressing problem testing against python 3.10 on Travis (skip more annex versions) [#6842](https://github.com/datalad/datalad/pull/6842) ([@yarikoptic](https://github.com/yarikoptic))
- XFAIL test_runner_parametrized_protocol on python3.8 when getting duplicate output [#6837](https://github.com/datalad/datalad/pull/6837) ([@yarikoptic](https://github.com/yarikoptic))
- BF: Make create's check for procedures work with several again [#6841](https://github.com/datalad/datalad/pull/6841) ([@adswa](https://github.com/adswa))
- Support older pytests [#6836](https://github.com/datalad/datalad/pull/6836) ([@jwodder](https://github.com/jwodder))
#### Authors: 3
- Adina Wagner ([@adswa](https://github.com/adswa))
- John T. Wodder II ([@jwodder](https://github.com/jwodder))
- Yaroslav Halchenko ([@yarikoptic](https://github.com/yarikoptic))
---
# 0.17.1 (Mon Jul 11 2022)
#### π Bug Fix
- DOC: minor fix - consistent DataLad (not Datalad) in docs and CHANGELOG [#6830](https://github.com/datalad/datalad/pull/6830) ([@yarikoptic](https://github.com/yarikoptic))
- DOC: fixup/harmonize Changelog for 0.17.0 a little [#6828](https://github.com/datalad/datalad/pull/6828) ([@yarikoptic](https://github.com/yarikoptic))
- BF: use --python-match minor option in new datalad-installer release to match outside version of Python [#6827](https://github.com/datalad/datalad/pull/6827) ([@christian-monch](https://github.com/christian-monch) [@yarikoptic](https://github.com/yarikoptic))
- Do not quote paths for ssh >= 9 [#6826](https://github.com/datalad/datalad/pull/6826) ([@christian-monch](https://github.com/christian-monch) [@yarikoptic](https://github.com/yarikoptic))
- Suppress DeprecationWarning to allow for distutils to be used [#6819](https://github.com/datalad/datalad/pull/6819) ([@yarikoptic](https://github.com/yarikoptic))
- RM(TST): remove testing of datalad.test which was removed from 0.17.0 [#6822](https://github.com/datalad/datalad/pull/6822) ([@yarikoptic](https://github.com/yarikoptic))
- Avoid import of nose-based tests.utils, make skip_if_no_module() and skip_if_no_network() allowed at module level [#6817](https://github.com/datalad/datalad/pull/6817) ([@jwodder](https://github.com/jwodder))
- BF(TST): use higher level asyncio.run instead of asyncio.get_event_loop in test_inside_async [#6808](https://github.com/datalad/datalad/pull/6808) ([@yarikoptic](https://github.com/yarikoptic))
#### Authors: 3
- Christian MΓΆnch ([@christian-monch](https://github.com/christian-monch))
- John T. Wodder II ([@jwodder](https://github.com/jwodder))
- Yaroslav Halchenko ([@yarikoptic](https://github.com/yarikoptic))
---
# 0.17.0 (Thu Jul 7 2022) -- pytest migration
#### π« Enhancements and new features
- "log" progress bar now reports about starting a specific action as well. [#6756](https://github.com/datalad/datalad/pull/6756) (by @yarikoptic)
- Documentation and behavior of traceback reporting for log messages via `DATALAD_LOG_TRACEBACK` was improved to yield a more compact report. The documentation for this feature has been clarified. [#6746](https://github.com/datalad/datalad/pull/6746) (by @mih)
- `datalad unlock` gained a progress bar. [#6704](https://github.com/datalad/datalad/pull/6704) (by @adswa)
- When `create-sibling-gitlab` is called on non-existing subdatasets or paths it now returns an impossible result instead of no feedback at all. [#6701](https://github.com/datalad/datalad/pull/6701) (by @adswa)
- `datalad wtf` includes a report on file system types of commonly used paths. [#6664](https://github.com/datalad/datalad/pull/6664) (by @adswa)
- Use next generation metadata code in search, if it is available. [#6518](https://github.com/datalad/datalad/pull/6518) (by @christian-monch)
#### πͺ Deprecations and removals
- Remove unused and untested log helpers `NoProgressLog` and `OnlyProgressLog`. [#6747](https://github.com/datalad/datalad/pull/6747) (by @mih)
- Remove unused `sorted_files()` helper. [#6722](https://github.com/datalad/datalad/pull/6722) (by @adswa)
- Discontinued the value `stdout` for use with the config variable `datalad.log.target` as its use would inevitably break special remote implementations. [#6675](https://github.com/datalad/datalad/pull/6675) (by @bpoldrack)
- `AnnexRepo.add_urls()` is deprecated in favor of `AnnexRepo.add_url_to_file()` or a direct call to `AnnexRepo.call_annex()`. [#6667](https://github.com/datalad/datalad/pull/6667) (by @mih)
- `datalad test` command and supporting functionality (e.g., `datalad.test`) were removed. [#6273](https://github.com/datalad/datalad/pull/6273) (by @jwodder)
#### π Bug Fixes
- `export-archive` does not rely on `normalize_path()` methods anymore and became more robust when called from subdirectories. [#6745](https://github.com/datalad/datalad/pull/6745) (by @adswa)
- Sanitize keys before checking content availability to ensure that the content availability of files with URL- or custom backend keys is correctly determined and marked. [#6663](https://github.com/datalad/datalad/pull/6663) (by @adswa)
- Ensure saving a new subdataset to a superdataset yields a valid `.gitmodules` record regardless of whether and how a path constraint is given to the `save()` call. Fixes #6547 [#6790](https://github.com/datalad/datalad/pull/6790) (by @mih)
- `save` now repairs annex symlinks broken by a `git-mv` operation prior recording a new dataset state. Fixes #4967 [#6795](https://github.com/datalad/datalad/pull/6795) (by @mih)
#### π Documentation
- API documentation for log helpers, like `log_progress()` is now included in the renderer documentation. [#6746](https://github.com/datalad/datalad/pull/6746) (by @mih)
- New design document on progress reporting. [#6734](https://github.com/datalad/datalad/pull/6734) (by @mih)
- Explain downstream consequences of using `--fast` option in `addurls`. [#6684](https://github.com/datalad/datalad/pull/6684) (by @jdkent)
#### π Internal
- Inline code of `create-sibling-ria` has been refactored to an internal helper to check for siblings with particular names across dataset hierarchies in `datalad-next`, and is reintroduced into core to modularize the code base further. [#6706](https://github.com/datalad/datalad/pull/6706) (by @adswa)
- `get_initialized_logger` now lets a given `logtarget` take precedence over `datalad.log.target`. [#6675](https://github.com/datalad/datalad/pull/6675) (by @bpoldrack)
- Many uses of deprecated call options were replaced with the recommended ones. [#6273](https://github.com/datalad/datalad/pull/6273) (by @jwodder)
- Get rid of `asyncio` import by defining few noops methods from `asyncio.protocols.SubprocessProtocol` directly in `WitlessProtocol`. [#6648](https://github.com/datalad/datalad/pull/6648) (by @yarikoptic)
- Consolidate `GitRepo.remove()` and `AnnexRepo.remove()` into a single implementation. [#6783](https://github.com/datalad/datalad/pull/6783) (by @mih)
#### π‘ Tests
- Discontinue use of `with_testrepos` decorator other than for the deprecation cycle for `nose`. [#6690](https://github.com/datalad/datalad/pull/6690) (by @mih @bpoldrack) See [#6144](https://github.com/datalad/datalad/issues/6144) for full list of changes.
- Remove usage of deprecated `AnnexRepo.add_urls` in tests. [#6683](https://github.com/datalad/datalad/pull/6683) (by @bpoldrack)
- Minimalistic (adapters, no assert changes, etc) migration from `nose` to `pytest`.
Support functionality possibly used by extensions and relying on `nose` helpers is left in place to avoid affecting their run time and defer migration of their test setups.. [#6273](https://github.com/datalad/datalad/pull/6273) (by @jwodder)
#### Authors: 7
- Yaroslav Halchenko (@yarikoptic)
- Michael Hanke (@mih)
- Benjamin Poldrack (@bpoldrack)
- Adina Wagner (@adswa)
- John T. Wodder (@jwodder)
- Christian MΓΆnch (@christian-monch)
- James Kent (@jdkent)
# 0.16.7 (Wed Jul 06 2022)
#### π Bug Fix
- Fix broken annex symlink after git-mv before saving + fix a race condition in ssh copy test [#6809](https://github.com/datalad/datalad/pull/6809) ([@christian-monch](https://github.com/christian-monch) [@mih](https://github.com/mih) [@yarikoptic](https://github.com/yarikoptic))
- Do not ignore already known status info on submodules [#6790](https://github.com/datalad/datalad/pull/6790) ([@mih](https://github.com/mih))
- Fix "common data source" test to use a valid URL (maint-based & extended edition) [#6788](https://github.com/datalad/datalad/pull/6788) ([@mih](https://github.com/mih) [@yarikoptic](https://github.com/yarikoptic))
- Upload coverage from extension tests to Codecov [#6781](https://github.com/datalad/datalad/pull/6781) ([@jwodder](https://github.com/jwodder))
- Clean up line end handling in GitRepo [#6768](https://github.com/datalad/datalad/pull/6768) ([@christian-monch](https://github.com/christian-monch))
- Do not skip file-URL tests on windows [#6772](https://github.com/datalad/datalad/pull/6772) ([@christian-monch](https://github.com/christian-monch))
- Fix test errors caused by updated chardet v5 release [#6777](https://github.com/datalad/datalad/pull/6777) ([@christian-monch](https://github.com/christian-monch))
- Preserve final trailing slash in ``call_git()`` output [#6754](https://github.com/datalad/datalad/pull/6754) ([@adswa](https://github.com/adswa) [@yarikoptic](https://github.com/yarikoptic) [@christian-monch](https://github.com/christian-monch))
#### β οΈ Pushed to `maint`
- Make sure a subdataset is saved with a complete .gitmodules record ([@mih](https://github.com/mih))
#### Authors: 5
- Adina Wagner ([@adswa](https://github.com/adswa))
- Christian MΓΆnch ([@christian-monch](https://github.com/christian-monch))
- John T. Wodder II ([@jwodder](https://github.com/jwodder))
- Michael Hanke ([@mih](https://github.com/mih))
- Yaroslav Halchenko ([@yarikoptic](https://github.com/yarikoptic))
---
# 0.16.6 (Tue Jun 14 2022)
#### π Bug Fix
- Prevent duplicated result rendering when searching in default datasets [#6765](https://github.com/datalad/datalad/pull/6765) ([@christian-monch](https://github.com/christian-monch))
- BF(workaround): skip test_ria_postclonecfg on OSX for now ([@yarikoptic](https://github.com/yarikoptic))
- BF(workaround to #6759): if saving credential failed, just log error and continue [#6762](https://github.com/datalad/datalad/pull/6762) ([@yarikoptic](https://github.com/yarikoptic))
- Prevent reentry of a runner instance [#6737](https://github.com/datalad/datalad/pull/6737) ([@christian-monch](https://github.com/christian-monch))
#### Authors: 2
- Christian MΓΆnch ([@christian-monch](https://github.com/christian-monch))
- Yaroslav Halchenko ([@yarikoptic](https://github.com/yarikoptic))
---
# 0.16.5 (Wed Jun 08 2022)
#### π Bug Fix
- BF: push to github - remove datalad-push-default-first config only in non-dry run to ensure we push default branch separately in next step [#6750](https://github.com/datalad/datalad/pull/6750) ([@yarikoptic](https://github.com/yarikoptic))
- In addition to default (system) ssh version, report configured ssh; fix ssh version parsing on Windows [#6729](https://github.com/datalad/datalad/pull/6729) ([@yarikoptic](https://github.com/yarikoptic))
#### Authors: 1
- Yaroslav Halchenko ([@yarikoptic](https://github.com/yarikoptic))
---
# 0.16.4 (Thu Jun 02 2022)
#### π Bug Fix
- BF(TST): RO operations - add test directory into git safe.directory [#6726](https://github.com/datalad/datalad/pull/6726) ([@yarikoptic](https://github.com/yarikoptic))
- DOC: fixup of docstring for skip_ssh [#6727](https://github.com/datalad/datalad/pull/6727) ([@yarikoptic](https://github.com/yarikoptic))
- DOC: Set language in Sphinx config to en [#6727](https://github.com/datalad/datalad/pull/6727) ([@adswa](https://github.com/adswa))
- BF: Catch KeyErrors from unavailable WTF infos [#6712](https://github.com/datalad/datalad/pull/6712) ([@adswa](https://github.com/adswa))
- Add annex.private to ephemeral clones. That would make git-annex not assign shared (in git-annex branch) annex uuid. [#6702](https://github.com/datalad/datalad/pull/6702) ([@bpoldrack](https://github.com/bpoldrack) [@adswa](https://github.com/adswa))
- BF: require argcomplete version at least 1.12.3 to test/operate correctly [#6693](https://github.com/datalad/datalad/pull/6693) ([@yarikoptic](https://github.com/yarikoptic))
- Replace Zenodo DOI with JOSS for due credit [#6725](https://github.com/datalad/datalad/pull/6725) ([@adswa](https://github.com/adswa))
#### Authors: 3
- Adina Wagner ([@adswa](https://github.com/adswa))
- Benjamin Poldrack ([@bpoldrack](https://github.com/bpoldrack))
- Yaroslav Halchenko ([@yarikoptic](https://github.com/yarikoptic))
---
# 0.16.3 (Thu May 12 2022)
#### π Bug Fix
- No change for a PR to trigger release [#6692](https://github.com/datalad/datalad/pull/6692) ([@yarikoptic](https://github.com/yarikoptic))
- Sanitize keys before checking content availability to ensure correct value for keys with URL or custom backend [#6665](https://github.com/datalad/datalad/pull/6665) ([@adswa](https://github.com/adswa) [@yarikoptic](https://github.com/yarikoptic))
- Change a key-value pair in drop result record [#6625](https://github.com/datalad/datalad/pull/6625) ([@mslw](https://github.com/mslw))
- Link docs of datalad-next [#6677](https://github.com/datalad/datalad/pull/6677) ([@mih](https://github.com/mih))
- Fix `GitRepo.get_branch_commits_()` to handle branch names conflicts with paths [#6661](https://github.com/datalad/datalad/pull/6661) ([@mih](https://github.com/mih))
- OPT: AnnexJsonProtocol - avoid dragging possibly long data around [#6660](https://github.com/datalad/datalad/pull/6660) ([@yarikoptic](https://github.com/yarikoptic))
- Remove two too prominent create() INFO log message that duplicate DEBUG log and harmonize some other log messages [#6638](https://github.com/datalad/datalad/pull/6638) ([@mih](https://github.com/mih) [@yarikoptic](https://github.com/yarikoptic))
- Remove unsupported parameter create_sibling_ria(existing=None) [#6637](https://github.com/datalad/datalad/pull/6637) ([@mih](https://github.com/mih))
- Add released plugin to .autorc to annotate PRs on when released [#6639](https://github.com/datalad/datalad/pull/6639) ([@yarikoptic](https://github.com/yarikoptic))
#### Authors: 4
- Adina Wagner ([@adswa](https://github.com/adswa))
- Michael Hanke ([@mih](https://github.com/mih))
- MichaΕ Szczepanik ([@mslw](https://github.com/mslw))
- Yaroslav Halchenko ([@yarikoptic](https://github.com/yarikoptic))
---
# 0.16.2 (Thu Apr 21 2022)
#### π Bug Fix
- Demote (to level 1 from DEBUG) and speed-up API doc logging (parseParameters) [#6635](https://github.com/datalad/datalad/pull/6635) ([@mih](https://github.com/mih))
- Factor out actual data transfer in push [#6618](https://github.com/datalad/datalad/pull/6618) ([@christian-monch](https://github.com/christian-monch))
- ENH: include version of datalad in tests teardown Versions: report [#6628](https://github.com/datalad/datalad/pull/6628) ([@yarikoptic](https://github.com/yarikoptic))
- MNT: Require importlib-metadata >=3.6 for Python < 3.10 for entry_points taking kwargs [#6631](https://github.com/datalad/datalad/pull/6631) ([@effigies](https://github.com/effigies))
- Factor out credential handling of create-sibling-ghlike [#6627](https://github.com/datalad/datalad/pull/6627) ([@mih](https://github.com/mih))
- BF: Fix wrong key name of annex' JSON records [#6624](https://github.com/datalad/datalad/pull/6624) ([@bpoldrack](https://github.com/bpoldrack))
#### β οΈ Pushed to `maint`
- Fix typo in changelog ([@mih](https://github.com/mih))
- [ci skip] minor typo fix ([@yarikoptic](https://github.com/yarikoptic))
#### Authors: 5
- Benjamin Poldrack ([@bpoldrack](https://github.com/bpoldrack))
- Chris Markiewicz ([@effigies](https://github.com/effigies))
- Christian MΓΆnch ([@christian-monch](https://github.com/christian-monch))
- Michael Hanke ([@mih](https://github.com/mih))
- Yaroslav Halchenko ([@yarikoptic](https://github.com/yarikoptic))
---
# 0.16.1 (Fr Apr 8 2022) -- April Fools' Release
- Fixes forgotten changelog in docs
# 0.16.0 (Fr Apr 8 2022) -- Spring cleaning!
#### π« Enhancements and new features
- A new set of ``create-sibling-*`` commands reimplements the GitHub-platform support of ``create-sibling-github`` and adds support to interface three new platforms in a unified fashion: GIN (``create-sibling-gin``), GOGS (``create-sibling-gogs``), and Gitea (``create-sibling-gitea``). All commands rely on personal access tokens only for authentication, allow for specifying one of several stored credentials via a uniform ``--credential`` parameter, and support a uniform ``--dry-run`` mode for testing without network. [#5949](https://github.com/datalad/datalad/pull/5949) (by @mih)
- ``create-sibling-github`` now has supports direct specification of organization repositories via a ``[/]repo``syntax [#5949](https://github.com/datalad/datalad/pull/5949) (by @mih)
- ``create-sibling-gitlab`` gained a ``--dry-run`` parameter to match the corresponding parameters in ``create-sibling-{github,gin,gogs,gitea}`` [#6013](https://github.com/datalad/datalad/pull/6013) (by @adswa)
- The ``--new-store-ok`` parameter of ``create-sibling-ria`` only creates new RIA stores when explicitly provided [#6045](https://github.com/datalad/datalad/pull/6045) (by @adswa)
- The default performance of ``status()`` and ``diff()`` commands is improved by up to 700% removing file-type evaluation as a default operation, and simplifying the type reporting rule [#6097](https://github.com/datalad/datalad/pull/6097) (by @mih)
- ``drop()`` and ``remove()`` were reimplemented in full, conceptualized as the antagonist commands to ``get()`` and ``clone()``. A new, harmonized set of parameters (``--what ['filecontent', 'allkeys', 'datasets', 'all']``, ``--reckless ['modification', 'availability', 'undead', 'kill']``) simplifies their API. Both commands include additional safeguards. ``uninstall`` is replaced with a thin shim command around ``drop()`` [#6111](https://github.com/datalad/datalad/pull/6111) (by @mih)
- ``add_archive_content()`` was refactored into a dataset method and gained progress bars [#6105](https://github.com/datalad/datalad/pull/6105) (by @adswa)
- The ``datalad`` and ``datalad-archives`` special remotes have been reimplemented based on ``AnnexRemote`` [#6165](https://github.com/datalad/datalad/pull/6165) (by @mih)
- The ``result_renderer()`` semantics were decomplexified and harmonized. The previous ``default`` result renderer was renamed to ``generic``. [#6174](https://github.com/datalad/datalad/pull/6174) (by @mih)
- ``get_status_dict`` learned to include exit codes in the case of CommandErrors [#5642](https://github.com/datalad/datalad/pull/5642) (by @yarikoptic)
- ``datalad clone`` can now pass options to ``git-clone``, adding support for cloning specific tags or branches, naming siblings other names than ``origin``, and exposing ``git clone``'s optimization arguments [#6218](https://github.com/datalad/datalad/pull/6218) (by @kyleam and @mih)
- Inactive BatchedCommands are cleaned up [#6206](https://github.com/datalad/datalad/pull/6206) (by @jwodder)
- ``export-archive-ora`` learned to filter files exported to 7z archives [#6234](https://github.com/datalad/datalad/pull/6234) (by @mih and @bpinsard)
- ``datalad run`` learned to glob recursively [#6262](https://github.com/datalad/datalad/pull/6262) (by @AKSoo)
- The ORA remote learned to recover from interrupted uploads [#6267](https://github.com/datalad/datalad/pull/6267) (by @mih)
- A new threaded runner with support for timeouts and generator-based subprocess communication is introduced and used in ``BatchedCommand`` and ``AnnexRepo`` [#6244](https://github.com/datalad/datalad/pull/6244) (by @christian-monch)
- A new switch allows to enable librarymode and queries for the effective API in use [#6213](https://github.com/datalad/datalad/pull/6213) (by @mih)
- ``run`` and ``rerun`` now support parallel jobs via ``--jobs`` [#6279](https://github.com/datalad/datalad/pull/6279) (by @AKSoo)
- A new ``foreach-dataset`` plumbing command allows to run commands on each (sub)dataset, similar to ``git submodule foreach``
[#5517](https://github.com/datalad/datalad/pull/5517) (by @yarikoptic)
- The ``dataset`` parameter is not restricted to only locally resolvable file-URLs anymore [#6276](https://github.com/datalad/datalad/pull/6276) (by @christian-monch)
- DataLad's credential system is now able to query `git-credential` by specifying credential type `git` in the respective provider configuration [#5796](https://github.com/datalad/datalad/pull/5796) (by @bpoldrack)
- DataLad now comes with a git credential helper `git-credential-datalad` allowing Git to query DataLad's credential system [#5796](https://github.com/datalad/datalad/pull/5796) (by @bpoldrack and @mih)
- The new runner now allows for multiple threads [#6371](https://github.com/datalad/datalad/pull/6371) (by @christian-monch)
- A new configurationcommand provides an interface to manipulate and query the DataLad configuration. [#6306](https://github.com/datalad/datalad/pull/6306) (by @mih)
- Unlike the global Python-only datalad.cfg or dataset-specific Dataset.config configuration managers, this command offers a uniform API across the Python and the command line interfaces.
- This command was previously available in the mihextras extension as x-configuration, and has been merged into the core package in an improved version. [#5489](https://github.com/datalad/datalad/pull/5489) (by @mih)
- In its default dump mode, the command provides an annotated list of the effective configuration after considering all configuration sources, including hints on additional configuration settings and their supported values.
- The command line interface help-reporting has been sped up by ~20% [#6370](https://github.com/datalad/datalad/pull/6370) [#6378](https://github.com/datalad/datalad/pull/6378) (by @mih)
- ``ConfigManager`` now supports reading committed dataset configuration in bare repositories. Analog to reading ``.datalad/config`` from a worktree, ``blob:HEAD:.datalad/config`` is read (e.g., the config committed in the default branch). The support includes ``reload()` change detection using the gitsha of this file. The behavior for non-bare repositories is unchanged. [#6332](https://github.com/datalad/datalad/pull/6332) (by @mih)
- The CLI help generation has been sped up, and now also supports the completion of parameter values for a fixed set of choices [#6415](https://github.com/datalad/datalad/pull/6415) (by @mih)
- Individual command implementations can now declare a specific "on-failure" behavior by defining `Interface.on_failure` to be one of the supported modes (stop, continue, ignore). Previously, such a modification was only possible on a per-call basis. [#6430](https://github.com/datalad/datalad/pull/6430) (by @mih)
- The `run` command changed its default "on-failure" behavior from `continue` to `stop`. This change prevents the execution of a command in case a declared input can not be obtained. Previously, only an error result was yielded (and run eventually yielded a non-zero exit code or an `IncompleteResultsException`), but the execution proceeded and potentially saved a dataset modification despite incomplete inputs, in case the command succeeded. This previous default behavior can still be achieved by calling run with the equivalent of `--on-failure continue` [#6430](https://github.com/datalad/datalad/pull/6430) (by @mih)
- The ``run` command now provides readily executable, API-specific instructions how to save the results of a command execution that failed expectedly [#6434](https://github.com/datalad/datalad/pull/6434) (by @mih)
- `create-sibling --since=^` mode will now be as fast as `push --since=^` to figure out for which subdatasets to create siblings [#6436](https://github.com/datalad/datalad/pull/6436) (by @yarikoptic)
- When file names contain illegal characters or reserved file names that are incompatible with Windows systems a configurable check for ``save`` (``datalad.save.windows-compat-warning``) will either do nothing (`none`), emit an incompatibility warning (`warning`, default), or cause ``save`` to error (`error`) [#6291](https://github.com/datalad/datalad/pull/6291) (by @adswa)
- Improve responsiveness of `datalad drop` in datasets with a large annex. [#6580](https://github.com/datalad/datalad/pull/6580) (by @christian-monch)
- `save` code might operate faster on heavy file trees [#6581](https://github.com/datalad/datalad/pull/6581) (by @yarikoptic)
- Removed a per-file overhead cost for ORA when downloading over HTTP [#6609](https://github.com/datalad/datalad/pull/6609) (by @bpoldrack)
- A new module `datalad.support.extensions` offers the utility functions `register_config()` and `has_config()` that allow extension developers to announce additional configuration items to the central configuration management. [#6601](https://github.com/datalad/datalad/pull/6601) (by @mih)
- When operating in a dirty dataset, `export-to-figshare` now yields and impossible result instead of raising a RunTimeError [#6543](https://github.com/datalad/datalad/pull/6543) (by @adswa)
- Loading DataLad extension packages has been sped-up leading to between 2x and 4x faster run times for loading individual extensions and reporting help output across all installed extensions. [#6591](https://github.com/datalad/datalad/pull/6591) (by @mih)
- Introduces the configuration key `datalad.ssh.executable`. This key allows specifying an ssh-client executable that should be used by datalad to establish ssh-connections. The default value is `ssh` unless on a Windows system where `$WINDIR\System32\OpenSSH\ssh.exe` exists. In this case, the value defaults to `$WINDIR\System32\OpenSSH\ssh.exe`. [#6553](https://github.com/datalad/datalad/pull/6553) (by @christian-monch)
- create-sibling should perform much faster in case of `--since` specification since would consider only submodules related to the changes since that point. [#6528](https://github.com/datalad/datalad/pull/6528) (by @yarikoptic)
- A new configuration setting `datalad.ssh.try-use-annex-bundled-git=yes|no` can be used to influence the default remote git-annex bundle sensing for SSH connections. This was previously done unconditionally for any call to `datalad sshrun` (which is also used for any SSH-related Git or git-annex functionality triggered by DataLad-internal processing) and could incur a substantial per-call runtime cost. The new default is to not perform this sensing, because for, e.g., use as GIT_SSH_COMMAND there is no expectation to have a remote git-annex installation, and even with an existing git-annex/Git bundle on the remote, it is not certain that the bundled Git version is to be preferred over any other Git installation in a user's PATH. [#6533](https://github.com/datalad/datalad/pull/6533) (by @mih)
- `run` now yields a result record immediately after executing a command. This allows callers to use the standard `--on-failure switch` to control whether dataset modifications will be saved for a command that exited with an error. [#6447](https://github.com/datalad/datalad/pull/6447) (by @mih)
#### πͺ Deprecations and removals
- The ``--pbs-runner`` commandline option (deprecated in ``0.15.0``) was removed [#5981](https://github.com/datalad/datalad/pull/5981) (by @mih)
- The dependency to PyGithub was dropped [#5949](https://github.com/datalad/datalad/pull/5949) (by @mih)
- ``create-sibling-github``'s credential handling was trimmed down to only allow personal access tokens, because GitHub discontinued user/password based authentication [#5949](https://github.com/datalad/datalad/pull/5949) (by @mih)
- ``create-sibling-gitlab``'s ``--dryrun`` parameter is deprecated in favor or ``--dry-run`` [#6013](https://github.com/datalad/datalad/pull/6013) (by @adswa)
- Internal obsolete ``Gitrepo.*_submodule`` methods were moved to ``datalad-deprecated`` [#6010](https://github.com/datalad/datalad/pull/6010) (by @mih)
- ``datalad/support/versions.py`` is unused in DataLad core and removed [#6115](https://github.com/datalad/datalad/pull/6115) (by @yarikoptic)
- Support for the undocumented ``datalad.api.result-renderer`` config setting has been dropped [#6174](https://github.com/datalad/datalad/pull/6174) (by @mih)
- Undocumented use of ``result_renderer=None`` is replaced with ``result_renderer='disabled'`` [#6174](https://github.com/datalad/datalad/pull/6174) (by @mih)
- ``remove``'s ``--recursive`` argument has been deprecated [#6257](https://github.com/datalad/datalad/pull/6257) (by @mih)
- The use of the internal helper ``get_repo_instance()`` is discontinued and deprecated [#6268](https://github.com/datalad/datalad/pull/6268) (by @mih)
- Support for Python 3.6 has been dropped ([#6286](https://github.com/datalad/datalad/pull/6286) (by @christian-monch) and [#6364](https://github.com/datalad/datalad/pull/6364) (by @yarikoptic))
- All but one Singularity recipe flavor have been removed due to their limited value with the end of life of Singularity Hub [#6303](https://github.com/datalad/datalad/pull/6303) (by @mih)
- All code in module datalad.cmdline was (re)moved, only datalad.cmdline.helpers.get_repo_instanceis kept for a deprecation period (by @mih)
- ``datalad.interface.common_opts.eval_default`` has been deprecated. All (command-specific) defaults for common interface parameters can be read from ``Interface`` class attributes ([#6391](https://github.com/datalad/datalad/pull/6391) (by @mih)
- Remove unused and untested ``datalad.interface.utils`` helpers `cls2cmdlinename` and `path_is_under` [#6392](https://github.com/datalad/datalad/pull/6392) (by @mih)
- An unused code path for result rendering was removed from the CLI ``main()`` [#6394](https://github.com/datalad/datalad/pull/6394) (by @mih)
- ``create-sibling`` will require now ``"^"`` instead of an empty string for since option [#6436](https://github.com/datalad/datalad/pull/6436) (by @yarikoptic)
- `run` no longer raises a `CommandError` exception for failed commands, but yields an `error` result that includes a superset of the information provided by the exception. This change impacts command line usage insofar as the exit code of the underlying command is no longer relayed as the exit code of the `run` command call -- although `run` continues to exit with a non-zero exit code in case of an error. For Python API users, the nature of the raised exception changes from `CommandError` to `IncompleteResultsError`, and the exception handling is now configurable using the standard `on_failure` command argument. The original `CommandError` exception remains available via the `exception` property of the newly introduced result record for the command execution, and this result record is available via `IncompleteResultsError.failed`, if such an exception is raised. [#6447](https://github.com/datalad/datalad/pull/6447) (by @mih)
- Custom cast helpers were removed from datalad core and migrated to a standalone repository https://github.com/datalad/screencaster [#6516](https://github.com/datalad/datalad/pull/6516) (by @adswa)
- The `bundled` parameter of `get_connection_hash()` is now ignored and will be removed with a future release. [#6532](https://github.com/datalad/datalad/pull/6532) (by @mih)
- `BaseDownloader.fetch()` is logging download attempts on DEBUG (previously INFO) level to avoid polluting output of higher-level commands. [#6564](https://github.com/datalad/datalad/pull/6564) (by @mih)
#### π Bug Fixes
- ``create-sibling-gitlab`` erroneously overwrote existing sibling configurations. A safeguard will now prevent overwriting and exit with an error result [#6015](https://github.com/datalad/datalad/pull/6015) (by @adswa)
- ``create-sibling-gogs`` now relays HTTP500 errors, such as "no space left on device" [#6019](https://github.com/datalad/datalad/pull/6019) (by @mih)
- ``annotate_paths()`` is removed from the last parts of code base that still contained it [#6128](https://github.com/datalad/datalad/pull/6128) (by @mih)
- ``add_archive_content()`` doesn't crash with ``--key`` and ``--use-current-dir`` anymore [#6105](https://github.com/datalad/datalad/pull/6105) (by @adswa)
- ``run-procedure`` now returns an error result when a non-existent procedure name is specified [#6143](https://github.com/datalad/datalad/pull/6143) (by @mslw)
- A fix for a silent failure of ``download-url --archive`` when extracting the archive [#6172](https://github.com/datalad/datalad/pull/6172) (by @adswa)
- Uninitialized AnnexRepos can now be dropped [#6183](https://github.com/datalad/datalad/pull/6183) (by @mih)
- Instead of raising an error, the formatters tests are skipped when the ``formatters`` module is not found [#6212](https://github.com/datalad/datalad/pull/6212) (by @adswa)
- ``create-sibling-gin`` does not disable git-annex availability on Gin remotes anymore [#6230](https://github.com/datalad/datalad/pull/6230) (by @mih)
- The ORA special remote messaging is fixed to not break the special remote protocol anymore and to better relay messages from exceptions to communicate underlying causes [#6242](https://github.com/datalad/datalad/pull/6242) (by @mih)
- A ``keyring.delete()`` call was fixed to not call an uninitialized private attribute anymore [#6253](https://github.com/datalad/datalad/pull/6253) (by @bpoldrack)
- An erroneous placement of result keyword arguments into a ``format()`` method instead of ``get_status_dict()`` of ``create-sibling-ria`` has been fixed [#6256](https://github.com/datalad/datalad/pull/6256) (by @adswa)
- ``status``, ``run-procedure``, and ``metadata`` are no longer swallowing result-related messages in renderers [#6280](https://github.com/datalad/datalad/pull/6280) (by @mih)
- ``uninstall`` now recommends the new ``--reckless`` parameter instead of the deprecated ``--nocheck`` parameter when reporting hints [#6277](https://github.com/datalad/datalad/pull/6277) (by @adswa)
- ``download-url`` learned to handle Pathobjects [#6317](https://github.com/datalad/datalad/pull/6317) (by @adswa)
- Restore default result rendering behavior broken by Key interface documentation [#6394](https://github.com/datalad/datalad/pull/6394) (by @mih)
- Fix a broken check for file presence in the ``ConfigManager`` that could have caused a crash in rare cases when a config file is removed during the process runtime [#6332](https://github.com/datalad/datalad/pull/6332) (by @mih)
`- ``ConfigManager.get_from_source()`` now accesses the correct information when using the documented ``source='local'``, avoiding a crash [#6332](https://github.com/datalad/datalad/pull/6332) (by @mih)
- ``run`` no longer let's the internal call to `save` render its results unconditionally, but the parameterization f run determines the effective rendering format. [#6421](https://github.com/datalad/datalad/pull/6421) (by @mih)
- Remove an unnecessary and misleading warning from the runner [#6425](https://github.com/datalad/datalad/pull/6425) (by @christian-monch)
- A number of commands stopped to double-report results [#6446](https://github.com/datalad/datalad/pull/6446) (by @adswa)
- `create-sibling-ria` no longer creates an `annex/objects` directory in-store, when called with `--no-storage-sibling`. [#6495](https://github.com/datalad/datalad/pull/6495) (by @bpoldrack )
- Improve error message when an invalid URL is given to `clone`. [#6500](https://github.com/datalad/datalad/pull/6500) (by @mih)
- DataLad declares a minimum version dependency to ``keyring >= 20.0`` to ensure that token-based authentication can be used. [#6515](https://github.com/datalad/datalad/pull/6515) (by @adswa)
- ORA special remote tries to obtain permissions when dropping a key from a RIA store rather than just failing. Thus having the same permissions in the store's object trees as one directly managed by git-annex would have, works just fine now. [#6493](https://github.com/datalad/datalad/pull/6493) (by @bpoldrack )
- `require_dataset()` now uniformly raises `NoDatasetFound` when no dataset was found. Implementations that catch the previously documented `InsufficientArgumentsError` or the actually raised `ValueError` will continue to work, because `NoDatasetFound` is derived from both types. [#6521](https://github.com/datalad/datalad/pull/6521) (by @mih)
- Keyboard-interactive authentication is now possibly with non-multiplexed SSH connections (i.e., when no connection sharing is possible, due to lack of socket support, for example on Windows). Previously, it was disabled forcefully by DataLad for no valid reason. [#6537](https://github.com/datalad/datalad/pull/6537) (by @mih)
- Remove duplicate exception type in reporting of top-level CLI exception handler. [#6563](https://github.com/datalad/datalad/pull/6563) (by @mih)
- Fixes DataLad's parsing of git-annex' reporting on unknown paths depending on its version and the value of the `annex.skipunknown` config. [#6550](https://github.com/datalad/datalad/pull/6550) (by @bpoldrack)
- Fix ORA special remote not properly reporting on HTTP failures. [#6535](https://github.com/datalad/datalad/pull/6535) (by @bpoldrack)
- ORA special remote didn't show per-file progress bars when downloading over HTTP [#6609](https://github.com/datalad/datalad/pull/6609) (by @bpoldrack)
- `save` now can commit the change where file becomes a directory with a staged for commit file. [#6581](https://github.com/datalad/datalad/pull/6581) (by @yarikoptic)
- `create-sibling` will no longer create siblings for not yet saved new subdatasets, and will now create sub-datasets nested in the subdatasets which did not yet have those siblings. [#6603](https://github.com/datalad/datalad/pull/6603) (by @yarikoptic)
#### π Documentation
- A new design document sheds light on result records [#6167](https://github.com/datalad/datalad/pull/6167) (by @mih)
- The ``disabled`` result renderer mode is documented [#6174](https://github.com/datalad/datalad/pull/6174) (by @mih)
- A new design document sheds light on the ``datalad`` and ``datalad-archives`` special remotes [#6181](https://github.com/datalad/datalad/pull/6181) (by @mih)
- A new design document sheds light on ``BatchedCommand`` and ``BatchedAnnex`` [#6203](https://github.com/datalad/datalad/pull/6203) (by @christian-monch)
- A new design document sheds light on standard parameters [#6214](https://github.com/datalad/datalad/pull/6214) (by @adswa)
- The DataLad project adopted the Contributor Covenant COC v2.1 [#6236](https://github.com/datalad/datalad/pull/6236) (by @adswa)
- Docstrings learned to include Sphinx' "version added" and "deprecated" directives [#6249](https://github.com/datalad/datalad/pull/6249) (by @mih)
- A design document sheds light on basic docstring handling and formatting [#6249](https://github.com/datalad/datalad/pull/6249) (by @mih)
- A new design document sheds light on position versus keyword parameter usage [#6261](https://github.com/datalad/datalad/pull/6261) (by @yarikoptic)
- ``create-sibling-gin``'s examples have been improved to suggest ``push`` as an additional step to ensure proper configuration [#6289](https://github.com/datalad/datalad/pull/6289) (by @mslw)
- A new [document](http://docs.datalad.org/credentials.html) describes the credential system from a user's perspective [#5796](https://github.com/datalad/datalad/pull/5796) (by @bpoldrack)
- Enhance the [design document](http://docs.datalad.org/design/credentials.html) on DataLad's credential system [#5796](https://github.com/datalad/datalad/pull/5796) (by @bpoldrack)
- The documentation of the configuration command now details all locations DataLad is reading configuration items from, and their respective rules of precedence [#6306](https://github.com/datalad/datalad/pull/6306) (by @mih)
- API docs for datalad.interface.base are now included in the documentation [#6378](https://github.com/datalad/datalad/pull/6378) (by @mih)
- A new design document is provided that describes the basics of the command line interface implementation [#6382](https://github.com/datalad/datalad/pull/6382) (by @mih)
- The ``datalad.interface.base.Interface` class, the basis of all DataLad command implementations, has been extensively documented to provide an overview of basic principles and customization possibilities [#6391](https://github.com/datalad/datalad/pull/6391) (by @mih)
- `--since=^` mode of operation of `create-sibling` is documented now [#6436](https://github.com/datalad/datalad/pull/6436) (by @yarikoptic)
#### π Internal
- The internal ``status()`` helper was equipped with docstrings and promotes "breadth-first" reporting with a new parameter ``reporting_order`` [#6006](https://github.com/datalad/datalad/pull/6006) (by @mih)
- ``AnnexRepo.get_file_annexinfo()`` is introduced for more convenient queries for single files and replaces a now deprecated ``AnnexRepo.get_file_key()`` to receive information with fewer calls to Git [#6104](https://github.com/datalad/datalad/pull/6104) (by @mih)
- A new ``get_paths_by_ds()`` helper exposes ``status``' path normalization and sorting [#6110](https://github.com/datalad/datalad/pull/6110) (by @mih)
- ``status`` is optimized with a cache for dataset roots [#6137](https://github.com/datalad/datalad/pull/6137) (by @yarikoptic)
- The internal ``get_func_args_doc()`` helper with Python 2 is removed from DataLad core [#6175](https://github.com/datalad/datalad/pull/6175) (by @yarikoptic)
- Further restructuring of the source tree to better reflect the internal dependency structure of the code: ``AddArchiveContent`` is moved from ``datalad/interface`` to ``datalad/local`` ([#6188](https://github.com/datalad/datalad/pull/6188) (by @mih)), ``Clean`` is moved from ``datalad/interface`` to ``datalad/local`` ([#6191](https://github.com/datalad/datalad/pull/6191) (by @mih)), ``Unlock`` is moved from ``datalad/interface`` to ``datalad/local`` ([#6192](https://github.com/datalad/datalad/pull/6192) (by @mih)), ``DownloadURL`` is moved from ``datalad/interface`` to ``datalad/local`` ([#6217](https://github.com/datalad/datalad/pull/6217) (by @mih)), ``Rerun`` is moved from ``datalad/interface`` to ``datalad/local`` ([#6220](https://github.com/datalad/datalad/pull/6220) (by @mih)), ``RunProcedure`` is moved from ``datalad/interface`` to ``datalad/local`` ([#6222](https://github.com/datalad/datalad/pull/6222) (by @mih)). The interface command list is restructured and resorted [#6223](https://github.com/datalad/datalad/pull/6223) (by @mih)
- ``wrapt`` is replaced with functools' ``wraps``
[#6190](https://github.com/datalad/datalad/pull/6190) (by @yariktopic)
- The unmaintained ``appdirs`` library has been replaced with ``platformdirs`` [#6198](https://github.com/datalad/datalad/pull/6198) (by @adswa)
- Modelines mismatching the code style in source files were fixed [#6263](https://github.com/datalad/datalad/pull/6263) (by @AKSoo)
- ``datalad/__init__.py`` has been cleaned up [#6271](https://github.com/datalad/datalad/pull/6271) (by @mih)
- ``GitRepo.call_git_items`` is implemented with a generator-based runner [#6278](https://github.com/datalad/datalad/pull/6278) (by @christian-monch)
- Separate positional from keyword arguments in the Python API to match CLI with ``*`` [#6176](https://github.com/datalad/datalad/pull/6176) (by @yarikoptic), [#6304](https://github.com/datalad/datalad/pull/6304) (by @christian-monch)
- ``GitRepo.bare`` does not require the ConfigManager anymore [#6323](https://github.com/datalad/datalad/pull/6323) (by @mih)
- ``_get_dot_git()`` was reimplemented to be more efficient and consistent, by testing for common scenarios first and introducing a consistently applied ``resolved`` flag for result path reporting [#6325](https://github.com/datalad/datalad/pull/6325) (by @mih)
- All data files under ``datalad`` are now included when installing DataLad [#6336](https://github.com/datalad/datalad/pull/6336) (by @jwodder)
- Add internal method for non-interactive provider/credential storing [#5796](https://github.com/datalad/datalad/pull/5796) (by @bpoldrack)
- Allow credential classes to have a context set, consisting of a URL they are to be used with and a dataset DataLad is operating on, allowing to consider "local" and "dataset" config locations [#5796](https://github.com/datalad/datalad/pull/5796) (by @bpoldrack)
- The Interface method ``get_refds_path()`` was deprecated [#6387](https://github.com/datalad/datalad/pull/6387) (by @adswa)
- ``datalad.interface.base.Interface`` is now an abstract class [#6391](https://github.com/datalad/datalad/pull/6391) (by @mih)
- Simplified the decision making for result rendering, and reduced code complexity [#6394](https://github.com/datalad/datalad/pull/6394) (by @mih)
- Reduce code duplication in ``datalad.support.json_py`` [#6398](https://github.com/datalad/datalad/pull/6398) (by @mih)
- Use public `ArgumentParser.parse_known_args` instead of protected `_parse_known_args` [#6414](https://github.com/datalad/datalad/pull/6414) (by @yarikoptic)
- `add-archive-content` does not rely on the deprecated `tempfile.mktemp` anymore, but uses the more secure `tempfile.mkdtemp` [#6428](https://github.com/datalad/datalad/pull/6428) (by @adswa)
- AnnexRepo's internal `annexstatus` is deprecated. In its place, a new test helper assists the few tests that rely on it [#6413](https://github.com/datalad/datalad/pull/6413) (by @adswa)
- ``config`` has been refactored from ``where[="dataset"]`` to ``scope[="branch"]`` [#5969](https://github.com/datalad/datalad/pull/5969) (by @yarikoptic)
- Common command arguments are now uniformly and exhaustively passed to result renderers and filters for decision making. Previously, the presence of a particular argument depended on the respective API and circumstances of a command call. [#6440](https://github.com/datalad/datalad/pull/6440) (by @mih)
- Entrypoint processing for extensions and metadata extractors has been consolidated on a uniform helper that is about twice as fast as the previous implementations. [#6591](https://github.com/datalad/datalad/pull/6591) (by @mih)
#### π‘ Tests
- A range of Windows tests pass and were enabled [#6136](https://github.com/datalad/datalad/pull/6136) (by @adswa)
- Invalid escape sequences in some tests were fixed [#6147](https://github.com/datalad/datalad/pull/6147) (by @mih)
- A cross-platform compatible HTTP-serving test environment is introduced [#6153](https://github.com/datalad/datalad/pull/6153) (by @mih)
- A new helper exposes ``serve_path_via_http`` to the command line to deploy an ad-hoc instance of the HTTP server used for internal testing, with SSL and auth, if desired. [#6169](https://github.com/datalad/datalad/pull/6169) (by @mih)
- Windows tests were redistributed across worker runs to harmonize runtime [#6200](https://github.com/datalad/datalad/pull/6200) (by @adswa)
- ``Batchedcommand`` gained a basic test [#6203](https://github.com/datalad/datalad/pull/6203) (by @christian-monch)
- The use of ``with_testrepo`` is discontinued in all core tests [#6224](https://github.com/datalad/datalad/pull/6224) (by @mih)
- The new ``git-annex.filter.annex.process`` configuration is enabled by default on Windows to speed up the test suite [#6245](https://github.com/datalad/datalad/pull/6245) (by @mih)
- If the available Git version supports it, the test suite now uses ``GIT_CONFIG_GLOBAL`` to configure a fake home directory instead of overwriting ``HOME`` on OSX ([#6251](https://github.com/datalad/datalad/pull/6251) (by @bpoldrack)) and ``HOME`` and ``USERPROFILE`` on Windows [#6260](https://github.com/datalad/datalad/pull/6260) (by @adswa)
- Windows test timeouts of runners were addressed [#6311](https://github.com/datalad/datalad/pull/6311) (by @christian-monch)
- A handful of Windows tests were fixed ([#6352](https://github.com/datalad/datalad/pull/6352) (by @yarikoptic)) or disabled ([#6353](https://github.com/datalad/datalad/pull/6353) (by @yarikoptic))
- ``download-url``'s test under ``http_proxy`` are skipped when a session can't be established [#6361](https://github.com/datalad/datalad/pull/6361) (by @yarikoptic)
- A test for ``datalad clean`` was fixed to be invoked within a dataset [#6359](https://github.com/datalad/datalad/pull/6359) (by @yarikoptic)
- The new datalad.cli.tests have an improved module coverage of 80% [#6378](https://github.com/datalad/datalad/pull/6378) (by @mih)
- The ``test_source_candidate_subdataset`` has been marked as ``@slow`` [#6429](https://github.com/datalad/datalad/pull/6429) (by @yarikoptic)
- Dedicated ``CLI`` benchmarks exist now [#6381](https://github.com/datalad/datalad/pull/6381) (by @mih)
- Enable code coverage report for subprocesses [#6546](https://github.com/datalad/datalad/pull/6546) (by @adswa)
- Skip a test on annex>=10.20220127 due to a bug in annex. See https://git-annex.branchable.com/bugs/Change_to_annex.largefiles_leaves_repo_modified/
#### π§ Infra
- A new issue template using GitHub forms prestructures bug reports [#6048](https://github.com/datalad/datalad/pull/6048) (by @Remi-Gau)
- DataLad and its dependency stack were packaged for Gentoo Linux [#6088](https://github.com/datalad/datalad/pull/6088) (by @TheChymera)
- The readthedocs configuration is modernized to version 2 [#6207](https://github.com/datalad/datalad/pull/6207) (by @adswa)
- The Windows CI setup now runs on Appveyor's Visual Studio 2022 configuration [#6228](https://github.com/datalad/datalad/pull/6228) (by @adswa)
- The ``readthedocs-theme`` and ``Sphinx`` versions were pinned to re-enable rendering of bullet points in the documentation [#6346](https://github.com/datalad/datalad/pull/6346) (by @adswa)
- The PR template was updated with a CHANGELOG template. Future PRs should use it to include a summary for the CHANGELOG [#6396](https://github.com/datalad/datalad/pull/6396) (by @mih)
#### Authors: 11
- Michael Hanke (@mih)
- Yaroslav Halchenko (@yarikoptic)
- Adina Wagner (@adswa)
- Remi Gau (@Remi-Gau)
- Horea Christian (@TheChymera)
- MichaΕ Szczepanik (@mslw)
- Christian MΓΆnch (@christian-monch)
- John T. Wodder (@jwodder)
- Benjamin Poldrack (@bpoldrack)
- Sin Kim (@AKSoo)
- Basile Pinsard (@bpinsard)
---
# 0.15.6 (Sun Feb 27 2022)
#### π Bug Fix
- BF: do not use BaseDownloader instance wide InterProcessLock - resolves stalling or errors during parallel installs [#6507](https://github.com/datalad/datalad/pull/6507) ([@yarikoptic](https://github.com/yarikoptic))
- release workflow: add -vv to auto invocation ([@yarikoptic](https://github.com/yarikoptic))
- Fix version incorrectly incremented by release process in CHANGELOGs [#6459](https://github.com/datalad/datalad/pull/6459) ([@yarikoptic](https://github.com/yarikoptic))
- BF(TST): add another condition to skip under http_proxy set [#6459](https://github.com/datalad/datalad/pull/6459) ([@yarikoptic](https://github.com/yarikoptic))
#### Authors: 1
- Yaroslav Halchenko ([@yarikoptic](https://github.com/yarikoptic))
---
# 0.15.5 (Wed Feb 09 2022)
#### π Enhancement
- BF: When download-url gets Pathobject as path convert it to a string [#6364](https://github.com/datalad/datalad/pull/6364) ([@adswa](https://github.com/adswa))
#### π Bug Fix
- Fix AnnexRepo.whereis key=True mode operation, and add batch mode support [#6379](https://github.com/datalad/datalad/pull/6379) ([@yarikoptic](https://github.com/yarikoptic))
- DOC: run - adjust description for -i/-o to mention that it could be a directory [#6416](https://github.com/datalad/datalad/pull/6416) ([@yarikoptic](https://github.com/yarikoptic))
- BF: ORA over HTTP tried to check archive [#6355](https://github.com/datalad/datalad/pull/6355) ([@bpoldrack](https://github.com/bpoldrack) [@yarikoptic](https://github.com/yarikoptic))
- BF: condition access to isatty to have stream eval to True [#6360](https://github.com/datalad/datalad/pull/6360) ([@yarikoptic](https://github.com/yarikoptic))
- BF: python 3.10 compatibility fixes [#6363](https://github.com/datalad/datalad/pull/6363) ([@yarikoptic](https://github.com/yarikoptic))
- Remove two(!) copies of a test [#6374](https://github.com/datalad/datalad/pull/6374) ([@mih](https://github.com/mih))
- Warn just once about incomplete git config [#6343](https://github.com/datalad/datalad/pull/6343) ([@yarikoptic](https://github.com/yarikoptic))
- Make version detection robust to GIT_DIR specification [#6341](https://github.com/datalad/datalad/pull/6341) ([@effigies](https://github.com/effigies) [@mih](https://github.com/mih))
- BF(Q&D): do not crash - issue warning - if template fails to format [#6319](https://github.com/datalad/datalad/pull/6319) ([@yarikoptic](https://github.com/yarikoptic))
#### Authors: 5
- Adina Wagner ([@adswa](https://github.com/adswa))
- Benjamin Poldrack ([@bpoldrack](https://github.com/bpoldrack))
- Chris Markiewicz ([@effigies](https://github.com/effigies))
- Michael Hanke ([@mih](https://github.com/mih))
- Yaroslav Halchenko ([@yarikoptic](https://github.com/yarikoptic))
---
# 0.15.4 (Thu Dec 16 2021)
#### π Bug Fix
- BF: autorc - replace incorrect releaseTypes with "none" [#6320](https://github.com/datalad/datalad/pull/6320) ([@yarikoptic](https://github.com/yarikoptic))
- Minor enhancement to CONTRIBUTING.md [#6309](https://github.com/datalad/datalad/pull/6309) ([@bpoldrack](https://github.com/bpoldrack))
- UX: If a clean repo is dirty after a failed run, give clean-up hints [#6112](https://github.com/datalad/datalad/pull/6112) ([@adswa](https://github.com/adswa))
- Stop using distutils [#6113](https://github.com/datalad/datalad/pull/6113) ([@jwodder](https://github.com/jwodder))
- BF: RIARemote - set UI backend to annex to make it interactive [#6287](https://github.com/datalad/datalad/pull/6287) ([@yarikoptic](https://github.com/yarikoptic) [@bpoldrack](https://github.com/bpoldrack))
- Fix invalid escape sequences [#6293](https://github.com/datalad/datalad/pull/6293) ([@jwodder](https://github.com/jwodder))
- CI: Update environment for windows CI builds [#6292](https://github.com/datalad/datalad/pull/6292) ([@bpoldrack](https://github.com/bpoldrack))
- bump the python version used for mac os tests [#6288](https://github.com/datalad/datalad/pull/6288) ([@christian-monch](https://github.com/christian-monch) [@bpoldrack](https://github.com/bpoldrack))
- ENH(UX): log a hint to use ulimit command in case of "Too long" exception [#6173](https://github.com/datalad/datalad/pull/6173) ([@yarikoptic](https://github.com/yarikoptic))
- Report correct HTTP URL for RIA store content [#6091](https://github.com/datalad/datalad/pull/6091) ([@mih](https://github.com/mih))
- BF: Don't overwrite subdataset source candidates [#6168](https://github.com/datalad/datalad/pull/6168) ([@bpoldrack](https://github.com/bpoldrack))
- Bump sphinx requirement to bypass readthedocs defaults [#6189](https://github.com/datalad/datalad/pull/6189) ([@mih](https://github.com/mih))
- infra: Provide custom prefix to auto-related labels [#6151](https://github.com/datalad/datalad/pull/6151) ([@adswa](https://github.com/adswa))
- Remove all usage of exc_str() [#6142](https://github.com/datalad/datalad/pull/6142) ([@mih](https://github.com/mih))
- BF: obtain information about annex special remotes also from annex journal [#6135](https://github.com/datalad/datalad/pull/6135) ([@yarikoptic](https://github.com/yarikoptic) [@mih](https://github.com/mih))
- BF: clone tried to save new subdataset despite failing to clone [#6140](https://github.com/datalad/datalad/pull/6140) ([@bpoldrack](https://github.com/bpoldrack))
#### π§ͺ Tests
- RF+BF: use skip_if_no_module helper instead of try/except for libxmp and boto [#6148](https://github.com/datalad/datalad/pull/6148) ([@yarikoptic](https://github.com/yarikoptic))
- git://github.com -> https://github.com [#6134](https://github.com/datalad/datalad/pull/6134) ([@mih](https://github.com/mih))
#### Authors: 6
- Adina Wagner ([@adswa](https://github.com/adswa))
- Benjamin Poldrack ([@bpoldrack](https://github.com/bpoldrack))
- Christian MΓΆnch ([@christian-monch](https://github.com/christian-monch))
- John T. Wodder II ([@jwodder](https://github.com/jwodder))
- Michael Hanke ([@mih](https://github.com/mih))
- Yaroslav Halchenko ([@yarikoptic](https://github.com/yarikoptic))
---
# 0.15.3 (Sat Oct 30 2021)
#### π Bug Fix
- BF: Don't make create-sibling recursive by default [#6116](https://github.com/datalad/datalad/pull/6116) ([@adswa](https://github.com/adswa))
- BF: Add dashes to 'force' option in non-empty directory error message [#6078](https://github.com/datalad/datalad/pull/6078) ([@DisasterMo](https://github.com/DisasterMo))
- DOC: Add supported URL types to download-url's docstring [#6098](https://github.com/datalad/datalad/pull/6098) ([@adswa](https://github.com/adswa))
- BF: Retain git-annex error messages & don't show them if operation successful [#6070](https://github.com/datalad/datalad/pull/6070) ([@DisasterMo](https://github.com/DisasterMo))
- Remove uses of `__full_version__` and `datalad.version` [#6073](https://github.com/datalad/datalad/pull/6073) ([@jwodder](https://github.com/jwodder))
- BF: ORA shouldn't crash while handling a failure [#6063](https://github.com/datalad/datalad/pull/6063) ([@bpoldrack](https://github.com/bpoldrack))
- DOC: Refine --reckless docstring on usage and wording [#6043](https://github.com/datalad/datalad/pull/6043) ([@adswa](https://github.com/adswa))
- BF: archives upon strip - use rmtree which retries etc instead of rmdir [#6064](https://github.com/datalad/datalad/pull/6064) ([@yarikoptic](https://github.com/yarikoptic))
- BF: do not leave test in a tmp dir destined for removal [#6059](https://github.com/datalad/datalad/pull/6059) ([@yarikoptic](https://github.com/yarikoptic))
- Next wave of exc_str() removals [#6022](https://github.com/datalad/datalad/pull/6022) ([@mih](https://github.com/mih))
#### β οΈ Pushed to `maint`
- CI: Enable new codecov uploader in Appveyor CI ([@adswa](https://github.com/adswa))
#### π Internal
- UX: Log clone-candidate number and URLs [#6092](https://github.com/datalad/datalad/pull/6092) ([@adswa](https://github.com/adswa))
- UX/ENH: Disable reporting, and don't do superfluous internal subdatasets calls [#6094](https://github.com/datalad/datalad/pull/6094) ([@adswa](https://github.com/adswa))
- Update codecov action to v2 [#6072](https://github.com/datalad/datalad/pull/6072) ([@jwodder](https://github.com/jwodder))
#### π Documentation
- Design document on URL substitution feature [#6065](https://github.com/datalad/datalad/pull/6065) ([@mih](https://github.com/mih))
#### π§ͺ Tests
- BF(TST): remove reuse of the same tape across unrelated tests [#6127](https://github.com/datalad/datalad/pull/6127) ([@yarikoptic](https://github.com/yarikoptic))
- Fail Travis tests on deprecation warnings [#6074](https://github.com/datalad/datalad/pull/6074) ([@jwodder](https://github.com/jwodder))
- Ux get result handling broken [#6052](https://github.com/datalad/datalad/pull/6052) ([@christian-monch](https://github.com/christian-monch))
- enable metalad tests again [#6060](https://github.com/datalad/datalad/pull/6060) ([@christian-monch](https://github.com/christian-monch))
#### Authors: 7
- Adina Wagner ([@adswa](https://github.com/adswa))
- Benjamin Poldrack ([@bpoldrack](https://github.com/bpoldrack))
- Christian MΓΆnch ([@christian-monch](https://github.com/christian-monch))
- John T. Wodder II ([@jwodder](https://github.com/jwodder))
- Michael Burgardt ([@DisasterMo](https://github.com/DisasterMo))
- Michael Hanke ([@mih](https://github.com/mih))
- Yaroslav Halchenko ([@yarikoptic](https://github.com/yarikoptic))
---
# 0.15.2 (Wed Oct 06 2021)
#### π Bug Fix
- BF: Don't suppress datalad subdatasets output [#6035](https://github.com/datalad/datalad/pull/6035) ([@DisasterMo](https://github.com/DisasterMo) [@mih](https://github.com/mih))
- Honor datalad.runtime.use-patool if set regardless of OS (was Windows only) [#6033](https://github.com/datalad/datalad/pull/6033) ([@mih](https://github.com/mih))
- Discontinue usage of deprecated (public) helper [#6032](https://github.com/datalad/datalad/pull/6032) ([@mih](https://github.com/mih))
- BF: ProgressHandler - close the other handler if was specified [#6020](https://github.com/datalad/datalad/pull/6020) ([@yarikoptic](https://github.com/yarikoptic))
- UX: Report GitLab weburl of freshly created projects in the result [#6017](https://github.com/datalad/datalad/pull/6017) ([@adswa](https://github.com/adswa))
- Ensure there's a blank line between the class `__doc__` and "Parameters" in `build_doc` docstrings [#6004](https://github.com/datalad/datalad/pull/6004) ([@jwodder](https://github.com/jwodder))
- Large code-reorganization of everything runner-related [#6008](https://github.com/datalad/datalad/pull/6008) ([@mih](https://github.com/mih))
- Discontinue exc_str() in all modern parts of the code base [#6007](https://github.com/datalad/datalad/pull/6007) ([@mih](https://github.com/mih))
#### π§ͺ Tests
- TST: Add test to ensure functionality with subdatasets starting with a hyphen (-) [#6042](https://github.com/datalad/datalad/pull/6042) ([@DisasterMo](https://github.com/DisasterMo))
- BF(TST): filter away warning from coverage from analysis of stderr of --help [#6028](https://github.com/datalad/datalad/pull/6028) ([@yarikoptic](https://github.com/yarikoptic))
- BF: disable outdated SSL root certificate breaking chain on older/buggy clients [#6027](https://github.com/datalad/datalad/pull/6027) ([@yarikoptic](https://github.com/yarikoptic))
- BF: start global test_http_server only if not running already [#6023](https://github.com/datalad/datalad/pull/6023) ([@yarikoptic](https://github.com/yarikoptic))
#### Authors: 5
- Adina Wagner ([@adswa](https://github.com/adswa))
- John T. Wodder II ([@jwodder](https://github.com/jwodder))
- Michael Burgardt ([@DisasterMo](https://github.com/DisasterMo))
- Michael Hanke ([@mih](https://github.com/mih))
- Yaroslav Halchenko ([@yarikoptic](https://github.com/yarikoptic))
---
# 0.15.1 (Fri Sep 24 2021)
#### π Bug Fix
- BF: downloader - fail to download even on non-crippled FS if symlink exists [#5991](https://github.com/datalad/datalad/pull/5991) ([@yarikoptic](https://github.com/yarikoptic))
- ENH: import datalad.api to bind extensions methods for discovery of dataset methods [#5999](https://github.com/datalad/datalad/pull/5999) ([@yarikoptic](https://github.com/yarikoptic))
- Restructure cmdline API presentation [#5988](https://github.com/datalad/datalad/pull/5988) ([@mih](https://github.com/mih))
- Close file descriptors after process exit [#5983](https://github.com/datalad/datalad/pull/5983) ([@mih](https://github.com/mih))
#### β οΈ Pushed to `maint`
- Discontinue testing of hirni extension ([@mih](https://github.com/mih))
#### π Internal
- Add debugging information to release step [#5980](https://github.com/datalad/datalad/pull/5980) ([@jwodder](https://github.com/jwodder))
#### π Documentation
- Coarse description of the credential subsystem's functionality [#5998](https://github.com/datalad/datalad/pull/5998) ([@mih](https://github.com/mih))
#### π§ͺ Tests
- BF(TST): use sys.executable, mark test_ria_basics.test_url_keys as requiring network [#5986](https://github.com/datalad/datalad/pull/5986) ([@yarikoptic](https://github.com/yarikoptic))
#### Authors: 3
- John T. Wodder II ([@jwodder](https://github.com/jwodder))
- Michael Hanke ([@mih](https://github.com/mih))
- Yaroslav Halchenko ([@yarikoptic](https://github.com/yarikoptic))
---
# 0.15.0 (Tue Sep 14 2021) -- We miss you Kyle!
#### Enhancements and new features
- Command execution is now performed by a new `Runner` implementation that is
no longer based on the `asyncio` framework, which was found to exhibit
fragile performance in interaction with other `asyncio`-using code, such as
Jupyter notebooks. The new implementation is based on threads. It also supports
the specification of "protocols" that were introduced with the switch to the
`asyncio` implementation in 0.14.0. ([#5667][])
- `clone` now supports arbitrary URL transformations based on regular
expressions. One or more transformation steps can be defined via
`datalad.clone.url-substitute.