# Changelog ## [3.10.0] - 2024-10-20 - **cmd/shfmt** - Report the correct language variant in parser error messages - #1102 - Move `--filename` out of the parser options category - #1079 - **syntax** - Parse all CRLF line endings as LF, including inside heredocs - #1088 - Count skipped backslashes inside backticks in position column numbers - #1098 - Count skipped null bytes in position column numbers for consistency - **interp** - Fix a regression in `v3.9.0` which broke redirecting files to stdin - #1099 - Fix a regression in `v3.9.0` where `HandlerContext.Stdin` was never nil - Add an `Interactive` option to be used by interactive shells - #1100 - Support closing stdin, stdout, and stderr via redirections like `<&-` Consider [becoming a sponsor](https://github.com/sponsors/mvdan) if you benefit from the work that went into this release! ## [3.9.0] - 2024-08-16 This release drops support for Go 1.21 and includes many fixes. - **cmd/shfmt** - Switch the diff implementation to remove one dependency - **syntax** - Protect against overflows in position offset integers - **interp** - Use `os.Pipe` for stdin to prevent draining by subprocesses - #1085 - Support cancelling reads in builtins when stdin is a file - #1066 - Support the `nocaseglob` bash option - #1073 - Support the Bash 5.2 `@k` parameter expansion operator - Support the `test -O` and `test -G` operators on non-Windows - #1080 - Support the `read -s` builtin flag - #1063 - **expand** - Add support for case insensitive globbing - #1073 - Don't panic when pattern words are nil - #1076 A special thanks to @theclapp for their contributors to this release! Consider [becoming a sponsor](https://github.com/sponsors/mvdan) if you benefit from the work that went into this release! ## [3.8.0] - 2024-02-11 This release drops support for Go 1.19 and 1.20 and includes many features and bugfixes, such as improving EditorConfig support in `shfmt`. - **cmd/shfmt** - Support EditorConfig language sections such as `[[shell]]` - #664 - Add `--apply-ignore` for tools and editors - #1037 - **syntax** - Allow formatting redirects before all command argumetnts - #942 - Support brace expansions with uppercase letters - #1042 - Unescape backquotes in single quotes within backquotes - #1041 - Better error when using `function` in POSIX mode - #993 - Better column numbers for escapes inside backquotes - #1028 - **interp** - Support parentheses in classic test commands - #1036 - Determine access to a directory via `unix.Access` - #1033 - Support subshells with `FuncEnviron` as `Env` - #1043 - Add support for `fs.DirEntry` via `ReadDirHandler2` - **expand** - Add support for `fs.DirEntry` via `ReadDir2` - Support zero-padding in brace expansions - #1042 ## [3.7.0] - 2023-06-18 - **syntax** - Correctly parse `$foo#bar` as a single word - #1003 - Make `&>` redirect operators an error in POSIX mode - #991 - Avoid producing invalid shell when minifying some heredocs - #923 - Revert the simplification of `${foo:-}` into `${foo-}` - #970 - **interp** - Add `ExecHandlers` to support layering multiple middlewares - #964 - Add initial support for the `select` clause - #969 - Support combining the `errexit` and `pipefail` options - #870 - Set `EUID` just like `UID` - #958 - Replace panics on unimplemented builtins with errors - #999 - Tweak build tags to support building for `js/wasm` - #983 - **syntax/typedjson** - Avoid `reflect.Value.MethodByName` to reduce binary sizes - #961 ## [3.6.0] - 2022-12-11 This release drops support for Go 1.17 and includes many features and fixes. - **cmd/shfmt** - Implement `--from-json` as the reverse of `--to-json` - [#900] - Improve the quality of the `--to-json` output - [#900] - Provide detected language when erroring with `-ln=auto` - [#803] - **syntax** - Don't require peeking two bytes after `echo *` - [#835] - Simplify `${name:-}` to the equivalent `${name-}` - [#849] - Don't print trailing whitespaces on nested subshells - [#814] - Don't print extra newlines in some case clauses - [#779] - Don't indent comments preceding case clause items - [#917] - Allow escaped newlines before unquoted words again - [#873] - Parse a redirections edge case without spaces - [#879] - Give a helpful error when `<<<` is used in POSIX mode - [#881] - Forbid `${!foo*}` and `${!foo@}` in mksh mode - [#929] - Batch allocations less aggressively in the parser - **syntax/typedjson** - Expose `--from-json` and `--to-json` as Go APIs - [#885] - **expand** - Improve support for expanding array keys and values - [#884] - Don't panic on unsupported syntax nodes - [#841] - Don't panic on division by zero - [#892] - Properly expand unquoted parameters with spaces - [#886] - Trim spaces when converting strings to integers - [#928] - **interp** - Add initial implementation for `mapfile` and `readarray` - [#863] - Improve matching patterns against multiple lines - [#866] - Support `%b` in the `printf` builtin - [#955] - Display all Bash options in `shopt` - [#877] - **pattern** - Add `EntireString` to match the entire string using `^$` - [#866] ## [3.5.1] - 2022-05-23 - **cmd/shfmt** - Fix the Docker publishing script bug which broke 3.5.0 - [#860] - **interp** - Support multi-line strings when pattern matching in `[[` - [#861] - Invalid glob words are no longer removed with `nullglob` - [#862] - **pattern** - `Regexp` now returns the typed error `SyntaxError` - [#862] ## [3.5.0] - 2022-05-11 This release drops support for Go 1.16 and includes many new features. - **cmd/shfmt** - Switch to `-ln=auto` by default to detect the shell language - Add support for long flags, like `--indent` for `-i` - **syntax** - Allow extglob wildcards as function names like `@() { ... }` - Add support for heredocs surrounded by backquotes - Add support for backquoted inline comments - Add `NewPos` to create `Pos` values externally - Support escaped newlines with CRLF line endings - `Minify` no longer omits a leading shebang comment - Avoid printing escaped newlines in non-quoted words - Fix some printer edge cases where comments weren't properly spaced - **fileutil** - Add `Shebang` to extract the shell language from a `#!` line - **expand** - Reimplement globstar `**` globbing for correctness - Replace `os.Stat` as the last direct use of the filesystem - **interp** - Add `CallHandler` to intercept all interpreted `CallExpr` nodes - Add `ReadDirHandler` to intercept glob expansion filesystem reads - Add `StatHandler` to intercept `os.Stat` and `os.Lstat` calls - Always surface exit codes from command substitutions - Add initial and incomplete support for `set -x` - Add support for `cd -` as `cd "$OLDPWD"` - Avoid panic on `set - args` ## [3.4.3] - 2022-02-19 - **cmd/shfmt** - New Docker `v3` tag to track the latest stable version - Don't duplicate errors when walking directories - **interp** - Properly handle empty paths in the `test` builtin - Allow unsetting global vars from inside a function again - Use `%w` to wrap errors in `Dir` ## [3.4.2] - 2021-12-24 - The tests no longer assume what locales are installed - **interp** - Keep `PATH` list separators OS-specific to fix a recent regression - Avoid negative elapsed durations in the `time` builtin ## [3.4.1] - 2021-11-23 - **syntax** - Don't return an empty string on empty input to `Quote` - **expand** - Properly sort in `ListEnviron` to avoid common prefix issues - **interp** - `export` used in functions now affects the global scope - Support looking for scripts in `$PATH` in `source` - Properly slice arrays in parameter expansions ## [3.4.0] - 2021-10-01 This release drops support for Go 1.15, which allows the code to start benefitting from `io/fs`. - **cmd/shfmt** - Walks directories ~10% faster thanks to `filepath.WalkDir` - **syntax** - Add `Quote` to mirror `strconv.Quote` for shell syntax - Skip null characters when parsing, just like Bash - Rewrite fuzzers with Go 1.18's native fuzzing - **fileutil** - Add `CouldBeScript2` using `io/fs.DirEntry` - **expand** - Skip or stop at null characters, just like Bash - **interp** - Set `GID` just like `UID` - Add support for `read -p` - Add support for `pwd` flags - Create random FIFOs for process substitutions more robustly - Avoid leaking an open file when interpreting `$( word` - Parse `$\"` correctly within double quotes - A few fixes where minification would break programs - Printing of heredocs within `<()` no longer breaks them - Printing of single statements no longer adds empty lines - Error on invalid parameter names like `${1a}` - **interp** - `Runner.Dir` is now always an absolute path - **shell** - `Expand` now supports expanding a lone `~` - `Expand` and `SourceNode` now have default timeouts - **cmd/shfmt** - Add `-sr` to print spaces after redirect operators - Don't skip empty string values in `-tojson` - Include comment positions in `-tojson` ## [2.4.0] - 2018-05-16 - Publish as a JS package, [mvdan-sh](https://www.npmjs.com/package/mvdan-sh) - **syntax** - Add `DebugPrint` to pretty-print a syntax tree - Fix comment parsing and printing in some edge cases - Indent `<<-` heredoc bodies if indenting with tabs - Add support for nested backquotes - Relax parser to allow quotes in arithmetic expressions - Don't rewrite `declare foo=` into `declare foo` - **interp** - Add support for `shopt -s globstar` - Replace `Runner.Env` with an interface - **shell** - Add `Expand` as a fully featured version of `os.Expand` - **cmd/shfmt** - Set appropriate exit status when `-d` is used ## [2.3.0] - 2018-03-07 - **syntax** - Case clause patterns are no longer forced on a single line - Add `ExpandBraces`, to perform Bash brace expansion on words - Improve the handling of backslashes within backquotes - Improve the parsing of Bash test regexes - **interp** - Support `$DIRSTACK`, `${param[@]#word}`, and `${param,word}` - **cmd/shfmt** - Add `-d`, to display diffs when formatting differs - Promote `-exp.tojson` to `-tojson` - Add `Pos` and `End` fields to nodes in `-tojson` - Inline `StmtList` fields to simplify the `-tojson` output - Support `-l` on standard input ## [2.2.1] - 2018-01-25 - **syntax** - Don't error on `${1:-default}` - Allow single quotes in `${x['str key']}` as well as double quotes - Add support for `${!foo[@]}` - Don't simplify `foo[$x]` to `foo[x]`, to not break string indexes - Fix `Stmt.End` when the end token is the background operator `&` - Never apply the negation operator `!` to `&&` and `||` lists - Apply the background operator `&` to entire `&&` and `||` lists - Fix `StopAt` when the stop string is at the beginning of the source - In `N>word`, check that `N` is a valid numeric literal - Fix a couple of crashers found via fuzzing - **cmd/shfmt** - Don't error if non-bash files can't be written to ## [2.2.0] - 2018-01-18 - Tests on Mac and Windows are now ran as part of CI - **syntax** - Add `StopAt` to stop lexing at a custom arbitrary token - Add `TranslatePattern` and `QuotePattern` for pattern matching - Minification support added to the printer - see `Minify` - Add ParamExp.Names to represent `${!prefix*}` - Add TimeClause.PosixFormat for its `-p` flag - Fix parsing of assignment values containing `=` - Fix parsing of parameter expansions followed by a backslash - Fix quotes in parameter expansion operators like `${v:-'def'}` - Fix parsing of negated declare attributes like `declare +x name` - Fix parsing of `${#@}` - Reject bad parameter expansion operators like `${v@WRONG}` - Reject inline array variables like `a=(b c) prog` - Reject indexing of special vars like `${1[3]}` - Reject `${!name}` when in POSIX mode - Reject multiple parameter expansion actions like `${#v:-def}` - **interp** - Add Bash brace expansion support, including `{a,b}` and `{x..y}` - Pattern matching actions are more correct and precise - Exported some Runner internals, including `Vars` and `Funcs` - Use the interpreter's `$PATH` to find binaries - Roll our own globbing to use our own pattern matching code - Support the `getopts` sh builtin - Support the `read` bash builtin - Numerous changes to improve Windows support - **shell** - New experimental package with high-level utility functions - Add `SourceFile` to get the variables declared in a script - Add `SourceNode` as a lower-level version of the above - **cmd/shfmt** - Add `-mn`, which minifies programs via `syntax.Minify` ## [2.1.0] - 2017-11-25 - **syntax** - Add `Stmts`, to parse one statement at a time - Walk no longer ignores comments - Parameter expansion end fixes, such as `$foo.bar` - Whitespace alignment can now be kept - see `KeepPadding` - Introduce an internal newline token to simplify the parser - Fix `Block.Pos` to actually return the start position - Fix mishandling of inline comments in two edge cases - **interp** - Expose `Fields` to expand words into strings - First configurable modules - cmds and files - Add support for the new `TimeClause` - Add support for namerefs and readonly vars - Add support for associative arrays (maps) - More sh builtins: `exec return` - More bash builtins: `command pushd popd dirs` - More `test` operators: `-b -c -t -o` - Configurable kill handling - see `KillTimeout` - **cmd/shfmt** - Add `-f` to just list all the shell files found - Add `-kp` to keep the column offsets in place - **cmd/gosh** - Now supports a basic interactive mode ## [2.0.0] - 2017-08-30 - The package import paths were moved to `mvdan.cc/sh/...` - **syntax** - Parser and Printer structs introduced with functional options - Node positions are now independent - `Position` merged into `Pos` - All comments are now attached to nodes - Support `mksh` - MirBSD's Korn Shell, used in Android - Various changes to the AST: - `EvalClause` removed; `eval` is no longer parsed as a keyword - Add support for Bash's `time` and `select` - Merge `UntilClause` into `WhileClause` - Moved `Stmt.Assigns` to `CallExpr.Assigns` - Remove `Elif` - chain `IfClause` nodes instead - Support for indexed assignments like `a[i]=b` - Allow expansions in arithmetic expressions again - Unclosed heredocs now produce an error - Binary ops are kept in the same line - see `BinaryNextLine` - Switch cases are not indented by default - see `SwitchCaseIndent` - **cmd/shfmt** - Add `-s`, which simplifies programs via `syntax.Simplify` - Add `-ln `, like `-ln mksh` - Add `-bn` to put binary ops in the next line, like in v1 - Add `-ci` to indent switch cases, like in v1 - **interp** - Some progress made, though still experimental - Most of POSIX done - some builtins remain to be done ## [1.3.1] - 2017-05-26 - **syntax** - Fix parsing of `${foo[$bar]}` - Fix printer regression where `> >(foo)` would be turned into `>>(foo)` - Break comment alignment on any line without a comment, fixing formatting issues - Error on keywords like `fi` and `done` used as commands ## [1.3.0] - 2017-04-24 - **syntax** - Fix backslashes in backquote command substitutions - Disallow some test expressions like `[[ a == ! b ]]` - Disallow some parameter expansions like `${$foo}` - Disallow some arithmetic expressions like `((1=3))` and `(($(echo 1 + 2)))` - Binary commands like `&&`, `||` and pipes are now left-associative - **fileutil** - `CouldBeScript` may now return true on non-regular files such as symlinks - **interp** - New experimental package to interpret a `syntax.File` in pure Go ## [1.2.0] - 2017-02-22 - **syntax** - Add support for escaped characters in bash regular expressions - **fileutil** - New package with some code moved from `cmd/shfmt`, now importable - New funcs `HasShebang` and `CouldBeScript` - Require shebangs to end with whitespace to reject `#!/bin/shfoo` ## [1.1.0] - 2017-01-05 - **syntax** - Parse `[[ a = b ]]` like `[[ a == b ]]`, deprecating `TsAssgn` in favour of `TsEqual` - Add support for the `-k`, `-G`, `-O` and `-N` unary operators inside `[[ ]]` - Add proper support for `!` in parameter expansions, like `${!foo}` - Fix a couple of crashes found via fuzzing - **cmd/shfmt** - Rewrite `[[ a = b ]]` into the saner `[[ a == b ]]` (see above) ## [1.0.0] - 2016-12-13 - **syntax** - Stable release, API now frozen - `Parse` now reads input in chunks of 1KiB - **cmd/shfmt** - Add `-version` flag ## [0.6.0] - 2016-12-05 - **syntax** - `Parse` now takes an `io.Reader` instead of `[]byte` - Invalid UTF-8 is now reported as an error - Remove backtracking for `$((` and `((` - `Walk` now takes a func literal to simplify its use ## [0.5.0] - 2016-11-24 - **cmd/shfmt** - Remove `-cpuprofile` - Don't read entire files into memory to check for a shebang - **syntax** - Use `uint32` for tokens and positions in nodes - Use `Word` and `Lit` pointers consistently instead of values - Ensure `Word.Parts` is never empty - Add support for expressions in array indexing and parameter expansion slicing ## [0.4.0] - 2016-11-08 - Merge `parser`, `ast`, `token` and `printer` into a single package `syntax` - Use separate operator types in nodes rather than `Token` - Use operator value names that express their function - Keep `;` if on a separate line when formatting - **cmd/shfmt** - Allow whitespace after `#!` in a shebang - **syntax** - Implement operator precedence for `[[ ]]` - Parse `$(foo)` and ``foo`` as the same (`shfmt` then converts the latter to the former) - Rename `Quoted` to `DblQuoted` for clarity - Split `((foo))` nodes as their own type, `ArithmCmd` - Add support for bash parameter expansion slicing ## [0.3.0] - 2016-10-26 - Add support for bash's `coproc` and extended globbing like `@(foo)` - Improve test coverage, adding tests to `cmd/shfmt` and bringing `parser` and `printer` close to 100% - Support empty C-style for loops like `for ((;;)) ...` - Support for the `>|` redirect operand - **cmd/shfmt** - Fix issue where `.sh` and `.bash` files might not be walked if running on a directory - Fix issue where `-p` was not obeyed when formatting stdin - **parser** - `$''` now generates an `ast.SglQuoted`, not an `ast.Quoted` - Support for ambiguous `((` like with `$((` - Improve special parameter expansions like `$@` or `$!` - Improve bash's `export` `typeset`, `nameref` and `readonly` - `<>`, `>&` and `<&` are valid POSIX - Support for bash's `^`, `^^`, `,` and `,,` operands inside `${}` ## [0.2.0] - 2016-10-13 - Optimizations all around, making `shfmt` ~15% faster - **cmd/shfmt** - Add `-p` flag to only accept POSIX Shell programs (`parser.PosixConformant`) - **parser** - Add support for ambiguous `$((` as in `$((foo) | bar)` - Limit more bash features to `PosixConformant` being false - Don't parse heredoc bodies in nested expansions and contexts - Run tests through `bash` to confirm the presence of a parse error - **ast** - Add `Walk(Visitor, Node)` function ## [0.1.0] - 2016-09-20 Initial release. [3.10.0]: https://github.com/mvdan/sh/releases/tag/v3.10.0 [3.9.0]: https://github.com/mvdan/sh/releases/tag/v3.9.0 [3.8.0]: https://github.com/mvdan/sh/releases/tag/v3.8.0 [3.7.0]: https://github.com/mvdan/sh/releases/tag/v3.7.0 [3.6.0]: https://github.com/mvdan/sh/releases/tag/v3.6.0 [#779]: https://github.com/mvdan/sh/issues/779 [#803]: https://github.com/mvdan/sh/issues/803 [#814]: https://github.com/mvdan/sh/issues/814 [#835]: https://github.com/mvdan/sh/issues/835 [#841]: https://github.com/mvdan/sh/issues/841 [#849]: https://github.com/mvdan/sh/pull/849 [#863]: https://github.com/mvdan/sh/pull/863 [#866]: https://github.com/mvdan/sh/pull/866 [#873]: https://github.com/mvdan/sh/issues/873 [#877]: https://github.com/mvdan/sh/issues/877 [#879]: https://github.com/mvdan/sh/pull/879 [#881]: https://github.com/mvdan/sh/issues/881 [#884]: https://github.com/mvdan/sh/issues/884 [#885]: https://github.com/mvdan/sh/issues/885 [#886]: https://github.com/mvdan/sh/issues/886 [#892]: https://github.com/mvdan/sh/issues/892 [#900]: https://github.com/mvdan/sh/pull/900 [#917]: https://github.com/mvdan/sh/pull/917 [#928]: https://github.com/mvdan/sh/issues/928 [#929]: https://github.com/mvdan/sh/pull/929 [#955]: https://github.com/mvdan/sh/pull/955 [3.5.1]: https://github.com/mvdan/sh/releases/tag/v3.5.1 [#860]: https://github.com/mvdan/sh/issues/860 [#861]: https://github.com/mvdan/sh/pull/861 [#862]: https://github.com/mvdan/sh/pull/862 [3.5.0]: https://github.com/mvdan/sh/releases/tag/v3.5.0 [3.4.3]: https://github.com/mvdan/sh/releases/tag/v3.4.3 [3.4.2]: https://github.com/mvdan/sh/releases/tag/v3.4.2 [3.4.1]: https://github.com/mvdan/sh/releases/tag/v3.4.1 [3.4.0]: https://github.com/mvdan/sh/releases/tag/v3.4.0 [3.3.1]: https://github.com/mvdan/sh/releases/tag/v3.3.1 [3.3.0]: https://github.com/mvdan/sh/releases/tag/v3.3.0 [3.2.4]: https://github.com/mvdan/sh/releases/tag/v3.2.4 [3.2.2]: https://github.com/mvdan/sh/releases/tag/v3.2.2 [3.2.1]: https://github.com/mvdan/sh/releases/tag/v3.2.1 [3.2.0]: https://github.com/mvdan/sh/releases/tag/v3.2.0 [3.1.2]: https://github.com/mvdan/sh/releases/tag/v3.1.2 [3.1.1]: https://github.com/mvdan/sh/releases/tag/v3.1.1 [3.1.0]: https://github.com/mvdan/sh/releases/tag/v3.1.0 [3.0.2]: https://github.com/mvdan/sh/releases/tag/v3.0.2 [3.0.1]: https://github.com/mvdan/sh/releases/tag/v3.0.1 [3.0.0]: https://github.com/mvdan/sh/releases/tag/v3.0.0 [2.6.4]: https://github.com/mvdan/sh/releases/tag/v2.6.4 [2.6.3]: https://github.com/mvdan/sh/releases/tag/v2.6.3 [2.6.2]: https://github.com/mvdan/sh/releases/tag/v2.6.2 [2.6.1]: https://github.com/mvdan/sh/releases/tag/v2.6.1 [2.6.0]: https://github.com/mvdan/sh/releases/tag/v2.6.0 [2.5.1]: https://github.com/mvdan/sh/releases/tag/v2.5.1 [2.5.0]: https://github.com/mvdan/sh/releases/tag/v2.5.0 [2.4.0]: https://github.com/mvdan/sh/releases/tag/v2.4.0 [2.3.0]: https://github.com/mvdan/sh/releases/tag/v2.3.0 [2.2.1]: https://github.com/mvdan/sh/releases/tag/v2.2.1 [2.2.0]: https://github.com/mvdan/sh/releases/tag/v2.2.0 [2.1.0]: https://github.com/mvdan/sh/releases/tag/v2.1.0 [2.0.0]: https://github.com/mvdan/sh/releases/tag/v2.0.0 [1.3.1]: https://github.com/mvdan/sh/releases/tag/v1.3.1 [1.3.0]: https://github.com/mvdan/sh/releases/tag/v1.3.0 [1.2.0]: https://github.com/mvdan/sh/releases/tag/v1.2.0 [1.1.0]: https://github.com/mvdan/sh/releases/tag/v1.1.0 [1.0.0]: https://github.com/mvdan/sh/releases/tag/v1.0.0 [0.6.0]: https://github.com/mvdan/sh/releases/tag/v0.6.0 [0.5.0]: https://github.com/mvdan/sh/releases/tag/v0.5.0 [0.4.0]: https://github.com/mvdan/sh/releases/tag/v0.4.0 [0.3.0]: https://github.com/mvdan/sh/releases/tag/v0.3.0 [0.2.0]: https://github.com/mvdan/sh/releases/tag/v0.2.0 [0.1.0]: https://github.com/mvdan/sh/releases/tag/v0.1.0