# Cat, Less, Tail and Head
**Table of Contents**
* [cat](#cat)
* [Concatenate files](#concatenate-files)
* [Accepting input from stdin](#accepting-input-from-stdin)
* [Squeeze consecutive empty lines](#squeeze-consecutive-empty-lines)
* [Prefix line numbers](#prefix-line-numbers)
* [Viewing special characters](#viewing-special-characters)
* [Writing text to file](#writing-text-to-file)
* [tac](#tac)
* [Useless use of cat](#useless-use-of-cat)
* [Further Reading for cat](#further-reading-for-cat)
* [less](#less)
* [Navigation commands](#navigation-commands)
* [Further Reading for less](#further-reading-for-less)
* [tail](#tail)
* [linewise tail](#linewise-tail)
* [characterwise tail](#characterwise-tail)
* [multiple file input for tail](#multiple-file-input-for-tail)
* [Further Reading for tail](#further-reading-for-tail)
* [head](#head)
* [linewise head](#linewise-head)
* [characterwise head](#characterwise-head)
* [multiple file input for head](#multiple-file-input-for-head)
* [combining head and tail](#combining-head-and-tail)
* [Further Reading for head](#further-reading-for-head)
* [Text Editors](#text-editors)
## cat
```bash
$ cat --version | head -n1
cat (GNU coreutils) 8.25
$ man cat
CAT(1) User Commands CAT(1)
NAME
cat - concatenate files and print on the standard output
SYNOPSIS
cat [OPTION]... [FILE]...
DESCRIPTION
Concatenate FILE(s) to standard output.
With no FILE, or when FILE is -, read standard input.
...
```
* For below examples, `marks_201*` files contain 3 fields delimited by TAB
* To avoid formatting issues, TAB has been converted to spaces using `col -x` while pasting the output here
#### Concatenate files
* One or more files can be given as input and hence a lot of times, `cat` is used to quickly see contents of small single file on terminal
* To save the output of concatenation, just redirect stdout
```bash
$ ls
marks_2015.txt marks_2016.txt marks_2017.txt
$ cat marks_201*
Name Maths Science
foo 67 78
bar 87 85
Name Maths Science
foo 70 75
bar 85 88
Name Maths Science
foo 68 76
bar 90 90
$ # save stdout to a file
$ cat marks_201* > all_marks.txt
```
#### Accepting input from stdin
```bash
$ # combining input from stdin and other files
$ printf 'Name\tMaths\tScience \nbaz\t56\t63\nbak\t71\t65\n' | cat - marks_2015.txt
Name Maths Science
baz 56 63
bak 71 65
Name Maths Science
foo 67 78
bar 87 85
$ # - can be placed in whatever order is required
$ printf 'Name\tMaths\tScience \nbaz\t56\t63\nbak\t71\t65\n' | cat marks_2015.txt -
Name Maths Science
foo 67 78
bar 87 85
Name Maths Science
baz 56 63
bak 71 65
```
#### Squeeze consecutive empty lines
```bash
$ printf 'hello\n\n\nworld\n\nhave a nice day\n'
hello
world
have a nice day
$ printf 'hello\n\n\nworld\n\nhave a nice day\n' | cat -s
hello
world
have a nice day
```
#### Prefix line numbers
```bash
$ # number all lines
$ cat -n marks_201*
1 Name Maths Science
2 foo 67 78
3 bar 87 85
4 Name Maths Science
5 foo 70 75
6 bar 85 88
7 Name Maths Science
8 foo 68 76
9 bar 90 90
$ # number only non-empty lines
$ printf 'hello\n\n\nworld\n\nhave a nice day\n' | cat -sb
1 hello
2 world
3 have a nice day
```
* For more numbering options, check out the command `nl`
```bash
$ whatis nl
nl (1) - number lines of files
```
#### Viewing special characters
* End of line identified by `$`
* Useful for example to see trailing spaces
```bash
$ cat -E marks_2015.txt
Name Maths Science $
foo 67 78$
bar 87 85$
```
* TAB identified by `^I`
```bash
$ cat -T marks_2015.txt
Name^IMaths^IScience
foo^I67^I78
bar^I87^I85
```
* Non-printing characters
* See [Show Non-Printing Characters](http://docstore.mik.ua/orelly/unix/upt/ch25_07.htm) for more detailed info
```bash
$ # NUL character
$ printf 'foo\0bar\0baz\n' | cat -v
foo^@bar^@baz
$ # to check for dos-style line endings
$ printf 'Hello World!\r\n' | cat -v
Hello World!^M
$ printf 'Hello World!\r\n' | dos2unix | cat -v
Hello World!
```
* the `-A` option is equivalent to `-vET`
* the `-e` option is equivalent to `-vE`
* If `dos2unix` and `unix2dos` are not available, see [How to convert DOS/Windows newline (CRLF) to Unix newline (\n)](https://stackoverflow.com/questions/2613800/how-to-convert-dos-windows-newline-crlf-to-unix-newline-n-in-a-bash-script)
#### Writing text to file
```bash
$ cat > sample.txt
This is an example of adding text to a new file using cat command.
Press Ctrl+d on a newline to save and quit.
$ cat sample.txt
This is an example of adding text to a new file using cat command.
Press Ctrl+d on a newline to save and quit.
```
* See also how to use [heredoc](http://mywiki.wooledge.org/HereDocument)
* [How can I write a here doc to a file](https://stackoverflow.com/questions/2953081/how-can-i-write-a-here-doc-to-a-file-in-bash-script)
* See also [difference between Ctrl+c and Ctrl+d to signal end of stdin input in bash](https://unix.stackexchange.com/questions/16333/how-to-signal-the-end-of-stdin-input-in-bash)
#### tac
```bash
$ whatis tac
tac (1) - concatenate and print files in reverse
$ tac --version | head -n1
tac (GNU coreutils) 8.25
$ seq 3 | tac
3
2
1
$ tac marks_2015.txt
bar 87 85
foo 67 78
Name Maths Science
```
* Useful in cases where logic is easier to write when working on reversed file
* Consider this made up log file, many **Warning** lines but need to extract only from last such **Warning** upto **Error** line
* See [GNU sed chapter](./gnu_sed.md#lines-between-two-regexps) for details on the `sed` command used below
```bash
$ cat report.log
blah blah
Warning: something went wrong
more blah
whatever
Warning: something else went wrong
some text
some more text
Error: something seriously went wrong
blah blah blah
$ tac report.log | sed -n '/Error:/,/Warning:/p' | tac
Warning: something else went wrong
some text
some more text
Error: something seriously went wrong
```
* Similarly, if characters in lines have to be reversed, use the `rev` command
```bash
$ whatis rev
rev (1) - reverse lines characterwise
```
#### Useless use of cat
* `cat` is used so frequently to view contents of a file that somehow users think other commands cannot handle file input
* [UUOC](https://en.wikipedia.org/wiki/Cat_(Unix)#Useless_use_of_cat)
* [Useless Use of Cat Award](http://porkmail.org/era/unix/award.html)
```bash
$ cat report.log | grep -E 'Warning|Error'
Warning: something went wrong
Warning: something else went wrong
Error: something seriously went wrong
$ grep -E 'Warning|Error' report.log
Warning: something went wrong
Warning: something else went wrong
Error: something seriously went wrong
```
* Use [input redirection](http://wiki.bash-hackers.org/howto/redirection_tutorial) if a command doesn't accept file input
```bash
$ cat marks_2015.txt | tr 'A-Z' 'a-z'
name maths science
foo 67 78
bar 87 85
$ tr 'A-Z' 'a-z' < marks_2015.txt
name maths science
foo 67 78
bar 87 85
```
* However, `cat` should definitely be used where **concatenation** is needed
```bash
$ grep -c 'foo' marks_201*
marks_2015.txt:1
marks_2016.txt:1
marks_2017.txt:1
$ # concatenation allows to get overall count in one-shot in this case
$ cat marks_201* | grep -c 'foo'
3
```
#### Further Reading for cat
* [cat Q&A on unix stackexchange](https://unix.stackexchange.com/questions/tagged/cat?sort=votes&pageSize=15)
* [cat Q&A on stackoverflow](https://stackoverflow.com/questions/tagged/cat?sort=votes&pageSize=15)
## less
```bash
$ less --version | head -n1
less 481 (GNU regular expressions)
$ # By default, pager is used to display the man pages
$ # and usually, pager is linked to less command
$ type pager less
pager is /usr/bin/pager
less is /usr/bin/less
$ realpath /usr/bin/pager
/bin/less
$ realpath /usr/bin/less
/bin/less
$ diff -s /usr/bin/pager /usr/bin/less
Files /usr/bin/pager and /usr/bin/less are identical
```
* `cat` command is NOT suitable for viewing contents of large files on the Terminal
* `less` displays contents of a file, automatically fits to size of Terminal, allows scrolling in either direction and other options for effective viewing
* Usually, `man` command uses `less` command to display the help page
* The navigation commands are similar to `vi` editor
#### Navigation commands
Commonly used commands are given below, press `h` for summary of options
* `g` go to start of file
* `G` go to end of file
* `q` quit
* `/pattern` search for the given pattern in forward direction
* `?pattern` search for the given pattern in backward direction
* `n` go to next pattern
* `N` go to previous pattern
#### Further Reading for less
* See `man less` for detailed info on commands and options. For example:
* `-s` option to squeeze consecutive blank lines
* `-N` option to prefix line number
* `less` command is an [improved version](https://unix.stackexchange.com/questions/604/isnt-less-just-more) of `more` command
* [differences between most, more and less](https://unix.stackexchange.com/questions/81129/what-are-the-differences-between-most-more-and-less)
* [less Q&A on unix stackexchange](https://unix.stackexchange.com/questions/tagged/less?sort=votes&pageSize=15)
## tail
```bash
$ tail --version | head -n1
tail (GNU coreutils) 8.25
$ man tail
TAIL(1) User Commands TAIL(1)
NAME
tail - output the last part of files
SYNOPSIS
tail [OPTION]... [FILE]...
DESCRIPTION
Print the last 10 lines of each FILE to standard output. With more
than one FILE, precede each with a header giving the file name.
With no FILE, or when FILE is -, read standard input.
...
```
#### linewise tail
Consider this sample file, with line numbers prefixed
```bash
$ cat sample.txt
1) Hello World
2)
3) Good day
4) How are you
5)
6) Just do-it
7) Believe it
8)
9) Today is sunny
10) Not a bit funny
11) No doubt you like it too
12)
13) Much ado about nothing
14) He he he
15) Adios amigo
```
* default behavior - display last 10 lines
```bash
$ tail sample.txt
6) Just do-it
7) Believe it
8)
9) Today is sunny
10) Not a bit funny
11) No doubt you like it too
12)
13) Much ado about nothing
14) He he he
15) Adios amigo
```
* Use `-n` option to control number of lines to filter
```bash
$ tail -n3 sample.txt
13) Much ado about nothing
14) He he he
15) Adios amigo
$ # some versions of tail allow to skip explicit n character
$ tail -5 sample.txt
11) No doubt you like it too
12)
13) Much ado about nothing
14) He he he
15) Adios amigo
```
* when number is prefixed with `+` sign, all lines are fetched from that particular line number to end of file
```bash
$ tail -n +10 sample.txt
10) Not a bit funny
11) No doubt you like it too
12)
13) Much ado about nothing
14) He he he
15) Adios amigo
$ seq 13 17 | tail -n +3
15
16
17
```
#### characterwise tail
* Note that this works byte wise and not suitable for multi-byte character encodings
```bash
$ # last three characters including the newline character
$ echo 'Hi there!' | tail -c3
e!
$ # excluding the first character
$ echo 'Hi there!' | tail -c +2
i there!
```
#### multiple file input for tail
```bash
$ tail -n2 report.log sample.txt
==> report.log <==
Error: something seriously went wrong
blah blah blah
==> sample.txt <==
14) He he he
15) Adios amigo
$ # -q option to avoid filename in output
$ tail -q -n2 report.log sample.txt
Error: something seriously went wrong
blah blah blah
14) He he he
15) Adios amigo
```
#### Further Reading for tail
* `tail -f` and related options are beyond the scope of this tutorial. Below links might be useful
* [look out for buffering](http://mywiki.wooledge.org/BashFAQ/009)
* [Piping tail -f output though grep twice](https://stackoverflow.com/questions/13858912/piping-tail-output-though-grep-twice)
* [tail and less](https://unix.stackexchange.com/questions/196168/does-less-have-a-feature-like-tail-follow-name-f)
* [tail Q&A on unix stackexchange](https://unix.stackexchange.com/questions/tagged/tail?sort=votes&pageSize=15)
* [tail Q&A on stackoverflow](https://stackoverflow.com/questions/tagged/tail?sort=votes&pageSize=15)
## head
```bash
$ head --version | head -n1
head (GNU coreutils) 8.25
$ man head
HEAD(1) User Commands HEAD(1)
NAME
head - output the first part of files
SYNOPSIS
head [OPTION]... [FILE]...
DESCRIPTION
Print the first 10 lines of each FILE to standard output. With more
than one FILE, precede each with a header giving the file name.
With no FILE, or when FILE is -, read standard input.
...
```
#### linewise head
* default behavior - display starting 10 lines
```bash
$ head sample.txt
1) Hello World
2)
3) Good day
4) How are you
5)
6) Just do-it
7) Believe it
8)
9) Today is sunny
10) Not a bit funny
```
* Use `-n` option to control number of lines to filter
```bash
$ head -n3 sample.txt
1) Hello World
2)
3) Good day
$ # some versions of head allow to skip explicit n character
$ head -4 sample.txt
1) Hello World
2)
3) Good day
4) How are you
```
* when number is prefixed with `-` sign, all lines are fetched except those many lines to end of file
```bash
$ # except last 9 lines of file
$ head -n -9 sample.txt
1) Hello World
2)
3) Good day
4) How are you
5)
6) Just do-it
$ # except last 2 lines
$ seq 13 17 | head -n -2
13
14
15
```
#### characterwise head
* Note that this works byte wise and not suitable for multi-byte character encodings
```bash
$ # if output of command doesn't end with newline, prompt will be on same line
$ # to highlight working of command, the prompt for such cases is not shown here
$ # first two characters
$ echo 'Hi there!' | head -c2
Hi
$ # excluding last four characters
$ echo 'Hi there!' | head -c -4
Hi the
```
#### multiple file input for head
```bash
$ head -n3 report.log sample.txt
==> report.log <==
blah blah
Warning: something went wrong
more blah
==> sample.txt <==
1) Hello World
2)
3) Good day
$ # -q option to avoid filename in output
$ head -q -n3 report.log sample.txt
blah blah
Warning: something went wrong
more blah
1) Hello World
2)
3) Good day
```
#### combining head and tail
* Despite involving two commands, often this combination is faster than equivalent sed/awk versions
```bash
$ head -n11 sample.txt | tail -n3
9) Today is sunny
10) Not a bit funny
11) No doubt you like it too
$ tail sample.txt | head -n2
6) Just do-it
7) Believe it
```
#### Further Reading for head
* [head Q&A on unix stackexchange](https://unix.stackexchange.com/questions/tagged/head?sort=votes&pageSize=15)
## Text Editors
For editing text files, the following applications can be used. Of these, `gedit`, `nano`, `vi` and/or `vim` are available in most distros by default
Easy to use
* [gedit](https://wiki.gnome.org/Apps/Gedit)
* [geany](http://www.geany.org/)
* [nano](http://nano-editor.org/)
Powerful text editors
* [vim](https://github.com/vim/vim)
* [vim learning resources](https://github.com/learnbyexample/scripting_course/blob/master/Vim_curated_resources.md) and [vim reference](https://github.com/learnbyexample/vim_reference) for further info
* [emacs](https://www.gnu.org/software/emacs/)
* [atom](https://atom.io/)
* [sublime](https://www.sublimetext.com/)
Check out [this analysis](https://github.com/jhallen/joes-sandbox/tree/master/editor-perf) for some performance/feature comparisons of various text editors