Markdown enhancement

This commit is contained in:
kargakis 2015-06-16 18:05:32 +03:00
parent 5686447a22
commit a147da5557

View file

@ -62,7 +62,7 @@ Scope:
- If you are halfway through typing a command but change your mind, hit **alt-#** to add a `#` at the beginning and enter it as a comment (or use **ctrl-a**, **#**, **enter**). You can then return to it later via command history. - If you are halfway through typing a command but change your mind, hit **alt-#** to add a `#` at the beginning and enter it as a comment (or use **ctrl-a**, **#**, **enter**). You can then return to it later via command history.
- Use `xargs` (or `parallel`). It's very powerful. Note you can control how many items execute per line (`-L`) as well as parallelism (`-P`). If you're not sure if it'll do the right thing, use `xargs echo` first. Also, `-I{}` is handy. Examples: - Use `xargs` (or `parallel`). It's very powerful. Note you can control how many items execute per line (`-L`) as well as parallelism (`-P`). If you're not sure if it'll do the right thing, use `xargs echo` first. Also, `-I{}` is handy. Examples:
``` ```bash
find . -name '*.py' | xargs grep some_function find . -name '*.py' | xargs grep some_function
cat hosts | xargs -I{} ssh root@{} hostname cat hosts | xargs -I{} ssh root@{} hostname
``` ```
@ -82,7 +82,7 @@ Scope:
- In Bash scripts, use `set -x` for debugging output. Use strict modes whenever possible. Use `set -e` to abort on errors. Use `set -o pipefail` as well, to be strict about errors (though this topic is a bit subtle). For more involved scripts, also use `trap`. - In Bash scripts, use `set -x` for debugging output. Use strict modes whenever possible. Use `set -e` to abort on errors. Use `set -o pipefail` as well, to be strict about errors (though this topic is a bit subtle). For more involved scripts, also use `trap`.
- In Bash scripts, subshells (written with parentheses) are convenient ways to group commands. A common example is to temporarily move to a different working directory, e.g. - In Bash scripts, subshells (written with parentheses) are convenient ways to group commands. A common example is to temporarily move to a different working directory, e.g.
``` ```bash
# do something in current dir # do something in current dir
(cd /some/other/dir; other-command) (cd /some/other/dir; other-command)
# continue in original dir # continue in original dir
@ -91,7 +91,7 @@ Scope:
- In Bash, note there are lots of kinds of variable expansion. Checking a variable exists: `${name:?error message}`. For example, if a Bash script requires a single argument, just write `input_file=${1:?usage: $0 input_file}`. Arithmetic expansion: `i=$(( (i + 1) % 5 ))`. Sequences: `{1..10}`. Trimming of strings: `${var%suffix}` and `${var#prefix}`. For example if `var=foo.pdf`, then `echo ${var%.pdf}.txt` prints `foo.txt`. - In Bash, note there are lots of kinds of variable expansion. Checking a variable exists: `${name:?error message}`. For example, if a Bash script requires a single argument, just write `input_file=${1:?usage: $0 input_file}`. Arithmetic expansion: `i=$(( (i + 1) % 5 ))`. Sequences: `{1..10}`. Trimming of strings: `${var%suffix}` and `${var#prefix}`. For example if `var=foo.pdf`, then `echo ${var%.pdf}.txt` prints `foo.txt`.
- The output of a command can be treated like a file via `<(some command)`. For example, compare local `/etc/hosts` with a remote one: - The output of a command can be treated like a file via `<(some command)`. For example, compare local `/etc/hosts` with a remote one:
``` ```sh
diff /etc/hosts <(ssh somehost cat /etc/hosts) diff /etc/hosts <(ssh somehost cat /etc/hosts)
``` ```
@ -116,7 +116,7 @@ Scope:
- A few other options relevant to ssh are security sensitive and should be enabled with care, e.g. per subnet or host or in trusted networks: `StrictHostKeyChecking=no`, `ForwardAgent=yes` - A few other options relevant to ssh are security sensitive and should be enabled with care, e.g. per subnet or host or in trusted networks: `StrictHostKeyChecking=no`, `ForwardAgent=yes`
- To get the permissions on a file in octal form, which is useful for system configuration but not available in `ls` and easy to bungle, use something like - To get the permissions on a file in octal form, which is useful for system configuration but not available in `ls` and easy to bungle, use something like
``` ```sh
stat -c '%A %a %n' /etc/timezone stat -c '%A %a %n' /etc/timezone
``` ```
@ -152,12 +152,12 @@ Scope:
- Know basic `awk` and `sed` for simple data munging. For example, summing all numbers in the third column of a text file: `awk '{ x += $3 } END { print x }'`. This is probably 3X faster and 3X shorter than equivalent Python. - Know basic `awk` and `sed` for simple data munging. For example, summing all numbers in the third column of a text file: `awk '{ x += $3 } END { print x }'`. This is probably 3X faster and 3X shorter than equivalent Python.
- To replace all occurrences of a string in place, in one or more files: - To replace all occurrences of a string in place, in one or more files:
``` ```sh
perl -pi.bak -e 's/old-string/new-string/g' my-files-*.txt perl -pi.bak -e 's/old-string/new-string/g' my-files-*.txt
``` ```
- To rename many files at once according to a pattern, use `rename`. For complex renames, [`repren`](https://github.com/jlevy/repren) may help. - To rename many files at once according to a pattern, use `rename`. For complex renames, [`repren`](https://github.com/jlevy/repren) may help.
``` ```sh
# Recover backup files foo.bak -> foo: # Recover backup files foo.bak -> foo:
rename 's/\.bak$//' *.bak rename 's/\.bak$//' *.bak
# Full rename of filenames, directories, and contents foo -> bar: # Full rename of filenames, directories, and contents foo -> bar:
@ -177,7 +177,7 @@ Scope:
- Also for binary files, `strings` (plus `grep`, etc.) lets you find bits of text. - Also for binary files, `strings` (plus `grep`, etc.) lets you find bits of text.
- To convert text encodings, try `iconv`. Or `uconv` for more advanced use; it supports some advanced Unicode things. For example, this command lowercases and removes all accents (by expanding and dropping them): - To convert text encodings, try `iconv`. Or `uconv` for more advanced use; it supports some advanced Unicode things. For example, this command lowercases and removes all accents (by expanding and dropping them):
``` ```sh
uconv -f utf-8 -t utf-8 -x '::Any-Lower; ::Any-NFD; [:Nonspacing Mark:] >; ::Any-NFC; ' < input.txt > output.txt uconv -f utf-8 -t utf-8 -x '::Any-Lower; ::Any-NFD; [:Nonspacing Mark:] >; ::Any-NFC; ' < input.txt > output.txt
``` ```
@ -228,35 +228,35 @@ Scope:
A few examples of piecing together commands: A few examples of piecing together commands:
- It is remarkably helpful sometimes that you can do set intersection, union, and difference of text files via `sort`/`uniq`. Suppose `a` and `b` are text files that are already uniqued. This is fast, and works on files of arbitrary size, up to many gigabytes. (Sort is not limited by memory, though you may need to use the `-T` option if `/tmp` is on a small root partition.) See also the note about `LC_ALL` above. - It is remarkably helpful sometimes that you can do set intersection, union, and difference of text files via `sort`/`uniq`. Suppose `a` and `b` are text files that are already uniqued. This is fast, and works on files of arbitrary size, up to many gigabytes. (Sort is not limited by memory, though you may need to use the `-T` option if `/tmp` is on a small root partition.) See also the note about `LC_ALL` above.
``` ```sh
cat a b | sort | uniq > c # c is a union b cat a b | sort | uniq > c # c is a union b
cat a b | sort | uniq -d > c # c is a intersect b cat a b | sort | uniq -d > c # c is a intersect b
cat a b b | sort | uniq -u > c # c is set difference a - b cat a b b | sort | uniq -u > c # c is set difference a - b
``` ```
- Summing all numbers in the third column of a text file (this is probably 3X faster and 3X less code than equivalent Python): - Summing all numbers in the third column of a text file (this is probably 3X faster and 3X less code than equivalent Python):
``` ```sh
awk '{ x += $3 } END { print x }' myfile awk '{ x += $3 } END { print x }' myfile
``` ```
- If want to see sizes/dates on a tree of files, this is like a recursive `ls -l` but is easier to read than `ls -lR`: - If want to see sizes/dates on a tree of files, this is like a recursive `ls -l` but is easier to read than `ls -lR`:
``` ```sh
find . -type f -ls find . -type f -ls
``` ```
- Use `xargs` or `parallel` whenever you can. Note you can control how many items execute per line (`-L`) as well as parallelism (`-P`). If you're not sure if it'll do the right thing, use xargs echo first. Also, `-I{}` is handy. Examples: - Use `xargs` or `parallel` whenever you can. Note you can control how many items execute per line (`-L`) as well as parallelism (`-P`). If you're not sure if it'll do the right thing, use xargs echo first. Also, `-I{}` is handy. Examples:
``` ```sh
find . -name '*.py' | xargs grep some_function find . -name '*.py' | xargs grep some_function
cat hosts | xargs -I{} ssh root@{} hostname cat hosts | xargs -I{} ssh root@{} hostname
``` ```
- Say you have a text file, like a web server log, and a certain value that appears on some lines, such as an `acct_id` parameter that is present in the URL. If you want a tally of how many requests for each `acct_id`: - Say you have a text file, like a web server log, and a certain value that appears on some lines, such as an `acct_id` parameter that is present in the URL. If you want a tally of how many requests for each `acct_id`:
``` ```sh
cat access.log | egrep -o 'acct_id=[0-9]+' | cut -d= -f2 | sort | uniq -c | sort -rn cat access.log | egrep -o 'acct_id=[0-9]+' | cut -d= -f2 | sort | uniq -c | sort -rn
``` ```
- Run this function to get a random tip from this document (parses Markdown and extracts an item): - Run this function to get a random tip from this document (parses Markdown and extracts an item):
``` ```sh
function taocl() { function taocl() {
curl -s https://raw.githubusercontent.com/jlevy/the-art-of-command-line/master/README.md | curl -s https://raw.githubusercontent.com/jlevy/the-art-of-command-line/master/README.md |
pandoc -f markdown -t html | pandoc -f markdown -t html |