From 8fb514dc59283a636aa55405e7436c101ea3d751 Mon Sep 17 00:00:00 2001 From: Joshua Levy Date: Wed, 20 May 2015 10:20:08 -0700 Subject: [PATCH] Formatting and intro. --- README.md | 64 +++++++++++++++++++++++++++++++++++++------------------ 1 file changed, 43 insertions(+), 21 deletions(-) diff --git a/README.md b/README.md index d802c9f..e382bf0 100644 --- a/README.md +++ b/README.md @@ -1,10 +1,13 @@ -# The Linux Command Line +# The Command Line -This is a selection of command-line tips that I've found useful over the years when working on Linux. It's organized as a bit of a journey, from command-line novice to expert. Some tips are elementary, and some sophisticated enough few people know them. The goal is broad coverage of important tips that a technical user will find useful or time-saving. It's a bit long, and users certainly don't need to know all of them, but I've done my best to review that each item is worth reading in terms of projected time savings, if you use Linux heavily. +### Time-saving tips for Linux -I originally wrote this list as an answer [on Quora](http://www.quora.com/What-are-some-time-saving-tips-that-every-Linux-user-should-know/answer/Joshua-Levy), but given the interest there it seems perhaps it's worth moving it to Github, where people more talented than I can more easily suggest improvements. If you see an error or something that could be better, please submit a PR! :) +This is a selection of command-line tips that I've found useful over the years when working on Linux. It's organized as a journey, from command-line novice to expert. Some tips are elementary, and some sophisticated enough very few people know them. + +The goal is broad coverage of important tips that a technical user will find useful or time-saving. It's a bit long, and users certainly don't need to know all of them, but I've done my best to review that each item is worth reading in terms of projected time savings, if you use Linux heavily. It's written for Linux, but many items apply equally to MacOS. The items are intentionally brief. For more information on a command, try man, aptitude/yum, or Google. + +I originally wrote much of this as an answer [on Quora](http://www.quora.com/What-are-some-time-saving-tips-that-every-Linux-user-should-know/answer/Joshua-Levy), but given the interest there. it seems it's worth moving the list to Github, where people more talented than I can readily suggest improvements. If you see an error or something that could be better, please submit a PR! -The items are intentionally brief. For more information on a command, try man, aptitude/yum, or Google. ## Basics @@ -24,6 +27,7 @@ The items are intentionally brief. For more information on a command, try man, a - Learn to use apt-get or yum (depending on distro) to find and install packages. + ## Everyday use - In bash, use Ctrl-R to search through command history. @@ -35,9 +39,10 @@ The items are intentionally brief. For more information on a command, try man, a - If you are halfway through typing a command but change your mind, hit Alt-# to add a # at the beginning and enter it as a comment (or use Ctrl-A, #, enter). You can then return to it later via command history. - Use xargs (or parallel). It's very powerful. Note you can control how many items execute per line (-L) as well as parallelism (-P). If you're not sure if it'll do the right thing, use xargs echo first. Also, -I{} is handy. Examples: - +``` find . -name \*.py | xargs grep some_function cat hosts | xargs -I{} ssh root@{} hostname +``` - pstree -p is a helpful display of the process tree. @@ -52,14 +57,18 @@ The items are intentionally brief. For more information on a command, try man, a - In bash scripts, use set -x for debugging output. Use set -e to abort on errors. - Consider using set -o pipefail as well, to be strict about errors (though this topic is a bit subtle). For more involved scripts, also use trap. - In bash scripts, subshells (written with parentheses) are convenient ways to group commands. A common example is to temporarily move to a different working directory, e.g. - +``` # do something in current dir (cd /some/other/dir; other-command) # continue in original dir +``` - In bash, note there are lots of kinds of variable expansion. Checking a variable exists: ${name:?error message}. For example, if a bash script requires a single argument, just write input_file=${1:?usage: $0 input_file}. Arithmetic expansion: i=$(( (i + 1) % 5 )). Sequences: {1..10}. Trimming of strings: ${var%suffix} and ${var#prefix}. For example if var=foo.pdf, then echo ${var%.pdf}.txt prints "foo.txt". -- The output of a command can be treated like a file via <(some command). For example, compare local /etc/hosts with a remote one: diff /etc/hosts <(ssh somehost cat /etc/hosts) +- The output of a command can be treated like a file via <(some command). For example, compare local /etc/hosts with a remote one: +``` +diff /etc/hosts <(ssh somehost cat /etc/hosts) +``` - Know about "here documents" in bash, as in cat <; ::Any-NFC; ' < input.txt > output.txt +``` - To split files into pieces, see split (to split by size) and csplit (to split by a pattern). @@ -167,27 +182,32 @@ stat -c '%A %a %n' /etc/timezone ## One-liners - It is remarkably helpful sometimes that you can do set intersection, union, and difference of text files via sort/uniq. Suppose a and b are text files that are already uniqued. This is fast, and works on files of arbitrary size, up to many gigabytes. (Sort is not limited by memory, though you may need to use the -T option if /tmp is on a small root partition.) See also the note about LC_ALL above. - +``` cat a b | sort | uniq > c # c is a union b cat a b | sort | uniq -d > c # c is a intersect b cat a b b | sort | uniq -u > c # c is set difference a - b +``` - Summing all numbers in the third column of a text file (this is probably 3X faster and 3X less code than equivalent Python): - +``` awk '{ x += $3 } END { print x }' myfile +``` - If want to see sizes/dates on a tree of files, this is like a recursive "ls -l" but is easier to read than "ls -lR": -find . -type f -ls - -- Use xargs. It's very powerful. Note you can control how many items execute per line (-L) as well as parallelism (-P). If you're not sure if it'll do the right thing, use xargs echo first. Also, -I{} is handy. Examples: +``` + find . -type f -ls +``` +- Use xargs or parallel for doing many things. Note you can control how many items execute per line (-L) as well as parallelism (-P). If you're not sure if it'll do the right thing, use xargs echo first. Also, -I{} is handy. Examples: +``` find . -name \*.py | xargs grep some_function cat hosts | xargs -I{} ssh root@{} hostname +``` - Say you have a text file, like a web server log, and a certain value that appears on some lines, such as an acct_id parameter that is present in the URL. If you want a tally of how many requests for each acct_id: - +``` cat access.log | egrep -o 'acct_id=[0-9]+' | cut -d= -f2 | sort | uniq -c | sort -rn - +``` ## Obscure but useful commands @@ -246,4 +266,6 @@ find . -type f -ls - fortune, ddate, and sl: um, well, it depends on whether you consider steam locomotives and Zippy quotations "useful" -Disclaimer: Just because you can do something in Bash, doesn't necessarily mean you should. ;) +## Disclaimer + +Just because you *can* do something in Bash, doesn't necessarily mean you should. ;)