PaleoEarthLabs wiki

One bucket of limited wisdom

User Tools

Site Tools


Unix tools - tips and tricks

A collection of quick-and-dirty tricks to do things with the standard command line tools in *Nix and cygwin. I am copying stuff here as I use these snippets regularly so this is a central repo for the ageing brain.


Print unique lines based on column (from Stackoverflow)

awk -F, '!seen[$1]++' Input.csv

Pass variable from bash into awk (Stackoverflow)

This is the best way to do it. It uses the -v option: (P.S. use a space after -v or it will be less portable. E.g., awk -v var= not awk -vvar)
variable="line one\nline two"
awk -v var="$variable" 'BEGIN {print var}'
line one
line two
# --- Multiple vars
# This should be compatible with most awk and variable is 
# available in the BEGIN block as well:
awk -v a="$var1" -v b="$var2" 'BEGIN {print a,b}'

Removing/adding whitespace

Sometimes one just wants to remove linebreaks in a file and have all data on one line (e.g. as input for a a “for loop”:

echo $(<file.txt) | tr -d ' '

If the troff part of the expression is removed, then the echo just pads with a space (from Stackoverflow

software/unixtools.txt · Last modified: 2018/01/22 13:29 by christian