Bash

Time the script

start=$(date +%s.%N)

# some important stuff


end=$(date +%s.%N)
runtime=$( echo "$end - $start" | bc -l )

Read stdin into a variable

Source

INPUT_VAR=$(</dev/stdin)

This allows something like:

echo 'hello world' | ./script.sh

Running commands in parallel

Use GNU parallel:

mplayer [--gnu] ::: file1.mp4 file2.mp4

Rename files from uppercase to lowercase

Source

for i in *; do mv $i `echo $i | tr [:upper:] [:lower:]`; done

Find Latest modified file

Source

find . -type f -printf '%T@ %p\n' | sort -n | tail -1 | cut -f2- -d" "
  • %T@ gives you the modification time like a unix timestamp
  • sort -n sorts numerically
  • tail -1 takes the last line (highest timestamp)
  • cut -f2 -d" " cuts away the first field (the timestamp) from the output

Se also: mail

Variable expansion from string

Access script parameters by iteration

$ for parameter in $(eval echo "{1..$#}"); do
  echo "Number: ${parameter}"
  echo "Value:  ${!parameter}"
done

Trim whitespaces

Source

var="    abc    "
# remove leading whitespace characters
var="${var#"${var%%[![:space:]]*}"}"
# remove trailing whitespace characters
var="${var%"${var##*[![:space:]]}"}"
echo "===$var==="

# remove both (simpler)
var="$(echo $var | xargs)"

Set environment

# see `man set`
set -eo pipefail

exit on command non-zero status

set -e  # or
set -o errexit

Error on unset variables

set -u  # or
set -o nounset

History

ls *.jp
^jp^jpg       # Replaces only the first occurence
!!:s/jp/jpg/  # Replaces only the first occurence
!ls:s/jp/jpg/ # most recent ls command, only first occurence

Test for remote open ports

Source

nc -z <host> <port>
echo $?

For a quick interactive check

$ nc -z -v -w5 <host> <port>
...

Add or remove trailing slash

# Add
STR="/i/am/a/path"

length=${#STR}
last_char=${STR:length-1:1}

[[ $last_char != "/" ]] && STR="${STR}"; :
echo "${STR}" # => /i/am/a/path/

# Remove
STR="/i/am/a/path/"

length=${#STR}
last_char=${STR:length-1:1}

[[ $last_char == "/" ]] && STR="${STR:0:length-1}"; :

echo "${STR}" # => /i/am/a/path

or short for removing:

STR=${STR%/}

VI bindings in bash

Source

# file: ~/.bashrc
set -o vi

Readline VI Editing Mode Cheat Sheet

Variablesubstition in find

Source

To assign the find placeholder {} to a variable, you need to do the following:

# _ = $0
# {} = $1  # read `man bash`
$ find . -iname "*.xml" -exec bash -c 'echo "$1"' _ {} \;

A more detailed explanation

Variable substituion

Source

# Removs -._ from the string
echo ${string//[-._]/}

Commandline parsing

Source

#!/usr/bin/env bash

POSITIONAL=()
while [[ $# -gt 0 ]]; do
  key="$1"

  case $key in
      -e|--extension)
      EXTENSION="$2"
      shift # past argument
      ;;
      -s|--searchpath)
      SEARCHPATH="$2"
      shift # past argument
      ;;
      -l|--lib)
      LIBPATH="$2"
      shift # past argument
      ;;
      --default)
      DEFAULT=YES
      ;;
      *)    # unknown option
      POSITIONAL+=("$1") # save it in an array for later
      ;;
  esac
  shift # past argument/value
done
set -- "${POSITIONAL[@]}" # restore positional parameters

echo FILE EXTENSION  = "${EXTENSION}"
echo SEARCH PATH     = "${SEARCHPATH}"
echo LIBRARY PATH    = "${LIBPATH}"
echo DEFAULT         = "${DEFAULT}"
echo "Number files in SEARCH PATH with EXTENSION:" $(ls -1 "${SEARCHPATH}"/*."${EXTENSION}" | wc -l)
if [[ -n $1 ]]; then
  echo "Last line of file specified as non-opt/last argument:"
  tail -1 "$1"
fi

Logg input/output

Source

Log the input and output of a session:

script <filename>

Script full path

Source

# Method A
SCRIPTPATH="$( cd $(dirname '$0') ; pwd -P )"

# Method B
SCRIPT_PATH=$(dirname "$(realpath -s '$0')")

Exit script on key press

Source

while true; do
  read -t 1 -N 1
  case "$k" in
    $'\x09') # TAB
      exit 0
      ;;
    $'\x7f') # Back-Space
      exit 0
      ;;
    $'\x01') # Ctrl+A
      exit 0
      ;;
    $'\x1b') # ESC
      exit 0
  esac
done

Version comparing

Source

#!/bin/bash
# Return values
# 0 = '='
# 1 = '>'
# 2 = '<'
vercomp () {
    if [[ $1 == $2 ]]
    then
        return 0
    fi
    local IFS=.
    local i ver1=($1) ver2=($2)
    # fill empty fields in ver1 with zeros
    for ((i=${#ver1[@]}; i<${#ver2[@]}; i++))
    do
        ver1[i]=0
    done
    for ((i=0; i<${#ver1[@]}; i++))
    do
        if [[ -z ${ver2[i]} ]]
        then
            # fill empty fields in ver2 with zeros
            ver2[i]=0
        fi
        if ((10#${ver1[i]} > 10#${ver2[i]}))
        then
            return 1
        fi
        if ((10#${ver1[i]} < 10#${ver2[i]}))
        then
            return 2
        fi
    done
    return 0
}

Exiting

Scripts

# Syntax
exit <errorcode>

# Example
exit 1

Functions

# Syntax
return <errorcode>

# Example
return 1

Help text

Source

Include a header like this

#!/bin/bash
###
### my-script — does one thing well
###
### Usage:
###   my-script <input> <output>
###
### Options:
###   <input>   Input file to read.
###   <output>  Output file to write. Use '-' for stdout.
###   -h        Show this message.
###

This function will print out all lines pre-prended with three octothorpes.

#
# Help function
help() {
  sed -rn 's/^###  ?//;T;p' "$0"
}

Tipps and tricks

Repeat until success

Source

$ while ! cat missingfile
do
    echo waiting for missingfile
    sleep 10
done
$ until cat missingfile
do
    echo waiting for missingfile
    sleep 10
done

Loop through array

Source

## declare an array variable
declare -a arr=("element1" "element2" "element3")

## or
declare -a arr=("element1"
                "element2" "element3"
                "element4"
                )

## now loop through the above array
for i in "${arr[@]}"
do
   echo "$i"
   # or do whatever with individual element of the array
done

# You can access them using echo "${arr[0]}", "${arr[1]}" also

Decimal octal interpration

Source

When facing the error

Value too great for base (error token is "08")

or something similar, you basically using octal numbers instead of decimal numbers. You can define the base of the variable you're using like this:

${variable#0}

DU - Disk usage output formation

Source

format the output of du and sort the files by size:

dusk() { du -sk "$@" | sort -rn | sed -E ':a; s/([[:digit:]]+)([[:digit:]]{3})/\1,\2/; ta' | awk -F'\t' '{printf "%10s %s\n",$1,substr($0,length($1)+2)}';}
dusk *

Performance measuring

While time provides some basic measurement for efficency comparison, a more suited tool is hyperfine which can run several commands and compare their average speed:

hyperfine [--warmup 10] --shell bash "$command1" "$command2"