Shell Scripting

• History of various Shells
• Shell vs Bash
• Is bash scripting the same as shell scripting?
  • sh = Bourne Shell
  • bash = Bourne Again Shell
• Unix had the Bourne Shell, which was modified in Linux and is called Bash (Bourne Again SHell).
• Bash Reference Manual
• Unix vs Linux
• Shell scripting crash course (beginners)
• Absolute vs Relative Path in Linux: What’s the Difference?
• The Missing Semester of Your CS Education
• Google’s Shell Style Guide
• Shell Script Best Practices
• Advanced Bash-Scripting Guide
• Check Shell scripts: shellcheck.net

---

• The # is called ‘sharp’ and the ! is called a ‘bang’, so together (#!) they’re called a ‘shebang’ or ‘hashbang’.
• A shebang is used to declare the dialect being used by the script.
• To find the location of the dialect, use the ‘which’ command. Eg: which bash Output: /usr/bin/bash

• Add this as the first line in every Shell script.

  #!/usr/bin/bash
  

• Running/executing a Shell script

  • Two ways
    • ./script.sh (Preferred, as the terminal can decide the interpreter to be used.)
    • sh script.sh
  • Difference between ./script.sh and sh script.sh.
    • About the Use of Dot-Slash (./) in Commands
  • [What is the difference between sourcing (‘.’ or ‘source’) and executing a file in bash? (./test.sh vs . ./test.sh or source ./test.sh)]
    • What does ‘source’ do?
    • What is the difference between executing a Bash script vs sourcing it?

• Error handling (set -Eeuo pipefail, trap, etc.): How to Trap Errors in Bash Scripts on Linux

• Debugging/printing commands to terminal: What does set -x do?

• Output to terminal

  echo Selena Gomez
  

• Variables

  • Allowed: letters, numbers and underscores

  NAME="Harsh Kapadia"  # no spaces between var name, '=' and var value allowed.
  echo "My name is $NAME"
  echo "My name is ${NAME}"
  

  • Strings in bash can be defined with ' and " delimiters, but they are not equivalent. Strings delimited with ' are literal strings and will not substitute variable values whereas " delimited strings will.

  $ name=selena
  
  $ echo 'Hey ${name}!'
  Hey ${name}!
  
  $ echo "Hey ${name}!"
  Hey Selena!
  
  $ echo "Hey '${name}'!"
  Hey 'Selena'!
  

  • ${} and $() are different.
    • ${abc} is variable/parameter substitution.
    • $(abc) is command substitution and will execute whatever is inside the (), which means it is usually used to run a command or store the output of a command.
    • More info and good practices
  • Parsing command-line arguments
  • set in Bash to set and clear arguments and set --
  • Built-in Shell variables
    • Built-in Shell variables $#, $*, etc
    • What is the difference between “$@” and “$*” in Bash?
    • Special parameters
  • Environment variables
  • Output redirection
    • What does “ 2>&1 “ mean?
    • How does cmd > /dev/null 2>&1 work?
    • Order of redirections
      • There is a difference between cmd 2>&1 > file.txt and cmd > file.txt 2>&1.
    • what is the difference between >&2 and &>2
    • Shell Redirection 101
    • Difference between 2>&1 > output.log and 2>&1 | tee output.log
    • Redirection differences between &> >& and 2>&1
    • What is the differences between &> and 2>&1
    • Creating temp file vs process substitution vs variable expansion?
    • Using grep with pipe and ampersand to filter errors from find
      • |& is the shorthand of 2>&1 |
    • Close File Descriptors in Bash
    • More info

• User input

  read -p "Enter your name: " NAME # 'p' stands for prompt
  echo "Hey $NAME, nice to meet you!"
  

• If statement

  if [ "$NAME" == "Selena Gomez" ] # The spaces are important!
  then
  	echo "Hey Sel!"
  fi # ending if statement
  
  # Or
  
  if [[ "$NAME" == "Selena Gomez" ]]; then
  	echo "Hey Sel!";
  fi;
  
  if command -v "apt-get" > /dev/null; then
  	echo "The command 'apt-get' is available!"
  else
  	echo "The command 'apt-get' is not available."
  fi # From: https://unix.stackexchange.com/a/602841
  

  NOTE:

  • Use if cmd; then .. to check exit code, or if [ "$(cmd)" = .. ] to check output.
  • More on if-else
  • When are square brackets required in a Bash if statement?
  • Quoting variables
    • Difference between quoting variables in shell script “if” statements?
    • When to wrap quotes around a shell variable?
  • What is the bash command: command?
    • which vs command -v in Bash

Difference between (), (()), [] and [[]]

• If-elif-else statement

  if [[ "$NAME" == "Selena Gomez" ]] # The spaces are important!
  then
  	echo "Selly!"
  elif [[ "$NAME" == "Selena" ]]
  then
  	echo "Yay it's Sel!"
  else
  	echo "Hiya!"
  fi # ending if statement
  

• Comparisons

  • val1 -eq val2: Returns true if the values are equal
  • val1 -ne val2: Returns true if the values are not equal
  • val1 -gt val2: Returns true if val1 is greater than val2
  • val1 -ge val2: Returns true if val1 is greater than or equal to val2
  • val1 -lt val2: Returns true if val1 is less than val2
  • val1 -le val2: Returns true if val1 is less than or equal to val2

  NUM_1=22
  NUM_2=7
  NUM_3=1992
  
  if [ "$NUM_1" -gt "$NUM_2" ]
  then
  	echo "$NUM_1 is greater than $NUM_2!"
  fi
  

  NOTE:

  • Understanding boolean operators in bash script
  • Comparisons
    • For string comparisons, use >, <, == (Bash) or = (Bash and Shell)
    • For numerical comparisons, use -eq, -lt, -gt, etc.
    • Shell equality operators (=, ==, -eq)
    • How do I negate a test with regular expressions in a bash script?

• File conditions

  • -d file True if the file is a directory
  • -e file True if the file exists (note that this is not particularly portable, thus -f is generally used)
  • -f file True if the provided string is a file
  • -g file True if the group id is set on a file
  • -r file True if the file is readable
  • -s file True if the file has a non-zero size
  • -u True if the user id is set on a file
  • -w True if the file is writable
  • -x True if the file is an executable
  • -L True if a symlink exists (The actual file being symlinked to may or may not exist.) (Source)
  • -z True if string length is zero

  FILE="text.txt"
  
  if [[ -f "$FILE" ]]; then
  	echo "$FILE is a file!";
  elif [[ ! -f "$FILE" ]]; then
  	echo "$FILE is not a file : (";
  fi;
  

• Case statements

  read -p "Are you 21 or over? Y/N " ANSWER
  
  case "$ANSWER" in
  	[yY] | [yY][eE][sS]) # Accepts 'y', 'Y', 'yes', 'YES', 'Yes', 'yEs'...
  		echo "You can have a beer ( :"
  		;; # ';;' is equivalent to a 'break' statement in C language
  	[nN] | [nN][oO])
  		echo "Sorry, no drinking"
  		;;
  	*) # Default
  		echo "Please enter y/yes or n/no"
  		;;
  esac
  

• For loop

  NAMES="Selena Felix Lucas Evelien Sean Elisabeth Robert Sonny"
  
  for NOME in $NAMES
  	do
  		echo "Hello $NOME! ( :"
  done
  
  # touch file_1.md file_2.md file_3.md
  FILES=$(ls *.md)
  
  for FOLE in $FILES
  	do
  		echo "Renaming $FOLE to test_$FOLE"
  		mv $FOLE test_$FOLE
  done
  
  max=10
  for (( i=2; i <= $max; ++i )) # The whitespaces in this line don't matter.
  	do
  		echo "$i"
  done
  

• While loop

  # Read from file, line by line
  LINE=1
  
  while read -r CURRENT_LINE
  	do
  		echo "$LINE: $CURRENT_LINE"
  		((LINE++)) # '(())': https://superuser.com/questions/1533900/difference-between-and-or-and-in-bash
  done < "./test_f1.md"
  

  More on the Bash While Loop

Difference between (), (()), [] and [[]]

• String manipulation and sub-strings

  • Delete the shortest substring match from the front of ${str}: ${str#substring}
  • Delete the shortest substring match from the back of ${str}: ${str%substring}
  • Delete the longest substring match from the front of ${str}: ${str##substring}
  • Delete the longest substring match from the back of ${str}: ${str%%substring}

  $ str="Hey Selena! Hey Lucas!"
  
  $ echo ${str#*Hey}
  Selena! Hey Lucas!
  
  $ echo ${str%Hey*}
  Hey Selena!
  
  $ echo ${str##*Hey}
  Lucas!
  
  $ echo ${str%%Hey*}
  
  

  • Parameter Expansion (Eg: ${parameter+str}, ${parameter:?str})
    • How to check if a variable is set in Bash (unset/unbound variable error)
  • Bash sub-string matching
  • Get a sub-string based on index numbers (${str:start_idx:substr_length})
  • Delete the last character of a string using string manipulation in shell script (${var::-1})
  • See if you can use ${variable//search/replace} instead.
  • How to delete a substring using shell script

NOTE:

• The asterisk (*) is a catchall to match any and all characters. (More info)

• Internal Field Separator (IFS)

  • Used to separate a string based on a certain character.
  • Default value: Three character string comprising a space, tab and newline.

  $ string="foo bar foobar"
  
  $ for i in $string
  > do
  >    echo "'$i' is the substring"
  > done
  'foo' is the substring
  'bar' is the substring
  'foobar' is the substring
  

  • Custom values can be set.
  • More info and use cases
  • Unset IFS - unexpected behaviour
  • How do I split a string on a delimiter in Bash?

• Functions

  function greet()
  {
  	echo "Hello $1! Your age is $2!"
  }
  greet "Selena" "29"
  

  • More about Bash Functions
  • How to store output of a function in a variable in bash
  • Return value in a Bash function
  • How to pass array as an argument to a function in Bash
  • Passing arrays as parameters in bash
  • Access arguments to Bash script inside a function
    • How to access command line arguments of the caller inside a function?

• Create a folder and write to a file

  mkdir "hello"
  touch "hello/world.md"
  echo "# Hello World" >> "hello/world.md"
  echo "Created hello/world.md"
  echo "Contents:"
  cat "hello/world.md"
  

• Arrays

  • Using Arrays in Bash
  • Bash For Loop Array: Iterate Through Array Values
  • Appending to Arrays in Bash
  • How to split a string into an array in Bash?
  • How do I split a string on a delimiter in Bash?
  • Double quote array expansions to avoid re-splitting elements.
  • Convert a text string in bash to array
  • What is the difference between “$@” and “$*” in Bash?
  • Indexed (declare -a) and Associative (declare -A) arrays