Browse: Home / Bash Shell Scripting

Bash Shell Scripting

Contents

[ hide ]

  1. 1 Introduction
  2. 2 Basics
  3. 2.1 Moving around the filesystem
  4. 2.2 Hello World
  5. 2.3 Variables
  6. 2.4 String Manipulation
  7. 2.5 Functions
  8. 2.6 Commands parameters (arguments)
  9. 3 Loops and control
  10. 3.1 basic
  11. 3.2 loop each line in file
  12. 4 Output redirection
  13. 5 Learning the commands
  14. 5.1 grep
  15. 5.2 rsync
  16. 6 Processes and Lifecycle Management
  17. 6.1 Useful variables
  18. 6.2 Manage child process
  19. 6.3 List child processes of a given PID
  20. 6.4 Signals
  21. 7 BASH Skeleton Script (zenbash.sh)

Introduction

Bash is a very versatile language, one that gets neglected often and gets crafted quickly and hurriedly as a glue between the application layer and the operating system shell / runtime environment. In my experience, learning bash is never a bad thing.

One should not rely on bash to create an enterprise application, but where the need arise a good developer should be able to craft a well-written bash script that is easy to read, easy to maintain, and doesn’t violate good programming practices (where possible)

NOTE:

I am in the processing of converting this page to use wp-wiki, which is an excellent wiki plugin.
Check below for updated bash template, this is way more useful than what I posted here ages ago.

Basics

Moving around the filesystem

The cd command is used to move around directories.

cd /absolute/path # brings you to a dir relative to the root (/) fs
cd relativePath   # brings you to a dir relative to your current dir
echo $PWD         # env variable that tells you where you are
cd -              # this brings you back to the current directory

Hello World

Bash is a scripting language. So really you can program in it without having to write any file. Hello world is as simple as typing the following in the console:

echo "Hello World"

To store it in files, we create a file like so (for example, save this as helloWorld.sh):

!/bin/bash
echo "Hello World"

important -> as you are learning bash, don’t rely too much on graphical editor. learn the vi editor (or better yet, vim – the vi improved).

Once the file is created, it needs to be made executable, and then it can be run in the following manner

chmod 700 helloWorld.sh
./helloWorld.sh

Variables

Variables are weakly typed, they can be number of strings.

#sets a variable
varname="value"
 
#prints a variable
echo "varname's value is $varname"

Default variables are variables that are automatically created by bash without you having to explicitly declare them.

$#     # number of command line parameter passed (argv.length)
$?     # return code of last run command
$!     # pid of last spawned sub process

String Manipulation

A practical example is extracting filename, directory name, and extension from a given file path. Example illustrates this quite well:

# given this file path
thePath="/opt/someDir/someCoolFile.tar.gz"
 
# doing this will give you...
dirPath=${thePath%/*}    # /opt/someDir
fileName=${thePath##*/}  # someCoolFile.tar.gz
fileBase=${fileName%%.*} # someCoolFile
fileExt=${fileName#*.}   # .tar.gz
 
# So how does this work anyway?
${variable%pattern}      # Trim the shortest match from the end
${variable##pattern}     # Trim the longest match from the beginning
${variable%%pattern}     # Trim the longest match from the end
${variable#pattern}      # Trim the shortest match from the beginning

Functions

Functions are a means of grouping a given set of commands together into what is known as a function. By grouping commands together into a function, the feature offered by that function becomes reusable.

Declaration 

 
function aFunction()
{
	param1=$1
	param2=$2
	param3=$3
	echo "$param1 is $param2 but not $param3"
	return 0
}

Invocation 

aFunction "param1" "param2" "param3"

Dynamic invocation

Continuing from the example above, we can also call the helloWorld function that was just declared in this way:

funcName="aFunction"
$funcName "dynamic invocation FOOBAR"

[1]A word of caution

I will put this in blockquote to put the emphasis:

A bash command can invoke any program, such as Java, Python, C and so on.
Java, Python and C can also make system calls, e.g. it is possible to invoke ‘ls’ from Java, Python and C.
However it is not possible to call functions declared within the script from these programs.

I will illustrate this with a combination of bash script and python program.

Suppose, we have this test.sh script:

#!/bin/bash
# in this script, we declare 2 functions, 
 
function helloWorld()
{
	echo "hello world 1"
}
function butHelloWorld2()
{
	echo "this will be called anyway"
}
 
# the script then calls the python script,
python aProgram.py
 
# and then calls buHelloWorld2
butHelloWorld2

The script aProgram.py look like this:

import os
import subprocess
echo "the 2 attempt below will not be successful"
os.system('helloWorld')
subprocess.call("helloWorld",shell=True)

If we then execute test.sh

$ ./test.sh

The result will NOT print “hello world 1?. Meaning that it is impossible for aProgram.py to call the bash function helloWorld() that way.

But since we can make system call, we can always call the script from within the python program in this manner:

subprocess.call(“./test.sh”,shell=True)
Now, because of the way test.sh is scripted, butHelloWorld2 will be called.

The interesting thing here is, ./test.sh will call aProgram.py. If we modify aProgram.py to invoke test.sh, then we have a case of two programs that invoke each other … to infinity. So yeah, hope you’re having fun here.

Commands parameters (arguments)

Commands parameters, or arguments are accessed with numbered variable like $1 (first argument) and $2 (2nd argument). See example script below:

example: helloargs.sh

basic numbered args

#!/bin/bash
echo "hello args a1=$1 a2=$2 a3=$3"

execution example:

$ ./helloargs.sh "hello" "world" "foo"

should prints ‘hello args a1=hello a2=world a3=foo

example: afunc.sh:

Args in function exceeding $9

Important note: command args beyond $9 must be enclosed by curly brackets.

#!/bin/bash
function aFunc()
{
	echo $1 $2 $3 $4 $5 $6 $7 $8 $9 ${10} ${11}
}
 
# pay close attention to the use of ${10} and ${11}
# instead of $10 and $11
aFunc ".1" ".2" ".3" ".4" ".5" ".6" ".7" ".8" ".9" "a" "b"

execution example:

$ ./afunc.sh
# should print: .1 .2 .3 .4 .5 .6 .7 .8 .9 a b

Loops and control

basic

# loop by count
for i in {1..100}; do
	echo "my count is $i"
done
 
# dynamic result from command result
for f in $(ls); do
	echo "file: $f"
done
 
# loop forever until terminated, sleep 5
# seconds in between each echo
while  true ; do
	echo "I will run forever until ctrl-C"
	sleep 5
done

loop each line in file

# This will print each line in a file:
while read line; do
	echo $line;
done < file

Output redirection

On a given command, there are 2 kind of output: stdout – which is the “standard output” and stderr – which is the “standard error output”. Both of these usually goes into the console, but they can be redirected to other destination, such as a file, another command, or /dev/null (nowhere).

Here’s some sample redirection to files.

# redirect stdout, and stderr separately
cmd > /tmp/cmd.log 2> /tmp/cmd-err.log
 
# pipe stderr to stdout
cmd 2>&1
 
# redirect cmd stdout&stderr to file (overwrite)
cmd &> /tmp/cmd.log
 
# redirect cmd stdout&stderr to file (APPEND)
cmd >> file 2>&1

Learning the commands

BASH as a scripting language in itself does not offer much utilities to do any kind of useful activity. What it provides is an environment and framework by which all available commands and executable on the machine can collaborate together.

In my opinion the true power in bash lies in being able to mix and match various executables regardless of their language or origin (C, C++,Java, python) usually via stdout and pipe. Therefore the more commands you learn, the more you can do in bash.

grep

Filters a pattern out of stuff.

# filter lines containing the word root from process list 
ps -elf | grep root
 
# filter lines that does NOT contain root
ps -elf | grep -v root
 
# filter lines that contains the word root or httpd
ps -elf | grep -E "root|httpd"
 
# in a directory containing a bunch of .txt files..
# search for lines in .txt file that contains the string 'hello'
grep "hello" *.txt 
 
# in a directory containing sub directories of log files..
# search for ERROR log lines in all log files in all sub directories
grep -r "ERROR" *.log

rsync

rsync can be used to synchronize data from one folder to another folder, whether they be on the same box, or on separate box. rsync is perfect for installation, backup, and distribution of file. With rsync, you can push data from one box to other boxes easily. Rsync is equipped with various smarts to decide whether the file need to be re-transferred or not. Common usage:

# basic rsync use
rsync -avz --perms "/path/to/sourceDir" "/path/to/targetDir"
 
# installs a folder checked out from svn (with .svn directories)
# and push it into target dir. Take backup of existing files, and
# preserve permission & date modified of target files.
rsync -avz --backup --suffix=.backup --exclude=".svn" --perms "$_srcDir/" "$_installDir"

Processes and Lifecycle Management

Process management, signals, child process, lifecycle, and so on

Useful variables

The following are a listing of special variables related to processes and lifecycle.

$$        #PID of current process
$PPID     #PID of parent of current process
$!        #PID of last backgrounded process (as mentioned above).
 
# Subprocess & PID, example using $! variable
. /fire/off/some/commands/somewhere.sh
childpid=$!
echo "child pid of spawned command is $childpid"

Manage child process

What you’ll want to do usually is spawning a child process, and continuously monitor the runtime status of that child process and exit only when the child process has finished running:

runChildProcess &
childpid=$!
# enter a loop that monitor childpid presence, and exit
# only when child pid is gone
while  true ; do
	ps -lfp $childpid;
	if  $? == 0 ;
		echo "child is still running..."
	else
		echo "child pid is terminated"
	break
fi
done
echo "parent process completes execution"

Or alternatively using wait:

runChildProcess &
childpid=$!
wait $childpid
echo "parent process completes execution"

List child processes of a given PID

#To check if a given $pid is still present or not,
ps -lfp $pid
if  $? == 0 ; then
	echo "process $pid is up"
else
	echo "process $pid is no longer running"
fi

Script derived from an Expert Exchange discussion (googled a long time ago from somewhere..)

# Function Definition
function printPIDFamily() {  -n "$1"  &amp;&amp; echo $1; for p in `pgrep -P $1`; do printPIDFamily $p; done; }
 
# Now to kill a PID found,
printPIDFamily | xargs kill

Signals

Signals are means by which processes can communicate by each other. The way this is done is, signals are send using the kill command. While signals are catched by registering signal handler.

Handler

This is an example of registering a handler for SIGTERM. You can read more about SIGTERM on Wikipedia , along with links to other signals.

on_die()
{
	echo "Dying..."
	# do cleanup here as necesary #
	# and then exit!
	exit 0
}
 
# register on_die on TERM signal (SIGTERM) and INTERRUPT
trap 'on_die' TERM

Sending Signal

On a process running with the above signal handler

kill $pid

BASH Skeleton Script (zenbash.sh)

This is my updated version of my skeleton script – use them at your own risk

#!/bin/bash
 
#@
#@ ==============================================================================
#@
#@ ZEN'S BASH TEMPLATE SCRIPT
#@ bash script template for creating scripts that supports commands 
#@ 
#@ example:
#@ zenbash.sh helloworld "HELL YEAH"
#@
#@ (c) 2010 zen sugiarto / zugiart.com
#@ you can find this script + my bash notes at zugiart.com/wiki/bash 
#@ 
#@ ==============================================================================
#@
 
# =============================================================================================================
# HELPER METHOD
# =============================================================================================================
 
 
function log()
{		
	# modify accordingly if you want msg to go to files etc
	echo " style="color: #780078">`date +'%Y.%m.%d %H:%M:%S'` $LOG_VMNAME | INFO  | $1"
}
 
function error()
{
	echo " style="color: #780078">`date +'%Y.%m.%d %H:%M:%S'` $LOG_VMNAME | ERROR | $1"
}
 
# =============================================================================================================
# CMD DEFINITION
# =============================================================================================================
 
#% 
#@ help 
#@  - show help for a given cmd. if not specified, shows all help cmd 
#@ 
#%  
function help() {
	cmd=$1
	if  "$cmd" == "" ; then
		cat $0 | grep "#@" | grep -v "grep \"#@\"" | cut -d'@' -f2-1000
	else
		strcmd="sed -n '/<doc:$cmd>/,/<\/doc:$cmd>/p' $0"
		eval $strcmd | grep -v "#%" | cut -d'@' -f2-1000
	fi
}
 
#% 
#@ helloworld 
#@  - prints a hello world message with additional  
#@ 
#% 
function helloworld() {
	msg=$1
	if  "$msg" == "" ; then error "param1:msg is mandatory"; return 1; fi
	echo "HELLO WORLD"
	echo "$msg"
}
 
#% 
#@ cmd1  
#@  - description of cmd1
#@
#% 
function cmd1()
{
	log "cmd1 is called"
}
 
 
# =============================================================================================================
# MAIN BLOCK
# =============================================================================================================
 
if  "$1" == "" ; then cmd="help"; else cmd=$1; fi
 
$cmd "$2" "$3" "$4" "$5" "$6" "$7" "$8" "$9" "${10}" "${11}" "${12}"
result=$?
if  $result == 127 ; then
	error "Command not found: $1"
	help
elif  $result != 0 ; then
	help $1
fi
exit $result
Usage

Try it in this order:

# when run on its own, this will print the help
./zenbash.sh
 
# when run with a command, but parameter is
# invalid, it will print that command's help message
./zenbash.sh helloworld
 
# when run correct, it will, well, run
./zenbash.sh helloworld "HELL YAH!"
 
# finally, open and modify the script to your liking
vim zenbash.sh
 
# enjoy

Now it’s important to point out that the comment structure provides the documentation for the help message. So in a way, this produces a slightly similar effect to javadoc or pydoc. But for lack of a better name, I shall call this ‘zenbash-doc’ The idea is:

#@ comment like this will show up in help message
#% 
#@ help msg for command 'cmdName'
#%

for more info, simply look at the help function in the script.

This page is a Wiki! Log in or register an account to edit.