Shell Scripting
Master shell scripting through hands-on exercises in an interactive Linux environment — from your first script to building real automation tools
Hello, Shell!
Welcome to the Shell Scripting Tutorial! On the top is a code editor; on the bottom is a real Linux terminal.
Shell scripting has a reputation for tricky syntax — even experienced developers regularly look up Bash quoting rules. If something feels confusing, that’s a sign you’re engaging with genuinely hard material, not a sign you’re doing it wrong. Every error message is a clue; every mistake is a step forward.
Why learn shell scripting?
Every time you repeat a task in the terminal — processing files, checking log files, running complex builds — you are a candidate for automation. A shell script captures those commands in a file so you can re-run, share, and schedule them without retyping anything. So learning shell scripting can supercharge your productivity as a developer.
Shell scripts are the foundation of Continuous Integration / Continuous Delivery (CI/CD) pipelines, Docker entrypoints, deployment scripts, and system administration. The skills you learn here transfer directly to real production workflows.
Two lines every script needs
Open morning.sh in the editor. It already has:
#!/bin/bash
set -e
Line 1 — the shebang (#!): When you run a file, Linux reads the
first two bytes to decide how to execute it. #! followed by a path
tells the OS which interpreter to use. Without it, the OS guesses —
and often guesses wrong. #!/bin/bash is the standard choice when
Bash is at /bin/bash (true on most Linux systems). For maximum
portability across systems where Bash may live elsewhere, you can also
use #!/usr/bin/env bash, which finds the first bash in your $PATH.
Line 2 — the safety net (set -e): By default, Bash happily
continues running after a failed command. set -e exits the script
when a command fails, preventing a cascade of confusing failures.
Always include it. (We’ll cover its edge cases in later steps —
for now, just know it makes scripts safer.)
New Concept: Command Substitution
You can capture the output of a command and use it as a string by wrapping it in $(...).
Try running this in your terminal right now: echo "I am $(whoami)"
Exploring Man Pages
Whenever you encounter an unfamiliar command or want to see all available options, the built-in manual is your first stop:
man date
man echo
man chmod
Each manual page is divided into sections: NAME, SYNOPSIS,
DESCRIPTION, and OPTIONS. Navigate with the arrow keys, search
with /keyword (then n for next match), and quit with q.
Try man date now to browse all available format specifiers — that’s
how you’d discover that +%A prints the full weekday name, +%H:%M
gives the time, and dozens of other options exist.
Your task
Add three commands to morning.sh:
- Print the literal string “Good morning!” using
echo. - Print “Today is “ followed by the current day. (Hint: the command
date +%Aoutputs the day of the week. Use command substitution!) - Print “You are logged in as: “ followed by your username. (Hint: use the
whoamicommand).
Then save (Ctrl+S / Cmd+S) and run in the terminal:
chmod +x morning.sh
./morning.sh
Breaking it down:
chmod +xgrants execute permission. Linux requires this explicit step before running a file as a program — a deliberate security feature so files don’t accidentally become executable../morning.sh— the./prefix means “look in the current directory.” The shell only searches directories listed in$PATHfor commands; your local folder is not in$PATHby default.$(date +%A)is command substitution: the shell runsdate +%Afirst, captures its output, and injects the result into your string. Any command can go inside$(...)— this is one of Bash’s most useful features.
#!/bin/bash
set -e
Hello, Shell! — Knowledge Check
Min. score: 80%
1. What is the purpose of the shebang line (#!/bin/bash) at the top of a shell script?
The shebang (#! followed by an interpreter path) is read by the OS kernel when you run a file. It tells the kernel to execute the file using the specified interpreter (here, /bin/bash). Without it, the OS may guess the wrong interpreter.
2. What does set -e do in a shell script?
By default, Bash continues executing even after a command fails. set -e (exit on error) exits the script when a command fails, preventing a cascade of confusing failures. We’ll cover its edge cases in later steps.
3. Which statements about $(...) command substitution are true? (Select all that apply)
(select all that apply)
Command substitution $(cmd) runs cmd and injects its stdout into the surrounding expression. It can be nested arbitrarily and does NOT require eval. It is one of Bash’s most powerful and widely-used features.
Navigating the Filesystem
Before you can automate tasks with scripts, you need to move around the filesystem confidently. In a GUI you click folders; in the shell you type commands. Let’s build muscle memory for the essential ones.
Where am I? What’s here?
pwd # Print Working Directory — your current location
ls # List what's in the current directory
ls -l # Long format — shows permissions, size, dates
Predict: Run ls now. You should see morning.sh from the
previous step. Now run ls -a. What extra entries appear?
Commit to your prediction, then run it. The . and .. entries
are special: . is the current directory, .. is the parent.
Files starting with . are “hidden” — ls skips them by default,
but ls -a shows everything.
Moving around with cd
cd /tmp # go to an absolute path
pwd # confirm you moved
cd .. # go up one level (to /)
pwd
cd ~ # go to your home directory (shortcut for $HOME)
pwd
Try each command above. Notice that cd with no output is normal —
it silently changes your location. Use pwd to confirm.
Important: Now return to the tutorial working directory:
cd /tutorial
Creating structure with mkdir
mkdir testdir # create one directory
Predict: Now try mkdir testdir/a/b — what happens?
The parent testdir/a/ doesn’t exist yet.
Try it and see — then use the fix:
mkdir -p testdir/a/b # -p creates parents too
The -p flag creates all missing parent directories at once.
Without it, mkdir requires every parent to already exist.
Clean up the test directory before moving on: rm -r testdir
Copying with cp
cp duplicates files. The original stays in place.
cp notes.txt notes_backup.txt # copy a file (try it!)
Predict: What happens if you try to copy a directory without any flags? Run:
mkdir temp_demo
cp temp_demo /tmp/backup
Will it (a) copy the whole directory, (b) copy just the name, or (c) fail with an error?
Try it — then read on. You need cp -r (recursive) to copy a
directory and everything inside it. Clean up: rm -r temp_demo
Moving and renaming with mv
mv does double duty — it moves and renames:
mv notes_backup.txt notes_copy.txt # rename (try it!)
ls # notes_backup.txt is gone,
# notes_copy.txt appeared
Unlike cp, mv works on directories without needing -r — it
just updates the path, it doesn’t copy data.
Removing with rm
rm notes_copy.txt # remove the copy we just made (no undo!)
rm -r directory/ # remove a directory and ALL its contents
rmdir empty_dir/ # remove ONLY if the directory is empty
Try the first command — notes_copy.txt from the mv example is
now gone. The other two are syntax references for the task below.
Predict: After building the project below, try running
rm myproject/ — without the -r flag — on a directory that
contains files. Will it (a) delete everything, (b) delete just
the directory, or (c) refuse with an error?
Try it and see. The shell protects you: without -r, rm refuses
to touch directories. This is intentional.
Your task — Build a project skeleton
Use the commands you just learned to create this directory structure
and manipulate files within it. We’ve provided notes.txt and data.csv
as starting materials.
- Create the directory tree:
myproject/src/,myproject/docs/,myproject/tests/(Hint:mkdir -pcan do this in one command) - Copy
notes.txtintomyproject/docs/ - Move
data.csvintomyproject/src/and rename it toinput.csv - Copy
morning.shintomyproject/src/as a backup - Create an empty file
myproject/tests/test_placeholder.txt(Hint:touchcreates empty files) - Remove the now-empty
myproject/tests/test_placeholder.txt - Verify your work:
ls -R myproject(the-Rflag lists recursively)
Project Notes ============= - Set up directory structure - Process log files - Write monitoring script
timestamp,level,message 08:12:01,INFO,server started 08:15:45,ERROR,request failed 08:18:33,ERROR,timeout
Navigating the Filesystem — Knowledge Check
Min. score: 80%
1. You run mkdir projects/backend/api but the projects/ directory doesn’t exist yet. What happens?
Without -p, mkdir requires all parent directories to already exist. If they don’t, it fails. The -p (parents) flag creates the entire chain of directories at once.
2. You run cp mydir /tmp/backup where mydir is a directory containing several files. What happens?
cp refuses to copy directories without the -r (recursive) flag. This is a safety feature — copying a large directory tree could be expensive, so the shell requires you to be explicit. mv, by contrast, works on directories without -r because moving just updates a path entry.
3. What is the difference between cp file.txt dir/ and mv file.txt dir/?
cp (copy) creates a second copy of the file — the original remains untouched. mv (move) relocates the file — it disappears from its original location. mv also doubles as a rename command when source and destination are in the same directory.
4. [Interleaved: Revisit Hello, Shell!] After running chmod +x morning.sh && ./morning.sh, you move the script: mv morning.sh scripts/morning.sh. Can you still run it with ./morning.sh?
./morning.sh means ‘run the file named morning.sh in the current directory.’ After mv moves it to scripts/, the file no longer exists at ./morning.sh. The execute permission does travel with the file (it’s a file attribute, not a path attribute), so ./scripts/morning.sh would work. This reinforces what ./ means from Hello, Shell!.
Pipes — Connecting Commands
The pipe operator | is one of the most powerful ideas in Unix.
It connects programs so that the output of one becomes the input of
the next, letting you build data-processing pipelines from small,
single-purpose tools. Data flows through memory from one process to
the next — no intermediate files needed.
But before you connect tools, you need to know what each one does on its own. First, explore each tool individually — then we’ll combine them with pipes.
Part 1: Meet your tools (one at a time)
wc -l — count lines of input
wc -l < /etc/hosts # how many lines are in /etc/hosts?
grep PATTERN file — print only lines that match a pattern
grep "WARN" server_log.txt # show only warning lines
sort — sort lines alphabetically; add -n for numeric order,
-r to reverse
echo -e "banana\napple\ncherry" | sort # → apple, banana, cherry
uniq -c — collapse consecutive duplicate lines and prefix each
with its count (always sort first so duplicates are adjacent)
echo -e "cat\ncat\ndog" | uniq -c # → 2 cat 1 dog
cut -d' ' -f<n> — extract the n-th space-separated field
cut -d' ' -f2 server_log.txt # extract the message type on each line
head -n — show only the first n lines
head -5 server_log.txt # the first 5 log entries
Explore the data
A file called server_log.txt is provided. Browse it first:
cat server_log.txt
Now try each tool individually on the log file. Run each command in the terminal and observe what it does:
grep "ERROR" server_log.txt # only ERROR lines
wc -l < server_log.txt # total line count
cut -d' ' -f2 server_log.txt # just the message types
head -3 server_log.txt # first 3 lines only
Tool isolation exercises
Save the result of each single tool to a file:
- grep practice: Use
grepto find all lines containing"WARN". Save togrep_result.txt. - cut practice: Use
cutto extract the second field (the message types: INFO, WARN, ERROR). Save tocut_result.txt. - head practice: Use
headto show only the first 3 lines of the log. Save tohead_result.txt.
Part 2: Building pipelines
Now that you know what each tool does alone, let’s connect them.
The pipe | takes the stdout of the left command and feeds
it directly into the stdin of the right command:
grep "ERROR" server_log.txt | wc -l # count ERROR lines
No intermediate files — data flows through memory. You can chain as many commands as you need.
Redirection connects commands to files:
grep "INFO" server_log.txt > info_only.txt # create/overwrite
echo "extra line" >> info_only.txt # append (safe)
wc -l < info_only.txt # read from file
Where do errors go? (stderr)
Every program has two output streams: stdout (normal output, file descriptor 1) and stderr (error messages, file descriptor 2). By default both appear on your terminal, which makes them look the same — but they are separate streams that can be redirected independently.
Try this sequence — but predict before you run each step:
Step A: Run a command that produces both normal output AND an error:
ls server_log.txt no_such_file.txt
You should see both a successful listing and an error message on your terminal.
Step B — Predict first! If you redirect stdout to a file with >, what
happens to the error message? Will it (a) go into the file, (b) still
appear on your terminal, or (c) disappear entirely?
Commit to your answer, then run:
ls server_log.txt no_such_file.txt > ls_out.txt
Were you right? If the error still appeared on screen, that’s the key
insight: > only captures stdout. The error traveled on a completely
separate stream.
Step C: Now redirect stderr separately:
ls server_log.txt no_such_file.txt > ls_out.txt 2> ls_err.txt
cat ls_out.txt # the successful listing
cat ls_err.txt # just the error message
Key insight: > only captures stdout. Errors travel on
stderr (2>), which is why they “leak through” regular
redirection.
Note: The tests below check that
ls_out.txtandls_err.txtexist with the expected content. Make sure you actually ran the commands from Steps B and C above!
Pipeline exercises
For each question, build a pipeline and save the result to the named
file using >. The tests below will check every file.
Tip:
wc -l server_log.txtprints15 server_log.txt(count + filename). To get just the number, redirect:wc -l < server_log.txtprints only15. Use the redirect form when saving counts to files.
- Count total lines: Feed
server_log.txtintowc -l. Save toline_count.txt. - Filter errors: Print only lines containing “ERROR”.
Save to
errors_only.txt. - Count errors: Pipe
grep "ERROR" server_log.txtintowc -l. Save toerror_count.txt. - Extract timestamps: Extract just the first field (the timestamps).
Save to
timestamps.txt. - Top message types: Find the 2 most frequent message types.
(Build step by step: extract field 2 → sort → count duplicates →
sort by count descending → top 2)
Save to
top_message_types.txt.
08:12:01 INFO server started on port 8080 08:12:03 INFO database connection established 08:14:22 WARN high memory usage detected (82%) 08:15:45 ERROR failed to process request /api/users 08:16:01 INFO request completed in 230ms 08:18:33 ERROR database timeout after 30s 08:19:02 WARN disk usage above threshold (91%) 08:20:15 INFO cache refreshed successfully 08:22:47 ERROR connection refused by upstream service 08:23:01 INFO retry succeeded for /api/users 08:25:00 INFO scheduled backup completed 08:27:12 WARN deprecated API endpoint called: /v1/legacy 08:30:00 INFO health check passed 08:31:44 ERROR out of memory on worker-3 08:32:01 INFO worker-3 restarted
Pipes — Knowledge Check
Min. score: 80%
1. [Interleaved: Revisit Hello, Shell!] A script starts with #!/bin/bash and set -e. The first command is cd /nonexistent. What happens?
set -e exits the script when any command returns a non-zero exit code. Since cd /nonexistent fails, the script stops immediately — which is exactly the safety net behavior we learned in Hello, Shell!.
2. In the pipeline grep 'ERROR' server_log.txt | wc -l, what does the | operator do?
The pipe | connects the stdout of the left command directly to the stdin of the right command, through memory. No intermediate file is created. This is the Unix philosophy: compose small, single-purpose tools into powerful pipelines.
3. What is the difference between > and >> for output redirection?
> creates or overwrites the file without warning — existing content is lost. >> appends new content after existing content. Always prefer >> when preserving existing data matters.
4. What does grep 'WARN' server_log.txt | head -n 3 | sort -r do?
grep 'WARN' server_log.txt searches the file from top to bottom and streams out every line containing the word ‘WARN’. | head -n 3 acts as a gatekeeper. It accepts the first 3 lines it receives from grep and then immediately closes the gate, discarding the rest of the matches. | sort -r receives only those 3 lines. It sorts those specific three lines in reverse alphabetical order and prints the final result to your screen.
5. You run ./script.sh > output.txt but error messages still appear on your terminal. Why?
Programs have two separate output streams: stdout (file descriptor 1) and stderr (file descriptor 2). The > operator only redirects stdout. To capture stderr, use 2>. To capture both to one file: > file.txt 2>&1.
6. The pipeline cut -d' ' -f2 server_log.txt | sort | uniq -c | sort -rn chains four small tools. Which principle does this best illustrate?
Each tool in the pipeline does one thing well: cut extracts fields, sort orders lines, uniq collapses duplicates, sort -rn ranks by count. They work together by passing text through pipes. Text is the universal interface that lets these tools compose freely — this is the Unix Philosophy in action.
Variables & The Quoting Trap
Variables store values for reuse. In Bash, you assign with = and
read with $.
The spaces rule — easy to break, hard to debug
color="blue" # correct
color = "blue" # WRONG — shell sees three words: "color", "=", "blue"
There must be no spaces around =. The shell interprets color = "blue" as running a command named color with arguments = and blue.
The quoting problem
When you write $variable, the shell replaces it with the value —
then word-splits the result on any characters in $IFS (the
Internal Field Separator, which defaults to space, tab, and newline).
This causes chaos when values contain spaces:
file="my report.txt"
wc -l $file # shell splits into: wc -l my report.txt (TWO args!)
wc -l "$file" # correct: one argument, treated as a unit
Rule: always double-quote your variables unless you have a specific reason not to.
See the bug (Predict → Debug)
buggy.sh has a deliberate bug related to what you just learned.
Before running it, open buggy.sh in the editor and read it carefully.
The variable filename is set to "my report.txt" — a value with a space.
Look at every line that uses $filename. Can you spot which line will
break? Predict the exact error message you’ll see, then run:
bash buggy.sh
Was your prediction correct? The error message tells you exactly what Bash tried to do — and why it failed.
Fix it:
- Diagnose why
wc -lis throwing an error based on what you just learned. - Fix the syntax and run the script again.
Build your own
Open inventory.sh and write a script from scratch that:
- Declares a variable for a project name and another for a version number.
- Uses command substitution
$(...)to dynamically count the number of.shfiles in the current directory and save it to a variable. (Hint: tryls *.sh | wc -l. This works for simple filenames; production scripts usefindinstead.) - Uses
echoto print a single string combining all three variables, e.g.,Project: mytools v1.0 — 5 scripts found
#!/bin/bash
set -e
# This script has a bug — can you find it?
filename="my report.txt"
echo "creating a test file..."
echo "important data" > "$filename"
# Something below is broken — can you find it?
line_count=$(wc -l $filename)
echo "Line count: $line_count"
rm "$filename"
#!/bin/bash
set -e
# Create variables for a project name and version, then count .sh files
Variables & Quoting — Knowledge Check
Min. score: 80%
1. [Interleaved: Revisit Pipes] You want to count only lines containing ERROR in server.log and save that number to a variable. Which is correct?
Command substitution $(...) captures a command’s stdout into a variable. The | pipe connects two commands’ stdin/stdout — it can’t assign to a variable by itself. ${...} is for variable/parameter expansion, not command execution.
2. Which variable assignment is syntactically correct in Bash?
Bash requires no spaces around = in variable assignment.
3. A variable dir contains the value my documents. What happens when you run ls $dir (unquoted)?
Without quotes, Bash word-splits the expanded value on spaces. ls $dir becomes ls my documents — two arguments. The fix is always "$dir".
4. [Interleaved: Revisit Hello, Shell!] What does the #!/bin/bash line at the very top of a script tell the Operating System?
As we saw in Hello, Shell!, the shebang (#!) followed by a path tells the OS which program to use to run the script. Without it, the OS might guess the wrong interpreter.
Conditionals — Making Decisions
Scripts need to react to different situations. Bash’s if statement
runs commands conditionally based on whether a test succeeds.
Syntax
if [ condition ]; then
# runs when condition is true
elif [ other_condition ]; then
# runs when first is false but this is true
else
# runs when all conditions are false
fi
Why the spaces inside [ ] are mandatory
[ is a shell builtin command (a synonym for test) — not special
syntax. Like any command, its arguments must be separated by spaces:
[ -f "$file" ] # correct: "[" receives "-f" and "$file" as args
[-f "$file"] # WRONG: shell tries to run a command named "[-f"
You can confirm this with type -a [, which shows both the builtin
and the external /usr/bin/[ binary. Bash always uses the builtin.
Common tests (Your Toolbox)
| Test | Meaning |
|---|---|
-f path |
Path exists and is a regular file |
-z "$var" |
String is empty (zero length) |
"$a" = "$b" |
Strings are equal |
$x -eq $y |
Integers are equal |
$x -gt $y |
Integer greater than |
! condition |
Logical NOT |
Important: use -eq, -lt, -gt for numbers; use = and !=
for strings. Mixing them gives wrong results silently!
Pro Tip: [[ ]] vs [ ]
While [ ] is the standard POSIX way, Bash also provides [[ ]]. It is more powerful because:
- It doesn’t require quoting variables to prevent word splitting.
- It supports Regex matching with
=~. - It’s less prone to subtle syntax errors.
For Bash scripts,
[[ ]]is generally preferred.
Discover a trap first
Before we start, try this experiment. Predict what happens, then run:
grep -c "NONEXISTENT" server_log.txt
echo "Did this print?"
Both lines should run fine. Now try it with set -e active:
bash -c 'set -e; grep -c "NONEXISTENT" server_log.txt; echo "Did this print?"'
What happened? grep -c found zero matches and returned exit
code 1. With set -e, that non-zero exit code killed the entire
script — echo never ran. But this isn’t really an error; it’s
just “no matches found.” This is a common trap: grep treats “no
matches” as failure.
The fix is || true — it means “if the command fails, succeed
anyway.” The skeleton below uses this idiom. We’ll cover || fully
in a later step.
Your task
We are providing a skeleton file health_check.sh. To help you structure your thinking, we’ve left blanks (_____) where the tests should go. Look at the “Common tests” toolbox above to fill them in logically:
- First blank: We want to exit if the file does not exist. How do you negate a file existence check?
- Second blank: We want to mark CRITICAL if
error_countis greater than 3. - Third blank: We want to mark WARNING if
error_countis greater than 0.
chmod +x health_check.sh
./health_check.sh server_log.txt # should report CRITICAL (4 errors)
./health_check.sh nonexistent.txt # should print an error and exit 1
#!/bin/bash
set -e
file="${1:-server_log.txt}"
# Step 1: Check if the file exists
if [ _____ ]; then
echo "Error: $file not found" >&2
exit 1
fi
# Step 2: Count ERROR lines
# Note: grep -c exits with code 1 when no matches are found.
# The "|| true" prevents set -e from killing the script in that case.
error_count=$(grep -c "ERROR" "$file" || true)
# Step 3: Decide severity
if [ _____ ]; then
echo "CRITICAL: $error_count errors found"
elif [ _____ ]; then
echo "WARNING: $error_count errors found"
else
echo "OK: no errors found"
fi
Conditionals — Knowledge Check
Min. score: 80%
1. Inside a Bash if statement, you want to check whether server.log has more than 100 lines. Which is syntactically correct?
$(wc -l < server.log) uses command substitution and redirection to capture the line count as a plain integer for arithmetic comparison. Using $(cat server.log) would capture the file’s entire content, not a count. The grep -c and pipe variants are syntactically invalid here — [ is a command and its arguments must follow command syntax.
2. Why are spaces required inside [ ] test brackets, like [ -f "$file" ]?
[ is a shell builtin command (synonym for test) — not special syntax. Like any command, arguments must be separated by spaces. You can verify with type -a [, which shows both the builtin and the external /usr/bin/[ binary; Bash uses the builtin by default.
3. You want to compare two integer variables $count and $max in a Bash conditional. Which test is correct?
Bash uses -eq, -lt, -gt, -le, -ge, -ne for integer comparisons. The = and == operators do string comparison.
4. [Interleaved: Revisit Pipes] Which operator is used to append output to an existing file without overwriting it?
As we learned in the Pipes step, > overwrites a file, while >> appends to it. Both are forms of output redirection.
Loops — Repeating Work
Loops eliminate repetition. Let’s look at iterating over globs (file expansions).
for f in *.sh; do # expands to all matching filenames
echo "Found: $f"
done
Accumulating totals
A common pattern is keeping running counts across loop iterations using arithmetic expansion $(( ... )):
passed=0
# ... inside loop:
passed=$((passed + 1))
Your task
Open batch_check.sh. We’ve provided the skeleton — the loop
structure, counters, and summary line are already in place. Your
job is to fill in the body of the loop (the three blanks):
- First blank: Capture the first line of the current file
into the variable
first. (Hint:head -1 "$f"prints the first line. Wrap it in$(...)to capture the output.) - Second blank: Test whether
firstequals exactly#!/bin/bash. (Hint: use=for string comparison inside[ ]. Remember to quote both sides!) - Third blank: The
elsebranch — print a fail message and increment thefailedcounter. (Mirror the structure of the pass branch above it.)
Before running, predict: How many .sh files are in the directory
right now? Which ones have a proper #!/bin/bash shebang and which
don’t? (Hint: look at the files created in earlier steps — including
no_shebang.sh that we’ve provided.) Write down your expected
pass/fail counts, then run:
chmod +x batch_check.sh
./batch_check.sh
Does the output match your prediction? If not, check which files surprised you — that’s where the learning happens.
#!/bin/bash
set -e
passed=0
failed=0
for f in *.sh; do
# Blank 1: Capture the first line of "$f" into variable "first"
first=_____
# Blank 2: Check if "first" equals exactly "#!/bin/bash"
if [ _____ ]; then
echo "pass $f"
passed=$((passed + 1))
else
# Blank 3: Print a fail message and increment "failed"
_____
_____
fi
done
total=$((passed + failed))
echo "Checked $total files: $passed passed, $failed failed"
set -e
Loops — Knowledge Check
Min. score: 80%
1. You write for f in *.log; do wc -l $f; done. One of the log files is named error log.txt (with a space). What happens when the loop processes it?
Without quotes, $f undergoes word-splitting on IFS characters (space, tab, newline). wc - l error log.txt becomes two arguments. The fix is always "$f". This is the exact same quoting rule from the Variables step — it applies everywhere variables are used.
2. In a for f in *.sh loop, when does the shell substitute *.sh?
Shell glob expansion happens before the loop executes. The shell replaces *.sh with the list of matching filenames, and the loop iterates over that fixed list.
3. What does $((counter + 1)) do in Bash?
$(( )) is arithmetic expansion — Bash evaluates the expression and substitutes the result as a string. The expression $((counter + 1)) does not change counter; you must assign it back: counter=$((counter + 1)). Expressions that use assignment operators like $((counter++)) or $((counter += 1)) do modify the variable in place as a side effect, but for the simple a + b form shown here, you always assign back.
4. [Interleaved: Revisit Variables & Quoting] Inside a loop, you use wc -l $f. If a file is named data 2024.txt, how does Bash interpret the unquoted $f?
As we learned in the Variables step, unquoted variables undergo word-splitting. The space in the filename breaks it into two arguments, likely causing wc to fail. Always use "$f" to treat the value as a single unit.
5. [Interleaved: Revisit Navigating the Filesystem] Your loop creates a directory for each .sh file: mkdir results/$f. But results/ doesn’t exist yet. What happens?
As we learned in the Filesystem step, mkdir requires all parent directories to already exist. Without -p, it fails. The fix is either mkdir -p results/"$f" or creating results/ before the loop.
Arguments & Special Variables
When you run ./script.sh one two three, the shell sets special
variables automatically:
| Variable | Contains |
|---|---|
$0 |
The script’s own name (great for usage messages) |
$1, $2, … |
Positional arguments |
$# |
Total number of arguments passed |
$@ |
All positional arguments (properly word-safe only when quoted as "$@") |
Looping over arguments
"$@" expands to all arguments as separate, properly-quoted words. You can loop over them like this:
for f in "$@"; do
echo "Processing: $f"
done
Your task
Now we remove the training wheels. Write file_info.sh completely from scratch.
Requirements:
- Input Validation: Check if the number of arguments (
$#) is equal to 0. If it is, print a usage message (e.g.,echo "Usage: $0 <file1>...") andexit 1. - Iteration: Loop over all arguments passed to the script using a
forloop and"$@". - Conditionals: Inside the loop, for each file:
- Check if it is a directory (
-d). If so, print<name>: directory. - Otherwise, check if the file does NOT exist (
! -f). If so, print<name>: not found. - Else (it’s a real file), use
wc -l < "$f"to count the lines and print<name>: <N> lines.
- Check if it is a directory (
Tip: Think about the flow of data. Combine what you learned in the Conditionals step with the for loop shown above.
Test your script with:
chmod +x file_info.sh
./file_info.sh server_log.txt morning.sh /tmp nope.txt
#!/bin/bash
set -e
# Write your code below!
Arguments & Special Variables — Knowledge Check
Min. score: 80%
1. [Interleaved: Revisit Conditionals] Your script receives a filename as $1. You want to check if the file exists before processing it. Which conditional is correct?
-f tests whether a path exists and is a regular file. "$1" must be quoted to safely handle filenames with spaces. Together [ -f "$1" ] is the standard idiom — applying the file-test knowledge from the Conditionals step to incoming script arguments.
2. What does $# contain when a script is called as ./deploy.sh app v1.2?
$# is the count of positional arguments, not counting the script name ($0). For ./deploy.sh app v1.2, $# is 2.
3. Why use "$@" (quoted) instead of $@ (unquoted) when looping over arguments?
Without quotes, $@ is subject to word splitting. "$@" preserves each argument as a single unit, regardless of spaces.
Functions — Reusable Building Blocks
Functions let you name a block of code and call it anywhere, just like external commands.
greet() {
local name="$1"
echo "Hello, ${name}!"
}
greet "engineer" # → Hello, engineer!
Rule of Thumb: Always use local for variables declared inside a function so they don’t leak out and overwrite global variables.
Functions receive $1, $2, etc. independently of the script’s own arguments.
Return Values
Functions exit with a numeric status code (0–255) set by return.
By convention, return 0 means success and any non-zero value means
failure — which lets you use functions directly in if statements.
You can return specific non-zero codes (e.g., return 2 for bad
arguments) to give callers richer information. To return data
(strings, numbers), use echo inside the function and capture it
outside with $(...) — return only carries an exit code, not data.
Your task
Write toolkit.sh and create these three functions:
to_upper: Echoes its argument converted to uppercase. (Tool hint:echo "$1" | tr '[:lower:]' '[:upper:]')file_ext: Echoes the file extension of its argument. (Tool hint:echo "${1##*.}"strips everything up to the last dot)is_number: Checks if its argument is a valid integer using the Regex test[[ "$1" =~ ^-?[0-9]+$ ]]. If true,return 0. Else,return 1.
Write a small script below the functions to test them, ensuring they work!
Watch out for
set -e:is_numberreturns 1 (failure) for non-numbers. If you callis_number abcas a bare command,set -ewill kill your script. Always test it inside anifor with&&/||— e.g.,if is_number "$val"; then ....
#!/bin/bash
set -e
Functions — Knowledge Check
Min. score: 80%
1. [Interleaved: Revisit Loops] A function process_all is called as process_all file1.txt "my report.txt". Inside, it runs for f in $@; do. How many iterations does the loop perform?
Without quotes, $@ undergoes word-splitting, breaking my report.txt into my and report.txt. The fix is "$@" — the same quoting rule from the Variables step applies everywhere, including inside functions. Always write for f in "$@"; do.
2. What problem does the local keyword solve inside a Bash function?
Without local, any variable set inside a function modifies the global scope. local constrains the variable to the function’s scope.
3. A function count_words should return a number to the caller. Which is the correct Bash pattern?
In Bash, return only carries exit codes (0–255). To pass data back, the function should echo the value and the caller captures it with $(...).
Case Statements & Exit Codes
case — readable multi-way branching
When you need to check one variable against many possible values,
case is cleaner than if/elif:
case "$input" in
start) echo "Starting..." ;;
stop) echo "Stopping..." ;;
*) echo "Unknown: $input" ;;
esac
Exit codes: the language of success and failure
Every command exits with a number. 0 always means success; any other value means failure.
exit 0 # success
exit 1 # general error
exit 2 # misuse / wrong arguments
Conditional chaining: && and ||
Because every command returns an exit code, you can chain
commands without a full if/then/fi block:
mkdir output && echo "Directory created" # runs echo only if mkdir succeeds
cd /target || exit 1 # exits script if cd fails
&&(AND): The right-hand command runs only if the left-hand command succeeds (exit code 0).||(OR): The right-hand command runs only if the left-hand command fails (non-zero exit code).
This is widely used in professional scripts for concise error
handling. Note: set -e does not trigger for commands that
are not the last in a &&/|| chain — those are treated as
intentional control flow.
Your task
Write service.sh — a simulated service controller.
Use a case statement to check the first argument $1.
Requirements:
- If
start— create a PID file usingtouch /tmp/my_service.pid && echo "Starting service...", exit 0. - If
stop— remove the PID file usingrm /tmp/my_service.pid 2>/dev/null || true, printStopping service..., exit 0. - If
status— check if/tmp/my_service.pidexists (-f). If yes: printService is running, exit 0. If no: printService is stopped, exit 1. - Anything else (or empty) — print usage instructions to stderr (
>&2) and exit 2.
#!/bin/bash
set -e
Case Statements & Exit Codes — Knowledge Check
Min. score: 80%
1. What does cd /project || exit 1 do?
|| (OR) runs the right-hand command only if the left-hand command fails. If cd /project succeeds, Bash skips exit 1 entirely. If it fails, the script exits. The counterpart && (AND) runs the right side only on success: mkdir out && echo "Done" prints only if mkdir worked.
2. In a Bash case statement, what does the * pattern in the last branch do?
* in a case branch acts as a catch-all default, matching any value that didn’t match the earlier patterns.
3. What is the universal meaning of exit code 0 in Unix/Linux?
Exit code 0 always means success in Unix. Non-zero values indicate failure. This contrasts with how most languages evaluate boolean truthiness in code (where 0 is false and non-zero is true), even though languages like C and Java also use return 0 / exit(0) to indicate process success to the OS.
4. [Interleaved: Revisit Arguments] Which special variable contains the number of arguments passed to the script?
As we practiced in the Arguments step, $# gives you the count of arguments, which is essential for input validation before your script starts its work.
Build a Log Monitor
Time to combine everything into a real tool. This is a retrieval practice exercise: you have all the knowledge, now you must retrieve it from memory and synthesize it.
Before you write any code, look at server_log.txt one more time
and predict: How many ERROR, WARN, and INFO lines are there? What
severity status should your script report? What exit code should it
return? Write your predictions down — you’ll check them against your
script’s actual output.
Challenge
Write monitor.sh — a log-monitoring tool that analyzes
server_log.txt and produces a complete status report.
Requirements:
- Accept an optional filename argument. If not provided, default to
server_log.txt. - Validate that the file exists; if not, print to stderr and exit.
- Print a header:
=== Log Monitor Report === - Summary section — write a function called
count_by_levelthat takes a log level (e.g., “ERROR”) and the filename, and echoes the count. Use it to report:- Total entries
- Count of
ERROR,WARN, andINFOentries
- Error details: Loop over ERROR lines and print each one.
(Remember:
grep -cexits with code 1 when there are zero matches. Use|| trueto preventset -efrom killing your script — just like in the health_check step.) - Severity assessment: Use a
casestatement on the error count:0→ printStatus: HEALTHY,1|2|3→Status: WARNING,*(anything else) →Status: CRITICAL. (Note:caseuses glob patterns, not numeric ranges. Use|to match multiple values:1|2|3)matches 1, 2, or 3.) - Exit with code 0 if no errors are found, and code 1 if errors are present.
Design Approach
Don’t just write code immediately. In learning science, planning reduces cognitive load. Sketch your script out in comments first:
# 1. Handle arguments and default file
# 2. Check if file exists
# 3. Print Header
# 4. Calculate counts using grep/wc
# ...
Once your structure is clear, write the bash code.
When NOT to use Shell Scripting
Shell scripting is powerful for text processing and automation, but it has real limits. Knowing when not to use a tool is as important as knowing how to use it. Switch to Python (or another general-purpose language) when:
- You need complex data structures (dictionaries, nested lists, objects) — Bash only has strings and flat arrays.
- Robust error handling is critical — Bash’s
set -ehas many subtle exceptions that can bite you. - Your script exceeds ~100 lines — maintainability degrades quickly without functions, types, and proper scoping.
- You need cross-platform support — Bash behaves differently on macOS vs Linux, and isn’t available on Windows by default.
Bash is a glue language: brilliant for orchestrating other programs and processing text streams. Use it for that, and reach for a real programming language when the task outgrows it.
#!/bin/bash
set -e
Shell Scripting Mastery — Final Comprehensive Review
Min. score: 80%
1. [Interleaved: Revisit Hello, Shell!] Scenario: A developer wrote the following deployment script but forgot to include set -e at the top:
#!/bin/bash
cd /var/www/production_app
git pull origin main
rm -rf temp_cache/*
systemctl restart app
cd command fails because the directory was recently renamed. What happens next?
Without set -e, Bash continues executing every line regardless of failures. The cd fails silently, and the script proceeds in whatever directory it was already in — potentially running git pull and rm -rf in the wrong location. This is exactly why set -e is a critical safety net.
2. [Interleaved: Revisit Pipes] Scenario: You are given a massive log file, server.log. You need to find out how many times the user “admin” triggered a “WARN” event. Which pipeline correctly filters and counts these logs?
Chaining grep | grep | wc -l pipes each command’s stdout directly into the next command’s stdin — the standard way to build multi-stage filters in Bash.
3. Scenario: A junior developer writes the following script:
#!/bin/bash
DIR = "/tmp/build"
line 2: DIR: command not found. Why does Bash produce this specific error?
Bash parses each line as: Command → Argument 1 → Argument 2. The spaces around = make Bash see three words: DIR (the command to run), = (first argument), and "/tmp/build" (second argument). Since no program named DIR exists, Bash reports ‘command not found.’ The fix is DIR="/tmp/build" with no spaces.
4. Scenario: A deployment script runs the following logic to check for a required environment file:
if [ -d ".env" ]; then
echo "Environment file loaded."
else
echo "Fatal: Missing .env file!"
exit 1
fi
.env file exists as a standard text file in the same directory as the script, yet the script exits with the “Fatal” message. Why?
The -d flag tests if a path is a directory, not a regular file. The correct test for a regular file is -f. Hidden files (starting with .) are perfectly visible to [ / test — the -a flag in ls is unrelated to Bash conditionals.
5. Scenario: Consider the following loop running in a directory that contains exactly one file named 01 Financial Report.csv:
for f in *.csv; do
wc -l $f
done
$f is unquoted inside the loop body, what is the exact sequence of “files” the wc -l command will attempt to process?
Without quotes, Bash performs word-splitting on the expanded variable. It treats the spaces as delimiters, passing three separate arguments to wc.
6. Scenario: A script deploy.sh requires exactly three arguments: environment, version, and region. A developer wrote this validation check:
if [ "$@" -ne 3 ]; then
echo "Error: Expected 3 arguments."
exit 1
fi
$@ expands to the argument values themselves (e.g., staging v2.1 us-west), not a count. Comparing a string to the integer 3 with -ne produces an error or wrong result. The correct variable for counting arguments is $#, which holds the numeric count.
7. Scenario: Trace the execution of the following script. What will the final echo statement print to the terminal?
target_dir="/var/www/html"
setup_temp() {
target_dir="/tmp/workspace"
}
setup_temp
echo "Deploying to $target_dir"
Bash variables are global by default — unlike C++ or Java, there is no block scoping. The function setup_temp overwrites the global target_dir. To prevent this, the function should declare local target_dir="/tmp/workspace" so the change stays inside the function.
8. Scenario: You are writing a script health_check.sh that checks database connectivity. If the database is unreachable, you need the CI/CD pipeline running the script to immediately halt. What is the standard Unix mechanism to communicate this failure back to the CI/CD environment?
Exit codes are the standard Unix mechanism for communicating success or failure to the calling environment (CI/CD, other scripts, make, etc.). exit 1 signals failure; exit 0 signals success. Printing to stdout/stderr is for human-readable messages — the pipeline does not parse those. return only works inside functions, not to terminate a script.
9. Scenario: Read the following script named start_server.sh:
#!/bin/bash
LOG_LEVEL="${1:-INFO}"
PORT="${2:-8080}"
echo "Starting on port $PORT with level $LOG_LEVEL"
./start_server.sh DEBUG, what will be printed to the terminal?
$1 receives DEBUG, so LOG_LEVEL is set to DEBUG. $2 is empty, so ${2:-8080} falls back to its default value 8080. The :- operator substitutes the default only when the variable is unset or empty — it does not cause an error.
10. [Interleaved: Revisit Case Statements & Exit Codes] Scenario: A deployment script contains this line:
cd /var/www/app && git pull && systemctl restart app
/var/www/app directory does not exist. What happens?
&& runs the next command only if the previous one succeeded (exit code 0). Since cd fails, Bash stops the chain immediately — git pull and systemctl restart are never executed. This is a safe pattern for critical operations where each step depends on the previous one.