Shell Scripting - Automating the Command Line

---

Dark Mode

Show Highlights

Audio Clips

Start here: If you are new to shell scripting, begin with the Interactive Shell Scripting Tutorial — hands-on exercises in a real Linux system. This article is a reference to deepen your understanding afterward.

If you have ever found yourself performing the same repetitive tasks on your computer—renaming batches of files, searching through massive text logs, or configuring system environments—then shell scripting is the magic wand you need. Shell scripting is the bedrock of system administration, software development workflows, and server management.

In this detailed educational article, we will explore the concepts, syntax, and power of shell scripting, specifically focusing on the most ubiquitous UNIX shell: Bash.

Basics

What is the Shell?

To understand shell scripting, you first need to understand the “shell”.

An operating system (like Linux, macOS, or Windows) acts as a middleman between the physical hardware of your computer and the software applications you want to run. It abstracts away the complex details of the hardware so developers can write functional software.

The kernel is the core of the operating system that interacts directly with the hardware. The shell, on the other hand, is a command-line interface (CLI) that serves as the primary gateway for users to interact with a computer’s operating system. While many modern users are accustomed to graphical user interfaces (GUIs), the shell is a program that specifically takes text-based user commands and passes them to the operating system to execute. In the context of this course, mastering the shell is like becoming a “wizard” who can construct and manipulate complex software systems simply by typing words.

Motivation: Why the Shell is Essential

As a software engineer, you need to be familiar with the ecosystem of tools that help you build software efficiently. The Linux ecosystem offers a vast array of specialized tools that allow you to write programs faster and debug log files by combining small, powerful commands. Understanding the shell increases your productivity in a professional environment and provides a foundation for learning other domain-specific scripting languages. Furthermore, the shell allows you to program directly on the operating system without the overhead of additional interpreters or heavy libraries.

The Unix Philosophy

The shell’s power is rooted in the Unix philosophy, which dictates:

1. Write programs that do one thing and do it well.
2. Write programs to work together.
3. Write programs to handle text streams, because that is a universal interface.

By treating data as a sequence of characters or bytes—similar to a conveyor belt rather than a truck—the shell allows parallel processing and the composition of complex behaviors from simple parts.

Essential UNIX Commands

Before writing scripts, you need to know the fundamental commands that you will be stringing together. These are the building blocks of any UNIX environment.

1. File Handling

These are the foundational tools for interacting with the POSIX filesystem:

• ls: List directory contents (files and other directories).
• cd: Change the current working directory (e.g., use .. to move to a parent folder).
• pwd: Print the name of the current/working directory so you don’t get lost.
• mkdir: Create a new directory.
• cp: Copy files. Use -r (recursive) to copy a directory and its contents.
• mv: Move or rename files and directories.
• rm: Remove (delete) files. Use -r to remove a directory and its contents recursively.
• rmdir: Remove empty directories (only works on empty ones).
• touch: Create an empty file or update timestamps.

2. Text Processing and Data Manipulation

Unix treats text streams as a universal interface, and these tools allow you to transform that data:

• cat: Concatenate and print files to standard output.
• grep: Search for patterns using regular expressions.
• sed: Stream editor for filtering and transforming text (commonly search-and-replace).
• tr: Translate or delete characters (e.g., changing case or removing digits).
• sort: Sort lines of text files alphabetically; add -n for numeric order, -r to reverse.
• uniq: Filter adjacent duplicate lines; the -c flag prefixes each line with its occurrence count. Because it only compares consecutive lines, you almost always pipe sort first so that duplicates are adjacent.
• wc: Word count (lines, words, characters).
• cut: Extract specific sections/fields from lines.
• comm: Compare two sorted files line by line.
• head / tail: Output the first or last part of files.
• awk: Advanced pattern scanning and processing language.

3. Permissions, Environment, and Documentation

These tools manage how your shell operates and how you access information:

• man: Access the manual pages for other commands. This is arguably the most useful command, providing built-in documentation for every other command in the system.
• chmod: Change file mode bits (permissions). Files in a Unix-like system have three primary types of permissions: read (r), write (w), and execute (x). For security reasons, the system requires an explicit execute permission because you do not want to accidentally run a file from an unknown source. Permissions are often read in “bits” for the owner (u), group (g), and others (o).
• which / type: Locate the binary or type for a command.
• export: Set environment variables. The PATH variable is especially important; it tells the shell which directories to search for executable programs. You can temporarily update it using export or make it permanent by adding the command to your ~/.bashrc or ~/.profile file.
• source / .: Execute commands from a file in the current shell environment.

4. System, Networking, and Build Tools

Tools used for remote work, debugging, and automating the construction process:

• ssh: Secure shell to connect to remote machines like SEASnet.
• scp: Securely copy files between hosts.
• wget / curl: Download files or data from the internet.
• make: Build automation tool that uses shell-like syntax to manage the incremental build process of complex software, ensuring that only changed files are recompiled.
• gcc / clang: C/C++ compilers.
• tar: Manipulate tape archives (compressing/decompressing).

The Power of I/O Redirection and Piping

The true power of the shell comes from connecting commands. Every shell program typically has three standard stream ports:

1. Standard Input (stdin / 0): Usually the keyboard.
2. Standard Output (stdout / 1): Usually the terminal screen.
3. Standard Error (stderr / 2): Where error messages go, also usually the terminal.

Redirection

You can redirect these streams using special operators:

• >: Redirects stdout to a file, overwriting it. (e.g., echo "Hello" > file.txt)
• >>: Redirects stdout to a file, appending to it without overwriting.
• <: Redirects stdin from a file. (e.g., cat < input.txt)
• 2>: Redirects stderr to a specific file to specifically log errors.
• 2>&1: Redirects stderr to the standard output stream. Note: order matters — command > file.txt 2>&1 sends both streams to the file, whereas command 2>&1 > file.txt only redirects stdout to the file while stderr still goes to the terminal.

Piping

The pipe operator | is the most powerful composition tool. It takes the stdout of the command on the left and sends it directly into the stdin for the command on the right.

Example: cat access.log | grep "ERROR" | wc -l This pipeline reads a log file, filters only the lines containing “ERROR”, and then counts how many lines there are.

Here Documents and Here Strings

Sometimes you need to feed a block of text directly into a command without creating a temporary file. A here document (<<) lets you embed multi-line input inline, up to a chosen delimiter:

cat <<EOF
Server: production
Version: 1.4.2
Status: running
EOF

The shell expands variables inside the block (just like double quotes). To suppress expansion, quote the delimiter: <<'EOF'.

A here string (<<<) feeds a single expanded string to a command’s standard input — a concise alternative to echo "text" | command:

grep "ERROR" <<< "08:15:45 ERROR failed to connect"

Process Substitution

Advanced shell users often utilize process substitution to treat the output of a command as a file. The syntax looks like <(command). For example, H < <(G) >> I allows you to refer to the standard output of command G as a file, redirect it into the standard input of H, and append the output to I.

Writing Your First Shell Script

When you find yourself typing the same commands repeatedly, you should create a shell script. A shell script is written in a plain text file (often ending in .sh) and contains a sequence of commands that the shell executes as a program.

Interpreted Nature

Unlike a compiled language like C++, which is compiled into machine code before execution, shell scripts are interpreted at runtime rather than ahead of time. This allows for rapid prototyping. Bash always reads at least one complete line of input, and reads all lines that make up a compound command (such as an if block or for loop) before executing any of them. This means a syntax error on a later line inside a multi-line compound block is caught before the block starts executing — but an error in a branch that is never reached at runtime may go unnoticed. Use bash -n script.sh to check for syntax errors without running the script.

The Shebang

Every script should start with a “shebang” (#!). This tells the operating system which interpreter should be used to run the script. For Bash scripts, the first line should be:

#!/bin/bash

Execution Permissions

By default, text files are not executable for security reasons. Execute permission is required only if you want to run the script directly as a command:

chmod +x myscript.sh
./myscript.sh

Alternatively, you can bypass the execute-permission requirement entirely by passing the file as an argument to the Bash interpreter directly — no chmod needed:

bash myscript.sh

You can also run a script’s commands within the current shell (inheriting and potentially modifying its environment) using source or the . builtin: source myscript.sh.

Debugging Scripts

When a script behaves unexpectedly, Bash has built-in tracing modes that let you see exactly what the shell is doing:

• bash -n script.sh: Reads the script and checks for syntax errors without executing any commands. Always run this first when a script refuses to start.
• bash -x script.sh (or set -x inside the script): Prints a trace of each command and its expanded arguments to stderr before executing it — indispensable for logic bugs. Each traced line is prefixed with +.
• bash -v script.sh (or set -v): Prints each line of input exactly as read, before expansion — useful for seeing the raw source being interpreted.

You can combine flags: bash -xv script.sh. To turn tracing on for only a section of a script, use set -x before that section and set +x after it.

Error Handling (set -e and Exit Status)

By default, a Bash script will continue executing even if a command fails. Every command returns a numerical code known as an Exit Status; 0 generally indicates success, while any non-zero value indicates an error or failure. Continuing after a failure can be dangerous and lead to unexpected behavior. To prevent this, you should typically include set -e at the top of your scripts:

#!/bin/bash
set -e

This tells the shell to exit immediately if any simple command fails, making your scripts safer and more predictable.

Syntax and Programming Constructs

Bash is a full-fledged programming language, but because it is an interpreted scripting language rather than a compiled language (like C++ or Java), its syntax and scoping rules are quite different.

5. Scripting Constructs

In our scripts, we also treat these keywords as “commands” for building logic:

• #! (Shebang): An OS-level interpreter directive on the first line of a script file — not a Bash keyword or command. When the OS executes the file, it reads #! and uses the rest of that line as the interpreter path. Within Bash itself, any line starting with # is simply a comment and is ignored.
• read: Read a line from standard input into a variable. Common flags: -p "prompt" displays a prompt on the same line, -s silently hides typed input (useful for passwords), and -n 1 returns after exactly one character instead of waiting for Enter.
• if / then / elif / else / fi: Conditional execution.
• for / do / done / while: Looping constructs.
• case / in / esac: Multi-way branching on a single value.
• local: Declare a variable scoped to the current function.
• return: Exit a function with a numeric status code.
• exit: Terminate the script with a specific status code.

Variables

You can assign values to variables without declaring a type. Note that there are no spaces around the equals sign in Bash.

NAME="Ada"
echo "Hello, $NAME"

Parameter Expansion — Default Values and String Manipulation

Beyond simple $VAR substitution, Bash supports a powerful set of parameter expansion operators that let you handle missing values and manipulate strings entirely within the shell, without spawning external tools.

Default values:

# Use "server_log.txt" if $1 is unset or empty
file="${1:-server_log.txt}"

# Use "anonymous" if $NAME is unset or empty, AND assign it
NAME="${NAME:=anonymous}"

String trimming — remove a pattern from the start (#) or end (%) of a value:

path="/home/user/project/main.sh"
filename="${path##*/}"    # removes longest prefix up to last /  → "main.sh"
noext="${filename%.*}"    # removes shortest suffix from last .  → "main"

The double form (## / %%) removes the longest match; the single form (# / %) removes the shortest.

Search and replace:

msg="Hello World World"
echo "${msg/World/Earth}"    # replaces first match  → "Hello Earth World"
echo "${msg//World/Earth}"   # replaces all matches  → "Hello Earth Earth"

Scope Differences

Unlike C++ or Java, Bash lacks strict block-level scoping (like {} blocks). Variables assigned anywhere in a script — including inside if statements and loops — remain accessible throughout the entire script’s global scope. There are, however, several important isolation boundaries:

• Function-level scoping: variables declared with the local builtin inside a Bash function are visible only to that function and its callees.
• Subshells: commands grouped with ( list ), command substitutions $(...), and background jobs run in a subshell — a copy of the shell environment. Any variable assignments made inside a subshell do not propagate back to the parent shell.
• Per-command environment: a variable assignment placed immediately before a simple command (e.g., VAR=value command) is only visible to that command for its duration, leaving the surrounding scope untouched.

Arithmetic

Math in Bash is slightly idiosyncratic. While a language like C++ operates directly on integers with + or /, arithmetic in Bash needs to be enclosed within $(( ... )) or evaluated using the let command.

x=5
y=10
sum=$((x + y))
echo "The sum is $sum"

Control Structures: If-Statements and Loops

Bash supports standard control flow constructs.

If-Statements:

if [ "$sum" -gt 10 ]; then
    echo "Sum is greater than 10"
elif [ "$sum" -eq 10 ]; then
    echo "Sum is exactly 10"
else
    echo "Sum is less than 10"
fi

[ is a shell builtin command: The single bracket [ is not special syntax — it is a builtin command, a synonym for test. Because Bash implements it internally, its arguments must be separated by spaces just like any other command: [ -f "$file" ] is correct, but [-f "$file"] tries to run a command named [-f, which fails. This is why the spaces inside brackets are mandatory, not just stylistic. (An external binary /usr/bin/[ also exists on most systems, but Bash uses its builtin by default — you can verify with type -a [.)

The following table covers the most important tests available inside [ ]:

Test	Meaning
-f path	Path exists and is a regular file
-d path	Path exists and is a directory
-z "$var"	String is empty (zero length)
"$a" = "$b"	Strings are equal
"$a" != "$b"	Strings are not equal
$x -eq $y	Integers are equal
$x -gt $y	Integer greater than
$x -lt $y	Integer less than
! condition	Logical NOT (negates the test)

Important: use -eq, -lt, -gt for numbers and = / != for strings. Mixing them produces wrong results silently.

[ vs [[: The double bracket [[ ... ]] is a Bash keyword with additional power: it does not perform word splitting on variables, allows && and || inside the condition, and supports regex matching with =~. Prefer [[ ]] in new Bash scripts.

Loops:

for i in 1 2 3 4 5; do
    echo "Iteration $i"
done

For numeric ranges, the C-style for loop (the arithmetic for command) is often cleaner:

for (( i=1; i<=5; i++ )); do
    echo "Iteration $i"
done

This is a distinct looping construct from the standalone (( )) arithmetic compound command. In this form, expr1 is evaluated once at start, expr2 is tested before each iteration (loop runs while non-zero), and expr3 is evaluated after each iteration — the same semantics as C’s for loop.

Loop control keywords:

• break: Exit the loop immediately, regardless of the remaining iterations.
• continue: Skip the rest of the current iteration and jump to the next one.

for f in *.log; do
    [ -s "$f" ] || continue    # skip empty files
    grep -q "ERROR" "$f" || continue
    echo "Errors found in: $f"
done

Quoting and Word Splitting

How you quote text profoundly changes how Bash interprets it — this is one of the most common sources of bugs in shell scripts.

• Single quotes ('...'): All characters are literal. No variable or command substitution occurs. echo 'Cost: $5' prints exactly Cost: $5.
• Double quotes ("..."): Spaces are preserved, but $VARIABLE and $(command) are still expanded. echo "Hello $USER" prints Hello Ada.

A critical pitfall is word splitting: when you reference an unquoted variable, the shell splits its value on whitespace and treats each word as a separate argument. Consider:

FILE="my report.pdf"
rm $FILE      # WRONG: shell splits into two args: "my" and "report.pdf"
rm "$FILE"    # CORRECT: the entire value is passed as one argument

Always quote variable references with double quotes to protect against word splitting.

Command Substitution

Command substitution captures the standard output of a command and uses it as a value in-place. The modern syntax is $(command):

TODAY=$(date +%Y-%m-%d)
echo "Backup started on: $TODAY"

The shell runs the inner command in a subshell, then replaces the entire $(...) expression with its output. This is the standard way to assign the results of commands to variables.

Positional Parameters and Special Variables

Scripts receive command-line arguments via positional parameters. If you run ./backup.sh /src /dest, then inside the script:

Variable	Value	Description
$0	./backup.sh	Name of the script itself
$1	/src	First argument
$2	/dest	Second argument
$#	2	Total number of arguments passed
$@	/src /dest	All arguments as separate, properly-quoted words
$?	(exit code)	Exit status of the most recent command

When iterating over all arguments, always use "$@" (quoted). Without quotes, $@ is subject to word splitting and arguments containing spaces are silently broken into multiple words:

for f in "$@"; do
    echo "Processing: $f"
done

Command Chaining with && and ||

Because every command returns an exit status, you can chain commands conditionally without writing a full if/then/fi block:

• && (AND): The right-hand command runs only if the left-hand command succeeds (exit code 0). mkdir output && echo "Directory created" — only prints if mkdir succeeded.
• || (OR): The right-hand command runs only if the left-hand command fails (non-zero exit code). cd /target || exit 1 — exits the script immediately if the directory cannot be entered.

This compact chaining idiom is widely used in professional scripts for concise, readable error handling.

Background Jobs

Appending & to a command runs it asynchronously — the shell launches it in the background and immediately returns to the prompt without waiting for it to finish:

./long_running_build.sh &
echo "Build started, continuing with other work..."

Two special variables are useful when managing background processes:

• $$: The process ID (PID) of the current shell itself. Often used to create unique temporary file names: tmp_file="/tmp/myscript.$$".
• $!: The PID of the most recently backgrounded job. Use it to wait for or kill a specific background process.

The jobs command lists all active background jobs; fg brings the most recent one back to the foreground, and bg resumes a stopped job in the background.

Functions — Reusable Building Blocks

When the same logic appears in multiple places, extract it into a function. Functions in Bash work like small scripts-within-a-script: they accept positional arguments via $1, $2, etc. — independently of the outer script’s own arguments — and can be called just like any other command.

greet() {
    local name="$1"
    echo "Hello, ${name}!"
}

greet "engineer"   # → Hello, engineer!

The local Keyword

Without local, any variable set inside a function leaks into and overwrites the global script scope. Always declare function-internal variables with local to prevent subtle bugs:

process() {
    local result="$1"   # visible only inside this function
    echo "$result"
}

Returning Values from Functions

The return statement only carries a numeric exit code (0–255), not data. To pass a string back to the caller, have the function echo the value and capture it with command substitution:

to_upper() {
    echo "$1" | tr '[:lower:]' '[:upper:]'
}

loud=$(to_upper "hello")   # loud="HELLO"

You can also use functions directly in if statements, because a function’s exit code is treated as its truth value: return 0 is success (true), return 1 is failure (false).

Case Statements — Readable Multi-Way Branching

When you need to check one variable against many possible values, a case statement is far cleaner than a chain of if/elif:

case "$command" in
    start)   echo "Starting service..."  ;;
    stop)    echo "Stopping service..."  ;;
    status)  echo "Checking status..."   ;;
    *)       echo "Unknown command: $command" >&2; exit 2 ;;
esac

Each branch ends with ;;. The * pattern is the catch-all default, matching any value not handled by earlier branches. The block closes with esac (case backwards).

Exit Codes — The Language of Success and Failure

Every command — including your own scripts — exits with a number. 0 always means success; any non-zero value means failure. This is the opposite of most programming languages where 0 is falsy. Conventional exit codes are:

Code	Meaning
0	Success
1	General error
2	Misuse — wrong arguments or invalid input

Meaningful exit codes make scripts composable: other scripts, CI pipelines, and tools like make can call your script and take action based on the result. For example, ./monitor.sh || alert_team only triggers the alert when your monitor exits non-zero.

Shell Expansions — Brace Expansion and Globbing

The shell performs several rounds of expansion on a command line before executing it. Understanding the order helps you predict and control what the shell does.

Brace Expansion

First comes brace expansion, which generates arbitrary lists of strings. It is a purely textual operation — no files need to exist:

mkdir project/{src,tests,docs}      # creates three directories at once
cp config.yml config.yml.{bak,old}  # copies to two names simultaneously
echo {1..5}                          # → 1 2 3 4 5  (sequence expression)

Brace expansion happens before all other expansions, so you can combine it freely with variables and globbing.

Supercharging Scripts with Regular Expressions

Because the UNIX philosophy is heavily centered around text streams, text processing is a massive part of shell scripting. Regular Expressions (RegEx) is a vital tool used within shell commands like grep, sed, and awk to find, validate, or transform text patterns quickly.

Globbing vs. Regular Expressions: These look similar but are entirely different systems. Globbing (filename expansion) uses *, ?, and [...] to match filenames — the shell expands these before the command runs (e.g., rm *.log deletes all .log files). The three special pattern characters are: * matches any string (including empty), ? matches any single character, and [ opens a bracket expression [...] that matches any one of the enclosed characters — e.g., [a-z] matches any lowercase letter, and [!a-z] matches any character that is not a lowercase letter. Regular Expressions use ^, $, .*, [0-9]+, and similar constructs — they are pattern languages used by tools like grep, sed, and awk, and also natively by Bash itself via the =~ operator inside [[ ]] conditionals (which evaluates POSIX extended regular expressions directly without spawning an external tool). Critically, * means “match anything” in globbing, but “zero or more of the preceding character” in RegEx.

RegEx allows you to match sub-strings in a longer sequence. Critical to this are anchors, which constrain matches based on their location:

• ^ : Start of string. (Does not allow any other characters to come before).
• $ : End of string.

Example: ^[a-zA-Z0-9]{8,}$ validates a password that is strictly alphanumeric and at least 8 characters long, from the exact beginning of the string to the exact end.

Conclusion

Shell scripting is an indispensable skill for anyone working in tech. By viewing the shell as a set of modular tools (the “Infinity Stones” of your development environment), you can combine simple operations to perform massive, complex tasks with minimal effort. Start small by automating a daily chore on your machine, and before you know it, you will be weaving complex UNIX tools together with ease!

Quiz

Shell Commands — What Does It Do?

Match each shell command to its purpose

What does ls do?

Lists files and directories in the current (or specified) directory

ls (list) shows the contents of a directory. Common flags: -l for a detailed long listing, -a to include hidden files (those starting with .), and -lh for human-readable file sizes.

What does mkdir do?

Creates a new directory

mkdir (make directory) creates one or more directories. Use mkdir -p path/to/nested/dir to create all intermediate directories at once without errors if they already exist.

What does cp do?

Copies a file or directory to a new location

cp source dest copies a file. To copy a directory and all its contents recursively, use cp -r source/ dest/. The original file is left unchanged.

What does mv do?

Moves or renames a file or directory

mv old new renames a file if both paths are on the same filesystem, or moves it otherwise. Unlike cp, the original is removed — there is no -r flag needed for directories.

What does rm do?

Permanently deletes files or directories

rm file deletes a file. Use rm -r dir/ to delete a directory and all its contents recursively. There is no trash or undo — deleted files are gone immediately.

What does less do?

Displays file contents one screen at a time, with scrolling

less file opens an interactive pager. Navigate with arrow keys or Page Up/Down, search with /keyword, and quit with q. Unlike cat, it doesn’t dump everything to the terminal at once — essential for large files.

What does cat do?

Prints the contents of a file to standard output

cat (concatenate) reads one or more files and writes them to stdout. It’s commonly used to view small files, combine files (cat a b > c), or feed file contents into a pipeline.

What does sed do?

Edits a stream of text — most commonly used for find-and-replace

sed (stream editor) applies editing commands to each line of input. The most common usage is substitution: sed 's/old/new/g' replaces every occurrence of old with new. The g flag means global (all matches per line, not just the first).

What does grep do?

Searches text and prints lines that match a pattern

grep pattern file prints every line containing the pattern. Key flags: -i (case-insensitive), -r (recursive search through directories), -n (show line numbers), -v (invert — show lines that do NOT match).

What does head do?

Prints the first lines of a file (default: 10)

head file shows the first 10 lines. Use head -n 20 file to show the first 20 lines. Useful for quickly inspecting the start of a file or checking a CSV header.

What does tail do?

Prints the last lines of a file (default: 10)

tail file shows the last 10 lines. Use tail -n 20 for more. The -f flag (follow) continuously streams new lines as they are appended — the standard way to monitor a live log file.

What does wc do?

Counts lines, words, and bytes in a file

wc file prints line count, word count, and byte count. Use flags to isolate each: -l (lines), -w (words), -c (bytes). For example, wc -l access.log tells you how many requests are in a log file.

What does sort do?

Sorts lines of text alphabetically (or numerically)

sort file outputs lines in ascending alphabetical order. Key flags: -n (numeric sort), -r (reverse/descending), -u (unique — remove duplicate lines). Combine with pipes: cat file | sort -n | tail -5 finds the five largest numbers.

What does cut do?

Extracts specific columns or fields from each line of text

cut -d',' -f2 file.csv splits each line on , and prints the second field. Use -d to set the delimiter and -f to choose the field(s). Essential for quickly extracting a column from a CSV or colon-separated file like /etc/passwd.

What does ssh do?

Opens an encrypted remote shell session on another machine

ssh user@host connects to host as user over the SSH protocol. Everything — including your password and the session data — is encrypted. You can also use it to run a single remote command: ssh user@host 'df -h'.

What does htop do?

Shows an interactive, real-time view of running processes and system resource usage

htop is an improved alternative to top. It displays CPU, memory, and swap usage at the top, with a scrollable list of processes below. You can sort by CPU or memory, search for a process, and kill processes interactively — all with keyboard shortcuts.

What does pwd do?

Prints the absolute path of the current working directory

pwd (print working directory) tells you exactly where you are in the filesystem. Useful when you’re deep in a directory tree or when writing scripts where you need to capture the script’s launch location.

What does chmod do?

Changes the read, write, and execute permissions of a file or directory

chmod (change mode) controls who can read, write, or execute a file. chmod +x script.sh adds execute permission for everyone. You can also use numeric notation: chmod 755 file sets owner=rwx, group=rx, others=rx — a common pattern for executable scripts.

Workout Complete!

Your Score: 0/18

Come back later to improve your recall!

Shell Commands Flashcards

Which Shell command would you use for the following scenarios?

You need to see a list of all the files and folders in your current directory. What command do you use?

ls

The ls command lists directory contents. You can also use flags like ls -l for a detailed list or ls -a to see hidden files.

You are currently in your home directory and need to navigate into a folder named ‘Documents’. Which command achieves this?

cd Documents

The cd (change directory) command is used to change the current working directory in the command line operating system.

You want to quickly view the entire contents of a small text file named ‘config.txt’ printed directly to your terminal screen.

cat config.txt

The cat (concatenate) command reads data from the file and gives its content as output. It is widely used to display file contents.

You need to find every line containing the word ‘ERROR’ inside a massive log file called ‘server.log’.

grep 'ERROR' server.log

grep searches for patterns in each file. It prints each line that matches the given pattern, making it invaluable for parsing logs.

You wrote a new bash script named ‘script.sh’, but when you try to run it, you get a ‘Permission denied’ error. How do you make the file executable?

chmod +x script.sh

The chmod command changes file modes or Access Control Lists. The +x flag specifically adds execute permissions to the file.

You want to rename a file from ‘draft_v1.txt’ to ‘final_version.txt’ without creating a copy.

mv draft_v1.txt final_version.txt

The mv command is used to move files or directories, but it is also the standard command used to rename a file by moving it to a new name in the same directory.

You are starting a new project and need to create a brand new, empty folder named ‘src’ in your current location.

mkdir src

mkdir stands for ‘make directory’. It creates a new directory with the specified name, provided you have the correct permissions.

You want to view the contents of a very long text file called ‘manual.txt’ one page at a time so you can scroll through it.

less manual.txt

The less command is a terminal pager program that allows viewing (but not editing) the contents of a text file one screen at a time, allowing both forward and backward navigation.

You need to create an exact duplicate of a file named ‘report.pdf’ and save it as ‘report_backup.pdf’.

cp report.pdf report_backup.pdf

The cp (copy) command is used to copy files or directories from one location to another.

You have a temporary file called ‘temp_data.csv’ that you no longer need and want to permanently delete from your system.

rm temp_data.csv

The rm (remove) command deletes files or directories. Use with caution, as deleted files are generally not recoverable.

You want to quickly print the phrase ‘Hello World’ to the terminal or pass that string into a pipeline.

echo 'Hello World'

The echo command outputs the strings it is being passed as arguments. It is commonly used in scripts to display messages or write data to a file using redirection.

You want to know exactly how many lines are contained within a file named ‘essay.txt’.

wc -l essay.txt

The wc (word count) command prints newline, word, and byte counts for a file. Adding the -l flag restricts the output to only the line count.

You need to perform an automated find-and-replace operation on a stream of text to change the word ‘apple’ to ‘orange’.

sed 's/apple/orange/g'

The sed (stream editor) s: Stands for ‘substitute’. /: The delimiter that separates the search and replacement terms. apple: The string you want to find. orange: The string you want to replace it with. g: Stands for ‘global’. This tells sed to replace every instance of ‘apple’ on a line, rather than just the first one it finds.

You have a space-separated log file and want a tool to extract and print only the 3rd column of data.

awk '{print $3}'

awk is a versatile command-line utility and programming language designed for text processing and data extraction, particularly excelling at handling structured, column-based data.

You want to store today’s date (formatted as YYYY-MM-DD) in a variable called TODAY so you can use it to name a backup file dynamically.

TODAY=$(date +%Y-%m-%d)

Command substitution $(...) executes the inner command in a subshell and replaces itself with the command’s standard output. This is the standard way to capture the result of a command into a variable.

A variable FILE holds the value my report.pdf. Running rm $FILE fails with a ‘No such file or directory’ error for both ‘my’ and ‘report.pdf’. How do you fix this?

rm "$FILE"

Without quotes, the shell performs word splitting on the expanded variable, breaking my report.pdf into two separate arguments: my and report.pdf. Wrapping the variable in double quotes ("$FILE") prevents word splitting and passes the entire value as a single argument.

You are writing a script that requires exactly two arguments. How do you check how many arguments were passed to the script so you can print a usage error if the count is wrong?

$#

$# expands to the total count of positional arguments passed to the script. A typical guard looks like: if [ $# -ne 2 ]; then echo 'Usage: script.sh src dest'; exit 1; fi. Other useful special variables: $1, $2 (individual args), $0 (script name), $@ (all args).

You want to create a directory called ‘build’ and then immediately run cmake .. inside it, but only if the directory creation succeeded — all in a single command.

mkdir build && cd build && cmake ..

The && (AND) operator runs the next command only if the previous one succeeded (exit code 0). If mkdir fails, neither cd nor cmake will execute. This is a concise alternative to a full if/then/fi block for simple success-guarded sequences.

At the start of a script, you need to change into /deploy/target. If that directory doesn’t exist, the script must abort immediately — write a defensive one-liner.

cd /deploy/target || exit 1

The || (OR) operator runs the right-hand command only if the left-hand command fails (non-zero exit code). This is the standard idiom for aborting a script when a critical precondition is not met, without needing a full if statement.

You want to delete all files ending in .tmp in the current directory using a single command, without listing each filename explicitly.

rm *.tmp

The * wildcard is a glob pattern — it is expanded by the shell itself (filename expansion) before rm runs. Globbing is entirely separate from regular expressions: in RegEx, * means ‘zero or more of the preceding character’, not ‘match anything’.

Workout Complete!

Your Score: 0/20

Come back later to improve your recall!

Shell Pipelines

Practice connecting UNIX commands together with pipes to solve real tasks.

You want to count how many lines in server.log contain the word ‘ERROR’.

grep 'ERROR' server.log | wc -l

grep filters and prints only the matching lines; wc -l counts them. Without the pipe you would have to save a temp file and count it separately.

You have a file names.txt with one name per line. Print only the unique names, sorted alphabetically.

sort names.txt | uniq

uniq only removes adjacent duplicates, so sort must come first to bring duplicates together. Then uniq collapses consecutive identical lines into one.

You have a file names.txt with one name per line. Print each unique name alongside a count of how many times it appears.

sort names.txt | uniq -c

sort brings duplicates together so they are adjacent; uniq -c then collapses consecutive identical lines into one and prefixes each with its count.

List all running processes and show only those belonging to user tobias.

ps aux | grep tobias

ps aux prints every running process with its owner; piping to grep tobias filters to lines that contain that username.

Print the 3rd line of config.txt without using sed or awk.

head -3 config.txt | tail -1

head -3 keeps only the first three lines; tail -1 then takes the very last of those — which is line 3 of the original file.

List the 5 largest files in the current directory, with the biggest first, showing only their names.

ls -S | head -5

ls -S sorts directory entries by file size (largest first); head -5 keeps only the first 5 lines of that output.

You want to replace every occurrence of http:// with https:// in links.txt and save the result to links_secure.txt.

sed 's|http://|https://|g' links.txt > links_secure.txt

Here we redirect sed’s stdout to a file instead of piping it onward, but sed reads from a file argument, transforms the stream, and writes to stdout — which the shell redirects with >.

Print only the unique error lines from access.log that contain the word ‘ERROR’, sorted alphabetically.

grep 'ERROR' access.log | sort | uniq

grep filters to matching lines; sort brings identical lines together; uniq collapses consecutive duplicates into one — a classic 3-step de-duplication pipeline.

Count the total number of files (not directories) inside the current directory tree.

find . -type f | wc -l

find . -type f recursively lists every regular file, one per line; wc -l counts those lines.

Show the 10 most recently modified files in the current directory, newest first.

ls -t | head -10

ls -t sorts directory entries by modification time (newest first); head -10 keeps only the first 10 lines of that output.

Extract the second column from comma-separated data.csv, sort the values, and print only the unique ones.

cut -d',' -f2 data.csv | sort | uniq

cut -d',' splits each line on commas; -f2 keeps the second field; sort brings duplicates together; uniq removes consecutive duplicates.

Convert the contents of readme.txt to uppercase and save the result to readme_upper.txt.

cat readme.txt | tr 'a-z' 'A-Z' > readme_upper.txt

cat reads the file and feeds it into the pipeline; tr 'a-z' 'A-Z' translates every lowercase letter to uppercase; > redirects the final output to a new file.

Print every line from app.log that does NOT contain the word ‘DEBUG’.

grep -v 'DEBUG' app.log

The -v flag inverts the match — grep prints every line that does not match the pattern. No pipe is needed here, but grep -v ... | ... is a common pipeline building block.

You have two files, file1.txt and file2.txt. Print all lines from both files that contain the word ‘success’, sorted alphabetically with duplicates removed.

grep 'success' file1.txt file2.txt | sort | uniq

grep can take multiple file arguments and searches all of them, prefixing each match with the filename; sort orders the results; uniq removes duplicate lines.

Workout Complete!

Your Score: 0/14

Come back later to improve your recall!

Shell Scripting & UNIX Philosophy Quiz

Test your conceptual understanding of shell environments, data streams, and scripting paradigms beyond basic command memorization.

A developer needs to parse a massive log file, extract IP addresses, sort them, and count unique occurrences. Instead of writing a 500-line Python script, they use cat | awk | sort | uniq -c. Why is this approach fundamentally preferred in the UNIX environment?

Correct Answer:

Explanation

The UNIX design philosophy revolves around writing programs that ‘do one thing and do it well’, and having them work together by handling text streams. Pipelines allow complex tasks to be solved elegantly by chaining these simple tools together.

A script runs a command that generates both useful output and a flood of permission error messages. The user runs script.sh > output.txt, but the errors still clutter the terminal screen while the useful data goes to the file. What underlying concept explains this behavior?

Correct Answer:

Explanation

In UNIX, processes have three default streams: stdin, stdout, and stderr. Redirection with > only captures stdout. To capture the errors, you would need to redirect stderr specifically (e.g., using 2>).

A C++ developer writes a Bash script with a for loop. Inside the loop, they declare a variable temp_val. After the loop finishes, they try to print temp_val expecting it to be undefined or empty, but it prints the last value assigned in the loop. Why did this happen?

Correct Answer:

Explanation

Bash does not use block-level scoping. Unless explicitly declared as local inside a function, variables assigned within control structures like loops or conditionals bleed into the rest of the script.

You want to use a command that requires two file inputs (like diff), but your data is currently coming from the live outputs of two different commands. Instead of creating temporary files on the disk, you use the <(command) syntax. What is this concept called and what does it achieve?

Correct Answer:

Explanation

Process substitution <() allows the standard output of a command to be treated as a file. The operating system creates a temporary file descriptor (like /dev/fd/63) that the consuming command can read from as if it were a physical file.

A script contains entirely valid Python code, but the file is named script.sh and has #!/bin/bash at the very top. When executed via ./script.sh, the terminal throws dozens of ‘command not found’ and syntax errors. What is the fundamental misunderstanding here?

Correct Answer:

Explanation

In UNIX systems, file extensions are largely superficial. The shebang (#!) on the first line is what tells the operating system’s program loader which interpreter program (e.g., Bash, Python, Node) to use to parse the rest of the file.

A developer uses the regular expression [0-9]{4} to validate that a user’s input is exactly a four-digit PIN. However, the system incorrectly accepts ‘12345’ and ‘A1234’. What crucial RegEx concept did the developer omit?

Correct Answer:

Explanation

Without anchors, a regular expression looks for a match anywhere within the string. Because ‘12345’ contains ‘1234’, the pattern matches. Adding ^ (start) and $ (end) ensures the pattern must account for the entire string.

You are designing a data pipeline in the shell. Which of the following statements correctly describe how UNIX handles data streams and command chaining? (Select all that apply)

Correct Answers:

Explanation

Pipes (|) link stdout to stdin, >> appends stdout to a file, and < feeds a file into stdin. By default, pipes do NOT pass stderr (errors will still print to the screen). >! is not the standard way to redirect both streams in Bash (typically >& or &> is used).

You’ve written a shell script deploy.sh but it throws a ‘Permission denied’ error or fails to run when you type ./deploy.sh. Which of the following are valid reasons or necessary steps to successfully execute a script as a standalone program? (Select all that apply)

Correct Answers:

Explanation

Scripts require execute permissions (chmod +x) to be run as programs, and a shebang tells the OS how to interpret the text. They are interpreted, not compiled. The ./ explicitly tells the OS to look in the current directory, bypassing the need for the directory to be in the $PATH.

In Bash, exit codes are crucial for determining if a command succeeded or failed. Which of the following statements are true regarding how Bash handles exit statuses and control flow? (Select all that apply)

Correct Answers:

Explanation

In UNIX, an exit code of 0 means success, while non-zero (like 1) means failure/error. if statements work by running a command and proceeding if its exit code is 0. set -e is a common safety mechanism to fail fast if an error occurs.

When you type a command like python or grep into the terminal, the shell knows exactly what program to run without you providing the full file path. How does the $PATH environment variable facilitate this, and how is it managed? (Select all that apply)

Correct Answers:

Explanation

$PATH is an explicit list of directories, not a global search crawler (which would be extremely slow). The OS checks these directories in order. Modifications in the terminal are bound to that specific session unless saved in a configuration file like .bashrc.

A developer writes LOGFILE="access errors.log" and then runs wc -l $LOGFILE. The command fails with ‘No such file or directory’ errors for both ‘access’ and ‘errors.log’. What is the root cause?

Correct Answer:

Explanation

When a variable is expanded without double quotes, the shell splits the result on whitespace. wc -l $LOGFILE is interpreted as wc -l access errors.log — two separate arguments. The fix is to always quote variable expansions: wc -l "$LOGFILE".

A script is invoked with ./deploy.sh production 8080 myapp. Inside the script, which variable holds the value 8080?

Correct Answer:

Explanation

$0 is the script name (./deploy.sh), $1 is the first argument (production), $2 is the second argument (8080), and $3 would be myapp. $# holds the total argument count (3).

A script contains the line: cd /deploy/target && ./run_tests.sh && echo 'All tests passed!'. If ./run_tests.sh exits with a non-zero status code, what happens next?

Correct Answer:

Explanation

The && operator short-circuits on failure. Since ./run_tests.sh returned a non-zero exit code, the shell skips all remaining &&-linked commands. Only if every command in the chain succeeds does echo 'All tests passed!' execute.

Which of the following statements correctly describe Bash quoting and command substitution behavior? (Select all that apply)

Correct Answers:

Explanation

Single quotes suppress all expansion; double quotes allow variable and command substitution while protecting spaces. Quoting "$VARIABLE" is essential to prevent word splitting on values containing spaces. Single and double quotes are NOT interchangeable when variable expansion is needed. $(command) is command substitution — the standard way to capture command output into a variable.

Arrange the pipeline fragments to build a command that extracts all ERROR lines from a log, sorts them, removes duplicates, and counts how many unique errors remain.

Drag fragments into the answer area in the correct order (some items are distractors that should not be used):

→ Drop here →

Correct order:
grep 'ERROR' server.log|sort|uniq|wc -l

Explanation

grep filters for ERROR lines, sort orders them (required before uniq), uniq removes adjacent duplicates, and wc -l counts the remaining lines. Each | pipes stdout to the next command’s stdin — the UNIX philosophy of chaining single-purpose tools. cat is unnecessary here (UUOC — Useless Use of Cat) and > would redirect to a file rather than pipe.

Arrange the lines to write a shell script that validates a command-line argument, prints an error to stderr if missing, and exits with a non-zero code.

Drag lines into the solution area in the correct order (some items are distractors that should not be used):

↓ Drop here ↓

Correct order:
#!/bin/bash
if [ $# -lt 1 ]; then
echo "Error: no filename given" >&2
exit 1
fi
echo "Processing $1..."

Explanation

The shebang (#!/bin/bash) must be the first line. The if [ $# -lt 1 ] test checks argument count, >&2 redirects echo to stderr, and exit 1 terminates with a failure code. fi closes the if block (no semicolon). The distractor return 1 only works inside functions, not at the top level of a script. fi; with a trailing semicolon is unnecessary and unconventional.

Arrange the pipeline fragments to find the 5 most frequently occurring IP addresses in an access log.

Drag fragments into the answer area in the correct order (some items are distractors that should not be used):

→ Drop here →

Correct order:
grep -oE '[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+' access.log|sort|uniq -c|sort -rn|head -5

Explanation

The grep -oE extracts all IPv4 addresses (one per line). sort groups identical IPs together (required for uniq). uniq -c counts each group. sort -rn sorts by count in descending numeric order. head -5 takes the top 5. tail -5 would give the least frequent, and wc -l would just count total lines.

Arrange the fragments to redirect both stdout and stderr of a deployment script into a single log file.

Drag fragments into the answer area in the correct order (some items are distractors that should not be used):

→ Drop here →

Correct order:
./deploy.sh>output.log2>&1

Explanation

> output.log redirects stdout to the file first, then 2>&1 redirects stderr to wherever stdout currently points (the file). Order matters: 2>&1 > output.log would send stderr to the original stdout (terminal) and only stdout to the file. The distractor 1>&2 goes the wrong direction — it sends stdout to stderr. Using 2> output.log alone still lets stdout reach the terminal.

Arrange the pipeline to count how many files under src/ contain the word TODO.

Drag fragments into the answer area in the correct order (some items are distractors that should not be used):

→ Drop here →

Correct order:
grep -rl 'TODO' src/|wc -l

Explanation

grep -rl (recursive, files-with-matches) prints one matching filename per line — each file counted once regardless of how many matches it contains. wc -l then counts those filenames. The distractor grep -r (without -l) prints every matching line, so wc -l would count occurrences, not files. sort is unnecessary here since we only need a count.

Arrange the fragments to grant execute permission on a script and immediately run it.

Drag fragments into the answer area in the correct order (some items are distractors that should not be used):

→ Drop here →

Correct order:
chmod +x script.sh&&./script.sh

Explanation

chmod +x script.sh adds the execute bit, making the file runnable as a program. && ensures the script only runs if chmod succeeded. ./script.sh executes it using the shebang to select the interpreter. Using || instead of && would run the script only when chmod fails — the opposite of the intent. sh script.sh works but bypasses the shebang, hardcoding the interpreter to sh regardless of what the script specifies.

Workout Complete!

Your Score: 0/20

After finishing these quizzes, you are now ready to practice in a real Linux system. Try the Interactive Shell Scripting Tutorial!