Command Line Arguments and Exit Codes

sys.argv

import sys

print(sys.argv)
print("script:", sys.argv[0])
print("arguments:", sys.argv[1:])

$ python report.py data.csv --verbose -n 10
['report.py', 'data.csv', '--verbose', '-n', '10']
script: report.py
arguments: ['data.csv', '--verbose', '-n', '10']

• sys.argv[0] is the script name, not the first argument.
• Every element is a string, including "10".
• The shell has already split on whitespace and handled quoting.
• The list is never empty.

$ python show.py "two words" one
['show.py', 'two words', 'one']        the quotes grouped the first argument

A minimal script

"""Count the lines in a file."""

import sys
from pathlib import Path


def count_lines(path):
    with open(path, encoding="utf-8") as handle:
        return sum(1 for _ in handle)


def main(argv=None):
    argv = sys.argv[1:] if argv is None else argv

    if not argv:
        print("usage: count.py FILE", file=sys.stderr)
        return 2

    path = Path(argv[0])
    if not path.is_file():
        print(f"not a file: {path}", file=sys.stderr)
        return 1

    print(count_lines(path))
    return 0


if __name__ == "__main__":
    sys.exit(main())

Three habits worth copying: the work is in a function, main takes argv so it can be tested, and it returns an exit code instead of calling sys.exit itself.

def test_main():
    assert main([]) == 2                     # no arguments
    assert main(["missing.txt"]) == 1        # not a file
    assert main(["real.txt"]) == 0           # success

Exit codes

$ python count.py notes.txt
42
$ echo $?
0

$ python count.py
usage: count.py FILE
$ echo $?
2

$ python count.py notes.txt && echo "worked"     # only runs on success
$ python count.py missing  || echo "failed"      # only runs on failure

A program that always exits zero cannot be used in a shell pipeline, a build script or a scheduled job, because nothing downstream can tell whether it worked.

import sys

sys.exit(0)                     # success
sys.exit(1)                     # failure
sys.exit("fatal: no input")     # prints to stderr, exits with 1

stdout and stderr

import sys

print("the result")                                  # stdout: the answer
print("warning: using defaults", file=sys.stderr)    # stderr: diagnostics

$ python report.py > results.txt        results go to the file, warnings to the screen
$ python report.py 2> errors.txt        warnings go to the file, results to the screen
$ python report.py | sort               only the results are piped

Results to stdout, everything else to stderr. That split is what lets a user redirect the output of your program without losing its warnings, and it costs one keyword argument.

Reading from standard input

"""Uppercase every line arriving on standard input."""

import sys

for line in sys.stdin:
    print(line.rstrip().upper())

$ cat notes.txt | python upper.py
$ python upper.py < notes.txt
$ echo "hello" | python upper.py

import sys


def read_input(paths):
    """Read from the given files, or from stdin when there are none."""
    if not paths:
        yield from sys.stdin
        return
    for path in paths:
        with open(path, encoding="utf-8") as handle:
            yield from handle


def main(argv=None):
    argv = sys.argv[1:] if argv is None else argv
    count = sum(1 for _ in read_input(argv))
    print(count)
    return 0

Accepting either file names or piped input is the behaviour every standard command line tool has. It is four lines.

import sys

if sys.stdin.isatty():
    print("interactive: waiting for a person to type")
else:
    print("piped: reading redirected input")

Parsing simple arguments by hand

import sys


def main(argv=None):
    argv = list(sys.argv[1:] if argv is None else argv)

    verbose = False
    limit = None
    paths = []

    while argv:
        arg = argv.pop(0)
        if arg in ("-v", "--verbose"):
            verbose = True
        elif arg in ("-n", "--limit"):
            if not argv:
                print("--limit needs a value", file=sys.stderr)
                return 2
            try:
                limit = int(argv.pop(0))
            except ValueError:
                print("--limit must be a whole number", file=sys.stderr)
                return 2
        elif arg in ("-h", "--help"):
            print(__doc__ or "usage: tool.py [-v] [-n N] FILE...")
            return 0
        elif arg.startswith("-"):
            print(f"unknown option: {arg}", file=sys.stderr)
            return 2
        else:
            paths.append(arg)

    print(f"verbose={verbose} limit={limit} paths={paths}")
    return 0


print(main(["-v", "-n", "5", "a.txt", "b.txt"]))
print(main(["--limit", "x"]))

This works, and it is already forty lines that do not validate types, generate help or support --limit=5. Beyond two or three options, use argparse.

Handling Ctrl+C

import sys
import time


def main():
    try:
        for i in range(100):
            print(f"working {i}", end="\r")
            time.sleep(0.1)
        print()
        return 0
    except KeyboardInterrupt:
        print("\ninterrupted", file=sys.stderr)
        return 130


if __name__ == "__main__":
    sys.exit(main())

Without the handler, Ctrl+C prints a traceback, which looks like a crash rather than a deliberate stop.

Environment variables

import os
import sys


def get_config():
    return {
        "debug": os.environ.get("DEBUG", "0") == "1",
        "workers": int(os.environ.get("WORKERS", "4")),
        "output": os.environ.get("OUTPUT_DIR", "."),
    }


def require(name):
    value = os.environ.get(name)
    if not value:
        sys.exit(f"fatal: {name} is not set")
    return value


print(get_config())

$ DEBUG=1 WORKERS=8 python tool.py         # set for one run only

The usual precedence is: command line argument, then environment variable, then configuration file, then a built in default. Secrets belong in environment variables, never in source code or in sys.argv, which is visible in the process list.

Progress and interactivity

import sys
import time


def progress(current, total, width=30):
    filled = int(width * current / total)
    bar = "#" * filled + "-" * (width - filled)
    percent = 100 * current / total
    print(f"\r[{bar}] {percent:5.1f}%", end="", file=sys.stderr, flush=True)


for i in range(1, 21):
    progress(i, 20)
    time.sleep(0.05)
print(file=sys.stderr)

Progress output goes to stderr so it does not pollute piped results, and flush=True is needed because stderr may be buffered.

import sys


def confirm(question, default=False):
    if not sys.stdin.isatty():
        return default                 # non-interactive: do not block
    suffix = " [Y/n] " if default else " [y/N] "
    answer = input(question + suffix).strip().lower()
    if not answer:
        return default
    return answer.startswith("y")


# if confirm("Delete every temporary file?"):
#     ...

A complete tool

"""wordcount.py - count lines, words and characters.

usage: wordcount.py [-l] [-w] [-c] [FILE ...]
       reads standard input when no file is given
"""

import sys
from pathlib import Path


def count(lines):
    result = {"lines": 0, "words": 0, "chars": 0}
    for line in lines:
        result["lines"] += 1
        result["words"] += len(line.split())
        result["chars"] += len(line)
    return result


def main(argv=None):
    argv = list(sys.argv[1:] if argv is None else argv)

    if "-h" in argv or "--help" in argv:
        print(__doc__)
        return 0

    flags = {a for a in argv if a.startswith("-")}
    paths = [a for a in argv if not a.startswith("-")]

    unknown = flags - {"-l", "-w", "-c"}
    if unknown:
        print(f"unknown option(s): {', '.join(sorted(unknown))}", file=sys.stderr)
        return 2

    show = flags or {"-l", "-w", "-c"}
    totals = {"lines": 0, "words": 0, "chars": 0}
    status = 0

    sources = paths or [None]
    for path in sources:
        try:
            if path is None:
                result = count(sys.stdin)
                label = "-"
            else:
                with open(path, encoding="utf-8") as handle:
                    result = count(handle)
                label = path
        except OSError as error:
            print(f"wordcount: {error.filename}: {error.strerror}", file=sys.stderr)
            status = 1
            continue

        for key in totals:
            totals[key] += result[key]

        parts = []
        if "-l" in show:
            parts.append(f"{result['lines']:>8}")
        if "-w" in show:
            parts.append(f"{result['words']:>8}")
        if "-c" in show:
            parts.append(f"{result['chars']:>8}")
        print("".join(parts), label)

    if len(paths) > 1:
        parts = []
        if "-l" in show:
            parts.append(f"{totals['lines']:>8}")
        if "-w" in show:
            parts.append(f"{totals['words']:>8}")
        if "-c" in show:
            parts.append(f"{totals['chars']:>8}")
        print("".join(parts), "total")

    return status


if __name__ == "__main__":
    try:
        sys.exit(main())
    except KeyboardInterrupt:
        sys.exit(130)

Common mistakes

• Treating sys.argv[0] as the first argument.
• Using an argument without checking that it was supplied, and getting IndexError.
• Forgetting that every argument is a string.
• Printing errors to stdout, so they end up inside redirected output.
• Always exiting zero, hiding failures from the shell.
• Letting Ctrl+C print a traceback.
• Passing a secret as a command line argument, where other users can see it.

Best practices

• Put the work in functions and drive them from main(argv=None).
• Return an exit code from main and pass it to sys.exit.
• Results to stdout, diagnostics and progress to stderr.
• Read from stdin when no file is named.
• Handle KeyboardInterrupt and exit with 130.
• Take secrets from the environment, not from arguments.

Practice

1. Write a script that takes two file names and reports which is larger, with proper exit codes.
2. Make a script work with both a file argument and piped input.
3. Write a tool that fails cleanly, printing to stderr and returning 1, when a file is missing.
4. Add Ctrl+C handling to a long running loop.
5. Write tests for main(argv) covering success, missing arguments and a bad file.

Conclusion

sys.argv is a list of strings starting with the script name. Send results to stdout, problems to stderr, and return a meaningful exit code - those three habits are what separate a script from a usable command line tool.