File Modes and Context Managers

The mode string

"r" is the default, so open(path) and open(path, "r") are the same.

The dangerous one is "w". It empties the file the instant it is opened, before you write anything. Opening a file with "w" to "check something" destroys it.

The x mode prevents accidental overwrites

try:
    with open("report.txt", "x", encoding="utf-8") as handle:
        handle.write("fresh report\n")
except FileExistsError:
    print("report.txt already exists; refusing to overwrite")

Text mode and binary mode

with open("data.txt", "r", encoding="utf-8") as handle:
    content = handle.read()
    print(type(content))          # <class 'str'>

with open("image.png", "rb") as handle:
    content = handle.read()
    print(type(content))          # <class 'bytes'>

# encoding is meaningless in binary mode
# open("f", "rb", encoding="utf-8")     # ValueError

with open("copy.png", "wb") as writer, open("image.png", "rb") as reader:
    writer.write(reader.read())

Newline translation

with open("out.txt", "w", encoding="utf-8") as handle:
    handle.write("line\n")            # on Windows this becomes \r\n on disk

with open("out.txt", "rb") as handle:
    print(handle.read())              # shows what is actually stored

with open("out.txt", "w", encoding="utf-8", newline="") as handle:
    handle.write("line\n")            # no translation; \n stays \n

In text mode Python translates \n to the platform line ending on write and back on read, so your code sees \n everywhere. Pass newline="" when a format specifies its own line endings, which is exactly why the csv module requires it.

The file object

with open("notes.txt", encoding="utf-8") as handle:
    print(handle.name)         # notes.txt
    print(handle.mode)         # r
    print(handle.encoding)     # utf-8
    print(handle.readable(), handle.writable(), handle.seekable())
    print(handle.closed)       # False

print(handle.closed)           # True

What with actually does

with open("notes.txt") as handle:
    content = handle.read()

# is equivalent to
handle = open("notes.txt")
try:
    content = handle.read()
finally:
    handle.close()

Any object defining __enter__ and __exit__ can be used with with. That pair of methods is the context manager protocol, and files are simply the most common example.

class Timer:
    def __enter__(self):
        import time
        self.start = time.perf_counter()
        return self                     # this is what `as` binds

    def __exit__(self, exc_type, exc_value, traceback):
        import time
        self.elapsed = time.perf_counter() - self.start
        print(f"took {self.elapsed:.4f}s")
        return False                    # False: do not suppress exceptions


with Timer() as timer:
    total = sum(range(1_000_000))

print(timer.elapsed)

The context managers note covers the protocol fully, including how __exit__ can suppress an exception and how contextlib.contextmanager lets you write one as a generator.

Several files at once

with open("a.txt", encoding="utf-8") as first, \
     open("b.txt", encoding="utf-8") as second:
    print(first.read(), second.read())


# Parenthesised form, Python 3.10 and later
with (
    open("a.txt", encoding="utf-8") as first,
    open("b.txt", encoding="utf-8") as second,
):
    pass

from contextlib import ExitStack

paths = ["a.txt", "b.txt", "c.txt"]

with ExitStack() as stack:
    handles = [stack.enter_context(open(p, encoding="utf-8")) for p in paths]
    for handle in handles:
        print(handle.name, len(handle.read()))

ExitStack manages a number of context managers that is not known until runtime, and still closes every one of them.

Buffering and flushing

with open("out.txt", "w", encoding="utf-8") as handle:
    handle.write("first\n")
    handle.flush()                # force it to disk now
    handle.write("second\n")
    # closing flushes the rest automatically

with open("out.txt", "w", encoding="utf-8", buffering=1) as handle:
    handle.write("line buffered\n")     # flushed on every newline

Writes are buffered for speed. Data is written to disk when the buffer fills, when you call flush(), or when the file is closed. This is why an unclosed file can lose data, and why with matters.

Writing safely by replacing

import os
from pathlib import Path


def write_atomic(path, text):
    """Write via a temporary file so a crash cannot leave a half written file."""
    path = Path(path)
    temporary = path.with_suffix(path.suffix + ".tmp")
    with open(temporary, "w", encoding="utf-8") as handle:
        handle.write(text)
        handle.flush()
        os.fsync(handle.fileno())
    os.replace(temporary, path)          # atomic on the same filesystem


write_atomic("settings.txt", "theme=dark\n")

The original file is replaced in one operation, so a reader sees either the old content or the new one, never a partial write. Use this for anything you would be upset to lose.

Common mistakes

• Using "w" when "a" was intended.
• Passing encoding= with a binary mode.
• Reading bytes and trying to use string methods on them.
• Forgetting newline="" when writing CSV, producing blank lines between rows on Windows.
• Assuming written data is on disk before the file is closed.
• Opening a file inside a loop and never closing it.

Best practices

• Use with for every file, without exception.
• Use "x" when a file must not already exist.
• Use binary mode for anything that is not text, and never guess.
• Use newline="" for CSV.
• Replace important files atomically rather than overwriting in place.

Practice

1. Show what happens to an existing file when it is opened with each of "r", "w", "a" and "x".
2. Copy a binary file and verify the copy is byte for byte identical.
3. Write a timing context manager and use it to measure two different operations.
4. Write the same text with and without newline="" and compare the bytes on disk.
5. Implement an atomic write and demonstrate that the original survives a failure mid-write.

Conclusion

The mode string answers four questions: read or write, must the file exist, is it truncated, and is it text or bytes. with guarantees the file is closed and flushed however the block ends, which is why it is the only form worth writing.