Packages and __init__.py

From module to package

A module   =  one .py file
A package  =  a directory containing modules

notesapp/
    __init__.py
    storage.py
    formatting.py
    search.py

import notesapp.storage
from notesapp import formatting
from notesapp.search import find

notesapp.storage.save("a note")
print(formatting.title("hello"))

__init__.py

The file __init__.py runs when the package is first imported. It may be completely empty, and often is. Its three real uses are:

1. Defining the public surface

# File: notesapp/__init__.py
"""A small note taking library."""

from .storage import save, load
from .formatting import title
from .search import find

__version__ = "1.0.0"
__all__ = ["save", "load", "title", "find"]

import notesapp

notesapp.save("a note")        # instead of notesapp.storage.save
print(notesapp.__version__)

Users of the package now import from one place. The internal file layout can be reorganised later without breaking anyone.

2. Package level constants

# File: notesapp/__init__.py
from pathlib import Path

PACKAGE_ROOT = Path(__file__).resolve().parent
DEFAULT_STORE = PACKAGE_ROOT / "data" / "notes.txt"
MAX_TITLE_LENGTH = 120

3. Keeping it empty

An empty __init__.py is a perfectly good choice. It marks the directory as a package and leaves users to import submodules explicitly, which keeps import cost low.

Since Python 3.3 a directory without __init__.py can still be imported, as a "namespace package". Include the file anyway: it makes the intent explicit, it makes tooling behave predictably, and it gives you somewhere to put __all__ later.

Nested packages

notesapp/
    __init__.py
    core/
        __init__.py
        storage.py
        models.py
    text/
        __init__.py
        formatting.py
        search.py
    cli/
        __init__.py
        __main__.py

import notesapp.core.storage
from notesapp.core import models
from notesapp.text.formatting import title
from notesapp.core.storage import save as save_note

Every directory in the chain needs its own __init__.py. The dotted path mirrors the folder structure exactly.

Relative imports

# File: notesapp/text/search.py

from .formatting import title          # same package: notesapp.text
from ..core.storage import load        # up one level, then into core
from ...other import thing             # up two levels

# Absolute equivalents
from notesapp.text.formatting import title
from notesapp.core.storage import load

Relative or absolute?

Use absolute imports by default, and relative imports for modules that clearly belong together inside one subpackage. Be consistent within a project.

The error everyone meets

$ python notesapp/text/search.py
ImportError: attempted relative import with no known parent package

Running a file inside a package by path means Python does not know it is part of a package, so relative imports have nothing to be relative to. Run it as a module instead:

$ python -m notesapp.text.search       # from the directory ABOVE notesapp

Making a package runnable

# File: notesapp/__main__.py
import sys

from .core.storage import load
from .text.formatting import title


def main(argv=None):
    argv = sys.argv[1:] if argv is None else argv
    if not argv:
        print("usage: python -m notesapp COMMAND", file=sys.stderr)
        return 2
    if argv[0] == "list":
        for note in load():
            print(title(note))
        return 0
    print(f"unknown command: {argv[0]}", file=sys.stderr)
    return 1


if __name__ == "__main__":
    sys.exit(main())

$ python -m notesapp list

A realistic layout

project/
    README.md
    notesapp/
        __init__.py          public API and version
        __main__.py          python -m notesapp
        core/
            __init__.py
            models.py        the data structures
            storage.py       reading and writing
        text/
            __init__.py
            formatting.py
            search.py
        errors.py            the exception hierarchy
    tests/
        test_storage.py
        test_search.py
    data/
        notes.txt

# File: notesapp/errors.py
class NotesError(Exception):
    """Base class for every error raised by notesapp."""


class StorageError(NotesError):
    """Reading or writing notes failed."""


class NoteNotFound(NotesError):
    """The requested note does not exist."""

One errors.py holding the whole exception hierarchy is a pattern worth adopting. Every other module imports from it, and callers catch NotesError to cover the entire package.

Import cost

# File: notesapp/__init__.py

# Eager: everything loads the moment anyone imports notesapp
from .core.storage import save, load
from .text.search import find


# Lazy: loaded only when actually used
def find(*args, **kwargs):
    from .text.search import find as _find
    return _find(*args, **kwargs)

A heavy __init__.py makes every import of the package slow, even for a program that needs one function. For a small package, eager imports are simpler and fine. For a large one, keep __init__.py thin.

Inspecting a package

import json

print(json.__name__)          # json
print(json.__package__)       # json
print(json.__path__)          # the directory - only packages have this
print(json.__file__)          # .../json/__init__.py

import math
print(hasattr(math, "__path__"))     # False - math is a module, not a package

Common mistakes

• Forgetting __init__.py in a nested directory and getting confusing import failures.
• Running a file inside a package by path and meeting the relative import error.
• Importing the package name from inside the package itself, creating a circular import.
• Putting heavy work in __init__.py, so every import is slow.
• Mixing relative and absolute imports inconsistently across one project.
• Naming a package after a standard library module.

Best practices

• Include __init__.py in every package directory, even when empty.
• Use __init__.py to define a small, stable public API with __all__.
• Prefer absolute imports; use relative imports only within a tight subpackage.
• Run package code with python -m package.module, never by file path.
• Keep one module for the exception hierarchy.
• Keep tests outside the package, in their own directory.

Practice

1. Build a package with two subpackages and import a function from each, both ways.
2. Write an __init__.py that re-exports three functions and set __all__.
3. Trigger the relative import error deliberately, then fix it with -m.
4. Add a __main__.py so your package runs as a command.
5. Add an errors.py with a base exception and two subclasses, and use it from two modules.

Conclusion

A package is a directory of modules with an __init__.py that defines what the outside world sees. Use absolute imports by default, relative imports within a subpackage, and run package code with python -m so relative imports resolve.