Reference Counting and Garbage Collection

Reference counting

import sys


class Tracked:
    def __init__(self, name):
        self.name = name

    def __del__(self):
        print(f"  {self.name} destroyed")


a = Tracked("first")
print("created")

b = a                 # count 2
del a                 # count 1, still alive
print("after del a")
del b                 # count 0, destroyed immediately
print("after del b")

created
after del a
  first destroyed
after del b

The object was freed the instant the last reference disappeared - not at some later point. This determinism is why files and locks are released predictably in CPython, and it is a property of CPython rather than of the Python language.

What changes the count

import sys

value = [1, 2, 3]
print(sys.getrefcount(value))        # 2 (the name, and getrefcount's argument)

other = value                        # +1
container = [value]                  # +1
d = {"key": value}                   # +1
print(sys.getrefcount(value))        # 5


def hold(x):
    print("inside:", sys.getrefcount(x))     # +1 while the call is running


hold(value)
print("after:", sys.getrefcount(value))

del other, container, d
print(sys.getrefcount(value))        # back to 2

The problem: cycles

class Node:
    def __init__(self, name):
        self.name = name
        self.partner = None

    def __del__(self):
        print(f"  {self.name} destroyed")


a = Node("A")
b = Node("B")

a.partner = b        # B's count is now 2
b.partner = a        # A's count is now 2

del a                # A's count drops to 1 - still referenced by b
del b                # B's count drops to 1 - still referenced by a

print("both names gone, neither object freed")

   a ──► [A] ──partner──► [B]
                ◄──partner──┘

   after del a and del b:

        [A] ──partner──► [B]
            ◄──partner──┘        unreachable, but each count is 1

Nothing can reach these objects any more, yet neither count reaches zero. Reference counting alone would leak them forever. This is precisely why the second collector exists.

The cycle collector

import gc

gc.collect()          # find and free unreachable cycles
print("collected")

  A destroyed
  B destroyed
collected

import gc

print(gc.isenabled())          # True by default
print(gc.get_threshold())      # (700, 10, 10)
print(gc.get_count())          # objects tracked in each generation

Generational collection

The collector works on the observation that most objects die young. New objects are checked frequently; anything that survives is promoted and checked less often. The default thresholds mean generation 0 is scanned after roughly 700 more allocations than deallocations.

import gc

gc.disable()                   # only for a measured, short lived reason
# ... allocation heavy work with no cycles ...
gc.enable()

print(gc.collect(generation=0))     # collect one generation only
print(len(gc.get_objects()))        # every tracked object

Disabling the collector does not disable reference counting. Non-cyclic garbage is still freed. Disable it only for a measured benefit in a phase you know creates no cycles, and turn it back on afterwards.

Finding what keeps an object alive

import gc


class Node:
    def __init__(self, name):
        self.name = name
        self.children = []


parent = Node("parent")
child = Node("child")
parent.children.append(child)

for referrer in gc.get_referrers(child):
    if isinstance(referrer, list):
        print("held by a list inside:", [n.name for n in referrer])

print(gc.get_referents(parent))      # what parent refers to

Weak references

import weakref


class Node:
    def __init__(self, name):
        self.name = name
        self.parent = None
        self.children = []

    def add(self, child):
        child.parent = weakref.ref(self)     # does NOT increase the count
        self.children.append(child)
        return child

    def __del__(self):
        print(f"  {self.name} destroyed")


root = Node("root")
leaf = root.add(Node("leaf"))

print(leaf.parent().name)          # call the weak reference to get the object

del root
print("root deleted")
print(leaf.parent())               # None - the target is gone

A weak reference points at an object without keeping it alive. Using one for the "back" direction of a parent and child relationship removes the cycle entirely, so plain reference counting can do its job.

import weakref


class Record:
    def __init__(self, key):
        self.key = key


cache = weakref.WeakValueDictionary()

record = Record("a")
cache["a"] = record
print(len(cache))          # 1

del record
print(len(cache))          # 0 - the entry vanished with the object

WeakValueDictionary is a cache that never keeps an object alive on its own. It is the correct structure for a lookup table that must not cause a memory leak.

__del__ and why to avoid it

class Risky:
    def __init__(self, path):
        self.handle = open(path, "w", encoding="utf-8")

    def __del__(self):
        self.handle.close()        # relies on WHEN this runs

• The timing is not guaranteed by the language, only by CPython's counting.
• An exception raised inside __del__ is swallowed and printed, not propagated.
• It may run during interpreter shutdown, when globals are already gone.
• An object with __del__ in a cycle could historically not be collected at all.

class Better:
    def __init__(self, path):
        self.handle = open(path, "w", encoding="utf-8")

    def close(self):
        self.handle.close()

    def __enter__(self):
        return self

    def __exit__(self, *exc):
        self.close()
        return False


with Better("out.txt") as f:
    f.handle.write("data\n")
# closed here, deterministically, by the with statement

Use a context manager. It states exactly when cleanup happens, and it does not depend on any implementation detail of the collector.

Practical memory leaks in Python

# 1. A global cache that only ever grows
CACHE = {}


def lookup(key):
    if key not in CACHE:
        CACHE[key] = expensive(key)      # nothing is ever removed
    return CACHE[key]


# Fix: bound it
from functools import lru_cache


@lru_cache(maxsize=1000)
def lookup(key):
    return expensive(key)

# 2. A list of results that is never cleared in a long running loop
results = []
while True:
    results.append(process())     # grows without limit
    break

# 3. A closure holding a large object alive
def make_handler(big_data):
    def handler(event):
        return event.upper()      # big_data is captured but never used
    return handler

Python does not leak memory in the C sense. What it does have is objects kept alive longer than intended - by caches, module level lists, closures and cycles. Those are design problems, not collector problems.

Measuring

import gc
import tracemalloc

tracemalloc.start()

data = [[i] * 100 for i in range(1000)]
snapshot = tracemalloc.take_snapshot()

for stat in snapshot.statistics("lineno")[:3]:
    print(stat)

print(f"objects tracked: {len(gc.get_objects()):,}")
tracemalloc.stop()

Common mistakes

• Believing Python has no garbage collection because it uses reference counting.
• Relying on __del__ for cleanup instead of a context manager.
• Creating parent and child cycles with strong references in both directions.
• Letting a module level cache grow without limit.
• Calling gc.collect() routinely instead of fixing what holds the references.
• Assuming del x frees memory; it removes one reference.

Best practices

• Let reference counting do its job; do not call gc.collect() in normal code.
• Use with for every resource rather than __del__.
• Use weakref for back references and for caches that must not retain objects.
• Bound every cache with lru_cache(maxsize=...) or an explicit eviction policy.
• Use tracemalloc when investigating growth, rather than guessing.

Practice

1. Create two objects referring to each other, delete both names, and show they survive until gc.collect().
2. Rewrite that pair using weakref so no collection is needed.
3. Demonstrate that an object is destroyed the instant its last reference goes.
4. Replace a __del__ based class with a context manager and explain what improved.
5. Use tracemalloc to find the line allocating the most memory in a small script.

Conclusion

Reference counting frees most objects immediately and deterministically; a generational collector exists only to break cycles. Neither needs your attention in normal code - what does need attention is anything that holds references longer than you intended.