Objects, References and Identity

Everything is an object

for value in [42, 3.5, "text", [1], {"a": 1}, len, int, type, None, print]:
    print(f"{type(value).__name__:<12}{id(value)}")

Numbers, strings, functions, classes, modules and None are all objects. There is no separate category of "primitive" values as there is in Java or C. That is why a function can be stored in a list and a class can be passed as an argument.

The three properties

value = [1, 2, 3]

print(id(value))          # identity - unique while the object lives
print(type(value))        # type     - fixed for the object's whole life
print(value)              # value    - may change, for a mutable object

Names are labels

a = [1, 2, 3]
b = a

print(id(a) == id(b))     # True - one object, two names

b.append(4)
print(a)                  # [1, 2, 3, 4]

b = [9]                   # rebinds b only
print(a)                  # [1, 2, 3, 4] - unchanged

a ──┐
    ├──► [1, 2, 3, 4]        after b = a and b.append(4)
b ──┘

a ─────► [1, 2, 3, 4]        after b = [9]
b ─────► [9]

Assignment binds a name. It never copies an object, and it never changes one.

Reference counting in action

import sys

value = [1, 2, 3]
print(sys.getrefcount(value))       # 2: the name, plus the argument to getrefcount

other = value
print(sys.getrefcount(value))       # 3

container = [value, value]
print(sys.getrefcount(value))       # 5

del other
print(sys.getrefcount(value))       # 4

Every object records how many references point at it. When that count reaches zero the object is destroyed immediately - which is the main way Python reclaims memory.

Passing arguments

def rebind(items):
    items = [9, 9]              # binds the LOCAL name to a new object
    return items


def mutate(items):
    items.append(9)             # changes the caller's object


original = [1, 2]

rebind(original)
print(original)                 # [1, 2] - untouched

mutate(original)
print(original)                 # [1, 2, 9] - changed

Python is neither "pass by value" nor "pass by reference" in the classical sense. The reference is passed by value: the function gets its own name pointing at the caller's object. Rebinding that name affects nothing outside; mutating the object affects everyone.

Interning and caching

a = 256
b = 256
print(a is b)          # True - small integers are pre-created

a = 257
b = 257
print(a is b)          # often False

a = "hello"
b = "hello"
print(a is b)          # True - simple string literals are interned

a = "hello world!"
b = "hello world!"
print(a is b)          # True at module level, may differ when built at runtime

a = "".join(["hel", "lo"])
print(a == "hello", a is "hello")     # True, then usually False

CPython caches the integers from -5 to 256 and interns strings that look like identifiers. This is an implementation detail, not a rule of the language. Never use is to compare values; use ==. The behaviour above is a favourite interview question precisely because relying on it is a bug.

import sys

a = sys.intern("some dynamic " + "string")
b = sys.intern("some dynamic string")
print(a is b)          # True - explicitly interned

Mutability, precisely

text = "hello"
print(id(text))
text += " world"
print(id(text))        # different - a NEW string was created

items = [1, 2]
print(id(items))
items += [3]
print(id(items))       # the SAME - the list was extended in place

a = [1, 2]
b = a
a += [3]               # in place: b sees it
print(b)               # [1, 2, 3]

a = [1, 2]
b = a
a = a + [3]            # new object: b does not
print(b)               # [1, 2]

Immutable objects: int, float, str, bytes, tuple, frozenset, bool, None. Everything else you are likely to meet is mutable.

Hashability

print(hash(42), hash("abc"), hash((1, 2)))
# print(hash([1, 2]))          # TypeError: unhashable type: 'list'

print(hash((1, 2)) == hash((1, 2)))       # True - equal values, equal hashes

# A tuple containing a list is not hashable
# print(hash((1, [2])))        # TypeError

The rule Python relies on: if a == b then hash(a) == hash(b). A mutable object could change after being used as a key, breaking that rule, so mutable built-ins are deliberately unhashable.

class Point:
    def __init__(self, x, y):
        self.x = x
        self.y = y

    def __eq__(self, other):
        return isinstance(other, Point) and (self.x, self.y) == (other.x, other.y)

    def __hash__(self):
        return hash((self.x, self.y))      # over values that must not change


locations = {Point(0, 0): "origin"}
print(locations[Point(0, 0)])              # origin

How attributes are stored

class Product:
    category = "stationery"          # on the class

    def __init__(self, name):
        self.name = name             # on the instance


p = Product("Notebook")

print(p.__dict__)                    # {'name': 'Notebook'}
print("category" in p.__dict__)      # False
print(Product.__dict__["category"])  # stationery
print(p.category)                    # found on the class

p.category = "office"                # creates an INSTANCE attribute
print(p.__dict__)                    # {'name': 'Notebook', 'category': 'office'}
print(Product.category)              # stationery - the class is untouched

Attribute lookup checks data descriptors on the class, then the instance dictionary, then the class and its ancestors in MRO order, then __getattr__.

import sys


class WithDict:
    def __init__(self, x, y):
        self.x = x
        self.y = y


class WithSlots:
    __slots__ = ("x", "y")

    def __init__(self, x, y):
        self.x = x
        self.y = y


a, b = WithDict(1, 2), WithSlots(1, 2)
print(sys.getsizeof(a) + sys.getsizeof(a.__dict__))
print(sys.getsizeof(b))              # noticeably smaller, no per instance dict

Measuring memory

import sys

print(sys.getsizeof(0))              # an int object still costs bytes
print(sys.getsizeof(""))
print(sys.getsizeof([]))
print(sys.getsizeof([1, 2, 3]))
print(sys.getsizeof({}))

inner = [1, 2, 3]
outer = [inner, inner, inner]
print(sys.getsizeof(outer))          # only the list of three references

getsizeof reports the size of the object itself, not of what it refers to. A list of a million strings reports only the size of the pointer array.

import sys


def deep_size(obj, seen=None):
    """Total size including referenced objects, counting each one once."""
    seen = set() if seen is None else seen
    if id(obj) in seen:
        return 0
    seen.add(id(obj))
    size = sys.getsizeof(obj)
    if isinstance(obj, dict):
        size += sum(deep_size(k, seen) + deep_size(v, seen) for k, v in obj.items())
    elif isinstance(obj, (list, tuple, set, frozenset)):
        size += sum(deep_size(item, seen) for item in obj)
    return size


print(deep_size([["a" * 100] * 10] * 10))

The is versus == summary

a = [1, 2]
b = [1, 2]
c = a

print(a == b, a is b)        # True False  - equal values, different objects
print(a == c, a is c)        # True True   - the same object

print(None is None)          # the correct None test
print([] is [])              # False - always two new objects
print(() is ())              # True  - the empty tuple is a singleton

• == asks the objects whether they are equal, by calling __eq__.
• is compares identities. It never calls any method and can never be overridden.
• Use is for None, True, False and deliberate identity checks. Use == everywhere else.

Common mistakes

• Using is to compare numbers or strings, and being misled by interning.
• Believing assignment copies a container.
• Expecting a function to be unable to change its arguments.
• Assuming getsizeof reports the full memory footprint.
• Defining __hash__ over attributes that later change.
• Relying on the small integer cache in production code.

Best practices

• Think in terms of names bound to objects, not variables holding values.
• Use == for values, is for None and identity.
• Return new objects from functions rather than mutating arguments.
• Base __hash__ only on immutable attributes.
• Use __slots__ for classes created in very large numbers.

Practice

1. Draw the name to object diagram for a = [1]; b = a; b = b + [2] and predict a.
2. Show a is b giving True for 256 and False for 257, and explain why you must not rely on it.
3. Track the reference count of an object as you add and remove references.
4. Write a class safely usable as a dictionary key, and one that is not, explaining the difference.
5. Measure the memory of a class with and without __slots__ for 100 000 instances.

Conclusion

Everything is an object with an identity, a type and a value. Names are labels; assignment rebinds them and never copies. Mutability decides whether a change is visible through other names, and hashability follows directly from immutability.