Dictionary Methods and Counting Patterns

The full method list

setdefault

setdefault returns the value for a key, inserting a default first if the key is absent. It is the tool for building a dictionary of collections.

groups = {}

# Verbose
for word in ["apple", "avocado", "banana"]:
    initial = word[0]
    if initial not in groups:
        groups[initial] = []
    groups[initial].append(word)

# With setdefault
groups = {}
for word in ["apple", "avocado", "banana"]:
    groups.setdefault(word[0], []).append(word)

print(groups)      # {'a': ['apple', 'avocado'], 'b': ['banana']}

d = {"a": 1}

print(d.setdefault("a", 99))     # 1  - already present, default ignored
print(d.setdefault("b", 99))     # 99 - inserted
print(d)                         # {'a': 1, 'b': 99}

get never changes the dictionary; setdefault may. That is the whole difference, and it is why setdefault is right for accumulating and wrong for a plain lookup.

Counting

text = "the cat sat on the mat the end"

# With get
counts = {}
for word in text.split():
    counts[word] = counts.get(word, 0) + 1
print(counts)

# With Counter, from the standard library
from collections import Counter

counts = Counter(text.split())
print(counts)                          # Counter({'the': 3, 'cat': 1, ...})
print(counts["the"])                   # 3
print(counts["missing"])               # 0 - never raises
print(counts.most_common(2))           # [('the', 3), ('cat', 1)]
print(sum(counts.values()))            # 8 - total words

from collections import Counter

print(Counter("mississippi"))          # Counter({'i': 4, 's': 4, 'p': 2, 'm': 1})
print(Counter([1, 1, 2, 3, 3, 3]).most_common(1))    # [(3, 3)]

a = Counter("aabbc")
b = Counter("abbbd")
print(a + b)          # counts added
print(a - b)          # counts subtracted, negatives dropped
print(a & b)          # minimum of each
print(a | b)          # maximum of each

defaultdict

from collections import defaultdict

groups = defaultdict(list)          # a missing key auto-creates an empty list
for word in ["apple", "avocado", "banana"]:
    groups[word[0]].append(word)

print(dict(groups))     # {'a': ['apple', 'avocado'], 'b': ['banana']}


totals = defaultdict(int)           # a missing key auto-creates 0
for item, amount in [("pen", 10), ("book", 50), ("pen", 5)]:
    totals[item] += amount

print(dict(totals))     # {'pen': 15, 'book': 50}


index = defaultdict(set)
for name, tag in [("a", "x"), ("b", "x"), ("a", "y")]:
    index[tag].add(name)
print(dict(index))      # {'x': {'a', 'b'}, 'y': {'a'}}

The one catch: merely reading a missing key creates it.

d = defaultdict(list)
print(len(d))          # 0
d["never set"]         # a read...
print(len(d))          # 1  ...that created a key

Choosing between the three

Inverting a dictionary

ages = {"Meera": 27, "Arun": 31, "Sara": 27}

# Simple inversion: later duplicates overwrite earlier ones
by_age = {age: name for name, age in ages.items()}
print(by_age)          # {27: 'Sara', 31: 'Arun'} - Meera was lost

# Keeping every name
from collections import defaultdict
by_age = defaultdict(list)
for name, age in ages.items():
    by_age[age].append(name)
print(dict(by_age))    # {27: ['Meera', 'Sara'], 31: ['Arun']}

Sorting a dictionary

scores = {"Meera": 92, "Arun": 78, "Sara": 85}

print(dict(sorted(scores.items())))                          # by key
print(dict(sorted(scores.items(), key=lambda p: p[1])))      # by value
print(dict(sorted(scores.items(), key=lambda p: -p[1])))     # by value, descending

print(max(scores, key=scores.get))                           # Meera
print(min(scores.values()), max(scores.values()))

max(scores, key=scores.get) reads oddly at first: it iterates the keys, scoring each by its value, and returns the winning key.

Aggregating records

sales = [
    {"region": "north", "amount": 1200},
    {"region": "south", "amount": 800},
    {"region": "north", "amount": 400},
    {"region": "east", "amount": 950},
]

from collections import defaultdict

totals = defaultdict(int)
counts = defaultdict(int)

for record in sales:
    totals[record["region"]] += record["amount"]
    counts[record["region"]] += 1

for region in sorted(totals, key=totals.get, reverse=True):
    average = totals[region] / counts[region]
    print(f"{region:<8}{totals[region]:>8,}{average:>10,.0f}")

Copying

import copy

original = {"a": [1, 2], "b": 3}

alias = original                    # not a copy
shallow = original.copy()           # or dict(original)
deep = copy.deepcopy(original)

original["a"].append(3)
print(shallow["a"])                 # [1, 2, 3] - the inner list is shared
print(deep["a"])                    # [1, 2]

Common mistakes

• Using setdefault for a plain read, quietly growing the dictionary.
• Reading a missing key from a defaultdict and creating it by accident.
• Inverting a dictionary with duplicate values and losing entries.
• Forgetting that update returns None.
• Using Counter and expecting a KeyError for a missing item; it returns 0.
• Assuming copy() protects nested values.

Best practices

• Reach for Counter the moment you write counts[x] = counts.get(x, 0) + 1.
• Reach for defaultdict the moment you write if key not in d: d[key] = [].
• Convert a defaultdict back with dict() before returning it from a public function, so callers do not inherit the auto-creation.
• Sort with key= rather than building intermediate lists.

Practice

1. Count word frequencies in a paragraph and print the five most common.
2. Group a list of file names by extension using both setdefault and defaultdict.
3. Invert a dictionary that has duplicate values without losing any keys.
4. Find the key with the largest value without sorting the whole dictionary.
5. Explain the one behaviour of defaultdict that can surprise you, with an example.

Conclusion

Counting, grouping and accumulating are the three jobs dictionaries do most. Counter handles the first, defaultdict the second, and setdefault covers the rest when you would rather not import anything.