Data Structures

weakref - module allows the Python programmer to create weak references to objects

Logging

import logging
import logging.handlers

# Sets up a basic textfile log, with formatting
logging.basicConfig(level=logging.DEBUG,
   format='%(asctime)s %(levelname)-8s %(message)s',
   datefmt='%m/%d/%y %H:%M:%S',
   filename=r'C:\temp\mylog.log',
   filemode='a')

# Log a few different events
logging.info('Just testing the water.')
logging.warning('Hmm, something is not right here')
logging.error("Oh no, now you're in for it")

The resulting text log:

02/14/08 22:19:03 INFO     Just testing the water.
02/14/08 22:19:03 WARNING  Hmm, something is not right here
02/14/08 22:19:03 ERROR    Oh no, now you're in for it

Add a few more lines and it sends you an email for any logs that are level "ERROR" or above:

email = logging.handlers.SMTPHandler('smtp.foo.com',
   'script@foo.com',('techart@bar.com'),'Error Report')
email.setLevel(logging.ERROR)

logging.getLogger('').addHandler(email)

Python String Templates

import string
template = string.Template("The $speed $color $thing1")
template.substitute(speed='quick', color='brown', thing1='fox')
# 'The quick brown fox'

Python Standard Library: Collections

Counter

Dict subclass for counting hashable items. Sometimes called a bag or multiset. Elements are stored as dictionary keys and their counts are stored as dictionary values.

Counter(['a', 'b', 'c', 'a'])
# Counter({'a': 2, 'c': 1, 'b': 1})

namedtuple

Returns a new subclass of tuple with named fields.

Point = namedtuple('Point', 'x y z')
Point.__doc__
# 'Point(x, y, z)'
p = Point(x=1, y=2, z=3)
print p.x, p.y, p.z
# 1 2 3

Python Standard Library: Functools

partial

New function with partial application of the given arguments and keywords

from functools import partial
def f(a, b=2):
    print a, b

f1 = partial(f, 'fixed_a')
f1(b=777)
# fixed_a 777

f2 = partial(f, b='fixed_b')
f2(777)
# 777 fixed_b

Python Standard Library: Itertools

chain

Return a chain object whose .next() method returns elements from the first iterable until it is exhausted, then elements from the next iterable, until all of the iterables are exhausted.

[x for x in chain([1, 2],[3, 4])]
# [1, 2, 3, 4]

izip

Works like the zip() function but consumes less memory by returning an iterator instead of a list.

[x for x in izip([1, 2],[3, 4])]
# [(1, 3), (2, 4)]

[x for x in izip([1, 2, 4],[3, 4])]
# [(1, 3), (2, 4)]

islice

Works like the slice() function but consumes less memory by returning an iterator instead of a list.

[x for x in islice([0, 1, 2, 3, 4, 5, 6, 7, 8, 9], 4)] # first n-elements
# [0, 1, 2, 3]

[x for x in islice([0, 1, 2, 3, 4, 5, 6, 7, 8, 9], 2, 5)]
# [2, 3, 4]

[x for x in islice([0, 1, 2, 3, 4, 5, 6, 7, 8, 9], 0, 10, 3)]
# [0, 3, 6, 9]

#same with count()
from itertools import islice, count
[x for x in islice(count(), 0, 10, 3)]
# [0, 3, 6, 9]

imap

Like map() except that it returns an iterator instead of a list and that it stops when the shortest iterable is exhausted instead of filling in None for shorter iterables.

[x for x in imap(lambda x: 2*x, [1,2,3])]
# [2, 4, 6]

combination

Return successive r-length combinations of elements in the iterable.

[x for x in combinations([1, 2, 3], 1)]
# [(1,), (2,), (3,)]

[x for x in combinations([1, 2, 3], 2)]
# [(1, 2), (1, 3), (2, 3)]

[x for x in combinations([1, 2, 3], 3)]
# [(1, 2, 3)]

[x for x in combinations([1, 2, 3], 4)]
# []

cycle

Return elements from the iterable until it is exhausted. Then repeat the sequence indefinitely.

for x in cycle(['one', 'two', 'three']):
    print x
  
one
two
three
one
two
three
one
two
......

repeat

Create an iterator which returns the element for the specified number of times. If not specified, returns the element endlessly.

for x in repeat(['one', 'two', 'three'], 2):
    print x
['one', 'two', 'three']
['one', 'two', 'three']

Much faster then

for x in ['one', 'two', 'three'] * 2:
    print x