Metaclasses¶

What is a Metaclass?¶

A metaclass is like a decorator but for classes
Specifically, a metaclass is a class that creates a class
It’s a class that gets invoked when it’s assigned to the magic attribute __metaclass__.

How Python normally creates class objects ¶

In Python code you create a class with the class keyword:

>>> class Fred(object):
...     hair_color = 'brown'
...
>>> type(Fred)
<type 'type'>
>>> Fred.__name__
'Fred'
>>> Fred.hair_color
'brown'

Here is an equivalent to what Python did behind the scenes:

>>> Fred = type('Fred', (object,), {'hair_color':'brown'})
>>> type(Fred)
<type 'type'>
>>> Fred.__name__
'Fred'
>>> Fred.hair_color
'brown'

Declaring __metaclass__ allows you to customize class creation by overriding the default behavior of type().

Getting Started ¶

Introducing the metaclass magic variable ¶

class __printer__(type):
    def __new__(meta, classname, bases, classDict):
        print 'Name            Type                 Value'
        print '-------------   ------------------   ---------------------------'
        for k, v in classDict.items():
            print '%-15s %-20s %r' % (k, type(v), v)

        return type.__new__(meta, classname, bases, classDict)

class Test(object):
    __metaclass__ = __printer__
    a = 47
    b = 'raspberry'
    def c(self): pass
    d = property(lambda self: None)

Expected output:

Name            Type                 Value
-------------   ------------------   ---------------------------
a               <type 'int'>         47
__module__      <type 'str'>         '__main__'
b               <type 'str'>         'raspberry'
__metaclass__   <type 'type'>        <class '__main__.__test__'>
d               <type 'property'>    <property object at 0x01A75850>
c               <type 'function'>    <function c at 0x01A17F30>

The __test__ metaclass prints out the members of its target classes. To make __test__ the metaclass of Test, we have to use the __metaclass__ magic variable.

All metaclasses inherit from class type. In most cases, you should override the __new__ magic method to modify the elements of the classDict argument. Modifying classDict has the effect of adding or modifying members of a class.

In most Python frameworks, metaclass usage is not explicit, because metaclasses are inherited by subclasses. This feature makes it easy to hide the metaclass machinery from your users. For example, Django’s Model class uses a metaclass called ModelBase. When you create models in Django, you inherit your model classes from Model and never have to explicitly use the __metaclass__ attribute.

Distinguishing different types of class members ¶

from inspect import isfunction, isdatadescriptor, isclass

def separate_members(classDict):
    d = dict(methods=[], properties=[], other=[])

    for k, v in classDict.items():
        if isfunction(v):
            d['methods'].append(k)
        elif isdatadescriptor(v):
            d['properties'].append(k)
        elif not isclass(v):
            d['other'].append(k)

    return d

Function separate_members shows you how to distinguish between methods, properties, and all other attributes from inside the metaclass. Note that methods defined in the target class appear as functions to the metaclass, hence we use isfunction instead of ismethod. Although we chose to use the inspect module for this example, it’s possible to achieve equivalent functionality with the isinstance built-in function in conjunction with the types module.

Now to make use of the separate_members function we defined above:

class __printer__(type):
    def __new__(meta, classname, bases, classDict):
        d = separate_members(classDict)

        for k, v in d.items():
            print '%s:' % k
            for name in v:
                print '- %s' % name
            print '-'*80

        return type.__new__(meta, classname, bases, classDict)

class Test(object):
    __metaclass__ = __printer__
    a = 47
    b = 'raspberry'
    def c(self): pass
    d = property(lambda self: None)
    e = 45.67
    f = [23, 24, 25]
    g = lambda self: None
    def h(self, v): self.v = v
    i = property(c, h)

Expected output:

other:
- a
- __module__
- b
- f
- e
--------------------------------------------------------------------------------
properties:
- d
- i
--------------------------------------------------------------------------------
methods:
- g
- h
- c
--------------------------------------------------------------------------------

We make use of the separate_members function we defined in the previous section to allow metaclass __test__ to print out the methods, descriptors, and other attributes of its target class.

Use Cases ¶

Generate attributes (Django)¶

from django.db import models

class Poll(models.Model):
    question = models.CharField(maxlength=200)
    pub_date = models.DateTimeField('date published')

class Choice(models.Model):
    poll = models.ForeignKey(Poll)
    choice = models.CharField(maxlength=200)
    votes = models.IntegerField()

This example was extracted from the official Django tutorial. The Model class uses a metaclass that generates additional attributes. For example, Poll objects have attributes choice_set and objects that were generated based on the declared class attributes.

Modify class attributes ¶

class GoBot(Robot):
    head      = Appendage()
    left_arm  = Appendage()
    right_arm = Appendage()

    cpu            = Component()
    matrix         = Component()
    flux_capacitor = Component()

print GoBot.appendages
print GoBot.components

Expected output:

[Appendage<head>, Appendage<left_arm>, Appendate<right_arm>]
[Component<cpu>, Component<matrix>, Component<flux_capacitor>]

Class Robot makes use of a metaclass (not shown) that causes all objects of type Appendage and Component to be added to the class attributes appendages and components, respectively.

Transform methods and properties ¶

class Transformer(Robot):
    __metaclass__ = __timer__

    def transform(self):
        "Transform into a GM vehicle of some kind"
        #...

    def attack(self, target):
        "Attack something with robotic fury"
        #...

t = Transformer()
t.transform()
t.attack(AdolfHitlerBot)

Expected output:

Took 3.715 seconds to transform!
Took 2.039 seconds to attack!

Class Transformer uses metaclass __timer__ (not shown), which transforms each of Transformer‘s methods to print out the execution duration after the completion of each method.

Recipes ¶

Automatic Debugger Methods ¶

This recipe shows you how you can focus your debugging on a specific class. In other words, have pdb run whenever any method of a class raises an exception.

class Example(object):
    __metaclass__ = __debugger__

    def divide(self, v):
        # exception if v == 0
        result = 100 / v
        return result

    def interpolate(self, *args):
        # exception if len(args) != 2
        result = "%s is %s!" % args
        return result

e = Example()
print e.divide(10)
print e.divide(0)       # enters debugger

In this recipe, we’ll explore how to transform all the methods of a class so that you automatically step into the debugger if and only if a method invocation triggers an error.

Prerequisites ¶

closures

Steps ¶

Generating Bitfield Properties ¶

We’ve all had to deal with libraries that use bitfields. It’s especially common when dealing with wrapper libraries for C/C++ libraries. For this example, assume that we have a Widget class that has a bitfield attribute style. After using this class for a while, you’ve gotten tired of having to write code like this:

w = Widget()
w.style = ENABLED | SUNKEN | TRANSPARENT

Instead, you wish you could just write code like this:

w = Widget()
w.enabled = True
w.sunken = True
w.transparent = True

The obvious way to do this would be to modify Widget to have a bunch of properties that modify the style attribute:

class Widget(object):
    style = 0

    enabled = property(
        lambda self: bool(self.style & ENABLED),
        lambda self, v: setattr(self, 'style',
            self.style | ENABLED if v else self.style ^ ENABLED),
    )
    simple = property(
        lambda self: bool(self.style & SIMPLE),
        lambda self, v: setattr(self, 'style',
            self.style | SIMPLE if v else self.style ^ SIMPLE),
    )
    sunken = property(
        lambda self: bool(self.style & SUNKEN),
        lambda self, v: setattr(self, 'style',
            self.style | SUNKEN if v else self.style ^ SUNKEN),
    )
    raised = property(
        lambda self: bool(self.style & RAISED),
        lambda self, v: setattr(self, 'style',
            self.style | RAISED if v else self.style ^ RAISED),
    )
    transparent = property(
        lambda self: bool(self.style & TRANSPARENT),
        lambda self, v: setattr(self, 'style',
            self.style | TRANSPARENT if v else self.style ^ TRANSPARENT),
    )

If you’re thinking “Gosh, it probably wasn’t fun to type all that”, you’d be totally right. Luckily for us, ever since Python 2.2 we’ve had a really good feature called metaclasses, which were created just for this kind of situation. Using a custom metaclass, we can rewrite the code like this:

class Widget(object):
    __metaclass__ = __bitproperties__
    style = 0
    bit_properties = ['enabled', 'simple', 'sunken', 'raised', 'transparent']

Here’s another possible way to rewrite it, still using metaclasses:

class Widget(object):
    __metaclass__ = __bitproperties__
    style = 0

    enabled       = bit_property(ENABLED)
    simple_border = bit_property(SIMPLE)
    sunken_border = bit_property(SUNKEN)
    raised_border = bit_property(RAISED)
    transparent_background = bit_property(TRANSPARENT)

Prerequisites ¶

Steps ¶

Metaclass Caveats ¶

Not as easy to use as frame hacks ¶

Sometimes what you want to do can be achieved using either a metaclass or a frame hack. How do you choose which to use? When prototyping, always go with the frame hack since it requires less effort. However, if you need your production code to be highly portable, use metaclasses. How you want your API to look is another consideration. Frame hacks add a different kind of structure to class definitions, and that may make your API look more elegant from a user’s perspective.

Might generate extra classes ¶

The __new__ method of a metaclass returns a new type object. Because of the way that Python’s import mechanism works, this may cause more than one class to be generated. In general, this shouldn’t be a problem. But it may be something you want to keep in mind.

Makes a mess of your code ¶

Excessive use of metaclasses can make your code harder to read and maintain. If you can achieve equivalent results using simpler approaches such as inheritance or frame hacks, use them instead.

Metaclass Exercises

Go Back