instance — Class

instance.__class__ is type(instance) is Class.

• instance(...)
  • Class.__call__(instance, ...)
• instance.attr
  • Class.__getattribute__(instance, 'attr')
  • object.__getattribute__(instance, 'attr')
    • Class.attr.__get__(instance, Class)
    • instance.__dict__['attr']
    • Class.attr
    • Class.__getattr__(instance, 'attr')

A new-style instance is a puppet — and the Class pulls the strings.

Class — metaclass (type)

Class.__class__ is type(Class) is type.

• Class(...)
  • type.__call__(Class, ...)
    • Class.__new__(Class, ...)
    • object.__new__(Class, ...)
      • Class.__init__(self, ...)
• Class.attr
  • type.__getattribute__(Class, 'attr')
    • Class.attr.__get__(None, Class)
    • Class.__dict__['attr']
    • Class.__base__.attr # actually MRO lookup

A new-style Class is basically just a puppet factory.

Example: time property

A property returns self on class access:

>>> import time
>>> time_prop = property(lambda self: time.time())
>>> class Class(object):
...     time = time_prop
...
>>> Class.time
<property object at 0x831c2d4>

but calls the getter function on instance access:

>>> instance = Class()
>>> instance.time
1175029167.593503
>>> instance.time
1175029168.477463

type instantiation

A class statement simply instantiates type:

• class C(object): x = 1
  • Class = type('Class', (object,), {'x': 1})

We can do it ourselves:

>>> Class = type('Class', (object,), dict(time=time_prop))
>>> Class().time
1175029698.4495051

Example: class decorator

This allows us to write a "class decorator":

def class_decorator(C):
    return type(C.__name__,
                C.__bases__,
                dict(C.__dict__, time=time_prop))

>>> class Class(object):
...     "will magically tells the time"
>>> Class = class_decorator(Class)
>>> Class().time
1175029821.2147801

This gives some power, but Class.__class__ is still type.

Subclassing type

If we instantiate a subclass of type, it becomes the metaclass:

>>> class TimedType(type):
...     time = time_prop
...
>>> TimedClass = TimedType('TimedClass', (object,), {'x': 1})
>>> TimedClass.__class__
<class '__main__.TimedType'>
>>> TimedType.time
<property object at 0x831c374>
>>> TimedClass.time
1175030904.9948061
>>> TimedClass().time
Traceback (most recent call last):
    TimedClass().time
AttributeError: 'TimedClass' object has no attribute 'time'

__class__ / __base__ relations

@@@ Move below?

<Diagram>

Liberation, M.C.Escher

class statement: explicit metaclass

The member __metaclass__ (if given) is called to construct the class:

class TimedClass(object):
    def __metaclass__(name, bases, members):
        return type(C.__name__,
                    C.__bases__,
                    dict(C.__dict__, time=time_prop))

But usually it will be a class:

class TimedClass(object):

__metaclass__ = TimedType

or directly:

class TimedClass(object):

class __metaclass__(type):

time = time_prop

class statement: inherited metaclass

• class Class(object): instantiates type because object.__class__ is type.

  • When there are no base classes, the global __metaclass__ is tried. So __metaclass__ = type at the beginning of a module makes all classes newstyle.
  • The last fallback is types.ClassType (the classic types metaclass). This will change to object in Python 3000.

• If TimedClass.__class__ is TimedType:

  >>> class SubClass(TimedClass):
  ...     pass
  ...
  >>> SubClass.__class__
  <class '__main__.TimedType'>
  >>> SubClass.time
  1175057391.6200049
  

• Multiple inheritance works but with limitations but leads to the dark side...

• Usually, the most convenient interface for a metaclass is to inherit from a base class using it.

Metaclass uses

• What:
  • Validation
  • Registration
  • Wrapping / Adaptation
  • Order-stamping trick
  • Other deep magic
• Why:
  • Hiding bugs (replacing them by very subtle ones?)
  • Classes that are also objects
  • Declarative programming
  • Domain specific languages
  • Don't Repeat Yourself!

Use in moderation!

"A day without objects is a day without job security."

Example: Enum

class Enum(int):
    class __metaclass__(type):
        def __new__(metaclass, name, bases, members):
            Class = type.__new__(metaclass, name, bases, members)
            Class._names = {}
            for k, v in members.items():
                if isinstance(v, (int, long)):
                    setattr(Class, k, Class(v))
                    Class._names[v] = k
            return Class

    def __str__(self):
        return '%s.%s' % (self.__class__.__name__,
                          self._names.get(self, int(self)))

>>> class FooBar(Enum):
...     FOO = 0
...     BAR = 1
>>> print FooBar(0), FooBar.BAR
FooBar.FOO FooBar.BAR

Example: inheriting dictionaries

class Dict(object):
    class __metaclass__(type):  # , UserDict.DictMixin
        __getitem__ = type.__getattribute__
        __setitem__ = type.__setattr__

>>> class A(Dict):
...     foo = 1
>>> class B(A):
...     bar = 2
>>> B['bar']
2
>>> B['foo']
1
>>> A['foo'] = 0
>>> B['foo']
0

Python 3000

PEP 3115

• class Class(*bases, metaclass=MetaClass, **kwargs): BODY
  1. Before the class body is executed:
     • locals_dict = MetaClass.__prepare__('Class', bases, **kwargs)
     • locals_dict = {}
  2. exec '''BODY''', globals(), locals_dict
  3. Class = MetaClass('Class', bases, locals_dict) as before

The main use case is a locals_dict that remembers the order of assignments.

Questions: and what does the turtle stand on?

"You are very clever, young man, very clever. But it is turtles all the way down!"

—An old lady unconvinced by Bertrand Russell's astronomy lecture.

• instance.__class__
  • instance.__class__.__getattribute__('__class__')?
    • ...?
• instance.__dict__['attr']
  • Class.__getattribute__('__dict__')?
    • ...?
• instance.attr
  • Class.__getattribute__(instance, 'attr')
    • type.__getattribute__(Class, '__getattribute__')?
      • ...?

The Pytrix has you!

• All the lookup rules discussed above (__getattribute__ &c.) are an illusion maintained inside the "Python Matrix".
• Outside the Pytrix, every Python object is represented by an object of the underlying reality:
  • struct PyObject` and related APIs in CPython.
  • org.python.core.PyObject in Jython.
  • "Object space" Python code simulating "Application space" Python code in PyPy.
  • ...
  • A capsule with nourishing liquid in far-future ZeroOneThon ?-;

We will focus on Jython in this lecture.

The illusion of free will

• Every operation inside the Pytrix is a method call on the implemeting object.
  • This method is written and called in Java and is found directly, and cannot be hooked from inside the Pytrix.
  • The method is called on the object simulating the instance, not the class. Here we are dealing with a Pytrix — implementation relation, and don't yet have an instance — Class relation.
  • This method MAY call hooks inside the Pytrix.
    • PyObject happens to call Class.__getattribute__ inside the Pytrix, creating the illusion inside the Pytrix that every attribute access works this way.

Autonomous ajents

"It's a turtle, for heaven's sake. It swims. That's what turtles are for."

—Small Gods, Terry Pratchett

• Some objects are native: operations on them are implemented in Java and don't call back into the Pytrix.
  • Functions implement __call__ natively.
  • object and type (or to be precise, Py.ObjectType and Py.TypeType) implement attribute access natively.
    • Subclasses of them do call back into the Pytrix - but what we called .__class__ and .__dict__ are always native .getType() and .getDict() calls.
  • This is what stops the recursion.
• For consistency, such objects expose their native method as __magic__ methods inside the Pytrix.
• See also PEP 252 and PEP 253 for the CPython specification and implementation overview.

instance_PyObject — Class_PyType

instance_PyObject.getType() is Class_PyType.

• Inside the Pytrix: instance(...)
  • instance_PyObject.__call__(...)
    • Class_PyType.__findattr__('__call__').__call__()
• instance.attr
  • instance_PyObject.__findattr__('attr')
    • Class_PyType.__findattr__('attr').__findattr__('__get__')
    • instance_PyObject.getDict().__finditem__('attr')
    • Class_PyType.lookup('attr')
    • Class_PyType.__findattr__('__getattr__').__call__(instance_PyObject, 'attr')

More about Jython

• The last "stable" release was 2.1 years ago.
  • Use 2.2b1 or SVN head!
• Compiles to Java bytecode.
• The beauty is that Java supports introspection ("reflection"), so you can just import any Java module from Jython and call it right away.
  • You can subclass in Python from Java classes!
    • Not sure about the other way around,
    • You can implement Java protocols from Python, which effective means you can supply Python callbacks to Java modules.

Java Reflection

>>> from org.python.core import PyObject, PyType

Hand with Reflecting Globe, M.C.Escher

Escaping the Pytrix

Print Gallery, M.C.Escher

Questions?

Snakes, M.C.Escher

Credits

• Once I was asked "why metaclasses don't work in Jython 2.1?".
  • For two weeks I stared at a page of code and then added 2 lines. Perhaps the most profound debugging experience I ever had.
    • The same 2 lines have been added during the same weeks in Jython's CVS :-).
• Moshe Zadka's PyPy lecture at August Penguin 2003.
• Gödel, Escher, Bach - an Ethernal Golden Braid by Douglas R. Hofstader, 1979
  • "In a way, this book is a statement of my religion"
  • #1 on the Jargon File bibliography since 1990 ;-)
  • This lecture was composed in parallel to reading GEB between patrols round an army base. If this lecture had any style, it came from the book.