Python Hacks

Python is my language of choice. Here is an incomplete collection of my works in it.

I've also given several lectures related to Python.

Stable

These are immediately useful and the API is expected to be stable.

Series

Documentation, Source

Lazy series abstraction tightly integrated with iterators.

This module exposes the duality between linked lists and iterators:

  • Simple linked lists have the drawback that their tail must be known and finite. This can be fixed by overloading head/tail access and making it lazy; the resulting facility is popular in functional programming languages. This is implemented by the Series classes.
  • Iterators have the drawback of being stateful (advancing them is destructive) but there is no way to capture the "remaining future" of an iterator at some moment. This can be fixed by making an iterator explicitly point to a series and allow separate iterators to be forked from it. This is implemented by the SeriesIter class.

Another useful thing about linked lists is that you can non-destructively cons items in front of them. This is implemented by the Series constructor (which is also used to represent the part of series that has already been computed) and is also accessible through the SeriesIter.unyield() method.

Dsets

Documentation, Source

A dictionary extended with set operations.

It has long been observed that sets can be emulated as dictionaries with dummy values (see dict.fromkeys()). The common operations, like dict.update() match directly. This module is simply a ''backport'' of the other goodies added in the sets module to dictionaries.

The main issue with applying set operations are collisions: given two values (which can be different) for the same key in two DSets, which value should we use? Choosing either one would be wrong in most applications, so the user is allowed to provide his own function for reconciling them. He has to provide them per call - experience showed that the desired operation chages frequently even when invoking the same method.

Alpha

These work but need improvement and/or redesign before they can be considered stable.

Autodep

Documentation, Source

Automagical attribute dependency tracking.

This module implements a framework for dependency-driven computation for object attributes. You simply inherit from Invalidator and wrap functions for computing attributes with autodep instead of property. The value of such an attribute is cached and deleted when any attribute it depends on is set or deleted. Moreover, the dependencies are automagically detected by noticing which attributes you accessed from your function.

This module works as designed but I feel it could be designed better...

Argexp

Documentation, Source

Command line parsing in the spirit of regular expressions / grammars.

This module defines an interface for "matcher" objects that can parse parts of arglists (sequences of strings) and be easily composed to describe complex command line syntaxes. Lots of such objects are provided - both elementary ones (e.g. matching one option) and compound ones (e.g. matching one of several alternative matchers). This hierarchical composition style is where this framework resembles regular expressions. It's a small domain-specific language of the kind that flourishes in LISP ;-).

For convenience, a shortcut syntax consisting of Python strings, tuples, dictionaries, etc. can be used to desribe the most useful matcher objects. Such "templates" can be used wherever a matcher object is expected and are converted to matcher objects. Templates tend to have many special cases and limitations, aimed to make the common case easy, while matcher objects are simpler and more orthogonal. See template() for full details.

Status:

This library was written before Optik entered the standard library; I still like my design better but I'm not sure when I'll continue working on it.

Parsing options seems to work well. Help output is missing. To implement it I would like to fisrt design a flexible pretty-printing framework.

Ripoff_Argexp

Source

My version of the comparison of command-line parsing libraries at http://www.python.org/sigs/getopt-sig/compare.html. The Ripoff command-line interface, implemented with argexp.

PEPs

I'm somewhat addicted to programming language design, so I'm writing some PEPs (Python Enhancement Proposals):

Concatenation By Multi-Argument Constructors.

Full PEP, reST source

Abstract

Python's current way to concatenate sequences is the + operator. For many operands it is sub-optimal (creating longer and longer temporary sequences) and it becomes awkward when you concatenate mixed types because you must explicitly convert every operand. There is no obvious way to combine dictionaries in a single expression at all. "Concatenating" iterators is supported but is placed in the itertools module.

This PEP proposes a consistent set of extensions to the list(), tuple(), set(), frozenset(), dict() and str() and unicode() constructors to concatenate several values by giving them multiple arguments. iter() is similarly extended to behave as itertools.ichain(). All arguments are converted in the same ways as a single argument is now converted.

Status:To be finished. I like the idea, think it has a good chance and plan to finish and post the PEP soon.

Construct For Writing Statements In Top-Down Order.

Full PEP, reST source

Abstract

This PEP proposes a single construct that achieves some of the benefits of embedding statements inside expressions without actually doing so - it simply allows to write statements in different order, communicating value through variable bindings. This allows it to scale to almost any scenario when one would wish for the power of statements inside expressions, unlike all the proposals to add yet another expression kind. However, it specifically does not aim for brevity.

Status:Unfinished. I posted the idea (but not in PEP form) and got the impression it won't be implemented. I'd like to finish it one day, for the record...

Refusing To Guess In The String Formatting Operation

Full PEP, reST source

Abstract

The string formatting operation - format % values - has a wart. In the face of non-tuple values, it assumes a singleton tuple around it. It's tempting to use this since singleton tuples look ugly but code using this easily breaks (when the single object happens to be tuple).

This PEP proposes several ways to fix this wart.

The problem was discussed repeatedly but the specific solutions proposed in this PEP need more feedback.

Status:Neglected. I got mixed responses for this when I posted it in March 2003. It doesn't make a lot of difference and I've got some better ideas for string formatting by now, so I don't think I'll push this PEP any more...