metatable module

Table data structure that supports the introduction of user-defined workflow combinators and the use of these combinators in concise workflow descriptions.

class metatable.metatable.metatable(iterable, name=None, header=False)[source]

Bases: object

Class for the extensible metatable data structure.

Parameters

iterable (Iterable) – Iterable of rows corresponding to the data in this instance.
name (Optional[str]) – Instance name.
header (Optional[bool]) – Header row consisting of column names.

>>> t = metatable([['a', 0], ['b', 1], ['c', 2]])
>>> list(iter(t))
[['a', 0], ['b', 1], ['c', 2]]

All rows in an instance can be updated in-place using a symbolic representation of the transformation that must be applied to each row.

>>> t = metatable([['char', 'num'], ['a', 0], ['b', 1]], header=True)
>>> t.update({1: column(0)})
[['char', 'num'], ['a', 'a'], ['b', 'b']]

Find more examples under the entries for the update and update_filter methods.

__iter__()[source]

Return this instance as an iterable.

>>> t = metatable([['a', 0], ['b', 1], ['c', 2]])
>>> list(iter(t))
[['a', 0], ['b', 1], ['c', 2]]

Return type: Iterable

map(function, iterable, progress)[source]

Internal method for mapping over the data in the table. This method can be redefined in derived classes to change how rows are processed (e.g., to introduce multiprocessing).

Parameters

function (Callable) – Function to apply to every item in the iterable.
iterable (Iterable) – Iterable of items to which the function should be applied (this should normally be the instance itself).
progress (Callable) – Function that returns its iterable input and reports progress.

>>> t = metatable([['a', 0], ['b', 1], ['c', 2]])
>>> list(t.map(lambda row: [[row[1], row[0]]], t, lambda _: _))
[[0, 'a'], [1, 'b'], [2, 'c']]

Return type: Iterable

update_filter(update, filter, header=None, strict=False, progress=lambda *a, **ka: ...)[source]

Perform update-then-filter operations across the entire table, based on symbolic expressions for the update and filter task(s). The result of the operation is returned.

Parameters

update (symbol) – Symbolic expression that represents an update operation (to be applied to every row).
filter (symbol) – Symbolic expression that represents a filter predicate (to be tested for every row).
header (Optional[list]) – Header row for the overall result of this method.
strict (Optional[bool]) – Drop columns that do not explicitly appear in the update expression.
progress (Optional[Callable]) – Function that returns its iterable input and reports progress.

>>> t = metatable([['a', 0], ['b', 1], ['c', 2]])
>>> t.update_filter({0: column(1)}, column(1) > symbolism.symbol(0))
[[1, 1], [2, 2]]

This instance is modified in-place, so iterating over it again yields the updated version.

>>> list(t)
[[1, 1], [2, 2]]

This method can be used in combination with the row class to introduce the row index into a column during the update.

>>> t = metatable([['a'], ['b'], ['c']])
>>> t.update_filter({3: row}, column(3) < 2)
[['a', None, None, 0], ['b', None, None, 1]]
>>> list(t)
[['a', None, None, 0], ['b', None, None, 1]]

Return type: list

update(update, header=None, strict=False, progress=lambda *a, **ka: ...)[source]

Update operation across the entire table, based on a symbolic expression for the update task(s).

Parameters

update (symbol) – Symbolic expression that represents an update operation (to be applied to every row).
header (Optional[list]) – Header row for the overall result of this method.
strict (Optional[bool]) – Drop columns that do not explicitly appear in the update expression.
progress (Optional[Callable]) – Function that returns its iterable input and reports progress.

>>> t = metatable([['a', 0], ['b', 1], ['c', 2]])
>>> t.update({0: column(1)}) # Replace first-column value with second-column value.
[[0, 0], [1, 1], [2, 2]]
>>> list(t)
[[0, 0], [1, 1], [2, 2]]

If a header row is present (and should be preserved when performing the update), this can be indicated using the header argument.

>>> t = metatable([['char', 'num'], ['a', 0], ['b', 1]], header=True)
>>> t.update({1: column(0)})
[['char', 'num'], ['a', 'a'], ['b', 'b']]

This method can be used in combination with the drop class in order to indicate that a column should be dropped during the update.

>>> t.update({0: drop})
[['num'], ['a'], ['b']]
>>> t.update({0: drop})
[[], [], []]
>>> t = metatable([['a', 0, True], ['b', 1, True], ['c', 2, False]])
>>> t.update([column(1), column(0), drop])
[[0, 'a'], [1, 'b'], [2, 'c']]

If the strict argument is assigned the value True, then columns that do not explicitly appear in the update task specification are dropped.

>>> t = metatable([['c', 'n', 'b'], ['a', 0, True], ['b', 1, True]], header=True)
>>> t.update([column(1), column(0)], strict=True, header=['n', 'c'])
[['n', 'c'], [0, 'a'], [1, 'b']]
>>> t.update([column(1)], strict=True)
[['n'], ['a'], ['b']]
>>> t.update([column(0)], strict=True, header=['c'])
[['c'], ['a'], ['b']]
>>> t.update({2: 'x'})
[['c', None, None], ['a', None, 'x'], ['b', None, 'x']]

Other common operations (such as the functions pre-defined within the symbolism library) can be used to introduce a new computed column (in which the entry for that column in every row is computed using zero or more of the values from that row found in the existing columns).

>>> t = metatable([['a', 0], ['b'], ['c', 2]])
>>> t.update({2: symbolism.is_(column(1), None)})
[['a', 0, False], ['b', None, True], ['c', 2, False]]

Return type: list

class metatable.metatable.row[source]

Bases: object

Symbolic representation of a row index (for use with methods such as metatable.update).

>>> t = metatable([['a'], ['b'], ['c']])
>>> t.update_filter({3: row}, column(3) < 2)
[['a', None, None, 0], ['b', None, None, 1]]

class metatable.metatable.drop[source]

Bases: object

Symbolic representation of a column drop operation (for use with methods such as metatable.update).

>>> t = metatable([['char', 'num'], ['a', 0], ['b', 1]], header=True)
>>> t.update({1: column(0)})
[['char', 'num'], ['a', 'a'], ['b', 'b']]
>>> t.update({0: drop})
[['num'], ['a'], ['b']]

class metatable.metatable.column(instance)[source]

Bases: symbolism.symbolism.symbol

Symbolic representation of a column specifier, such as a numerical index or an attribute name (for use with methods such as metatable.update).

>>> t = metatable([['a', 0], ['b', 1], ['c', 2]])
>>> t.update_filter({0: column(1)}, column(1) > symbolism.symbol(0))
[[1, 1], [2, 2]]