moo 無 otypes
Defining objects with moo oschema types
Table of Contents
Overview
moo otypes is way to produce instances of oschema types that tries to follow a valid-by-construction pattern. In general, otypes are native language types (eg those of Python) which derive from oschema schema data structures.
For example, in the oschema document we saw some basic use of the
general otypes pattern applied as C++ codegen in the structs.hpp.j2
and nljs.hpp.j2 templates. When instantiating a codegen'ed C++ struct
we have some immediate validity guarantees. They may then
automatically transfer to a JSON object that may be derived from the
C++ struct.
The rest of this document describes how moo applies the otypes pattern
in Python. With the use of the moo.otypes Python module, user code
may access Python classes corresponding to oschema types and in
instantiating them as Python objects gain some valid-by-construction
guarantees.
See also DUNE DAQ Command Object Creation which describes moo.otypes
in the context of one particular application.
Usage
The moo.otypes module provides high level functions to construct types
from oschema. They make use of lower-level Python metaprogramming for
the actual type construction which is also in this module.
Individual type construction
We may make an individual type from oschema data structure which is
expressed as keyword arguments to the moo.otypes.make_type() function.
For example:
import moo.otypes
Age = moo.otypes.make_type(
name="Age", doc="An age in years",
schema="number", dtype='i4', path='a.b')
myage = Age(42) # my forever age
print(f'{Age}, {myage}, {myage.pod()}')
import a.b
myage2 = Age(18)
print(f'{a.b.Age}, {myage2}, {myage2.pod()}')
['name', 'doc', 'schema', 'dtype', 'path']
{'name': 'Age', 'doc': 'An age in years', 'schema': 'number', 'dtype': 'i4', 'path': 'a.b'}
<class 'a.b.Age'>, <number Age: 42>, 42
['name', 'doc', 'schema', 'dtype', 'path']
{'name': 'Age', 'doc': 'An age in years', 'schema': 'number', 'dtype': 'i4', 'path': 'a.b'}
<class 'a.b.Age'>, <number Age: 18>, 18
The moo.otypes.make_type() directly returns a Python type object (aka
Python class) and it "places" the type object into the module tree
given the schema structure's .path attribute.
We may try to use our otype with invalid data:
myage = Age("older than the hills")
Which will return an error like:
Traceback (most recent call last):
File "<stdin>", line 8, in <module>
File "<number Age>", line 13, in __init__
File "/home/bv/dev/moo/moo/otypes.py", line 485, in update
self._value = numpy.array(val, dtype)
ValueError: invalid literal for int() with base 10: 'older than the hills'
This error illustrates how the valid-by-construction pattern works. It's not that magic is performed and the data is miraculously valid. Rather, we the developer are immediately punished if we attempt to violate the schema.
We may also produce an object from these (or more generic) types and
validate that object against schema. It is important to know that the
constraints asserted by these two validation procedures are somewhat
disjoint. As moo.otypes matures it may gain more rigor, but for now
the JSON Schema based validation on a final object will catch mistakes
that moo.otypes will miss during object construction.
Schema structure array
Creation of a system of related types is done with an array of data
structures describing its schema and the Python types are constructed
with the plural function moo.otypes.make_types(). For example:
import moo.otypes
schema = [dict(name="Pet", schema="enum",
symbols=["cat", "dog", "early personal computer"],
default="cat",
path="my.home.office",
doc="A kind of pet"),
dict(name="Desk", schema="record",
fields=[
dict(name="ontop", item="my.home.office.Pet"),
],
path="my.home.office",
doc="Model my desk")]
moo.otypes.make_types(schema)
from my.home.office import Desk, Pet
desk = Desk(ontop="cat")
print(f'{Desk}, {desk}, {desk.pod()}, {desk.ontop}')
<class 'my.home.office.Desk'>, <record Desk, fields: {ontop}>, {'ontop': 'cat'}, cat
The schema structure array is assumed to be sorted in topological
order of type dependencies. Here, this is implicitly assured by how
the schema array is formed. In more complex code one may rely on
moo.oschema.toposort() to produce a sorted schema array.
Load types from file
The highest layer function is moo.otypes.load_types(). It is a mere
convenience function that combines the moo method to load a schema
file and a call to make_types() on the result.
Any file format that moo supports may be used to provide schema, while Jsonnet is recommended. See the oschema document for more details on how to construct schema files.
We can see load_types() in action using a schema that is part of the
moo test suite:
import os
import moo.otypes
# Directly make a type in Python to use for an "any" below
moo.otypes.make_type(schema="string", name="Uni", path="test.basetypes")
from test.basetypes import Uni
# We locate and load a test schema file
here = os.path.join(os.path.dirname(__file__), "test")
types = moo.otypes.load_types("test-ogen-oschema.jsonnet", [here])
from app import Person
per = Person(email="foo@example.com", counts=[42],
affil=Uni("Snooty U"), mbti="judging")
print(f'{Person}, {per}, {per.affil}')
print(per.pod())
In the above we give load_types() a file system path (ie, [here]) so
that files in the moo test directory will be located. moo will search
this path when a file is given as a relative path.
When using the moo Python module the application may set a default
path instead of providing one to every moo.io.load() call (which
load_types() forwards to). An example of this usage is given below.
When using the moo CLI one may set the environment variable
MOO_LOAD_PATH to equivalently provide this default load path.
Help
Type information and any doc strings given in the schema are reflected
into the constructed Python types. This information can be displayed
using usual Python meta interrogation methods. For example:
import os
import moo.otypes
import moo.io
here = os.path.join(os.path.dirname(__file__), "test")
moo.io.default_load_path = [here]
types = moo.otypes.load_types("test-ogen-oschema.jsonnet")
from app import Person
help(Person)
Help on class Person in module app: class Person(moo.otypes._Record) | Person(*args, email: app.Email = None, email2: app.Email = 'me@example.com', counts: app.Counts = None, affil: app.Affiliation = None, mbti: app.MBTI = 'introversion') | | Record type Person with fields: "email", "email2", "counts", "affil", "mbti" | | Describe everything there is to know about an individual human | | Method resolution order: | Person | moo.otypes._Record | moo.otypes.BaseType | abc.ABC | builtins.object | | Methods defined here: | | __init__(self, *args, email: app.Email = None, email2: app.Email = 'me@example.com', counts: app.Counts = None, affil: app.Affiliation = None, mbti: app.MBTI = 'introversion') | Create a record type of Person | | ---------------------------------------------------------------------- | Data descriptors defined here: | | affil | | counts | | email | | email2 | | mbti | | ---------------------------------------------------------------------- | Data and other attributes defined here: | | __abstractmethods__ = frozenset() | | ---------------------------------------------------------------------- | Methods inherited from moo.otypes._Record: | | __repr__(self) | Return repr(self). | | pod(self) | Return record as plain old data. | | Will perform validation. | | update(self, *args, **kwds) | Update a record. | | An arg in args may be one of: | - a JSON string | - a dictionary | - an instance of a record of same type. | | kwds may be a dictionary. | | Dictionaries are taken to be field settings, values can be POD | or a typed object consistent with the field type. | | ---------------------------------------------------------------------- | Readonly properties inherited from moo.otypes._Record: | | field_names | Return list of field names | | fields | Return mapping of field name to field dict | | ---------------------------------------------------------------------- | Readonly properties inherited from moo.otypes.BaseType: | | ost | The object schema type | | ---------------------------------------------------------------------- | Data descriptors inherited from moo.otypes.BaseType: | | __dict__ | dictionary for instance variables (if defined) | | __weakref__ | list of weak references to the object (if defined)