Copying, templating and serialization

Variables, entities, and graphs can be copied, templated, serialized, or instantiated from serializations. Here we show how to use these methods. We also discuss, how to copy or template non-self-contained objects.

Imports

First let us import the required packages, classes, and functions:

import halerium.core as hal

from halerium.core import Variable, StaticVariable, Entity, Graph
from halerium.core.utilities.print import print_child_tree

Copying

Copying variables

One can copy variable, entity, and graph instances by calling their copy method. As an example, let us create a Variable called ‘v’:

v = Variable('v', shape=(3,), mean=(0.4, 1.4, 2.4), variance=(0.2, 1.2, 2.2))

We now call v‘s copy method to create a new variable with name ’w’ (which is passed as an argument to the method) and assign the result to the python variable w:

w = v.copy('w')

The copy w has the same shape and statistical properites as the original v, i.e. both are normally distributed with mean=(0.4, 1.4, 2.4) and variance=(0.2, 1.2, 2.2):

print("shapes are equal:", v.shape == w.shape)
print("means are equal:", all(v.mean.value == w.mean.value))
print("variances are equal:", all(v.variance.value == w.variance.value))

shapes are equal: True
means are equal: True
variances are equal: True

Note that v and w are not identical, but just i.i.d. So they may generally take on different values. This is in contrast to variable links, which identify the linked variables such that both always take on identical values.

Copying entities

As another example, let us create a ‘car’ entity:

car = Entity("car")
with car:
    Entity("body")
    Entity("engine")
    with car.engine:
        Variable("power", mean=160., variance=4.)
        Variable("weight", mean=2.4, variance=0.01)

print_child_tree(car)

car
├─body
└─engine
  ├─power
  └─weight

We now use copy on the entity “car” to create an entity called “enhanced_car”. The copy method takes the name of the copy as argument:

another_car = car.copy("another_car")

print_child_tree(car)
print_child_tree(another_car)

car
├─body
└─engine
  ├─power
  └─weight
another_car
├─body
└─engine
  ├─power
  └─weight

The thus created entity ‘another_car’ has the same structure as the original ‘car’, and all the variables also have the same statistical properties.

Copying graphs

This creates a graph that takes a car and enhances its power by a factor that is intrinsic to the enhancement process:

enhance_car_engine = Graph("enhance_car_engine")
with enhance_car_engine:
    with inputs:
        car = Entity("car")
        with car:
            Entity("body")
            Entity("engine")
            with car.engine:
                Variable("power", mean=160., variance=4.)
                Variable("weight", mean=2.4, variance=0.01)
    with outputs:
        enhanced_car = car.copy("enhanced_car")

    enhancement_factor = StaticVariable("enhancement_factor", mean=2.0, variance=0.3**2)

    enhanced_car.engine.power.mean = car.engine.power * enhancement_factor

print_child_tree(enhance_car_engine)

#hal.show(enhance_car_engine)

enhance_car_engine
├─inputs
│ └─car
│   ├─body
│   └─engine
│     ├─power
│     └─weight
├─outputs
│ └─enhanced_car
│   ├─body
│   └─engine
│     ├─power
│     └─weight
└─enhancement_factor

Now let us make a copy of that graph:

another_enhance_car_engine = enhance_car_engine.copy('another_enhance_car_engine')

print_child_tree(another_enhance_car_engine)

another_enhance_car_engine
├─inputs
│ └─car
│   ├─body
│   └─engine
│     ├─power
│     └─weight
├─outputs
│ └─enhanced_car
│   ├─body
│   └─engine
│     ├─power
│     └─weight
└─enhancement_factor

The resulting graph has the same structure as the original graph, and the variables of the copy have the same statistical properties as those of the original graph.

Templates

One can also create templates from an instance. This does not immediately create another instance, but the template can then be called to create new instances:

car = Entity("car")
with car:
    Entity("body")
    Entity("engine")
    with car.engine:
        Variable("power", mean=160., variance=4.)
        Variable("weight", mean=2.4, variance=0.01)

car_template = car.get_template("car_template")

another_car = car_template("another_car")

yet_another_car = car_template("yet_another_car")

print_child_tree(car)
print_child_tree(another_car)
print_child_tree(yet_another_car)

car
├─body
└─engine
  ├─power
  └─weight
another_car
├─body
└─engine
  ├─power
  └─weight
yet_another_car
├─body
└─engine
  ├─power
  └─weight

Serialization

Variables, entities, and graphs can also be serialized, i.e. their structure can be stored in a format suitable for storing on disk or transmitting to other programs.

Consider again the variable:

v = Variable('v', shape=(3,), mean=(0.4, 1.4, 2.4), variance=(0.2, 1.2, 2.2))

The dump_dict() method creates a specification dictionary, which contains only JSON-compatible standard types (list, dict, string, float) and fully specifies all properties of the scopetor:

v_dict = v.dump_dict()
display(v_dict)

{'version': '3.0.0',
 'scopetors': {'v': {'class': 'Variable',
   'global_name': 'v',
   'shape': [3],
   'dtype': 'float',
   'distribution': 'NormalDistribution',
   'variables': [],
   'code': "\nopConst0 = Const([0.4, 1.4, 2.4], shape=[3], dtype='float')\nv.set_distribution_parameter('mean', opConst0)\nopConst1 = Const([0.2, 1.2, 2.2], shape=[3], dtype='float')\nv.set_distribution_parameter('variance', opConst1)"}}}

The dump_string() method creates a JSON string that fully specifies all properties of the scopetor:

v_string = v.dump_string()
display(v_string)

'{"version": "3.0.0", "scopetors": {"v": {"class": "Variable", "global_name": "v", "shape": [3], "dtype": "float", "distribution": "NormalDistribution", "variables": [], "code": "\\nopConst0 = Const([0.4, 1.4, 2.4], shape=[3], dtype=\'float\')\\nv.set_distribution_parameter(\'mean\', opConst0)\\nopConst1 = Const([0.2, 1.2, 2.2], shape=[3], dtype=\'float\')\\nv.set_distribution_parameter(\'variance\', opConst1)"}}}'

For convenience, there is also the dump_file() method that takes a file name or file-like object, and then stores the scopetor specification as a JSON string in the associated file.

Deserialization

A variable, entity, or graph specification created by one of the aforementioned serialization methods can be used to create new instances.

One simply has to call the approriate class method from_specification with that specification dict, string, or file as argument. Optionally, one can give the new instance another halerium name. For example,

w = Variable.from_specification(v_dict, overwrite_name='w')
print("w=", w)
print("w.mean=", w.mean.value)

w= <halerium.Variable 'w'>
w.mean= [0.4 1.4 2.4]

Likewise,

w = Variable.from_specification(v_string, overwrite_name='w')
print("w=", w)
print("w.mean=", w.mean.value)

w= <halerium.Variable 'w'>
w.mean= [0.4 1.4 2.4]

One can also create an instance from a file holding a specification by providing from_specification with a file name or file object.

Copying, templating, and serialization of non-self-contained objects

For a variable, entity, or graph to be faithfully copied or stored in a template, specification dict, string, or file the object to be serialized must be fully self-contained. For example, none if its variable distribution paramerers (e.g. mean or variance) can depend on quantities outside the object itself.

For example, in the enhance_car_engine graph,

with Graph("enhance_car_engine") as enhance_car_engine:
    with inputs:
        car = Entity("car")
        with car:
            Entity("body")
            Entity("engine")
            with car.engine:
                Variable("power", mean=160., variance=4.)
                Variable("weight", mean=2.4, variance=0.01)
    with outputs:
        enhanced_car = car.copy("enhanced_car")

    enhancement_factor = StaticVariable("enhancement_factor", mean=2.0, variance=0.3**2)

    enhanced_car.engine.power.mean = car.engine.power * enhancement_factor

the entity enhance_car_engine.inputs.car is self-contained and can be copied in full:

another_car = enhance_car_engine.inputs.car.copy('another_car')
print(another_car.engine.power.mean.value)

160.0

The enhance_car_engine.outputs.enhanced_car, however, is not self-contained since its mean engine power depends on the input car engine power and the enhancement factor. Simply trying to copy it fails:

try:
    yet_another_car = enhance_car_engine.outputs.enhanced_car.copy('yet_another_car')
except Exception as error:
    print(error)

Scope <halerium.Entity 'enhance_car_engine/outputs/enhanced_car'> not self-contained. Not self-contained scopetors can only be serializedwith strict=False.

The error gives a hint, how to copy that car nevertheless:

yet_another_car = enhance_car_engine.outputs.enhanced_car.copy('yet_another_car', strict=False)
print(yet_another_car.engine.power.mean)

None

Thus, if an object to be copied, templated, or serialized does depend on outside quantities, these dependencies will be replaced by None in the copy, template, or serialization. In addition, a warning may be generated for each such replacement.