The data contract¶

This page lists what sources may hold and what attaching refuses and accepts, for anyone putting data on a spec. sources maps each name the spec declares to its numbers:

import specsolve as sps

result = sps.solve(
    'dispatch.yaml',
    {'load': 'load.parquet', 'cost': cost_frame, 'p_max': p_max_frame},
)

Preparing the data gets an instance into these shapes.

What a parameter accepts¶

For a parameter declared dims: [d1, d2], the value under its key is one of:

• a parquet path;
• a table exposing the Arrow PyCapsule protocol (polars, pandas, pyarrow, duckdb) with columns d1, d2, value;
• an int or float, one value for every coordinate the parameter covers;
• a dict of label to value, over one dimension;
• a sequence (list, tuple, np.ndarray) over one dimension, positional against its index.

A number, a dict and a sequence are dense, so declare a constant as dims: []. A sequence carries no labels, so its dimension's index comes from another source.

A pd.Series attaches by its index. An unnamed index attaches to the declared dimension, and a named one by its name. A name outside the declared dimensions raises.

A MultiIndex and an xr.DataArray are refused. Pass series.reset_index() or array.to_series().reset_index().

Where coordinates come from¶

Each dimension's index is a key in sources named after the dimension: a table with a column of that name, a parquet path, or a bare sequence of labels. Other columns of the table are ignored.

An index lists each label once, in order. shift reads that order. A label on two rows is refused: table.select('generator').unique(maintain_order=True) keeps the first row of each label of a table, and list(dict.fromkeys(labels)) the first of a bare sequence.

A datetime label is held in microseconds, in the time zone it arrives in. A column must be in its index's time zone. Under a dtype: datetime dimension, a date is the microsecond its day starts on.

A dimension nothing supplies raises. Attach never reads labels out of the parameters, so a misspelled label, such as gsa for gas, is refused by name.

A relation goes under its own name, as a table of its rows with one column per column it declares. A value matching no label is refused. A keyed relation holds one row per key; a bare relation holds each row at most once.

What attaching refuses and accepts¶

A coordinate has a value, or it has no row. A null or NaN value is refused.

A number the model computes is finite:

An infinite bound never binds either. A reported quotient by zero is absent (absence).

Refused¶

Accepted¶

The tables a solve reads¶

sps.tidy(spec, sources) returns the tables a solve attaches, after every check above:

An archive holds these tables under sources/, each with specsolve_run added, and each goes back into sources as it is.

Column types in an archive¶

An archive writes each column as a type that parquet readers agree on, in sources/ and in answer/:

A table read back from an archive attaches as the original did.

Growing or replacing the data¶

update and solve_over attach through the rules above.

Do not write into data while a model or its answers are in use. A model keeps tables without a copy, so writing into the numpy array under a polars frame changes what evaluate reads and what a save records. A parquet path is read once. Change data through update, or pass a copy.