Inference tracking (APIs)

Once deployed a model creates inferences and predictions based on its configuration and training._ Inference tracking_ identifies how VIANOPS ingests and stores model inferences to the inference table (as defined by Inference mapping). To submit ground truth (i.e., the actual values) for inferences, POST to the /v1/inference-tracking endpoint.

Inferences can be provided as raw json or from cached files (where files uploaded to cache may be csv, parquet, or json, or zip files containing supported file types. Keep in mind that feature set names, column names in feature sets, and table names can contain only lowercase letters (a-z) or numbers (0-9), or the characters underscore (_) or period (.). Cache keys provide access to files uploaded to cache via v1/cache/upload.

Note: When reading a high volume of inferences, the recommendation is to use parquet files (under 500MB). For example, upload parquet files containing inferences to /v1/cache/upload, then read inference files from cache using the returned `cache_key`. Parquet files provide enhanced performance (better compression, faster uploads) and maintain data types.

Inference tracking object

{
    "uuid": "000121d1-e790-46f9-b603-c168ezzzzzzz",
    "deployment": "0234_placeholder_deployment",
    "model_name": "0234_placeholder_deployment",
    "model_stage": "primary",
    "model_version": "1",
    "hostname": "STRING",
    "starttime": "1654703247907.512",
    "endtime": "1654703247921.791",
    "num_inferences": 4,
    "inputs": [
        {
            "feature_a": 1.0,
            "feature_b": 42,
            "feature_c": 42,
            "feature_d": 1.0,
        }
    ],
    "outputs": ["1"],  
   	"ground_truth": [],  
    "xformed_inputs": [],  
    "modified_ts": null,
    "modified_indexes": null,
    "inference_mapping_index": 1,
    "inference_mapping": [{"key": "value"}, {"key": "value"}],
}
Property Type Description
uuid string Platform-assigned ID unique ID for this inference tracking.
deployment string Name of the deployment for this inference tracking.
model_name string Name of the model for this inference tracking.
model_stage string Model stage for this inference tracking.
model_version string Model version for this inference tracking.
hostname string Host name for the model.
starttime datetime Timestamp (in Unix time in milliseconds) for the first inference values.
endtime datetime Timestamp (in Unix time in milliseconds) for the last inference values.
num_inferences integer Total number of inferences included in this inference tracking.
inputs array All input features and their values.
outputs array All output (prediction) features and their values.
ground_truth array All ground truth features and their values. Ground truth can be added later using /v1/inference-tracking/ground-truth.
xformed_inputs array All transformed features and their values.
modified_ts datetime Timestamp (in Unix time in milliseconds) for the last time the inferences were updated.
modified_indexes array List of modified indexes.
inference_mapping_index integer Inference mapping index related to the inference tracking.
inference_mapping object Inference mapping values (key-value pairs) related to the inference tracking.
TABLE OF CONTENTS