Onboard your model
Bring your model onto VIANOPS
VIANOPS can monitor any model that has input, output, and ground truth, regardless of whether it was developed and deployed outside of VIANOPS. To onboard your model to VIANOPS, you can create a placeholder model by providing essential metadata such as the model name, description, type, and owner, without the need to provide the actual model file (e.g., pkl file) to VIANOPS. This process can be accomplished using either the SDK or the REST API.
For more detailed information on how to bring your model onto VIANOPS, please refer to the following link Create a placeholder model deployment
Create a project
Create a project to group your models.
Add feature importance
You can pass feature importance data to a model and show the SHAP results in a graph on the model dashboard. You can either send a JSON object through the API or upload a file.
Ingest inference data
As VIANOPS uses a placeholder model as a reference to the actual model outside of VIANOPS, you will need to ingest inference data with or without ground truth from the data sources where the inference data is stored. These data sources can include cloud storage solutions such as AWS S3 or data warehouses like Snowflake.
Since drift monitoring requires comparing the target window and baseline window, you may need to ingest historical inference data for a specified period, such as the last three months, initially. You can then set up a data pipeline to automatically ingest future inference data.
For detailed information on how to ingest inference data, please refer to the following link:
To integrate your data sources and set up an automatic data pipeline for inference data ingestion, please refer to the following link:
Ingest ground truth
Ground truth for your model may not always be immediately available and may be deferred for days, weeks, or even months. VIANOPS provides support for deferred ground truth. You do not need to ingest ground truth at the same time as inference data with input features and output predictions. Instead, you can ingest ground truth when it becomes available, and VIANOPS will compute or refresh model performance for every day that has inferences with new or updated ground truth.
This process can be easily accomplished using either the SDK or the REST API. For more detailed information on how to ingest inference data, please refer to the following link:
Create segments
A segment refers to a slice or subsection of a data set. Segmentation allows for dedicated monitoring or observation of a specific customer, age group, state, and so on. It is a powerful tool for narrowing the scope and detecting patterns that may only be present in certain sections of the population but can impact the overall behavior of the model.
VIANOPS provides flexible ways to define segments. You can define segments for California, seniors in California, or the top 4 banks by specifying conditions such as
- Segment for California: state = CA;
- Segment for seniors in California: state = CA and age >= 62,
- Segment for the top four banks: bank_name = JPMorgan Chase, Banks of America, Citigroup, Wells Fargo.
Segments can be created at any time using either the SDK or the REST API. Once created, they can be selected in a policy to begin monitoring and comparison.
For more detailed information on how to create a segment, please refer to the following link: