Job maker objects
Objects and models for VIANOPS jobs, including dataloading, deployments, model performance, preprocessing, etc. See supported Job maker APIs.
vianops_client.models.jobmaker.dataloading
- pydantic model V1DataloadingJob
Bases:
BaseModelShow JSON schema
{ "title": "V1DataloadingJob", "type": "object", "properties": { "description": { "title": "Description", "default": "Data description.", "type": "string" }, "dryrun": { "title": "Dryrun", "description": "Dry run determines if data is NOT to be saved in a table. Default value is `False` and data is saved.", "default": false, "type": "boolean" }, "dtypes": { "title": "Dtypes", "description": "A list of pandas data types as a string instance.", "type": "string" }, "filetype": { "title": "Filetype", "description": "File type containing data to be loaded. Acceptable values include: `csv`, 'json`, and 'paquet`. Parquet file types are recommended over json or csv files. Default value is `parquet`.", "default": "parquet", "type": "string" }, "overwrite": { "title": "Overwrite", "description": "Determines how to manage data if already exists. Supported values: `replace` (overwrite existing data) and `append` (add new data to existing data).", "default": "Replace", "type": "string" }, "separator": { "title": "Separator", "description": "Data separator.", "default": ",", "type": "string" }, "setname": { "title": "Setname", "description": "Feature set name.", "type": "string" }, "tablename": { "title": "Tablename", "description": "Table name.", "type": "string" }, "srcurl": { "title": "Srcurl", "description": "Source url fron where data is to be loaded.", "type": "string" }, "upload_key": { "title": "Upload Key", "description": "Cache upload key.", "type": "string" }, "job_description": { "title": "Job Description", "description": "Job description.", "anyOf": [ { "type": "string" }, { "type": "object" } ] }, "job_type": { "title": "Job Type", "description": "Job type. Supports one value: dataloading (the default value).", "default": "dataloading", "enum": [ "dataloading" ], "type": "string" }, "maintain_archives": { "title": "Maintain Archives", "description": "If `true` creates an archive table of a snapshot of the data before it is overwritten or appended; otherwise, `false` (the default value).", "default": false, "type": "boolean" }, "connectionname": { "title": "Connectionname", "description": "Connection name must match one of the connections already in data source connection list.", "default": "Internal clickhouse DB", "type": "string" } }, "required": [ "setname" ] }
- Fields:
- Validators:
- field connectionname: str | None = 'Internal clickhouse DB'
Connection name must match one of the connections already in data source connection list.
- field description: str | None = 'Data description.'
- field dryrun: bool | None = False
Dry run determines if data is NOT to be saved in a table. Default value is False and data is saved.
- field dtypes: str | None = None
A list of pandas data types as a string instance.
- field filetype: str | None = 'parquet'
File type containing data to be loaded. Acceptable values include: csv, ‘json`, and ‘paquet`. Parquet file types are recommended over json or csv files. Default value is parquet.
- Validated by:
- field job_description: str | dict | None = None
Job description.
- field job_type: Literal['dataloading'] | None = 'dataloading'
Job type. Supports one value: dataloading (the default value).
- field maintain_archives: bool | None = False
If true creates an archive table of a snapshot of the data before it is overwritten or appended; otherwise, false (the default value).
- field overwrite: str | None = 'Replace'
Determines how to manage data if already exists. Supported values: replace (overwrite existing data) and append (add new data to existing data).
- field separator: str | None = ','
Data separator.
- field setname: str [Required]
Feature set name.
- Validated by:
- field srcurl: str | None = None
Source url fron where data is to be loaded.
- field tablename: str | None = None
Table name.
- field upload_key: str | None = None
Cache upload key.
vianops_client.models.jobmaker.deployment_mgt
- pydantic model V1CreateDeploymentModel
Bases:
BaseModelShow JSON schema
{ "title": "V1CreateDeploymentModel", "type": "object", "properties": { "model_name": { "title": "Model Name", "description": "The name of the model.", "type": "string" }, "deployment_name": { "title": "Deployment Name", "description": "The name of the deployment.", "type": "string" }, "model_version": { "title": "Model Version", "description": "The version of the model, e.g., 1.", "type": "string" }, "model_path": { "title": "Model Path", "description": "Path (location) for the model to deploy.", "type": "string" }, "model_run_id": { "title": "Model Run Id", "description": "Run id for the model to deploy.", "type": "string" } }, "required": [ "model_name" ], "example": { "deployment_name": "deployment_xyz", "model_name": "vianai.model_name", "model_version": "1", "model_path": "", "model_run_id": "abcdef-123456-xyzyui" } }
- Config:
schema_extra: dict = {‘example’: {‘deployment_name’: ‘deployment_xyz’, ‘model_name’: ‘vianai.model_name’, ‘model_version’: ‘1’, ‘model_path’: ‘’, ‘model_run_id’: ‘abcdef-123456-xyzyui’}}
- Fields:
- field deployment_name: str | None = None
The name of the deployment.
- field model_name: str [Required]
The name of the model.
- field model_path: str | None = None
Path (location) for the model to deploy.
- field model_run_id: str | None = None
Run id for the model to deploy.
- field model_version: str | None = None
The version of the model, e.g., 1.
- pydantic model V1DeploymentModel
Bases:
BaseModelShow JSON schema
{ "title": "V1DeploymentModel", "type": "object", "properties": { "deployment_detail": { "title": "Deployment Detail", "type": "object" }, "config": { "title": "Config", "type": "object" }, "name": { "title": "Name", "type": "string" }, "deployment_name": { "title": "Deployment Name", "type": "string" }, "user_id": { "title": "User Id", "type": "string" }, "run_id": { "title": "Run Id", "type": "string" }, "job_type": { "title": "Job Type", "type": "string" }, "models": { "title": "Models", "default": [], "type": "array", "items": { "$ref": "#/definitions/V1ModelModel" } }, "model_name": { "title": "Model Name", "type": "string" }, "model_version": { "title": "Model Version", "type": "string" }, "model_path": { "title": "Model Path", "type": "string" }, "model_uri": { "title": "Model Uri", "type": "string" }, "status": { "title": "Status", "type": "string" }, "status_msg": { "title": "Status Msg", "type": "string" }, "timestamp": { "title": "Timestamp", "type": "number" } }, "required": [ "name", "deployment_name", "model_name" ], "definitions": { "V1ModelModel": { "title": "V1ModelModel", "type": "object", "properties": { "model_name": { "title": "Model Name", "type": "string" }, "model_version": { "title": "Model Version", "type": "string" }, "model_path": { "title": "Model Path", "type": "string" }, "model_uri": { "title": "Model Uri", "type": "string" }, "deployment_mode": { "title": "Deployment Mode", "type": "string" }, "run_id": { "title": "Run Id", "type": "string" } }, "required": [ "model_name" ] } } }
- field config: dict | None = None
- field deployment_detail: dict | None = None
- field deployment_name: str [Required]
- field job_type: str | None = None
- field model_name: str [Required]
- field model_path: str | None = None
- field model_uri: str | None = None
- field model_version: str | None = None
- field models: List[V1ModelModel] | None = []
- field name: str [Required]
- field run_id: str | None = None
- field status: str | None = None
- field status_msg: str | None = None
- field timestamp: float | None = None
- field user_id: str | None = None
- pydantic model V1ModelModel
Bases:
BaseModelShow JSON schema
{ "title": "V1ModelModel", "type": "object", "properties": { "model_name": { "title": "Model Name", "type": "string" }, "model_version": { "title": "Model Version", "type": "string" }, "model_path": { "title": "Model Path", "type": "string" }, "model_uri": { "title": "Model Uri", "type": "string" }, "deployment_mode": { "title": "Deployment Mode", "type": "string" }, "run_id": { "title": "Run Id", "type": "string" } }, "required": [ "model_name" ] }
- Fields:
- field deployment_mode: str | None = None
- field model_name: str [Required]
- field model_path: str | None = None
- field model_uri: str | None = None
- field model_version: str | None = None
- field run_id: str | None = None
- pydantic model V1PlaceholderDeploymentResponse
Bases:
BaseModelShow JSON schema
{ "title": "V1PlaceholderDeploymentResponse", "type": "object", "properties": { "user_id": { "title": "User Id", "type": "string" }, "version_comment": { "title": "Version Comment", "type": "string" }, "deployment_version": { "title": "Deployment Version", "type": "string" }, "deployment_status": { "title": "Deployment Status", "type": "string" }, "deployment_mode": { "title": "Deployment Mode", "type": "string" }, "deployment_name": { "title": "Deployment Name", "type": "string" }, "deployment_kind": { "title": "Deployment Kind", "type": "string" }, "deployment_yaml": { "title": "Deployment Yaml", "type": "string" }, "deployment_detail": { "title": "Deployment Detail", "type": "object" }, "model_details": { "title": "Model Details", "default": [], "type": "array", "items": { "type": "object" } }, "status": { "title": "Status", "type": "string" }, "timestamp": { "title": "Timestamp", "type": "string" }, "internal_endpointurl": { "title": "Internal Endpointurl", "type": "string" }, "external_endpointurl": { "title": "External Endpointurl", "type": "string" } } }
- Fields:
- field deployment_detail: dict | None = None
- field deployment_kind: str | None = None
- field deployment_mode: str | None = None
- field deployment_name: str | None = None
- field deployment_status: str | None = None
- field deployment_version: str | None = None
- field deployment_yaml: str | None = None
- field external_endpointurl: str | None = None
- field internal_endpointurl: str | None = None
- field model_details: List[dict] | None = []
- field status: str | None = None
- field timestamp: str | None = None
- field user_id: str | None = None
- field version_comment: str | None = None
vianops_client.models.jobmaker.driftdetection
- pydantic model V1DriftDetectionJob
Bases:
BaseModelShow JSON schema
{ "title": "V1DriftDetectionJob", "type": "drift", "properties": { "deployment": { "title": "Deployment", "description": "The name of the deployment.", "type": "string" }, "model_name": { "title": "Model Name", "description": "The name of the model.", "type": "string" }, "job_type": { "title": "Job Type", "enum": [ "driftdetection" ], "type": "string" }, "model_version": { "title": "Model Version", "description": "The version of the model, e.g., 1.", "type": "string" }, "model_stage": { "title": "Model Stage", "description": "The stage of the model, e.g., primary.", "type": "string" }, "name": { "title": "Name", "description": "The name of the policy.", "type": "string" }, "description": { "title": "Description", "description": "Parameters passed to the drift detection job.", "type": "string" }, "type": { "title": "Type", "description": "The type of policy.", "type": "string" }, "policy": { "title": "Policy", "description": "drift policy for detection", "allOf": [ { "$ref": "#/definitions/V1DriftPolicy" } ] }, "drift_policies": { "title": "Drift Policies", "description": "Extending support for legacy drift policies.", "default": [], "type": "array", "items": { "type": "object" } }, "status": { "title": "Status", "description": "The status of policy, such as `active` or `inactive`.", "type": "string" }, "segments": { "title": "Segments", "description": "List of segments in drift detection policies.", "type": "array", "items": { "$ref": "#/definitions/V1SegmentModel" } }, "uuid": { "title": "Uuid", "description": "UUID for the drift policy", "type": "string" }, "created_ts": { "title": "Created Ts", "description": "Timestamp (Unix time in milliseconds) generated when the drift policy is created. You can specify the timestamp as a float or date time string value.", "anyOf": [ { "type": "number" }, { "type": "string", "format": "date-time" } ] }, "modified_ts": { "title": "Modified Ts", "description": "Timestamp (Unix time in milliseconds) generated when the drift policy is modified. You can specify the timestamp as a float or date time string value.", "anyOf": [ { "type": "number" }, { "type": "string", "format": "date-time" } ] }, "created_by": { "title": "Created By", "description": "User who created the segment.", "type": "string" }, "modified_by": { "title": "Modified By", "description": "User who last modified the segment.", "type": "string" } }, "required": [ "policy", "segments", "created_ts", "modified_ts", "created_by", "modified_by" ], "deployment": "vianai.d_mp_placeholder_deployment", "model_name": "vianai.d_mp_placeholder_deployment", "job_type": "driftdetection", "model_version": "1", "model_stage": "primary", "name": "vianai.d_mp_placeholder_deployment.vianai_d_mp_d_mp_placeholder_deployment_1_policy", "description": "Description of Policy", "policy": { "type": "feature-drift", "drift_type": "distance", "window_parameters": { "target": { "window_type": "week" }, "baseline": { "window_method": "last", "window_type": "week", "last_amount": 2 } }, "select_features_type": "custom", "feature_weightage": "equal", "feature_weights": { "trip_distance": 25, "travel_time": 25, "est_fare_amount": 25, "extra": 25 }, "drift_measure": "PSI", "warning_level": 0.1, "critical_level": 0.25, "schedule": "0 0 5 ? * *", "deployment_name": "vianai.d_mp_placeholder_deployment", "method": "preprocess" }, "status": "inactive", "segments": [ { "model_uuid": "ff0f98ae-5b59-48dd-8567-14efce48b4b4", "name": "vianai.d_mp_placeholder_deployment_segment_1", "description": "Segment to filter data for particular Pickup and Dropoff locations", "filters": [ { "feature_name": "PULocationID", "value": [ "256", "217" ], "operator": "=", "conjunction": "and", "grouped_filters": null }, { "feature_name": "DOLocationID", "value": [ "144", "148" ], "operator": "=", "conjunction": null, "grouped_filters": null } ], "id": 1, "status": "active", "created_ts": 1679496214358.969, "modified_ts": 1679496214358.969, "created_by": "vianai", "modified_by": "vianai" } ], "uuid": "76ce26d2-a41c-4390-b015-52d7a606aeec", "created_ts": 1679496214659.72, "modified_ts": 1679496214659.72, "created_by": "vianai", "modified_by": "vianai", "definitions": { "V1Target": { "title": "V1Target", "type": "object", "properties": { "window_type": { "title": "Window Type", "description": "The target window type. Accepted values include: `day`, `week`, `month`, `quarter`, and `year`.", "type": "string" }, "process_date": { "title": "Process Date", "description": "The process date.", "type": "string" }, "offset_type": { "title": "Offset Type", "description": "The policy`s offset type. Accepted values include: `week`, `month`, and `quarter`.", "type": "string" }, "offset": { "title": "Offset", "description": "The offset amount.", "type": "integer" }, "start_of_week": { "title": "Start Of Week", "description": "The start day of week. Supported values: `monday`, `tuesday`, `wednesday`, `thursday`, `friday`, `saturday`, `sunday`.", "type": "string" }, "quarter_start": { "title": "Quarter Start", "description": "The date to start the quarter.", "type": "string" }, "year_start": { "title": "Year Start", "description": "The date to start the year.", "type": "string" } }, "required": [ "window_type" ] }, "V1Baseline": { "title": "V1Baseline", "type": "object", "properties": { "window_method": { "title": "Window Method", "description": "The window method. Supported values: `prior`, `last`, `same_prior_year`, `training` .", "type": "string" }, "window_type": { "title": "Window Type", "description": "The baseline window type. Supported values: `day`, `week`, `month`, `quarter`, `year`, `training`.", "type": "string" }, "last_amount": { "title": "Last Amount", "description": "The last amount of `prior` or `last` for the given window type.", "type": "integer" }, "process_date": { "title": "Process Date", "description": "The process date.", "type": "string" }, "offset_type": { "title": "Offset Type", "description": "The policy`s offset type. Supported values: `week`, `month`, and `quarter`.", "type": "string" }, "offset": { "title": "Offset", "description": "The offset amount.", "type": "integer" }, "start_of_week": { "title": "Start Of Week", "description": "The start day of week. Supported values: `monday`, `tuesday`, `wednesday`, `thursday`, `friday`, `saturday`, `sunday`.", "type": "string" }, "quarter_start": { "title": "Quarter Start", "description": "The date to start the quarter.", "type": "string" }, "year_start": { "title": "Year Start", "description": "The date to start the year.", "type": "string" } }, "required": [ "window_method", "window_type" ] }, "V1WindowParameters": { "title": "V1WindowParameters", "type": "object", "properties": { "target": { "title": "Target", "description": "The target parameters.", "allOf": [ { "$ref": "#/definitions/V1Target" } ] }, "baseline": { "title": "Baseline", "description": "The baseline parameters.", "allOf": [ { "$ref": "#/definitions/V1Baseline" } ] } }, "required": [ "target", "baseline" ] }, "V1HotspotParamsModel": { "title": "V1HotspotParamsModel", "type": "object", "properties": { "method": { "title": "Method", "description": "Method for selecting top hotspot results. Currently supported values: `flat` (straight number limit) and `percent` (top value % of the hotspots results).", "enum": [ "flat", "percent" ], "type": "string" }, "value": { "title": "Value", "description": "Integer value that correlates to the selected method. If using percent as the selected method, number can never be greater than 100.", "type": "integer" }, "features": { "title": "Features", "description": "List of hotspot features (of categorical data type) to be used when performing hotspot analysis.", "type": "array", "items": { "type": "string" } } }, "required": [ "method", "value", "features" ], "example": { "hotspot": { "method": "flat", "value": 50, "features": [ "a", "b", "c" ] } } }, "V1DriftPolicy": { "title": "V1DriftPolicy", "type": "object", "properties": { "window_parameters": { "title": "Window Parameters", "description": "The window parameters, which include the target and baseline parameters.", "allOf": [ { "$ref": "#/definitions/V1WindowParameters" } ] }, "warning_level": { "title": "Warning Level", "description": "The warning level value. For distance based drift and performance drift, the expected value type is a float. For window based drift, the expected value type is an integer.", "anyOf": [ { "type": "number" }, { "type": "integer" } ] }, "critical_level": { "title": "Critical Level", "description": "The critical level value. For distance based drift and performance drift, the expected value type is a float. For window based drift, the expected value type is an integer.", "anyOf": [ { "type": "number" }, { "type": "integer" } ] }, "schedule": { "title": "Schedule", "description": "The cron scheduleer.", "type": "string" }, "deployment_name": { "title": "Deployment Name", "description": "The name of the deployment the policy belongs to.", "type": "string" }, "method": { "title": "Method", "description": "The drift method.", "type": "string" }, "hotspot_analysis": { "title": "Hotspot Analysis", "description": "Parameters that determine the features to run hotspot analysis against and level of traffic to include.", "default": { "method": "percent", "value": 100, "features": [] }, "allOf": [ { "$ref": "#/definitions/V1HotspotParamsModel" } ] }, "type": { "title": "Type", "description": "The drift category. Supported values: `feature-drift`, `prediction-drift`.", "type": "string" }, "drift_type": { "title": "Drift Type", "description": "The type of drift. Supported values: `distance`, `windowed`.", "type": "string" }, "select_features_type": { "title": "Select Features Type", "description": "The features to be included in drift policy. Supported values: `custom`, `all`.", "type": "string" }, "feature_weightage": { "title": "Feature Weightage", "description": "The weight category selected for the included features. Supported values: `equal`, `manual`.", "type": "string" }, "feature_weights": { "title": "Feature Weights", "description": "The specific weight given to each feature.", "type": "object" }, "drift_measure": { "title": "Drift Measure", "description": "The drift measure. Supported values: `PSI`, `JS`.", "type": "string" }, "baseline_bins": { "title": "Baseline Bins", "description": "The custom bin dictionary for which to run drift against per feature.", "type": "object" } }, "required": [ "window_parameters", "warning_level", "critical_level", "schedule", "method", "type", "drift_type" ], "example": { "window_parameters": { "target": { "window_type": "week" }, "baseline": { "window_method": "last", "window_type": "week", "last_amount": 2 } }, "warning_level": 0.1, "critical_level": 0.25, "schedule": "0 0 5 ? * *", "deployment_name": "d_mp_placeholder_deployment", "method": "preprocess", "hotspot": { "method": "flat", "value": 50, "features": [ "a", "b", "c" ] } } }, "V1BetweenOperatorModel": { "title": "V1BetweenOperatorModel", "type": "object", "properties": { "min": { "title": "Min", "description": "Minimum value of BETWEEN.", "type": "number" }, "max": { "title": "Max", "description": "Maximum value of BETWEEN.", "type": "number" } } }, "V1TopBottomOperatorModel": { "title": "V1TopBottomOperatorModel", "type": "object", "properties": { "direction": { "title": "Direction", "description": "Direction of field values. This field is case-insensitive. Supported directions: `ASC` (ascending), `DESC` (descending). Default is `DESC`.", "default": "DESC", "type": "string" }, "method": { "title": "Method", "description": "Whether it should be the top/bottom percent or flat amount. This field is case-insensitive. Supported methods: `percent`, `flat`. Default is `flat`.", "default": "flat", "type": "string" }, "value": { "title": "Value", "description": "Percent or flat number of rows to take.", "type": "number" } } }, "V1SegmentFilterModel": { "title": "V1SegmentFilterModel", "type": "object", "properties": { "feature_name": { "title": "Feature Name", "description": "Name of feature field to do the operation on.", "type": "string" }, "value": { "title": "Value", "description": "Values of operator doing the filter. Value types supported are based on operator.", "anyOf": [ { "type": "array", "items": { "anyOf": [ { "type": "string" }, { "type": "integer" }, { "type": "boolean" }, { "type": "number" } ] } }, { "type": "string" }, { "type": "integer" }, { "type": "boolean" }, { "type": "number" }, { "$ref": "#/definitions/V1BetweenOperatorModel" }, { "$ref": "#/definitions/V1TopBottomOperatorModel" } ] }, "operator": { "title": "Operator", "description": "Operation to perform on the feature field. This field is case-insensitive. Supported operators: `=`, `>`, `<`, `<=`, `>=`, `BETWEEN`, `TOP`, `BOTTOM`, `DISTINCT`", "type": "string" }, "conjunction": { "title": "Conjunction", "description": "Relationship to next filter. If there is no next filter, set to null. This field is case-insensitive. Supported values: `AND`, `OR`, `null`. Default value is `null`.", "type": "string" }, "grouped_filters": { "title": "Grouped Filters", "description": "List of grouped filters. Use for creating a nested filter within a filter.", "type": "array", "items": { "type": "object" } } }, "additionalProperties": false, "example": { "feature_name": "FEATURE NAME", "operator": "=", "value": [ "VALUE 1", "VALUE 2" ], "conjunction": "AND", "grouped_filters": [ { "feature_name": "FEATURE NAME", "operator": "=", "value": [ "VALUE 1", "VALUE 2" ], "conjunction": "AND" } ] } }, "V1SegmentModel": { "title": "V1SegmentModel", "type": "object", "properties": { "model_uuid": { "title": "Model Uuid", "description": "Unique identifier for model.", "type": "string" }, "name": { "title": "Name", "description": "Name of the segment.", "type": "string" }, "description": { "title": "Description", "description": "Description of the segment.", "type": "string" }, "filters": { "title": "Filters", "description": "List of segment filters, their conditions, and relationships.", "type": "array", "items": { "$ref": "#/definitions/V1SegmentFilterModel" } }, "id": { "title": "Id", "description": "Unique identifier generated by the platform for the segment.", "type": "integer" }, "status": { "title": "Status", "description": "Current status of the segment. This field is case-insensitive. Possible values: `active`, `inactive` (default), `archived`, `delete`.", "type": "string" }, "created_ts": { "title": "Created Ts", "description": "Timestamp (Unix time in milliseconds) generated when the segment is created. You can specify the timestamp as a float or date time string value.", "anyOf": [ { "type": "number" }, { "type": "string", "format": "date-time" } ] }, "modified_ts": { "title": "Modified Ts", "description": "Timestamp (Unix time in milliseconds) generated when the segment is modified. You can specify the timestamp as a float or date time string value.", "anyOf": [ { "type": "number" }, { "type": "string", "format": "date-time" } ] }, "created_by": { "title": "Created By", "description": "User who created the segment.", "type": "string" }, "modified_by": { "title": "Modified By", "description": "User who last modified the segment.", "type": "string" } }, "required": [ "model_uuid", "name", "filters", "id", "status", "created_ts", "modified_ts", "created_by", "modified_by" ], "example": { "id": 1, "model_uuid": "5efc9544-e84a-4a94-9e6d-5774e1c609ab", "name": "NAME", "description": "DESCRIPTION", "filters": [ { "grouped_filters": null, "feature_name": "FEATURE NAME", "operator": "=", "value": [ "VALUE 1", "VALUE 2" ], "conjunction": null } ], "status": "inactive", "created_ts": 1672531200000, "modified_ts": 1672531200000, "created_by": "user1", "modified_by": "user2" } } } }
- Config:
schema_extra: dict = {‘deployment’: ‘vianai.d_mp_placeholder_deployment’, ‘model_name’: ‘vianai.d_mp_placeholder_deployment’, ‘job_type’: ‘driftdetection’, ‘model_version’: ‘1’, ‘model_stage’: ‘primary’, ‘name’: ‘vianai.d_mp_placeholder_deployment.vianai_d_mp_d_mp_placeholder_deployment_1_policy’, ‘description’: ‘Description of Policy’, ‘type’: ‘drift’, ‘policy’: {‘type’: ‘feature-drift’, ‘drift_type’: ‘distance’, ‘window_parameters’: {‘target’: {‘window_type’: ‘week’}, ‘baseline’: {‘window_method’: ‘last’, ‘window_type’: ‘week’, ‘last_amount’: 2}}, ‘select_features_type’: ‘custom’, ‘feature_weightage’: ‘equal’, ‘feature_weights’: {‘trip_distance’: 25, ‘travel_time’: 25, ‘est_fare_amount’: 25, ‘extra’: 25}, ‘drift_measure’: ‘PSI’, ‘warning_level’: 0.1, ‘critical_level’: 0.25, ‘schedule’: ‘0 0 5 ? * *’, ‘deployment_name’: ‘vianai.d_mp_placeholder_deployment’, ‘method’: ‘preprocess’}, ‘status’: ‘inactive’, ‘segments’: [{‘model_uuid’: ‘ff0f98ae-5b59-48dd-8567-14efce48b4b4’, ‘name’: ‘vianai.d_mp_placeholder_deployment_segment_1’, ‘description’: ‘Segment to filter data for particular Pickup and Dropoff locations’, ‘filters’: [{‘feature_name’: ‘PULocationID’, ‘value’: [‘256’, ‘217’], ‘operator’: ‘=’, ‘conjunction’: ‘and’, ‘grouped_filters’: None}, {‘feature_name’: ‘DOLocationID’, ‘value’: [‘144’, ‘148’], ‘operator’: ‘=’, ‘conjunction’: None, ‘grouped_filters’: None}], ‘id’: 1, ‘status’: ‘active’, ‘created_ts’: 1679496214358.969, ‘modified_ts’: 1679496214358.969, ‘created_by’: ‘vianai’, ‘modified_by’: ‘vianai’}], ‘uuid’: ‘76ce26d2-a41c-4390-b015-52d7a606aeec’, ‘created_ts’: 1679496214659.72, ‘modified_ts’: 1679496214659.72, ‘created_by’: ‘vianai’, ‘modified_by’: ‘vianai’}
- Fields:
- field created_by: str [Required]
User who created the segment.
- field created_ts: float | datetime [Required]
Timestamp (Unix time in milliseconds) generated when the drift policy is created. You can specify the timestamp as a float or date time string value.
- field deployment: str | None = None
The name of the deployment.
- field description: str | None = None
Parameters passed to the drift detection job.
- field drift_policies: List[dict] | None = []
Extending support for legacy drift policies.
- field job_type: Literal['driftdetection'] | None = None
- field model_name: str | None = None
The name of the model.
- field model_stage: str | None = None
The stage of the model, e.g., primary.
- field model_version: str | None = None
The version of the model, e.g., 1.
- field modified_by: str [Required]
User who last modified the segment.
- field modified_ts: float | datetime [Required]
Timestamp (Unix time in milliseconds) generated when the drift policy is modified. You can specify the timestamp as a float or date time string value.
- field name: str | None = None
The name of the policy.
- field policy: V1DriftPolicy [Required]
drift policy for detection
- field segments: List[V1SegmentModel] [Required]
List of segments in drift detection policies.
- field status: str | None = None
The status of policy, such as active or inactive.
- field type: str | None = None
The type of policy.
- field uuid: str | None = None
UUID for the drift policy
vianops_client.models.jobmaker.job
- pydantic model V1BenchmarkOptimizeRequest
Bases:
BaseModelShow JSON schema
{ "title": "V1BenchmarkOptimizeRequest", "type": "object", "properties": { "job_description": { "title": "Job Description" } } }
- Fields:
- field job_description: Any | None = None
- pydantic model V1JobModel
Bases:
BaseModelShow JSON schema
{ "title": "V1JobModel", "type": "object", "properties": { "job_id": { "title": "Job Id", "description": "The job id.", "type": "string" }, "job_type": { "title": "Job Type", "description": "The job type. Supported values: `dataloading`, `dataprofiling`, `driftdetection`, `modelperformance`, `preprocessing`.", "type": "string" }, "user_id": { "title": "User Id", "description": "User id.", "type": "string" }, "run_id": { "title": "Run Id", "description": "Run id.", "type": "string" }, "status": { "title": "Status", "description": "Job status. Supported values: `queued`, `done`.", "type": "string" }, "statusdetails": { "title": "Statusdetails", "description": "Status details.", "type": "string" }, "result": { "title": "Result", "description": "Job results, if job status is `done`." }, "exception": { "title": "Exception", "description": "Exception details, if an exeption occurred while running the job.", "type": "string" }, "created": { "title": "Created", "description": "The date and time the job pod was created.", "type": "string", "format": "date-time" }, "started": { "title": "Started", "description": "The date and time the job started.", "type": "string" }, "finished": { "title": "Finished", "description": "The date and time the job finished.", "type": "string" }, "timestamp": { "title": "Timestamp", "description": "Job timestamp.", "type": "number" }, "configmap": { "title": "Configmap", "description": "Configuration map.", "type": "string" } }, "required": [ "job_id", "job_type", "user_id", "status", "statusdetails", "exception", "started", "finished", "timestamp" ] }
- Fields:
- field configmap: str | None = None
Configuration map.
- field created: datetime | None = None
The date and time the job pod was created.
- field exception: str [Required]
Exception details, if an exeption occurred while running the job.
- field finished: str [Required]
The date and time the job finished.
- field job_id: str [Required]
The job id.
- field job_type: str [Required]
The job type. Supported values: dataloading, dataprofiling, driftdetection, modelperformance, preprocessing.
- field result: Any | None = None
Job results, if job status is done.
- field run_id: str | None = None
Run id.
- field started: str [Required]
The date and time the job started.
- field status: str [Required]
Job status. Supported values: queued, done.
- field statusdetails: str [Required]
Status details.
- field timestamp: float [Required]
Job timestamp.
- field user_id: str [Required]
User id.
- pydantic model V1JobResponseList
Bases:
BaseModelShow JSON schema
{ "title": "V1JobResponseList", "type": "array", "items": { "$ref": "#/definitions/V1JobModel" }, "example": [ { "job_id": "job-queue-362526", "job_type": "job", "user_id": "user1", "status": "queued", "statusdetails": "", "result": "", "exception": "", "created": "2023-06-16 16:02:08.362534", "started": "", "finished": "", "timestamp": 1686931328362.5432, "configmap": "config-job-queue-362526" } ], "definitions": { "V1JobModel": { "title": "V1JobModel", "type": "object", "properties": { "job_id": { "title": "Job Id", "description": "The job id.", "type": "string" }, "job_type": { "title": "Job Type", "description": "The job type. Supported values: `dataloading`, `dataprofiling`, `driftdetection`, `modelperformance`, `preprocessing`.", "type": "string" }, "user_id": { "title": "User Id", "description": "User id.", "type": "string" }, "run_id": { "title": "Run Id", "description": "Run id.", "type": "string" }, "status": { "title": "Status", "description": "Job status. Supported values: `queued`, `done`.", "type": "string" }, "statusdetails": { "title": "Statusdetails", "description": "Status details.", "type": "string" }, "result": { "title": "Result", "description": "Job results, if job status is `done`." }, "exception": { "title": "Exception", "description": "Exception details, if an exeption occurred while running the job.", "type": "string" }, "created": { "title": "Created", "description": "The date and time the job pod was created.", "type": "string", "format": "date-time" }, "started": { "title": "Started", "description": "The date and time the job started.", "type": "string" }, "finished": { "title": "Finished", "description": "The date and time the job finished.", "type": "string" }, "timestamp": { "title": "Timestamp", "description": "Job timestamp.", "type": "number" }, "configmap": { "title": "Configmap", "description": "Configuration map.", "type": "string" } }, "required": [ "job_id", "job_type", "user_id", "status", "statusdetails", "exception", "started", "finished", "timestamp" ] } } }
- Config:
orm_mode: bool = True
schema_extra: dict = {‘example’: [{‘job_id’: ‘job-queue-362526’, ‘job_type’: ‘job’, ‘user_id’: ‘user1’, ‘status’: ‘queued’, ‘statusdetails’: ‘’, ‘result’: ‘’, ‘exception’: ‘’, ‘created’: ‘2023-06-16 16:02:08.362534’, ‘started’: ‘’, ‘finished’: ‘’, ‘timestamp’: 1686931328362.5432, ‘configmap’: ‘config-job-queue-362526’}]}
- Fields:
__root__ (List[vianops_client.models.jobmaker.job.V1JobModel])
- pydantic model V1TransformJobInput
Bases:
BaseModelShow JSON schema
{ "title": "V1TransformJobInput", "type": "object", "properties": { "setname": { "title": "Setname", "type": "string" }, "label": { "title": "Label", "type": "string" }, "xformers": { "title": "Xformers", "type": "array", "items": { "$ref": "#/definitions/V1TransformModel" } }, "combined": { "title": "Combined", "type": "boolean" }, "target_featureset_name": { "title": "Target Featureset Name", "type": "string" }, "split_by": { "title": "Split By", "type": "string" }, "split_args": { "title": "Split Args", "type": "object" } }, "required": [ "setname", "label", "xformers", "combined", "target_featureset_name", "split_by", "split_args" ], "definitions": { "V1TransformModel": { "title": "V1TransformModel", "type": "object", "properties": { "classname": { "title": "Classname", "type": "string" }, "columns": { "title": "Columns" }, "method": { "title": "Method", "type": "string" }, "initparams": { "title": "Initparams" }, "stateful": { "title": "Stateful", "type": "boolean" }, "serialized": { "title": "Serialized", "type": "string" } }, "required": [ "classname", "method", "stateful" ] } } }
- Fields:
- field combined: bool [Required]
- field label: str [Required]
- field setname: str [Required]
- field split_args: Dict [Required]
- field split_by: str [Required]
- field target_featureset_name: str [Required]
- field xformers: List[V1TransformModel] [Required]
- pydantic model V1TransformJobModel
Bases:
BaseModelShow JSON schema
{ "title": "V1TransformJobModel", "type": "object", "properties": { "job_id": { "title": "Job Id", "type": "string" }, "job_type": { "title": "Job Type", "type": "string" }, "job_description": { "title": "Job Description" }, "user_id": { "title": "User Id", "type": "string" }, "run_id": { "title": "Run Id", "type": "string" }, "status": { "title": "Status", "type": "string" }, "statusdetails": { "title": "Statusdetails", "type": "string" }, "result": { "title": "Result" }, "exception": { "title": "Exception", "type": "string" }, "created": { "title": "Created", "type": "string" }, "started": { "title": "Started", "type": "string" }, "finished": { "title": "Finished", "type": "string" }, "timestamp": { "title": "Timestamp", "type": "number" }, "configmap": { "title": "Configmap", "type": "string" } }, "required": [ "job_id", "job_type", "user_id", "status", "statusdetails", "exception", "created", "started", "finished", "timestamp" ] }
- Fields:
- field configmap: str | None = None
- field created: str [Required]
- field exception: str [Required]
- field finished: str [Required]
- field job_description: Any | None = None
- field job_id: str [Required]
- field job_type: str [Required]
- field result: Any | None = None
- field run_id: str | None = None
- field started: str [Required]
- field status: str [Required]
- field statusdetails: str [Required]
- field timestamp: float [Required]
- field user_id: str [Required]
- pydantic model V1TransformJobResult
Bases:
BaseModelShow JSON schema
{ "title": "V1TransformJobResult", "type": "object", "properties": { "success": { "title": "Success", "type": "boolean" }, "newsetname": { "title": "Newsetname", "type": "string" }, "input": { "$ref": "#/definitions/V1TransformJobInput" } }, "definitions": { "V1TransformModel": { "title": "V1TransformModel", "type": "object", "properties": { "classname": { "title": "Classname", "type": "string" }, "columns": { "title": "Columns" }, "method": { "title": "Method", "type": "string" }, "initparams": { "title": "Initparams" }, "stateful": { "title": "Stateful", "type": "boolean" }, "serialized": { "title": "Serialized", "type": "string" } }, "required": [ "classname", "method", "stateful" ] }, "V1TransformJobInput": { "title": "V1TransformJobInput", "type": "object", "properties": { "setname": { "title": "Setname", "type": "string" }, "label": { "title": "Label", "type": "string" }, "xformers": { "title": "Xformers", "type": "array", "items": { "$ref": "#/definitions/V1TransformModel" } }, "combined": { "title": "Combined", "type": "boolean" }, "target_featureset_name": { "title": "Target Featureset Name", "type": "string" }, "split_by": { "title": "Split By", "type": "string" }, "split_args": { "title": "Split Args", "type": "object" } }, "required": [ "setname", "label", "xformers", "combined", "target_featureset_name", "split_by", "split_args" ] } } }
- Fields:
- field input: V1TransformJobInput | None = None
- field newsetname: str | None = None
- field success: bool | None = None
- pydantic model V1TransformModel
Bases:
BaseModelShow JSON schema
{ "title": "V1TransformModel", "type": "object", "properties": { "classname": { "title": "Classname", "type": "string" }, "columns": { "title": "Columns" }, "method": { "title": "Method", "type": "string" }, "initparams": { "title": "Initparams" }, "stateful": { "title": "Stateful", "type": "boolean" }, "serialized": { "title": "Serialized", "type": "string" } }, "required": [ "classname", "method", "stateful" ] }
- Fields:
- field classname: str [Required]
- field columns: Any | None = None
- field initparams: Any = None
- field method: str [Required]
- field serialized: str | None = None
- field stateful: bool [Required]
- pydantic model V1TransformSplitArgs
Bases:
BaseModelShow JSON schema
{ "title": "V1TransformSplitArgs", "type": "object", "properties": { "train_size": { "title": "Train Size", "type": "number" }, "test_size": { "title": "Test Size", "type": "number" }, "validation_size": { "title": "Validation Size", "type": "number" } }, "required": [ "train_size", "test_size", "validation_size" ] }
- field test_size: float [Required]
- field train_size: float [Required]
- field validation_size: float [Required]
vianops_client.models.jobmaker.model_artifacts
- pydantic model V1ModelRepoAddMetaJob
Bases:
BaseModelShow JSON schema
{ "title": "V1ModelRepoAddMetaJob", "type": "object", "properties": { "model_name": { "title": "Model Name", "type": "string" }, "model_version": { "title": "Model Version", "type": "string" }, "kind": { "title": "Kind", "type": "string" }, "data": { "title": "Data", "type": "object" } }, "required": [ "model_name", "model_version", "kind" ] }
- field data: dict | None = None
- field kind: str [Required]
- field model_name: str [Required]
- field model_version: str [Required]
- pydantic model V1ModelRepoDeleteArtifactJob
Bases:
BaseModelShow JSON schema
{ "title": "V1ModelRepoDeleteArtifactJob", "type": "object", "properties": { "model_name": { "title": "Model Name", "type": "string" }, "model_version": { "title": "Model Version", "type": "string" }, "artifact_path": { "title": "Artifact Path", "type": "string" }, "run_id": { "title": "Run Id", "type": "string" } }, "required": [ "model_name", "model_version", "artifact_path", "run_id" ] }
- field artifact_path: str [Required]
- field model_name: str [Required]
- field model_version: str [Required]
- field run_id: str [Required]
- pydantic model V1ModelRepoGetMetaJob
Bases:
BaseModelShow JSON schema
{ "title": "V1ModelRepoGetMetaJob", "type": "object", "properties": { "model_name": { "title": "Model Name", "type": "string" }, "model_version": { "title": "Model Version", "type": "string" } }, "required": [ "model_name", "model_version" ] }
- Fields:
- field model_name: str [Required]
- field model_version: str [Required]
vianops_client.models.jobmaker.model_deployment
- pydantic model V1ModelDetails
Bases:
BaseModelShow JSON schema
{ "title": "V1ModelDetails", "type": "object", "properties": { "model_version": { "title": "Model Version", "description": "Version of the placeholder model to deploy.", "type": "string" }, "registered_model_version": { "title": "Registered Model Version", "description": "Version of registered model.", "type": "string" } }, "required": [ "model_version", "registered_model_version" ] }
- field model_version: str [Required]
Version of the placeholder model to deploy.
- field registered_model_version: str [Required]
Version of registered model.
- pydantic model V1PlaceholderDeployment
Bases:
BaseModelShow JSON schema
{ "title": "V1PlaceholderDeployment", "type": "object", "properties": { "deployment_name": { "title": "Deployment Name", "description": "Name of the model for this placeholder deployment.", "type": "string" }, "model_name": { "title": "Model Name", "description": "Name of the model for this placeholder deployment. Model name and deployment name are the same. (The endpoint replaces this name with the deployment name.)", "type": "string" }, "model_version": { "title": "Model Version", "description": "Version of the model to use in the placeholder deployment.", "type": "string" }, "model_details": { "title": "Model Details", "description": "Model details.", "type": "array", "items": { "$ref": "#/definitions/V1ModelDetails" } }, "featureset": { "title": "Featureset", "description": "Name of the feature set associated with the deployment.", "type": "string" }, "training_params": { "title": "Training Params", "description": "Model training data and meta data", "allOf": [ { "$ref": "#/definitions/V1TrainingParams" } ] } }, "required": [ "deployment_name", "model_name", "model_details" ], "example": { "deployment_name": "string", "model_name": "string", "model_version": "1", "model_details": [ { "model_version": "string", "registered_model_version": "string" } ], "featureset": "string", "training_params": { "model_class": "catboost", "model_name": "catboost.CatBoostRegressor", "experiment_type": "regression", "featureset": "string", "allcolumns": [ "column1", "column2", "column3", "column4", "column5" ], "continuouscolumns": [ "column1", "column2", "column5" ], "categoricalcolumns": [ "column3", "column4" ], "targetcolumn": "column2" } }, "definitions": { "V1ModelDetails": { "title": "V1ModelDetails", "type": "object", "properties": { "model_version": { "title": "Model Version", "description": "Version of the placeholder model to deploy.", "type": "string" }, "registered_model_version": { "title": "Registered Model Version", "description": "Version of registered model.", "type": "string" } }, "required": [ "model_version", "registered_model_version" ] }, "V1TrainingParams": { "title": "V1TrainingParams", "type": "object", "properties": { "model_class": { "title": "Model Class", "description": "Class of model (or framework) used to train the experiment. Default value: `sklearn`", "default": "sklearn", "type": "string" }, "model_name": { "title": "Model Name", "description": "Name of model.", "type": "string" }, "experiment_type": { "title": "Experiment Type", "description": "Type of prediction problem for the experiment. Supported values: `regression` or `classification`.", "type": "string" }, "featureset": { "title": "Featureset", "description": "Name of the feature set used to train the experiment.", "type": "string" }, "allcolumns": { "title": "Allcolumns", "description": "List of all columns in the training feature set. Includes the target column.", "type": "array", "items": { "type": "string" } }, "continuouscolumns": { "title": "Continuouscolumns", "description": "List of columns containing continuous data in the training feature set, if any. (Typically, these are numerical columns with integer or float data types.) Include target column if applicable. Leave empty brackets [] if there are no continuous columns.", "type": "array", "items": { "type": "string" } }, "categoricalcolumns": { "title": "Categoricalcolumns", "description": "List of all columns containing categorical data in the training feature set, if any. Include target column if applicable. Leave empty brackets [] if there are no categorical columns.", "type": "array", "items": { "type": "string" } }, "targetcolumn": { "title": "Targetcolumn", "description": "Name of the feature (column) the model is predicting.", "type": "string" } }, "required": [ "model_name", "experiment_type", "featureset" ] } } }
- Config:
schema_extra: dict = {‘example’: {‘deployment_name’: ‘string’, ‘model_name’: ‘string’, ‘model_version’: ‘1’, ‘model_details’: [{‘model_version’: ‘string’, ‘registered_model_version’: ‘string’}], ‘featureset’: ‘string’, ‘training_params’: {‘model_class’: ‘catboost’, ‘model_name’: ‘catboost.CatBoostRegressor’, ‘experiment_type’: ‘regression’, ‘featureset’: ‘string’, ‘allcolumns’: [‘column1’, ‘column2’, ‘column3’, ‘column4’, ‘column5’], ‘continuouscolumns’: [‘column1’, ‘column2’, ‘column5’], ‘categoricalcolumns’: [‘column3’, ‘column4’], ‘targetcolumn’: ‘column2’}}}
- Fields:
- Validators:
- field deployment_name: str [Required]
Name of the model for this placeholder deployment.
- field featureset: str | None = None
Name of the feature set associated with the deployment.
- field model_details: List[V1ModelDetails] [Required]
Model details.
- field model_name: str [Required]
Name of the model for this placeholder deployment. Model name and deployment name are the same. (The endpoint replaces this name with the deployment name.)
- Validated by:
- field model_version: str | None = None
Version of the model to use in the placeholder deployment.
- field training_params: V1TrainingParams | None = None
Model training data and meta data
- validator grouped_filters_validator » model_name
- pydantic model V1TrainingParams
Bases:
BaseModelShow JSON schema
{ "title": "V1TrainingParams", "type": "object", "properties": { "model_class": { "title": "Model Class", "description": "Class of model (or framework) used to train the experiment. Default value: `sklearn`", "default": "sklearn", "type": "string" }, "model_name": { "title": "Model Name", "description": "Name of model.", "type": "string" }, "experiment_type": { "title": "Experiment Type", "description": "Type of prediction problem for the experiment. Supported values: `regression` or `classification`.", "type": "string" }, "featureset": { "title": "Featureset", "description": "Name of the feature set used to train the experiment.", "type": "string" }, "allcolumns": { "title": "Allcolumns", "description": "List of all columns in the training feature set. Includes the target column.", "type": "array", "items": { "type": "string" } }, "continuouscolumns": { "title": "Continuouscolumns", "description": "List of columns containing continuous data in the training feature set, if any. (Typically, these are numerical columns with integer or float data types.) Include target column if applicable. Leave empty brackets [] if there are no continuous columns.", "type": "array", "items": { "type": "string" } }, "categoricalcolumns": { "title": "Categoricalcolumns", "description": "List of all columns containing categorical data in the training feature set, if any. Include target column if applicable. Leave empty brackets [] if there are no categorical columns.", "type": "array", "items": { "type": "string" } }, "targetcolumn": { "title": "Targetcolumn", "description": "Name of the feature (column) the model is predicting.", "type": "string" } }, "required": [ "model_name", "experiment_type", "featureset" ] }
- Fields:
- field allcolumns: List[str] | None = None
List of all columns in the training feature set. Includes the target column.
- field categoricalcolumns: List[str] | None = None
List of all columns containing categorical data in the training feature set, if any. Include target column if applicable. Leave empty brackets [] if there are no categorical columns.
- field continuouscolumns: List[str] | None = None
List of columns containing continuous data in the training feature set, if any. (Typically, these are numerical columns with integer or float data types.) Include target column if applicable. Leave empty brackets [] if there are no continuous columns.
- field experiment_type: str [Required]
Type of prediction problem for the experiment. Supported values: regression or classification.
- field featureset: str [Required]
Name of the feature set used to train the experiment.
- field model_class: str = 'sklearn'
Class of model (or framework) used to train the experiment. Default value: sklearn
- field model_name: str [Required]
Name of model.
- field targetcolumn: str | None = None
Name of the feature (column) the model is predicting.
vianops_client.models.jobmaker.modelperformance
- pydantic model V1ModelPerformanceJob
Bases:
BaseModelShow JSON schema
{ "title": "V1ModelPerformanceJob", "type": "object", "properties": { "deployment_name": { "title": "Deployment Name", "description": "The name of the deployment.", "type": "string" }, "model_name": { "title": "Model Name", "description": "The name of the model.", "type": "string" }, "job_type": { "title": "Job Type", "description": "Supports the model performance job.", "enum": [ "modelperformance" ], "type": "string" }, "model_version": { "title": "Model Version", "description": "The version of the model, e.g., 1.", "type": "string" }, "model_stage": { "title": "Model Stage", "description": "The stage of the model, e.g., primary.", "type": "string" }, "job_description": { "title": "Job Description", "description": "Parameters passed to the model performance job. For example, \"job_description\":{\"deployment_name\":\"deployment_xyz\",\"deployment_type\":\"vianai-v2\"}}" }, "method": { "title": "Method", "description": "The model performance method. Supports the value `preprocess` (the default value).", "default": "preprocess", "type": "string" } }, "required": [ "deployment_name", "model_name" ], "example": { "deployment_name": "deployment_xyz", "model_name": "vianai.model_name", "model_version": "1", "job_type": "modelperformance", "model_stage": "primary", "job_description": { "deployment_name": "deployment_xyz", "deployment_type": "vianai-v2" }, "method": "preprocess" } }
- Config:
schema_extra: dict = {‘example’: {‘deployment_name’: ‘deployment_xyz’, ‘model_name’: ‘vianai.model_name’, ‘model_version’: ‘1’, ‘job_type’: ‘modelperformance’, ‘model_stage’: ‘primary’, ‘job_description’: {‘deployment_name’: ‘deployment_xyz’, ‘deployment_type’: ‘vianai-v2’}, ‘method’: ‘preprocess’}}
- Fields:
- field deployment_name: str [Required]
The name of the deployment.
- field job_description: Any | None = None
Parameters passed to the model performance job. For example, “job_description”:{“deployment_name”:”deployment_xyz”,”deployment_type”:”vianai-v2”}}
- field job_type: Literal['modelperformance'] | None = None
Supports the model performance job.
- field method: str | None = 'preprocess'
The model performance method. Supports the value preprocess (the default value).
- field model_name: str [Required]
The name of the model.
- field model_stage: str | None = None
The stage of the model, e.g., primary.
- field model_version: str | None = None
The version of the model, e.g., 1.
vianops_client.models.jobmaker.preprocessing
- pydantic model V1PreprocessingJob
Bases:
BaseModelShow JSON schema
{ "title": "V1PreprocessingJob", "type": "object", "properties": { "inference_mapping": { "$ref": "#/definitions/V1InferenceMappingModel" }, "process_window": { "title": "Process Window", "defult": null, "allOf": [ { "$ref": "#/definitions/V1PreprocessingProcessWindow" } ] }, "job_type": { "title": "Job Type", "default": "preprocessing", "enum": [ "preprocessing" ], "type": "string" }, "method": { "title": "Method", "default": "preprocess", "type": "string" }, "refresh": { "title": "Refresh", "default": false, "type": "boolean" } }, "required": [ "inference_mapping" ], "definitions": { "V1InferenceMappingColumnSchema": { "title": "V1InferenceMappingColumnSchema", "type": "object", "properties": { "feature_id": { "title": "Feature Id", "description": "Id of the feature, generated by the platform for optimization.", "type": "integer" }, "name": { "title": "Name", "description": "Name of column.", "type": "string" }, "dtype": { "title": "Dtype", "description": "Data type of column. For example, `float64`.", "type": "string" }, "sql_type": { "title": "Sql Type", "description": "SQL data type of column. For example, `Float32`.", "type": "string" }, "feature_type": { "title": "Feature Type", "description": "Feature type of column. Acceptable values include `categorical` and `continuous`.", "type": "string" }, "segmentation": { "title": "Segmentation", "description": "If true (the default), the column is enabled for segmentation calculations.", "default": true, "type": "boolean" }, "drift": { "title": "Drift", "description": "If true (the default), the column is enabled for drift calculations.", "default": true, "type": "boolean" }, "hotspot": { "title": "Hotspot", "description": "If true (the default), the column is enabled for hotspot analysis.", "default": false, "type": "boolean" }, "rca": { "title": "Rca", "description": "List of columns included for root cause analysis (rca). Set by the platform based on feature configuration to identify features available for hotspot analysis.", "default": [], "type": "array", "items": {} }, "round": { "title": "Round", "description": "The number of decimals to round to on a continuous float column.", "type": "integer" } }, "required": [ "name", "dtype", "sql_type", "feature_type" ] }, "V1InferenceMappingSchema": { "title": "V1InferenceMappingSchema", "type": "object", "properties": { "target_col": { "title": "Target Col", "description": "Name of target column.", "type": "string" }, "identifier_cols": { "title": "Identifier Cols", "description": "List of identifier columns.", "default": [], "type": "array", "items": { "type": "string" } }, "columns": { "title": "Columns", "description": "List of all columns.", "type": "array", "items": { "$ref": "#/definitions/V1InferenceMappingColumnSchema" } }, "datetime_col": { "title": "Datetime Col", "description": "Name of datetime column", "default": "", "type": "string" }, "predict_proba_col": { "title": "Predict Proba Col", "description": "(Classification models only) Name of column that provides the predict probability.", "type": "string" } }, "required": [ "target_col", "columns" ] }, "V1InferenceMappingModel": { "title": "V1InferenceMappingModel", "type": "object", "properties": { "index": { "title": "Index", "description": "Index identifier.", "type": "integer" }, "deployment": { "title": "Deployment", "description": "Deployment name for the related model.", "type": "string" }, "model_name": { "title": "Model Name", "description": "Model name for the related model.", "type": "string" }, "model_version": { "title": "Model Version", "description": "Model version for the related model, e.g., 1.", "type": "string" }, "model_stage": { "title": "Model Stage", "description": "Model stage for the related model, e.g., primary.", "type": "string" }, "connection": { "title": "Connection", "description": "Database connection for the inference mapping.", "type": "string" }, "df_schema": { "title": "Df Schema", "description": "Dictionary containing dataframe schema for model, i.e., target column, identifier column(s), all columns, datetime column, and predict probability column (classification models only).", "allOf": [ { "$ref": "#/definitions/V1InferenceMappingSchema" } ] }, "identifier_cols_schema": { "title": "Identifier Cols Schema", "description": "List containing identifier columns schema for model." }, "inference_table": { "title": "Inference Table", "description": "Name of inference table for mapping.", "type": "string" }, "ground_truth_table": { "title": "Ground Truth Table", "description": "Name of ground truth table for mapping.", "type": "string" }, "joined_table": { "title": "Joined Table", "description": "Name of joined table for mapping.", "type": "string" }, "key_table": { "title": "Key Table", "description": "Name of key table for mapping.", "type": "string" }, "create_ddl": { "title": "Create Ddl", "description": "The query to create the ddl tables.", "type": "string" }, "postprocessors": { "title": "Postprocessors", "description": "One or more postprocessor (and related params) to include in inference mapping, to transform output data before sending to backend database. The available postprocessors include `PickHighestProbability`, `PickInferenceMapper`, and `PickProbablityThreshold`. For each postprocessor, specify the classname, method, and initparams.", "default": [], "type": "array", "items": { "type": "object" } }, "training_set_id": { "title": "Training Set Id", "description": "Training set id that needs to be specified for training data preprocessing.", "type": "integer" }, "training_table": { "title": "Training Table", "description": "Training table name that needs to be specified for training data preprocessing.", "type": "string" } }, "required": [ "deployment", "model_name", "model_version", "model_stage", "connection", "df_schema" ], "example": { "index": 7, "deployment": "vianai.jt_placeholder_deployment", "model_name": "vianai.jt_placeholder_deployment", "model_version": "1", "model_stage": "primary", "connection": "Internal clickhouse DB", "df_schema": { "target_col": "total_amount", "identifier_cols": [ "pk" ], "columns": [ { "name": "PULocationID", "dtype": "float64", "sql_type": "Float32", "feature_type": "categorical", "feature_id": 1, "drift": true, "segmentation": true }, { "name": "DOLocationID", "dtype": "float64", "sql_type": "Float32", "feature_type": "categorical", "feature_id": 2, "drift": true, "segmentation": true }, { "name": "trip_distance", "dtype": "float64", "sql_type": "Float32", "feature_type": "continuous", "feature_id": 3, "drift": true, "segmentation": true }, { "name": "travel_time", "dtype": "float64", "sql_type": "Float32", "feature_type": "continuous", "feature_id": 4, "drift": true, "segmentation": true }, { "name": "PUdatetime", "dtype": "datetime64[ns]", "sql_type": "", "feature_type": "unknown", "feature_id": 5, "drift": false } ], "datetime_col": "PUdatetime" }, "identifier_cols_schema": null, "inference_table": "16794179557100_dit", "ground_truth_table": "16794179557100_gt", "joined_table": "16794179557100_joined", "key_table": "16794179557100_key", "create_ddl": null, "postprocessors": [ { "classname": "vianai.postprocessing.classification.PickInferenceMapper", "method": "fit_transform", "initparams": { "mapper": "{1:'a',2:'b',3:'c'}" } } ] } }, "V1PreprocessingProcessWindow": { "title": "V1PreprocessingProcessWindow", "type": "object", "properties": { "start_date": { "title": "Start Date", "description": "Start date in string for process window (in string format), formatted as yyy-mm-dd.", "type": "string" }, "end_date": { "title": "End Date", "description": "End date in string for process window (in string format), formatted as yyy-mm-dd.", "type": "string" } }, "example": { "start_date": "2023-02-07", "end_date": "2023-03-16" } } } }
- Fields:
- field inference_mapping: V1InferenceMappingModel [Required]
- field job_type: Literal['preprocessing'] | None = 'preprocessing'
- field method: str | None = 'preprocess'
- field process_window: V1PreprocessingProcessWindow | None = None
- field refresh: bool | None = False
- pydantic model V1PreprocessingJoinJob
Bases:
BaseModelShow JSON schema
{ "title": "V1PreprocessingJoinJob", "type": "object", "properties": { "inference_mapping": { "$ref": "#/definitions/V1InferenceMappingModel" }, "process_window": { "title": "Process Window", "defult": null, "allOf": [ { "$ref": "#/definitions/V1PreprocessingJoinProcessWindow" } ] }, "method": { "title": "Method", "default": "join", "type": "string" }, "job_type": { "title": "Job Type", "default": "join_preprocessing", "enum": [ "join_preprocessing" ], "type": "string" } }, "required": [ "inference_mapping" ], "definitions": { "V1InferenceMappingColumnSchema": { "title": "V1InferenceMappingColumnSchema", "type": "object", "properties": { "feature_id": { "title": "Feature Id", "description": "Id of the feature, generated by the platform for optimization.", "type": "integer" }, "name": { "title": "Name", "description": "Name of column.", "type": "string" }, "dtype": { "title": "Dtype", "description": "Data type of column. For example, `float64`.", "type": "string" }, "sql_type": { "title": "Sql Type", "description": "SQL data type of column. For example, `Float32`.", "type": "string" }, "feature_type": { "title": "Feature Type", "description": "Feature type of column. Acceptable values include `categorical` and `continuous`.", "type": "string" }, "segmentation": { "title": "Segmentation", "description": "If true (the default), the column is enabled for segmentation calculations.", "default": true, "type": "boolean" }, "drift": { "title": "Drift", "description": "If true (the default), the column is enabled for drift calculations.", "default": true, "type": "boolean" }, "hotspot": { "title": "Hotspot", "description": "If true (the default), the column is enabled for hotspot analysis.", "default": false, "type": "boolean" }, "rca": { "title": "Rca", "description": "List of columns included for root cause analysis (rca). Set by the platform based on feature configuration to identify features available for hotspot analysis.", "default": [], "type": "array", "items": {} }, "round": { "title": "Round", "description": "The number of decimals to round to on a continuous float column.", "type": "integer" } }, "required": [ "name", "dtype", "sql_type", "feature_type" ] }, "V1InferenceMappingSchema": { "title": "V1InferenceMappingSchema", "type": "object", "properties": { "target_col": { "title": "Target Col", "description": "Name of target column.", "type": "string" }, "identifier_cols": { "title": "Identifier Cols", "description": "List of identifier columns.", "default": [], "type": "array", "items": { "type": "string" } }, "columns": { "title": "Columns", "description": "List of all columns.", "type": "array", "items": { "$ref": "#/definitions/V1InferenceMappingColumnSchema" } }, "datetime_col": { "title": "Datetime Col", "description": "Name of datetime column", "default": "", "type": "string" }, "predict_proba_col": { "title": "Predict Proba Col", "description": "(Classification models only) Name of column that provides the predict probability.", "type": "string" } }, "required": [ "target_col", "columns" ] }, "V1InferenceMappingModel": { "title": "V1InferenceMappingModel", "type": "object", "properties": { "index": { "title": "Index", "description": "Index identifier.", "type": "integer" }, "deployment": { "title": "Deployment", "description": "Deployment name for the related model.", "type": "string" }, "model_name": { "title": "Model Name", "description": "Model name for the related model.", "type": "string" }, "model_version": { "title": "Model Version", "description": "Model version for the related model, e.g., 1.", "type": "string" }, "model_stage": { "title": "Model Stage", "description": "Model stage for the related model, e.g., primary.", "type": "string" }, "connection": { "title": "Connection", "description": "Database connection for the inference mapping.", "type": "string" }, "df_schema": { "title": "Df Schema", "description": "Dictionary containing dataframe schema for model, i.e., target column, identifier column(s), all columns, datetime column, and predict probability column (classification models only).", "allOf": [ { "$ref": "#/definitions/V1InferenceMappingSchema" } ] }, "identifier_cols_schema": { "title": "Identifier Cols Schema", "description": "List containing identifier columns schema for model." }, "inference_table": { "title": "Inference Table", "description": "Name of inference table for mapping.", "type": "string" }, "ground_truth_table": { "title": "Ground Truth Table", "description": "Name of ground truth table for mapping.", "type": "string" }, "joined_table": { "title": "Joined Table", "description": "Name of joined table for mapping.", "type": "string" }, "key_table": { "title": "Key Table", "description": "Name of key table for mapping.", "type": "string" }, "create_ddl": { "title": "Create Ddl", "description": "The query to create the ddl tables.", "type": "string" }, "postprocessors": { "title": "Postprocessors", "description": "One or more postprocessor (and related params) to include in inference mapping, to transform output data before sending to backend database. The available postprocessors include `PickHighestProbability`, `PickInferenceMapper`, and `PickProbablityThreshold`. For each postprocessor, specify the classname, method, and initparams.", "default": [], "type": "array", "items": { "type": "object" } }, "training_set_id": { "title": "Training Set Id", "description": "Training set id that needs to be specified for training data preprocessing.", "type": "integer" }, "training_table": { "title": "Training Table", "description": "Training table name that needs to be specified for training data preprocessing.", "type": "string" } }, "required": [ "deployment", "model_name", "model_version", "model_stage", "connection", "df_schema" ], "example": { "index": 7, "deployment": "vianai.jt_placeholder_deployment", "model_name": "vianai.jt_placeholder_deployment", "model_version": "1", "model_stage": "primary", "connection": "Internal clickhouse DB", "df_schema": { "target_col": "total_amount", "identifier_cols": [ "pk" ], "columns": [ { "name": "PULocationID", "dtype": "float64", "sql_type": "Float32", "feature_type": "categorical", "feature_id": 1, "drift": true, "segmentation": true }, { "name": "DOLocationID", "dtype": "float64", "sql_type": "Float32", "feature_type": "categorical", "feature_id": 2, "drift": true, "segmentation": true }, { "name": "trip_distance", "dtype": "float64", "sql_type": "Float32", "feature_type": "continuous", "feature_id": 3, "drift": true, "segmentation": true }, { "name": "travel_time", "dtype": "float64", "sql_type": "Float32", "feature_type": "continuous", "feature_id": 4, "drift": true, "segmentation": true }, { "name": "PUdatetime", "dtype": "datetime64[ns]", "sql_type": "", "feature_type": "unknown", "feature_id": 5, "drift": false } ], "datetime_col": "PUdatetime" }, "identifier_cols_schema": null, "inference_table": "16794179557100_dit", "ground_truth_table": "16794179557100_gt", "joined_table": "16794179557100_joined", "key_table": "16794179557100_key", "create_ddl": null, "postprocessors": [ { "classname": "vianai.postprocessing.classification.PickInferenceMapper", "method": "fit_transform", "initparams": { "mapper": "{1:'a',2:'b',3:'c'}" } } ] } }, "V1PreprocessingJoinProcessWindow": { "title": "V1PreprocessingJoinProcessWindow", "type": "object", "properties": { "start_time": { "title": "Start Time", "description": "tart time for process window (in string format), with method join.", "type": "string" }, "end_time": { "title": "End Time", "description": "End time for process window (in string format), with method join.", "type": "string" }, "batch_size": { "title": "Batch Size", "description": "Batch size for process window (in integer format) with method join.", "type": "integer" } }, "example": { "start_time": "2023-03-21 11:00:00", "end_time": "2023-03-21 18:00:00", "batch_size": 60 } } } }
- Fields:
- field inference_mapping: V1InferenceMappingModel [Required]
- field job_type: Literal['join_preprocessing'] | None = 'join_preprocessing'
- field method: str | None = 'join'
- field process_window: V1PreprocessingJoinProcessWindow | None = None
- pydantic model V1PreprocessingJoinProcessWindow
Bases:
BaseModelShow JSON schema
{ "title": "V1PreprocessingJoinProcessWindow", "type": "object", "properties": { "start_time": { "title": "Start Time", "description": "tart time for process window (in string format), with method join.", "type": "string" }, "end_time": { "title": "End Time", "description": "End time for process window (in string format), with method join.", "type": "string" }, "batch_size": { "title": "Batch Size", "description": "Batch size for process window (in integer format) with method join.", "type": "integer" } }, "example": { "start_time": "2023-03-21 11:00:00", "end_time": "2023-03-21 18:00:00", "batch_size": 60 } }
- Config:
schema_extra: dict = {‘example’: {‘start_time’: ‘2023-03-21 11:00:00’, ‘end_time’: ‘2023-03-21 18:00:00’, ‘batch_size’: 60}}
- Fields:
- field batch_size: int | None = None
Batch size for process window (in integer format) with method join.
- field end_time: str = None
End time for process window (in string format), with method join.
- field start_time: str = None
tart time for process window (in string format), with method join.
- pydantic model V1PreprocessingProcessWindow
Bases:
BaseModelShow JSON schema
{ "title": "V1PreprocessingProcessWindow", "type": "object", "properties": { "start_date": { "title": "Start Date", "description": "Start date in string for process window (in string format), formatted as yyy-mm-dd.", "type": "string" }, "end_date": { "title": "End Date", "description": "End date in string for process window (in string format), formatted as yyy-mm-dd.", "type": "string" } }, "example": { "start_date": "2023-02-07", "end_date": "2023-03-16" } }
- Config:
schema_extra: dict = {‘example’: {‘start_date’: ‘2023-02-07’, ‘end_date’: ‘2023-03-16’}}
- Fields:
- field end_date: str = None
End date in string for process window (in string format), formatted as yyy-mm-dd.
- field start_date: str = None
Start date in string for process window (in string format), formatted as yyy-mm-dd.
- pydantic model V1PreprocessingTrainingJob
Bases:
BaseModelShow JSON schema
{ "title": "V1PreprocessingTrainingJob", "type": "object", "properties": { "method": { "title": "Method", "default": "training", "type": "string" }, "job_type": { "title": "Job Type", "default": "training_preprocessing", "enum": [ "training_preprocessing" ], "type": "string" } } }
- field job_type: Literal['training_preprocessing'] | None = 'training_preprocessing'
- field method: str | None = 'training'