Notifications (APIs)

Overview

The platform tracks events and activities that you can configure as triggering events for generating notifications (aka webhooks). For example, generate notifications based on model conditions detected by policies, such as emails for warning alerts and slack messages for critical alerts. When the policy detects the configured alert event (e.g., surpasses the threshold defined for the warning alert), the platform generates the configured notification.

To configure a notification, specify the data to review for events, the trigger conditions, and how frequently you want to look for those conditions. Identify the notification information and how and where to send the notification, such as a slack or email REST endpoint. You can specify a throttle to control how often the notification is sent.

Each notification query includes data from one of the VIANOPS internal data stores (connection):

  • Internal database—PostgresSQL; VIANAOPS metadata including but not limited to featuresets loaded via dataloading by default.
  • Model Store database—PostgresSQL; contains information for models, policies, segments, model alerts.
  • Inference Store database—PostgresSQL; contains inference, prediction, and (if provided) ground truth data.
  • Risk Store database—PostgresSQL; contains information for drift, explainability, model performance, etc.
  • Internal clickhouse database—Clickhouse; may contain feature data uploaded via REST API, v1/dataloading endpoint.

(Note that the Internal sqlite database is not used at this time.)

The query specifies the tables/columns and data to look in for defined event conditions. When the condition is true, the platform generates the configured notification. The REST endpoint may be the VIANOPS internal email webhook or an external webhook, such as Slack or Zapier.

Notification object

{
    "name": "drift_notify_sample",
    "description": "Notification on drift alert",
    "connection": "Risk Store DB",
    "query": "SELECT COUNT(*) AS \"count\" FROM myfeatures WHERE setname = 'my_featureset'",
    "condition": "query_result[\"cnt\"] > 0",
    "runschedule": "0 8 * * *",
    "restrequestmethod": "POST",
    "restendpoint": "http://webservices:5000/v1/alerts/test",
  	"restbodytemplate": "",
    "tags": [],
    "paused": true,
    "status": "Inactive",
    "alertsuppressiontime": 0,
    "edit_mode": "code"
}

{
  "data": {
    "h-0": "Property",
    "h-1": "Type",
    "h-2": "Description",
    "0-0": "index",
    "0-1": "integer",
    "0-2": "Assigned by the platform for the notification.",
    "1-0": "name",
    "1-1": "string",
    "1-2": "Name for the notification.",
    "2-0": "description",
    "2-1": "string",
    "2-2": "Description for the notification.",
    "3-0": "connection",
    "3-1": "string",
    "3-2": "Database connection string for accessing data to include in query. identifies internal database source. Supported values:  `Internal DB`, `Model Store DB`, `Inference Store DB`, `Risk Store DB`, `Internal clickhouse DB`.",
    "4-0": "query",
    "4-1": "string",
    "4-2": "Specifies the database data (from the `connection`) to review a defined `condition`. Supports SQL.",
    "5-0": "condition",
    "5-1": "string",
    "5-2": "Event condition (discovered in `query`) to use as a trigger for notifications. Use`query_results` (containing results of the query) to create the condition expression. Supports Python.",
    "6-0": "runschedule",
    "6-1": "cron string",
    "6-2": "Schedule for running the query. Defined as a cron expression with a maximum of 5 fields. Example cron expression generator: <https://crontab.cronhub.io/>. For example, the following cron string runs the notification query every day at 8am:  \n\"schedule\": \"0 8 \\* \\* \\*\"",
    "7-0": "restendpoint",
    "7-1": "string",
    "7-2": "REST endpoint (URL) to which defined results are sent.",
    "8-0": "restbodytemplate",
    "8-1": "string",
    "8-2": "Body to send to defined REST endpoint. JSON content template based on the [Jinja templating engine](https://jinja.palletsprojects.com/en/3.1.x/). May include the following variables for sending information to the defined restendpoint: `query_result`, `createdtime`, `createdby`, `tags`.  \n  \nExample `restbodytemplate` that creates an email notification message and includes the time the notification was created and results of the query:  \n  \n`{ \\\"addresses\\\":\\\"mary_smith@examplecompany.com\\\", \\\"subject\\\":\\\"alert – critical drift detected\\\",  \\\"body\\\":\\\"Critical drift detected at \\\"for\\\" \\\" }\",`  \n  \n(Time is checked based on how frequently the platform runs and updates cron.) Supports multiple semicolon-separated addresses.",
    "9-0": "restrequestmethod",
    "9-1": "string",
    "9-2": "REST method for the defined endpoint. Supported values: `GET`, `POST`, `PUT`, `PATCH`, `DELETE`.",
    "10-0": "tags",
    "10-1": "list",
    "10-2": "Comma-separated list of tags for this notification, if any.",
    "11-0": "lastrun",
    "11-1": "float",
    "11-2": "Identifies when the notification (query) last run. Unix timestamp in milliseconds, generated by the platform (read-only).",
    "12-0": "lastalert",
    "12-1": "float",
    "12-2": "Identifies when the last notification was generated. Unix timestamp in milliseconds, generated by the platform.",
    "13-0": "createdby",
    "13-1": "string",
    "13-2": "Name of platform user who created the notification.",
    "14-0": "createddate",
    "14-1": "float",
    "14-2": "Unix time in milliseconds generated when this notification was created.",
    "15-0": "paused",
    "15-1": "Boolean",
    "15-2": "Identifies if the notification is currently paused (True) or not paused (False).",
    "16-0": "status",
    "16-1": "string",
    "16-2": "Identifies if the notification is currently active (False) or inactive (True). If inactive, the notification will not run on the defined schedule.",
    "17-0": "alertsuppressiontime",
    "17-1": "integer",
    "17-2": "Number of minutes to wait before sending additional notifications.",
    "18-0": "edit_mode",
    "18-1": "string",
    "18-2": "Not used."
  },
  "cols": 3,
  "rows": 19,
  "align": [
    "left",
    "left",
    "left"
  ]
}

Example notification

{
  "name": "notification.example",
  "description": "This is an example notification",
  "connection": "Model Store DB",
  "query": "select count(*) as cnt from alerts where policy_uuid in (select uuid from public.policies where model_name = 'model_123' and deployment = 'deployment_xyz' and model_version = '1') and type = 'drift' and status = 'Open' and severity >= 1",
  "condition": "query_result[\"count\"] > 0",
  "runschedule": "*/1 * * * *",
  "restrequestmethod": "POST",
  "restendpoint": "http://eaiservice:5000/v1/eaiservice/pubevent?channel=collab.email.send&queue=True",
  "restbodytemplate": "{ \"addresses\":\"user1@company.com\", \"subject\":\"test message\", \"body\":\"email body cnt = \"}",
  "tags": [
    "tag1",
    "tag2"
  ],
  "alertsuppressiontime": 0,
  "edit_mode": "code"
}

For example, the following query looks at data in the alerts table (from the Model Store database) for model_123 when drift is detected with a severity of 1 or greater.

select count(*) as cnt from alerts where policy_uuid in 
(select uuid from public.policies where model_name = 'model_123' and 
deployment = 'deployment_xyz' and model_version = '1') and type = 'drift' and 
status = 'Open' and severity >= 1"

When the condition is true, the platform generates the configured notification. In this case, an email is sent to user1@company.com whenever the query results contain at least 1 instance of drift with severity >=1.

TABLE OF CONTENTS