Custom SLI-Provider

An SLI-provider is an implementation of a Keptn-service with a dedicated purpose. This type of service is responsible for querying an external data source for SLIs that Keptn then evaluates with SLOs. To configure a query for an indicator, Keptn provides the concept of an SLI configuration.

Create an SLI configuration that defines tool-specific queries for indicators. An example of an SLI configuration is:

spec_version: '1.0'
indicators:
  throughput: "metricSelector=builtin:service.requestCount.total:merge(\"dt.entity.service\"):sum&entitySelector=type(SERVICE),tag(keptn_project:$PROJECT),tag(keptn_stage:$STAGE),tag(keptn_service:$SERVICE),tag(keptn_deployment:$DEPLOYMENT)"
  error_rate: "metricSelector=builtin:service.errors.total.count:merge(\"dt.entity.service\"):avg&entitySelector=type(SERVICE),tag(keptn_project:$PROJECT),tag(keptn_stage:$STAGE),tag(keptn_service:$SERVICE),tag(keptn_deployment:$DEPLOYMENT)"

Note: This SLI configuration file is stored in Keptn’s configuration store using the keptn add-resource command.

The Keptn CloudEvents to which an SLI-provider must subscribe is:

sh.keptn.event.get-sli.triggered

Write your custom SLI-provider

Like a Keptn-service, an SLI-provider has the following characteristics:

has a subscription to an event (i.e., sh.keptn.event.get-sli.triggered)
sends a started event to inform Keptn about receiving the event and acting on it
processes functionality and can therefore leverage additional tools, e.g., through their REST interface
sends a finished event to inform Keptn about its execution status and the result

Subscription to Keptn event

An SLI-provider starts working, when it receives a Keptn CloudEvent of type:

sh.keptn.event.get-sli.triggered

Next to event meta-data such as project, stage, or service name, the event contains information about the indicators, time frame, and labels to query. For more details, please see the specification here and take a look at the example:

{
  "type": "sh.keptn.event.get-sli.triggered",
  "specversion": "1.0",
  "source": "https://github.com/keptn/keptn/lighthouse-service",
  "id": "f2b878d3-03c0-4e8f-bc3f-454bc1b3d79d",
  "time": "2019-06-07T07:02:15.64489Z",
  "contenttype": "application/json",
  "shkeptncontext": "08735340-6f9e-4b32-97ff-3b6c292bc509",
  "data": {
    "get-sli": {
        "customFilters": [
          { "key" : "tags", "value": "test-subject:true" }
        ],
        "indicators": ["throughput", "error_rate", "request_latency_p95"],
        "start": "2019-10-28T15:44:27.152330783Z",
        "end": "2019-10-28T15:54:27.152330783Z",
        "sliProvider": "dynatrace"
    },
    "project": "sockshop",
    "service": "carts",
    "stage": "dev",
    "labels": {
      "testId": "4711",
      "buildId": "build-17",
      "owner": "JohnDoe"
    }
  }
}

Distributor:

To subscribe your SLI-provider to the sh.keptn.event.action.triggered event, please follow Subscription to Keptn event.

Functionality

After receiving the sh.keptn.event.get-sli.triggered event, an SLI-provider must perform the following tasks:

Process the incoming event to get the project, stage, and service name. You also need the indicators and time frame to query.
Decide based on the sliProvider property whether the mentioned data source is supported by your SLI-provider. If the data source is not supported, no further task is required.
Send a started event: If the data source is supported, send a start event of type: sh.keptn.event.get-sli.started. This CloudEvent informs Keptn that your service takes care of fetching the SLIs.
Get the SLI configuration from Keptn’s configuration-service. This SLI configuration is identified by the resourceURI, which follows the pattern: [tool-name]/sli.yaml (e.g., dynatrace/sli.yaml).

Service URL: http://configuration-service.keptn.svc.cluster.local:8080
Endpoint: v1/project/{projectName}/stage/{stageName}/service/{serviceName}/resource/{resourceURI}

Process the SLI configuration and use the defined queries to retrieve the values of each indicator.
Send a finished event: Send a finished event of type: sh.keptn.event.get-sli.finished with the added properties:

Add to the header of the event:
- triggeredid: The value of this property is the id of the sh.keptn.event.get-sli.triggered event.
Add to the data block at least a value for status, result and indicatorValues in data.get-sli:
- status: [succeeded, errored, unknown] - The status of the task execution
- result: [pass, failed] - The result of a successful task execution
- indicatorValues: List of indicators and their measured values

{
  "type": "sh.keptn.event.get-sli.finished",
  "specversion": "1.0",
  "source": "dynatrace-service",
  "id": "ggb878d3-03c0-4e8f-bc3f-454bc1b3d888",
  "time": "2019-06-07T07:02:15.64489Z",
  "contenttype": "application/json",
  "shkeptncontext": "08735340-6f9e-4b32-97ff-3b6c292bc509",
  "triggeredid": "2b878d3-03c0-4e8f-bc3f-454bc1b3d79d",      # <- add triggeredid
  "data": {
    "get-sli": {                                             # <- add data.get-sli
      "status": "succeeded",
      "result": "pass",
      "indicatorValues": [
        {
          "metric":"request_latency_p95",
          "value": 1.1620000000000001,
          "success": true
        },
        {
          "metric":"error_rate",
          "value": 0,
          "success": true
        }
      ]
    },
    "project": "sockshop",
    "service": "carts",
    "stage": "staging",
    "labels": {
      "testId": "4711",
      "buildId": "build-17",
      "owner": "JohnDoe"
    }
  }
}

Deploy SLI-provider with distributor

A default deployment manifest is provided in the keptn-service-template-go template, see: deploy/service.yaml.

Change the deployment manifest for your SLI-provider and then apply it to the Kubernetes cluster where Keptn is running:

kubectl apply -f service.yaml -n keptn