Integration smartchecks

Overview

The smart-checks extend the health-check functionality and provide the way how to validate behavior and consistency of a specific domain. The data-integration is using built-in probes to check availability and data consistency of connected applications and data flow in between.

Diagnostic probes

Probes are evaluating several areas (metadata, activity, consumer and publisher) for each application data-source within the last 24 hours.

Statuses:

• ☀ Healthy (= Ok) - data-integration is fully working
• ⛈ Degraded (= Warning) - data-integration is still working, but with some limits
• ☠ Unhealthy (= Error) - data-integration isn't working

Metadata smartchecks

Code	Status	Message	Description
AgentNotEnabled	⛈ Degraded	DataAgent {id} isn't enabled.	Enabled flag on data-agent is null which is implicitly handled as enabled, but flag should be one-time initialized to true or false.
AgentIsDisabled	☠ Unhealthy	DataAgent {id} is disabled.	DataAgent is disabled. Data integration process is stopped. Customer can enable/disable integration process using this flag (via Connector Catalogue).
SourceNotEnabled	⛈ Degraded	DataSource {id} isn't enabled.	Enabled flag on data-source is null which is implicitly handled as enabled, but flag should be one-time initialized to true or false.
SourceIsDisabled	☠ Unhealthy	DataSource {id} is disabled.	DataSource is disabled. Data integration process is stopped. Customer can enable/disable integration process using this flag (via Connector Catalogue).
DsAppNotDefined	☠ Unhealthy	DataSource {id} isn't assigned to application.	DataSource must be assigned to integrated application providing integration capabilities.
DsClientNotDefined	⛈ Degraded	DataSource {id} isn't assigned to clientId.	DataSource must be assigned to service client (which is provided by DevOps application or Connector Catalogue)
ConsumerTypeNotDefined	⛈ Degraded	DataSource {id} isn't assigned to message-gateway consumer.	DataSource should be assigned to MessageGateway consumer. Set ConsumerType to 'Grpc' or 'Webhook'.
ConsumerCodeNotDefined	☠ Unhealthy	DataSource {id} is assigned to message-gateway, but consumer code isn't defined.	ConsumerType on data-source is assigned to MessageGateway consumer, but ConsumerCode isn't defined.

Activity smartchecks

Code	Status	Message	Description
DsNoActivity	⛈ Degraded	DataSource {id} has no activity/status (black hole disaster).	DataSource isn't providing status info. Default interval is 1 hour.
DsInvalidStatus	⛈ Degraded	DataSource {id} has invalid status ({failedStatusCount}/{statusInstances.Length}).	DataSource status contains atleast one invalid status (e.g. IsReady: false).
DsPublisherProblem	⛈ Degraded	DataSource {id} - publisher has {opLog1Instances.Length} problems (to publish and transform data).	OperatingLog contains atleast one problem belonging to data-source in publisher role.
DsConsumerFeedback	⛈ Degraded	DataSource {id} - publisher has {opLog2Instances.Length} problems (in consumer feedback).	OperatingLog contains atleast one problem belonging to consumer feedback to data-source in publisher role.
DsConsumerProblem	⛈ Degraded	DataSource {id} - consumer has {opLog2Instances.Length} problems (to consume data).	OperatingLog contains atleast one problem belonging to data-source in consumer role.

Consumer smartchecks

Code	Status	Message	Description
ConsumerNotDefined	☠ Unhealthy	DataSource {id} message-gateway consumer doesn't exist.	DataSource is using MessageGateway, but consumer doesn't exist.
ConsumerNotConnected	☠ Unhealthy	DataSource {id} message-gateway consumer isn't connected.	DataSource is using MessageGateway consumer, but it isn't connected.
ConsumerDegradedStatus	⛈ Degraded	DataSource {id} message-gateway consumer has 'Degraded' status (total:{totalMessages}/ready:{readyMessages}/unack:{unacknowledgedMessages}/dead:{unprocessableMessages}).	The lower limit of waiting or invalid messages has been exceeded.
ConsumerUnhealthyStatus	☠ Unhealthy	DataSource {id} message-gateway consumer has 'Unhealthy' status (total:{totalMessages}/ready:{readyMessages}/unack:{unacknowledgedMessages}/dead:{unprocessableMessages}).	The upper limit of waiting or invalid messages has been exceeded.

Publisher smartchecks

Code	Status	Message	Description
DsInputQueueAnyFailed	⛈ Degraded	DataSource {id} input queue has ANY failed status.	Input-queue of data-source contains atleast one failed item.
DsInputQueueAllFailed	☠ Unhealthy	DataSource {id} input queue has ALL failed status.	Input-queue of data-source contains only/all failed items.
DsInputQueueOldCompleted	⛈ Degraded	DataSource {id} input queue contains any older completed status.	Input-queue of data-source contains unprocessed item(s).
DsInputQueueTooManyItems	⛈ Degraded	DataSource {id} input queue processed {itemsTotalCount} records.	The lower limit of records in input-queue items has been exceeded.
DsInputQueueTooManyItems	☠ Unhealthy	DataSource {id} input queue processed {itemsTotalCount} records.	The upper limit of records in input-queue items has been exceeded.

Profile smartchecks

Code	Status	Message	Description
TenantSourceStatusNotDefined	☠ Unhealthy	DataSource {id} tenant status isn't defined. Application isn't probably configured well. Run initialization of application for given tenant again.
ConsumerSettingsNotDefined	⛈ Degraded	DataSource {id} consumer settings isn't defined. Application isn't probably configured well. Check initialization of application for given tenant again.

FAQ

How to handle AgentNotEnabled, AgentIsDisabled, SourceNotEnabled, SourceIsDisabled

External application / integration solution must provide one-time setup and enable data-agent and data-source during registration. It isn't possible to switch these flags multiple times, because the flags are used by customers to temporarily disable integration solution and enable it again.

See more:

• Register data-agent using REST API

How to handle DsAppNotDefined, DsClientNotDefined, ConsumerTypeNotDefined, ConsumerCodeNotDefined

External application / integration solution must provide integration settings (i.e. applicationCode, clientId, consumerCode, consumerType and consumerWebhookUrl).

• Cloud service (multi-tenant) must use global application code for applicationCode, clientId and consumerCode
• On-premise installation (single-tenant) must use global application code for applicationCode, but unique agentCode for clientId and consumerCode

Both must setup 'gRPC' or 'Webhook' to consumerType and for webhook also consumerWebhookUrl.

See more:

• Register MessageGateway consumer for data-agent using REST API

How to handle DsNoActivity

External application / integration solution must provide periodical data-source status (in 1 hour interval) in pasive or active mode.

Passive mode (recommended):
Consume ASOL.DataService.RefreshSourceStatus message-type with RefreshStatusEvent contract using RefreshStatusEventConsumer to provide current status.
Register AutoRefreshStatusInterval on data-agent definition using customProperties:

    ...
    "customProperties": [
       ...
       {
           "key": "AutoRefreshStatusInterval",
           "value": "0.01:00:00"
       }
    ],
    ...

Active mode:
Use own timer and provide current status using REST API in 1 hour interval.
Send data-agent/data-source status using REST API:
POST /api/v1/SourcingData/SetAgentStatus/{agentCode}

• agentCode - the code property on data-agent definition
• request body properties:
  • sourceId - the GUID identifier of data-source
  • operationId - the GUID identifier of virtual operation
  • isLive - the external application / integration solution is live (i.e. it is responding)
  • isReady - the external application / integration solution is ready (i.e. it is working)
  • verboseStatus - more detailed verbose info
• sample request body:

{
  "sourceId": "3496e289-42a1-41c2-a380-722f32725273",
  "operationId": "bd700096-6418-4b53-aec6-d90f1a4e439d",
  "isLive": true,
  "isReady": true,
  "verboseStatus": [
    "more detailed verbose info"
  ],
  "utcTimestamp": "2024-08-26T12:28:20.476Z"
}