Asynchronous workflows

Overview

An asynchronous workflow is used as follows:

A call must be made to create a new task. This call must contain, in the request body, the input for the workflow. The response contains the ID of the new task. The task is queued, then executed as soon as workflow's computing resources become available.
A call must be made to know about the progress (status) of the task and repeated after a interval of choice until the status of the task tells that the task is finished.
Once the task is finished, a call can be made to retrieve the task outcome. The response body is the output of the workflow or, in case of error, the description of the problem.

Info

This call is not strictly required. There can be cases in which all that is needed about a task is its final status, either completed—which denotes successful execution—or error, and the output of the workflow is of no interest. An example is a workflow that uploads its results to a Cloud storage or a workflow that indexes documents.

The traces of a task, for example its outcome, are kept for 24 hours after creation, then they are deleted automatically. The good practice however is to make a call to delete task traces as soon as the task is finished and, if needed, its outcome has been retrieved. This frees up memory and keeps the workflow efficient.

An asynchronous workflow also provides calls to:

Get the status of all the tasks.
Get the status of multiple tasks.
Get a list of tasks in given statuses.
Delete the tasks in given statuses.
Delete all the tasks.
Retrieve the input JSON, that is the JSON passed as the body of the task creation request.

All the possible API calls for an asynchronous workflow are described below.

Create a task

This call creates a new task for an asynchronous workflow.

Request:
- Verb: POST
- Endpoint:
  
  https://host/api/v1/runtime/workflow/workflowId/task[?verboseErrorOutput=true]
  
  where:
  - host is host name or IP address of the NL Flow runtime
  - workflowId is the workflow ID
  - verboseErrorOutput=true is an optional query string parameter that makes the workflow produce a verbose output in case of errors.
- HTTP headers:
  
  Mandatory:
  - X-API-KEY: API key
    
    where API key is one of the API keys associated with the workflow.
  - Content-type: application/json; charset=utf-8
  Optional:
  - X-Correlation-Id: Correlation ID
    
    For troubleshooting purposes. Correlation ID can be any unique string. It can be mentioned when asking for support in case of unexpected errors because support personnel can then track API calls based on this value when examining system logs.
- Body: the substance of what needs to be processed by the workflow for this task, that is an UTF-8 encoded JSON object compatible with the format of the workflow's input.
Response:
- HTTP status code: the HTTP status code upon success is 202 (Accepted).
- Body: the response body is a JSON object like this:
```
{
    "taskId": "taskId"
}
```
  where taskId is the ID of the newly created task. This value must be used in all the subsequent requests about the task.

Get the status of a task

This call gets the status of a given task of an asynchronous workflow.

Request:
- Verb: GET
- Endpoint:
  
  https://host/api/v1/runtime/workflow/workflowId/task/taskId/status
  
  where:
  - host is host name or IP address of the NL Flow runtime
  - workflowId is the workflow ID
  - taskId is the ID of the task obtained when creating the task
- HTTP headers:
  - X-API-KEY: API key
    
    where API key is one of the API keys associated with the workflow.
- Body: not required
Response:
- HTTP status code: the HTTP status code upon success is 200 (OK).
- Body: the response body is a JSON object like this:
```
{
    "status": "taskStatus"
}
```
  where taskStatus is the status of the task.

Get the list and status of all tasks

This call gets the list and the status for all the tasks of an asynchronous workflow.

Request:
- Verb: GET
- Endpoint:
  
  https://host/api/v1/runtime/workflow/workflowId/task/status
  
  where:
  - host is host name or IP address of the NL Flow runtime
  - workflowId is the workflow ID
- HTTP headers:
  - X-API-KEY: API key
    
    where API key is one of the API keys associated with the workflow.
- Body: not required
Response:
- HTTP status code: the HTTP status code upon success is 200 (OK).
- Body: the response body is a JSON object like this:
```
{
    "taskStatusMap": {
        "task1Id": "task1Status",
        "task2Id": "task2Status"
    }
}
```
  where the taskStatusMap object has a property for each task of the workflow; task1Status is the status of task with ID task1Id and task2Status is the status of task with ID task2Id.

Get the status of multiple tasks

This call gets the status of multiple tasks of an asynchronous workflow.

Request:
- Verb: GET
- Endpoint:
  
  https://host/api/v1/runtime/workflow/workflowId/task/status?taskIds=task1Id[,task2Id[,...]]
  
  where:
  - host is host name or IP address of the NL Flow runtime
  - workflowId is the workflow ID
  - task1Id, task2Id, etc. are the IDs of the tasks
- HTTP headers:
  - X-API-KEY: API key
    
    where API key is one of the API keys associated with the workflow.
- Body: not required
Response:
- HTTP status code: the HTTP status code upon success is 200 (OK).
- Body: the response body is a JSON object like this:
```
{
    "taskStatusMap": {
        "task1Id": "task1Status",
        "task2Id": "task2Status"
    }
}
```
  where task1Status is the status of task with ID task1Id and task2Status is the status of task with ID task2Id.

List the tasks in given statuses

This call gets the list of the tasks of an asynchronous workflow that are in given statuses.

Request:
- Verb: GET
- Endpoint:
  
  https://host/api/v1/runtime/workflow/workflowId/task/status?taskStatuses=status1[,status2[,...]]
  
  where:
  - host is host name or IP address of the NL Flow runtime
  - workflowId is the workflow ID
  - status1, status2, etc. are possible task statuses
- HTTP headers:
  - X-API-KEY: API key
    
    where API key is one of the API keys associated with the workflow.
- Body: not required
Response:
- HTTP status code: the HTTP status code upon success is 200 (OK).
- Body: the response body is a JSON object like this:
```
{
    "taskStatusMap": {
        "task1Id": "task1Status",
        "task2Id": "task2Status"
    }
}
```
  where the taskStatusMap object has a property for each task in one of the specified statuses; task1Status is the status of task with ID task1Id and task2Status is the status of task with ID task2Id.

Get the outcome of a task

This call gets the outcome of a finished task of an asynchronous workflow, that is the output of the workflow in case of success or the description of the problem in case of error.

Request:
- Verb: GET
- Endpoint:
  
  https://host/api/v1/runtime/workflow/workflowId/task/taskId/response
  
  where:
  - host is host name or IP address of the NL Flow runtime
  - workflowId is the workflow ID
  - taskId is the ID of the task obtained when creating the task
- HTTP headers:
  - X-API-KEY: API key
    
    where API key is one of the API keys associated with the workflow.
- Body: not required

Response:

HTTP status code: the HTTP status code upon success is 200 (OK).

Body: if the status of the task is COMPLETED, the response body is a JSON corresponding to the workflow output for that task.

If the status of the task is ERROR, the response body is a JSON containing the description of the problem, for example:

{
    "message": "Cannot execute workflow 5941302d-c109-4b20-aa1a-3370e1d91ce6 for taskId 6c3bb18c-acdc-4407-b4f4-5a0f4701bd38 because workflow does not become ready",
    "details": {
        "stream": {
            "id": "benthos",
            "desired": 1,
            "actual": 1,
            "status": "GREEN"
        },
        "services": [
            {
                "id": "executor-5941302d",
                "desired": 1,
                "actual": 0,
                "status": "RED",
                "rolloutStatus": "InProgress",
                "rolloutStatusReason": "AwaitingNewReplicasToBeAvailable",
                "replicasDetail": [
                    {
                        "name": "depl-executor-5941302d-5e1d319e-ee7c-4070-a00e-0ffa5e854c7ld4ch",
                        "ready": false,
                        "available": false,
                        "startedAt": "2023-07-17T12:30:20Z",
                        "age": "1h33m34.372636619s",
                        "status": "Running",
                        "atLatestSpec": true
                    }
                ]
            },
            {
                "id": "knowledge-m-63fd8140",
                "desired": 1,
                "actual": 1,
                "status": "GREEN",
                "rolloutStatus": "Complete",
                "replicasDetail": [
                    {
                        "name": "depl-knowledge-m-63fd8140-da9b70cc-42f3-486e-9b34-30f85a1e9zhxd",
                        "ready": true,
                        "available": true,
                        "startedAt": "2023-07-17T12:30:19Z",
                        "age": "1h33m36.372707126s",
                        "status": "Running",
                        "atLatestSpec": true
                    }
                ]
            }
        ]
    }
}

Delete a task

This call deletes any trace of given task of an asynchronous workflow.

Request:
- Verb: DELETE
- Endpoint:
  
  https://host/api/v1/runtime/workflow/workflowId/task/taskId
  
  where:
  - host is host name or IP address of the NL Flow runtime
  - workflowId is the workflow ID
  - taskId is the ID of the task obtained when creating the task
- HTTP headers:
  - X-API-KEY: API key
    
    where API key is one of the API keys associated with the workflow.
- Body: not required
Response:
- HTTP status code: the HTTP status code upon success is 204 (No content).
- Body: none

Delete all the tasks in given statuses

This call deletes any trace of all the tasks of an asynchronous workflow that are in given statuses.

Request:
- Verb: DELETE
- Endpoint:
  
  https://host/api/v1/runtime/workflow/workflowId/task?taskStatuses=status1[,status2[,...]]
  
  where:
  - host is host name or IP address of the NL Flow runtime
  - workflowId is the workflow ID
  - status1, status2, etc. are possible task statuses
- HTTP headers:
  - X-API-KEY: API key
    
    where API key is one of the API keys associated with the workflow.
- Body: not required
Response:
- HTTP status code: the HTTP status code upon success is 204 (No content).
- Body: none

Delete all the tasks

This call deletes any trace of all the tasks of an asynchronous workflow.

Request:
- Verb: DELETE
- Endpoint:
  
  https://host/api/v1/runtime/workflow/workflowId/task
  
  where:
  - host is host name or IP address of the NL Flow runtime
  - workflowId is the workflow ID
- HTTP headers:
  - X-API-KEY: API key
    
    where API key is one of the API keys associated with the workflow.
- Body: not required
Response:
- HTTP status code: the HTTP status code upon success is 204 (No content).
- Body: none

Get the workflow input for a given task

This resource returns the workflow input of a given task, that is the body of the task creation request.

Request:
- Verb: GET
- Endpoint:
  
  https://host/api/v1/runtime/workflow/workflowId/task/taskId/request
  
  where:
  - host is host name or IP address of the NL Flow runtime
  - workflowId is the workflow ID
  - taskId is the ID of the task obtained when creating the task
- HTTP headers:
  - X-API-KEY: API key
    
    where API key is one of the API keys associated with the workflow.
- Body: not required
Response:
- HTTP status code: the HTTP status code upon success is 200 (OK).
- Body: the response body is the JSON that was used as the request body when creating the task.

Possible statuses of a task

The status of a task can be:

SUBMITTED: the task was submitted and is going to be queued.
RUNNING: the task is either waiting in the queue or being executed.
COMPLETED: the task was completed, result can be retrieved.
ERROR: there was an error during task execution, the description of the problem can be retrieved.
DROPPED: the task was dropped because:
- The sum of the queue time and the execution time exceeded the workflow timeout.
Or:
- The workflow was unpublished.

How to know a workflow's ID

To get the ID of a workflow to use in endpoints access the NL Flow application and: