Skip to main content

Monitor tasks

In the previous lesson, you learnt about how Elara executes Tasks in a live solution. In this tutorial, you will learn how to use the EDK CLI to

  • monitor the state of tasks, and
  • access detailed logging of task instances.

This lesson will assume that you have an empty project and asset which you can to deploy to a workspace named 03_03_02_monitor_tasks with the following command:

edk template deploy -ycw 03_03_02_monitor_tasks

Define and deploy a template

To use a nullable datastream you will perform the following steps:

  1. define a data source using a similar pattern as your previous My Source definition.
  2. define a pipeline using the same pattern as your previous My Pipeline definition.
  3. add the data source and pipeline to a template

In an asset, perform the above steps to create the resulting Typescript code:

import { SourceBuilder, PipelineBuilder, Template, Add, Const } from "@elaraai/core"

const my_source = new SourceBuilder("My Source")
    .value({ value: 2n })

const my_pipeline = new PipelineBuilder("My Pipeline")
    .from(my_source.outputStream())
    .transform((stream) =>  Add(stream, Const(1)))

export default Template(my_source, my_pipeline)

Monitor the state of tasks

To view the states of all instances of all Tasks in your solution, you run the edk task list command. You can get help with usage of the command by running edk task list --help in the command-line.

Usage: edk task list [options]

list a summary of task statuses in the workspace

Options:
  -s, --server <server>        the server to target
  -w, --workspace <workspace>  workspace to query
  --since <since>              hours past to limit the search for (default is 1hr)
  --file <path>                get detailed results to a file
  -r, --raw                    raw output (default: false)
  -h, --help                   display help for command

Run edk task list in the command-line, providing your workspace using the -w option:

edk task list -w 03_03_02_monitor_tasks

Which will result in the following:

Listing tasks and their status available in tenant with identifier: XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX and workspace 03_03_02_monitor_tasks.


NAME                 TOTAL ENQUEUED RUNNING SUCCESS WARN ERROR CANCELLED
Pipeline.My Pipeline 1     0        0       1       0    0     0

Under the ENQUEUED, RUNNING, SUCCESS, WARN, ERROR, CANCELLED columns, the output table lists a count of Task instances in each state. If you are running your solution after a fresh deployment, there should be 1 count under RUNNING or SUCCESS. This is expected as you have an initial value set to your input datastream Writeable.My Source, so your Pipeline.My Pipeline has executed one task instance on that initial datastream value.

You will also notice that the only entry in the result is Pipeline.My Pipeline, and Writeable.My Source is missing. The reason that Writeable.My Source is missing is because a datasource by default will not require a task, since it is simply a value. There are exceptions to the rule which you will learn about later.

Access detailed information of a task

You can access detailed information about the the execution of tasks, including the start/end datetimes of the task. You use the command and provide a taskName argument.

Run edk task get with the pipeline name in the command-line to view the detailed executions for your pipeline Task.

edk task get "Pipeline.My Pipeline" -w 03_03_02_monitor_tasks

Which will result in:

Listing statuses for task Pipeline.My Pipeline available in tenant with identifier: XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX and workspace 03_03_02_monitor_tasks.


UUID                                 NAME                 REASON              STATUS  STARTEDAT                COMPLETEDAT
XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX Pipeline.My Pipeline new_task_definition success YYYY-MM-DDTHH:MM:SS.MSZ  YYYY-MM-DDTHH:MM:SS.MSZ

Access logs of a task

Run edk task logs with the pipeline name in the command-line to view the detailed logs for your pipeline Task.

edk task logs "Pipeline.My Pipeline" -w 03_03_02_monitor_tasks

Which will result in:

Listing task logs for Pipeline.My Pipeline available in tenant with identifier: XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX and workspace 03_03_02_monitor_tasks.


UUID                                 REASON              STATUS  LEVEL MESSAGE                                                                                                           LOGGEDAT
XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX new_task_definition success info  Worker XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX starting task instance XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX attempt 1 YYYY-MM-DDTHH:MM:SS.MSZ
XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX new_task_definition success info  Loading inputs ["input"]                                                                                          YYYY-MM-DDTHH:MM:SS.MSZ
XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX new_task_definition success info  Executing pipeline with 1 operations                                                                              YYYY-MM-DDTHH:MM:SS.MSZ
XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX new_task_definition success info  Performing transform operation                                                                                    YYYY-MM-DDTHH:MM:SS.MSZ
XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX new_task_definition success info  Saving outputs ["output"]                                                                                         YYYY-MM-DDTHH:MM:SS.MSZ
XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX new_task_definition success info  Task XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX complete                                                                YYYY-MM-DDTHH:MM:SS.MSZ

Example solution

The code for this tutorial is available below:

Next steps

You've now used the EDK CLI to generate summarised and detailed views of the states of Task instance execution. You can now monitor the state of your solution without having to read datastream values.

Continue to the next module to learn how to set up validations in a template to be performed in a live solution.

On this page