Skip to main content

Define a custom task

In this tutorial you will:

  • Create a data source,
  • Build a custom task that executes a shell script based on input data, and
  • Read the results of the custom task.

This lesson will assume that you have an empty project and asset which you can to deploy to a workspace named 08_01_02_define_a_custom_task with the following command:

edk template deploy -ycw 08_01_02_define_a_custom_task

Create the datasource

In this case we are going to use our custom task to do some simple mathematics. The input will be a floating-point number. Initially, our project will have just this datasource.

import { FloatType, SourceBuilder, Template } from "@elaraai/core"

const my_source = new SourceBuilder("My Source")
    .writeable(FloatType);

export default Template(my_source);

Build the custom task

A custom task is built by the CustomTaskBuilder class. You start by giving a task a name, and providing the inputs. This task will have just one input, from the datasource above.

The .input method

const custom_task = new CustomTaskBuilder("Quadratic")
    .input(
        "input.txt", // filename
        my_source.outputStream(), // datastream
        false, // executable
        float => Utf8Encode(Print(float)), // toBlob
    )

A custom task can execute one of:

  • a shell script (bash, on Ubuntu LTS)
  • a native linux program (on Ubuntu LTS)
  • a user-provided docker image

In this lesson we will look at the simplest option – a shell script. We will use bash builtin functions and cat to perform some simple mathematics. Our goal is to perform the following function:

f(x) = x * (x - 10)

Our input is in a file called input.txt. We can use the cat program to read that file. We can use $(( ... )) syntax to perform mathematical expressions in bash, and pipe the output to a file. One way to do calculate the formula above is using the awk command-line tool via this bash script:

awk '{x=$1; printf "%.14f\n", x * (10 - x)}' input.txt > output.txt

The awk program parses simple C-like expressions and produces a result. Understanding awk is not crucial to this lesson; just know that the above reads input.txt and writes the answer to a new file, output.txt. We can pass the above program above as a string to the .shell method on FunctionBuilder. This results in a file called output.txt.

The final thing to do is to instruct the FunctionBuilder to read the output file with the .output method, and parse it back to a floating-point number. Parsing is optional; you can simply return results with a BlobType data type. But in this case the result is numerical and it makes sense to parse it to a float immediately. To do so, we must first convert the blob to a string with Utf8Decode and then parse the string to a float with Parse. Putting it all together results in the following project definition:

import { CustomTaskBuilder, FloatType, Parse, Print, SourceBuilder, Template, Utf8Decode, Utf8Encode } from "@elaraai/core"

const my_source = new SourceBuilder("My Source")
    .writeable(FloatType);

const custom_task = new CustomTaskBuilder("Quadratic")
    .input(
        "input.txt", // filename
        my_source.outputStream(), // datastream
        false, // executable
        float => Utf8Encode(Print(float)), // toBlob
    )
    .shell(`awk '{x=$1; printf "%.14f\n", x * (10 - x)}' input.txt > output.txt`)
    .output(
        "output.txt", // filename
        blob => Parse(FloatType, Utf8Decode(blob)) // fromBlob
    )

export default Template(my_source, custom_task);

Running the task

Build and deploy the template. Populate the value, for example via:

edk stream replace "Writeable.My Source" --value 2 -w 08_01_02_define_a_custom_task

This should inject the value 2 into the function. This should calculate 2 * (10 - 2), or 16. Let's see what result we get:

$ edk stream get Custom.Quadratic.output.txt -w 08_01_02_define_a_custom_task

Additional exercises

Try propulating the writable data source with some other values and see what results you get.

  • Try providing unusual numbers, such as something negative
  • Try more extreme cases, such as Infinity or NaN (Recall, you use "Infinity" and "NaN" to represent those values as JSON in edk stream replace, and you will need to escape each " with a \ like \"NaN\").

You may have some experience programming in other languages.

  • If so, how would you go about running a program you created inside an Elara task?
  • Would you use a shell script on Ubuntu LTS?
  • Or would you need to provide an entire docker image to set up your environment to execute your code?

Example Solution

The final solution for this tutorial is available below:

Next Steps

In this tutorial, you learned how to create custom tasks. In the

next module you will learn how to create custom scenarios that you can optimize.

On this page