Skip to main content

Handle missing values

In this tutorial, you will:

  • define a type explicitly in a datastream to handle missing values,
  • observe how Elara treats missing values in an implicitly-typed datastream and
  • enforce non-nullable from a nullable datastream

This lesson will assume that you have an empty project and asset which you can to deploy to a workspace named 03_02_07_handle_missing_values with the following command:

edk template deploy -ycw 03_02_07_handle_missing_values

Why are nullable datastream's required?

There are often cases of missing values in data (also known as null values), and Templates must be written to handle these cases explicitly. The equivalent type for null in Elara is NullType.

Unless otherwise specified, primitive and collection types are not allowed to be NullType. For example, if you declare a datastream value as IntegerType such as in your datastream definition Writeable.My Source, a NullType value can't be ascribed to the datastream.

Often missing values are not expected or desired, in which case you do not need to make any changes to a datastream definition to handle missing values. The solution itself will fail to parse a missing value and generate a run-time error.

In other circumstances, you may want to allow missing values in a defined datastream. In these cirumstances, you use the Nullable() operator in the type definition of your datastream to make the value "nullable", which will allow the datastream to have NullType values.

Define a nullable datastream

To use a nullable type DataStream you will perform the following steps:

  1. define a datasource using a similar pattern as your previous My Source definition.
  2. in the definition for My Source, manually define the type as Nullable(IntegerType).
  3. define a pipeline using the same pattern as your previous My Pipeline definition.
  4. add the datasource to the template

In an asset, perform the above steps to create the resulting Typescript code:

import { SourceBuilder, Template, Nullable, IntegerType } from "@elaraai/core"

const my_source = new SourceBuilder("My Source")
    .value({
        value: 2n,
        // define the type as a Nullable Integer
        type: Nullable(IntegerType)
    })

export default Template(my_source)

Interact with a Nullable datastream

Once deployed, you can test if the the value of Writeable.My Source can be set to null using the following command:

edk stream replace "Writeable.My Source" --format "json" --value null -w 03_02_07_handle_missing_values

Using the following command:

edk stream get "Writeable.My Source" -w 03_02_07_handle_missing_values

Will result in the correct null value:

▹▹▹▹▹ Attempting to stream Writeable.My Source to stdout
null
✔ Download complete

With the declaration of the datastream type wrapped with Nullable(), the datastream value is allowed to be NullType. The datastream value is nullable.

A NullType isn't an empty stream

In previous lessons, you learnt that a datastream is allowed to be empty. This is not the same as setting a datastream value to NullType. This is an important distinction - an empty datastream signifies that no data is set, whereas a NullType datastream signifies that it is set as missing or null.

Further Reading

Read the

null tutorial for a more detailed understanding of working with null.

Example solution

The code for this tutorial is available below:

Next steps

In this tutorial, you ..., in the next lesson you will ...

On this page