Skip to main content

East expressions

East expressions are the primary way of adding logic to your Elara solutions. Expressions are created in the Elara Development Kit (EDK) and sent to the Elara platform as a part of the solution template. When performing a task involving user-defined logic, Elara will compile and execute your expressions as a part of that task.

This immediately provides some constraints for how the East programming language was designed and implemented. To integerate with the EDK, we need out-of-the-box support within VS Code and TypeScript (the language of the EDK). The expressions need to be easy to send accross the internet and easy to embed into tasks on the Elara platform. It needs to be easy to check correctness of expressions with the EDK, and any code evaluated on the Elara platform must be safe and secure.

Expressions in the EDK

The EDK is a TypeScript framework to build Elara solutions. Developers use the EDK to construct solutions Templates consisting of data streams, tasks, portal layouts, etc. The various components are constructed through Builders, which make heavy use of the fluent interfaces (a.k.a. the builder pattern).

Inside components like data pipelines or simulations Elara needs to evaluate your custom logic. This logic is provided inline in TypeScript using a combination fluent interfaces and East functions exported by the EDK. East expressions are enterred directly in TypeScript with functions like Add(1, 2). We do not use strings like "1 + 2" to representing East expressions, and there are no seperate East source files to worry about. That means East is a programming language without a syntax. East is a language built entirely inside TypeScript. We expect that it make take some time for you to get used to working with a language within another language.

Our choice to embed East directly within TypeScript is to minimize complexity for you and us:

  • The structure and logic of your solutions can live together in the same .ts source file, making it easy to reason about.
  • TypeScript can check the East types of your expressions and VS Code can provide live programming feedback, for example warning you when your East types don't match within a complex expression.
  • The process of constructing solution templates, checking correctness, etc is immensely simplified.

The consequence of this is that East expressions may "look" a little different to other programming languages you have learned, but they fundamentally use the same structure as any other language.

Simple expressions

At their core, East expressions are a simple "tree" of instructions to follow in order to compute some result. Consider these examples:

  1. Add(1, 2)
  2. Multiply(5, Add(1, 2))
  3. Add(Mutply(5, 1), 2)

The first represents 1 + 2 = 3. The second is 5 * (1 + 2) = 15, and the third is (5 * 1) + 2 = 7.

The first thing to note is that East doesn't support operators like +, -, * or /. Because of this, there are no need for "extra" parentheses like used in mathematical expressions - East expressions are either nested inside another expression, or are the unique "root" expression. The "leaves" of the tree are evaluated first, working our way up towards the root. In computer science, this tree is often called an "abstract syntax tree" (AST) because it's an abstract tree representation of a programming language's syntax after it is parsed and processed (East doesn't have a concrete syntax). Our expressions were originally named "Elara AST"s, and the acronym East quickly followed.

There is no limit to how deep or complex expressions can be. East is a simple expression language where each node is one of a number built-in or "fundamental" operations. Complex expressions are constructed out of these building blocks. All expressions have the TypeScript type EastFunction<T extends EastType>.

There are fundamental East functions for each of the East data types, including:

  • Logical operators for Boolean values
  • Mathematical operations on floats and integers
  • String manipulation and parsing functions
  • Construction and deconstruction of structs
  • Construction and deconstruction of variants
  • Array creation, lookup values, filter, map, reduce, sort, etc.
  • Set creation, check for inclusion, filter, map, reduce, union, intersect, etc.
  • Dictionary creation, lookup values, filter, map, reduce, etc.
  • Equality and comparison for every type

With these building blocks, it is possible to craft intricate logic and transform complex data. A single EDK project will involve tens to thousands of expressions, spread across many tasks to be executed on the Elara platform. Leveraging this, it is possible to construct any program imaginable with Elara.

Note that while the East expression language doesn't allow for defining and calling functions, it is possible to simulate this in TypeScript. One can simplify code and make EDK projects more modular but crafting TypeScript functions that return complex East expressions as a function of inputs. A simple example is const Double = (x: EastFunction<FloatType>) => Multiply(x, 2), which doubles the input x. TypeScript will helpfully infer the type of the output of expressions for us.

A note on values vs. expressions

One thing that we've glossed over is how we represent values in a the AST. Every node of the AST has TypeScript type EastFunction<T>. To make values (including JavaScript objects) unambiguous there is a special node called Const. A Const is a wrapper for a value, along with its type. For example Const(true) is an EastFunction<BooleanType> that contains value true and type BooleanType (the East type can optionally be inferred from the value).

The expressions like Add(1, 2) really are just shorthand for Add(Const(1, FloatType), Const(2, FloatType)). This "shorthand" is provided in most places a primitive value is expected. For more complex values (like structs or arrays) it will usually be necessary to use Const explicitly. Similarly, for more complex expressions like Struct it will be necessary to use Const even when the values are primitive.

The difference between values and Const can be tricky when first learning East. When in doubt, if you are experiencing problems entering an expression try wrap your values in Const to see if that helps.

Working with variables

The expressions we've seen so far are quite simple. In fact, they can all be evaluated in advance! We know that Add(1, 2) will always evaluate to 3.

In contrast, solutions deployed on Elara will be data driven and expressions will depend on run-time data. Like all programming languages, East uses variables to represent data that varies at run time. Each variable has a distinct name to identify, as well as a fixed EastType. A Variable<T> is one of the possible nodes in an expression; it is an EastFunction<T>. Variables are used extensively throughout the EDK.

Consider this simple pipeline that takes a datastream input_stream which contains an integer (input_stream is a Stream<IntegerType>) and increments it by 1:

new PipelineBuilder("increment")
    .from(input_stream)
    .transform(x => Add(x, 1n))

Here we see the EDK and East expressions working together. The .transform method is used to evaluate an arbitrary East expression on the inputs. In the above the EDK user has provided the function x => Add(x, 1n). The EDK will construct a variable representing the run-time value, and call this function with this variable, resulting in an EastFunction. TypeScript will infer the type of x is Variable<IntegerType> and that the return type of the expression is a EastFunction<IntegerType> (In this way we can automatically infer the output type of the pipeline is also IntegerType). Once deployed, the pipeline task will inject the value from input_stream into a variable of the given name and evaluate the East expression.

One thing to note is how in this case the name of the variable is generated automatically by the EDK. Note that the EDK does not "see" the name of the TypeScript variable x as it builds your solution template (it simply executes the JavaScript code without reference to the source). Only sometimes (in FunctionBuilder and ProcessBuilder) you will have the opportunity to name your the Variables yourself with the .let method. In either case, the name of the variable used is unimportant, as the output value does not depend on the Variable name.

Expressions that create new variables

Users can also create variables to use within expressions. Certain East functions will result in the creation of new variables. The most important one to learn is Let.

Like let in TypeScript/JavaScript, the Let function creates a new variable that can be accessed within the current scope. That means that any variables constructed by Let go "out of scope" once the expression is fully evaluated. East expressions do not have code blocks like most modern langauges, so the way Let works might seem a bit foreign at first. To clarify, here is an example of Let in action:

Let(
    Multiply(8n, 5n),
    x => Add(x, 2n)
)

What this does is first evaluate the first expression. Once that value is known it creates a new variable (the x above is a Variable<IntegerType>) and uses that variable when evaluating the second expression. The variable x is only "in scope" within the second expression. It immediately goes out of scope once the Let node has been fully evaluated. The Let expression returns the final result of the second expression. In this case we find that x = 8n * 5n or 40n, and return x + 2n which is 42n.

When is this useful? Sometimes a value is difficult to compute (the code is complex or it takes a long time to compute). When that value needs to be reused, it can be better to assign it to variable and refer to it multiple times rather than compute it all over again. This can result in simpler or faster code.

Expressions that branch

Code that "branches" is code that executes conditionally. The simplest example is the IfElse function, which returns results depending on if a "predicate" expression evaluates to true or false. For example, suppose we had two floating-point variables x and y and wanted to know which was bigger. We could construct an appropriate string depending on the run-time values via:

IfElse(
    Greater(x, y),
    Const("x is bigger"),
    Const("y is bigger"),
)

Note the usage of Const in the second and third expressions. The second expression (true branch) and third expression (false branch) can be arbitrarily complex. East will only evaluate the one of the two subexpressions based on the result of the predicate (it will not evaluate both the true branch and the false branch). IfElse(x, y, z) is similar to IF(x, y, z) in Excel and the "ternary" operator x ? y : z in TypeScript/JavaScript and many other languages.

Note that for branching functions the East data types of the results must be compatibile. For example, we cannot have the true branch of IfElse return a string and the false branch return an integer. The East type of the result must be well defined. Note that "compatible" doesn't mean "identical". East can automatically merge null with any type to make it nullable, or merge together different variants to result in a VariantType that covers all cases.

The other EastFunctions that branch are IfNull (for nullable types) and Match (for variants). These functions are more complex as they also inject variables into their branch expressions.

Expressions that loop

East expressions are a lot more powerful than some "simple" expression languages. In East it is possible to iterate over a collection to filter, transform and aggregate results. However, in East these are not achieved using a for or while loop as in many imperative languages. There are no while loops in East expressions (though you can use this and other imperative constructs in FunctionBuilder and ProcessBuilder).

Instead, East uses constructs from functional programming languages to iterate over collections, like Filter, Map and Reduce. Take for example Filter:

Filter(
    Const([1, 2, 3, 4, 5]),
    x => LessEqual(x, 3)
)

This operation loops over the values in the input array (1 through 5) and evaluates the expression LessEqual(x, 3) for each of them. This returns true for 1, 2 and 3 and returns false for 4 and 5. The Filter function retains the true values and excludes the false values, returning a new collection (the input collection is not modified). The final result is [1, 2, 3].

Note that although East does not support first-class functions, the EDK does have higher-order functions. The EDK manages variable mapping to enforce the appropriate scoping rules (referencial transparency) expected when functions map arguments to parameters as they are called. Advanved EDK users can use TypeScript to construct their own higher-order functions with care (by using Let on the arguments to ensure they are not calculated multiple times).

That covers the various "patterns" to expect when constructing East expressions with the EDK.

JSON representation of the AST

Here we provide some insight into how Elara represents expressions in JavaScript and JSON. It is included for completeness; you do not need to understand this to use Elara effectively. For more advanced programmers it might provide some context of how the system works and is designed. Otherwise, feel free to skip this section.

Let's suppose you wanted to encode the expression for 1 + 2 as JSON. Expressions are represented directly as an "abstract syntax tree", without relying on a source code language. The EDK provides the Add function for addition. Calling Add(1, 2) results in the following simple JavaScript object:

{ 
    ast_type: "Add",
    type: FloatType,
    first: Const(1),
    second: Const(2),
}

The Const function also creates simple JavaScript objects, like:

{ 
    ast_type: "Const",
    type: FloatType,
    value: 1,
}

Note that the value in Const is always encoded in a our canonical JSON format. For floats that are not NaN, Infinity or -Infinity this is just a number.

Finally, the types themselves are simple JavaScript objects, such as FloatType:

{
    type: "Float",
    value: null,
}

All of these objects are straightforward to serialize with JSON.toString() and deserialize with JSON.parse(). You can inspect the TypeScript definitions of EastFunction and EastType to learn more. Expressions like the above are included inside task definitions inside the solution template, which itself is a JSON document sent to the Elara platform to instantiate a workspace into a hosted solution. From there, expressions can be evaluated by a variety of techniques, from simple tree-walking interpretation of individual expressions to optimized native-code generation of entire tasks.

Next steps

Now that understand how East expressions allow you to perform safe and performant logic on the Elara platform, continue to the next section to learn how to use and manipulate all the primitive data types in East.

On this page