What is a function?

Functions are another way to transform data in Elara solutions. They are an alternative approach to pipelines, but result in a similar outcome.

How does a function differ from a pipeline?

The EDK provides a declarative interface for building solution Templates. You simply declare that a data stream or pipeline should exist, and the platform will build and manage them for you.

Similarly, pipelines allow you to write logic and data transformations in a declarative manner. Like SQL, a pipeline declares what result should be constructed during each operation. It is up to Elara precisely how the result is calculated, so long as the expected result is produced.

Functions allow you to transform data using an imperative, not declarative, interface. Imperative code is probably what you are used to writing as a computer programmer. Languages including TypeScript/JavaScript, C, C++, C#, Java, Python, Rust, Swift and more are imperative languages. In an imperative language, you provide a series of steps for the computer to follow.

This interface allows you to express your logic in a manner to which you may be accustomed. You are free to implement advanced algorithms yourself, if necessary. You can use for and while loops, reassign variables, and mutate arrays, sets and dictionaries. Unlike pipelines, functions are Turing complete. You can write any program imaginable within a function, and chain multiple functions together in a solution to obtain the results you want in a modular and observable manner.

You may think of pipelines as a high-level interface for data transformation, and functions as a lower-level interface.

What is a function?

A function is a type of task that runs on the Elara platform (like sources, pipelines, simulations, etc). It takes as input any number of data streams, and can produce any number of data streams. A simple function would have a single input and a single output.

The function has a "body", which is a block of code. The function task will load the inputs and execute this code, as written. Once the calculation is complete, the function can "return" data. This data is written to one or more data streams.

That is all a function is – a set of instructions to transform input data into output data.

How are functions written?

The code in a function is written in a combination of East expressions and function statements. Take TypeScript (or JavaScript) as an example of a language with both expressions and statements. Statements tend to start with keywords like let, const, if, for, while and switch, often end with a semicolon (;), and do not evaluate to a value (instead, they are instructions). Expressions are things like calculations, 1 + 2 or f(x) or x ? y : z, which do evaluate to a value. You can generally nest expressions inside other expressions (like 1 + (2 * 3)) but statements must exist at "top" level within a block of code. In TypeScript/JavaScript you cannot write f(let x = 3) because you cannot nest the let statement inside the expression f(...).

The body of a function build is pieced together with statements that somewhat resemble the statements in TypeScript or JavaScript. Instead of writing let x= 3 in FunctionBuilder, you use the .let method. The equivalent code looks like .let("x", (vars) => Const(3)), meaning "create a variable called x and assign the value 3 to it". The vars contains a list of variables in scope. The right hand side could be an expression of these variables, like .let("x", (vars) => Add(vars.y, 1)) meaning let x = y + 1. You will get more practice with this in the next lesson.

Next steps

Now you understand what a function is, in the next tutorial you will get some experience in creating functions with FunctionBuilder and running them on the Elara platform.