Filtering and Transforming

With East you can work with collections of data large and small. You do not write imperative for and while loops within East expressions to iterate over the values inside. Instead you use operations popularized by functional programming languages, like filtering and "mapping" data.

Filtering collections

East provides the Filter operation for filtering values of an array, set or dictionary. It behaves similarly to JavaScript's .filter method on arrays. For each entry in the collection, a predicate is evaluated which returns true to retain the element and false to discard it.

Filter works on arrays, sets and dictionaries and will return a new collection with the same East type. The input is not modified.

East function	Description	Example usage	Result
Filter	Filter an array	Filter([1n, 2n, 3n], value => Greater(value, 1n))	[2, 3,]
Filter	Filter a set	Filter(new Set(["a", "b", "c"]), key => Greater(key, "a"))	{"b", "c",}
Filter	Filter a dictionary	Filter(new Map([["a", 1n], ["b", 2n], ["c", 3n]]), (value, key) => Greater(value, 1n))	{"b": 1, "c": 2,}

Transforming collections

One common operation is applying a transformation to each value in a collection. For example, JavaScript's .map method on arrays returns a new array by applying a function to each value.

A related thing you might do is convert an array to a dictionary, or a set to an array, and so-on. East has functions ToArray, ToSet and ToDict to transform any collection to another. Each of these will iterate over their input collection and insert entries into the output. The user may provide expressions to transform the values and/or keys, as necessary. The input is never modified.

East function	Description	Example usage	Result
ToArray	Double each value in an array	ToArray([1n, 2n, 3n], value => Multiply(value, 2n))	[2, 4, 6,]
ToArray	Collect the keys of a set in an array	ToArray(new Set(["a", "b", "c"]))	["a", "b", "c",]
ToArray	Collect the values of a dictionary in an array	ToArray(new Map([["a", true], ["b", false], ["c", true]]))	[true, false, true,]
ToSet	Get the unique values of an array	ToSet(["a", "b", "a"])	{"a", "b",}
ToSet	Make set keys uppercase	ToSet(new Set(["a", "b", "c"]), key => Uppercase(key))	["A", "B", "C",]
ToSet	Collect the keys of a dictionary in a set	ToSet(new Map([["a", true], ["b", false], ["c", true]]), (value, key) => key)	["a", "b", "c",]
ToDict	Build a dictionary from array value to its index	ToDict(["a", "b", "a"], (value, key) => value, (value, key) => key)	{"a": 0, "b": 1, "c": 2}
ToDict	Build a mapping from set key to its uppercase value	ToDict(new Set(["a", "b", "c"]), key => key, key => Uppercase(key))	["a": "A", "b": "B", "c": "C",]
ToDict	Double each value in a dictionary	ToDict(new Map([["a", 1n], ["b", 2n], ["c", 3n]]), key => key, key => Multiply(key, 2))	["a": 2, "b": 4, "c": 6,]

In ToDict you specify the new key before the value. Note how the transformations of value and key are optional in ToArray, ToSet and ToDict (the identity function is presumed).

It is common to use Get on another collection within these expressions, using an appropriate key. This lets you "join" together data in different collections, a bit like joining tables in SQL.

Filtering and transforming at once

Often you want to both filter a collection, and apply a transformation to the values. It is perfectly OK to combine, for example, Filter and ToArray to achieve this.

One challenge that can arise is when filltering your data may "narrow" the resulting type. A good example is removing all the null values from an array. Suppose you had the array [1, 2, null, 4] and wanted to create a new array with the nulls removed, [1, 2, 4]. The Filter expression can do this, but the resulting type will still have a nullable value type.

A similar case arises with variants. Consider using the option variant types instead of nullable types. Suppose you had an array like [some(1), some(2), none, some(4)] and wanted to unwrap all the some values and discard the none values, to result in [1, 2, 4].

East provides the FilterMap function for applying a filter and transformation that may result in a different data type. It works using the option variant. Entries which transformed to some(x) are preserved as x, and entries which are transformed to none are discarded.

East function	Description	Example usage	Result
FilterMap	Extract the non-null values from a nullable array	FilterMap([1n, 2n, null, 4n], value => IfNull(value, none, x => some(x)))	[1, 2, 4,]
FilterMap	Extract the present values from an array of optional data	FilterMap([some(1n), some(2n), none, some(4n)], value => value)	[1, 2, 4,]

Note in the above examples an array of type ArrayType(IntegerType) is returned. Achieving the same result with Filter and ToArray would be challenging. This can be particularly useful when trying to extract certain variant cases from a collection, while discarding others.

The FilterMap expression also works with set and dictionary inputs. In this case a set or dictionary is returned.

Next steps

Continue to the next tutorial to understand how summarize data and group data together with reduction and aggergation operations.