Optimize a custom scenario
In this tutorial you will:
- Start with the custom scenario from the previous lesson,
- Add an optimization objective,
- Instruct the optimizer to optimize the input value, and
- Configure the optimizer.
This lesson will assume that you have an empty project and asset which you can to deploy to a workspace named 08_02_03_optimize_a_custom_scenario with the following command:
edk template deploy -ycw 08_02_03_optimize_a_custom_scenario
Optimization
Mathematical optimization is the processes of maximizing (or minimizing) the result of some function or procedure by altering the arguments or inputs.
You can apply Elara's optimizer to a custom task using custom scenarios.
The custom task will be executed many times with different.
The optimizer will explore the optimization space in order to maximize some objective that you specify.
The object is a real number – a FloatType value.
Preparation
We begin with the solution to the previous lesson.
import { CustomScenarioBuilder, FloatType, Parse, Print, SourceBuilder, Template, Utf8Decode, Utf8Encode } from "@elaraai/core"
const my_source = new SourceBuilder("My Source")
.writeable(FloatType);
const custom_scenario = new CustomScenarioBuilder("Quadratic")
.input(
"input.txt", // filename
my_source.outputStream(), // datastream
float => Utf8Encode(Print(float)), // toBlob
)
.shell(`awk '{x=$1; printf "%.14f\n", x * (10 - x)}' input.txt > output.txt`)
.output(
"output.txt", // filename
blob => Parse(FloatType, Utf8Decode(blob)) // fromBlob
)
export default Template(my_source, custom_scenario);
To custom_scenario we configure two more settings to enable optimization.
First we need to set the optimization objective, and second we need to instruct it which input(s) to optimize.
Optimization objective
You provide the objective to maximize using the .objective method.
The objective is an East expression of the (decoded) output files.
The objective must be a float.
For this tutorial, we simply want to maximize the output. We can add this objective function like so:
const custom_task = new CustomScenarioBuilder("Quadratic")
.input(
"input.txt", // filename
my_source.outputStream(), // datastream
float => Utf8Encode(Print(float)), // toBlob
)
.shell(`awk '{x=$1; printf "%.14f\\n", x * (10 - x)}' input.txt > output.txt`)
.output(
"output.txt", // filename
blob => Parse(FloatType, Utf8Decode(blob)) // fromBlob
)
.objective(outputs => outputs['output.txt'])
Optimized inputs
Next we need to instruct which inputs may be varied by the optimizer, with the remainder fixed. In this case there is only one input, but in more complex cases it is important to be able to specify precisely what may vary.
If you want to optimize a "scalar" input you can use .optimize.
With this you can specify a range of values to search with min and max (or you may specify a descrete set of possibilities with range).
We can specify this by adding a single new line of code
const custom_task = new CustomScenarioBuilder("Quadratic")
.input(
"input.txt", // filename
my_source.outputStream(), // datastream
float => Utf8Encode(Print(float)), // toBlob
)
.shell(`awk '{x=$1; printf "%.14f\\n", x * (10 - x)}' input.txt > output.txt`)
.output(
"output.txt", // filename
blob => Parse(FloatType, Utf8Decode(blob)) // fromBlob
)
.objective(outputs => outputs['output.txt'])
.optimize("input.txt", { min: 0, max: 10 });
If you want to optimize many values, say every value in a column of a table, you can use the .optimizeEvery method instead.
We do not use that here, but it works the same as in ScenarioBuilder.
Configuring the optimizer
There are multiple settings to configure and tune the optimizer. The most important to set is the maximum number of iterations. In this very simple case 20 iterations is sufficient to produce a good result. When there are more input parameters to optimize, or finding a good objective is difficult, you may require many more iterations.
Another important setting is the number of Monte Carlo trajectories to employ during optimization. This defaults to 20 trajectories (per optimization iteration). Our problem is deterministic, so a single trajectory is sufficient.
const custom_task = new CustomScenarioBuilder("Quadratic")
.input(
"input.txt", // filename
my_source.outputStream(), // datastream
float => Utf8Encode(Print(float)), // toBlob
)
.shell(`awk '{x=$1; printf "%.14f\\n", x * (10 - x)}' input.txt > output.txt`)
.output(
"output.txt", // filename
blob => Parse(FloatType, Utf8Decode(blob)) // fromBlob
)
.objective(outputs => outputs['output.txt'])
.optimize("input.txt", { min: 0, max: 10 })
.optimizationMaxIterations(20)
.optimizationTrajectories(1);
Running the optimized scenario
Deploy the template.
Optimization is a procedure that can take a long time.
You can observe the progress live using the task logs.
Once you have deployed, observe the progress using edk task logs:
edk task logs CustomOptimization.Quadratic -w 08_02_03_optimize_a_custom_scenario
This will produce output something like:
Listing task logs for CustomOptimization.Quadratic available in tenant with identifier: XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX and workspace 08_02_03_optimize_a_custom_scenario.
UUID REASON STATUS LEVEL MESSAGE LOGGEDATE
XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX new_task_definition success info Worker XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX starting task instance XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX attempt 1 YYYY-MM-DDTHH:MM:SS.MSZ
XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX new_task_definition success info Loading inputs ["input.input.txt"] YYYY-MM-DDTHH:MM:SS.MSZ
XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX new_task_definition success info Executing custom optimization task YYYY-MM-DDTHH:MM:SS.MSZ
XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX new_task_definition success info Executing command ["/bin/bash", "-c", "awk '{x=\$1; printf \"%.14f\\n\", x * (10 - x)}' input.txt > output.txt"] YYYY-MM-DDTHH:MM:SS.MSZ
XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX new_task_definition success info Using default docker image YYYY-MM-DDTHH:MM:SS.MSZ
XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX new_task_definition success info Initializing optimization for CustomOptimization.Quadratic YYYY-MM-DDTHH:MM:SS.MSZ
XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX new_task_definition success info Optimizing 1 parameter... YYYY-MM-DDTHH:MM:SS.MSZ
XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX new_task_definition success info Optimization iteration 1: objective = 0.0 YYYY-MM-DDTHH:MM:SS.MSZ
XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX new_task_definition success info Optimization iteration 2: objective = 24.99 YYYY-MM-DDTHH:MM:SS.MSZ
XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX new_task_definition success info Optimization iteration 3: objective = 0.0 YYYY-MM-DDTHH:MM:SS.MSZ
XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX new_task_definition success info Optimization iteration 4: objective = 22.44 YYYY-MM-DDTHH:MM:SS.MSZ
XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX new_task_definition success info Optimization iteration 5: objective = 25.0 YYYY-MM-DDTHH:MM:SS.MSZ
XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX new_task_definition success info Optimization iteration 6: objective = 21.99555555555555 YYYY-MM-DDTHH:MM:SS.MSZ
XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX new_task_definition success info Optimization iteration 7: objective = 24.15972222222222 YYYY-MM-DDTHH:MM:SS.MSZ
XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX new_task_definition success info Optimization iteration 8: objective = 14.11 YYYY-MM-DDTHH:MM:SS.MSZ
XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX new_task_definition success info Optimization iteration 9: objective = 7.7775 YYYY-MM-DDTHH:MM:SS.MSZ
XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX new_task_definition success info Optimization iteration 10: objective = 20.30555555555556 YYYY-MM-DDTHH:MM:SS.MSZ
XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX new_task_definition success info Optimization iteration 11: objective = 21.45305555555556 YYYY-MM-DDTHH:MM:SS.MSZ
XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX new_task_definition success info Optimization iteration 12: objective = 24.86145061728395 YYYY-MM-DDTHH:MM:SS.MSZ
XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX new_task_definition success info Optimization iteration 13: objective = 24.94425154320988 YYYY-MM-DDTHH:MM:SS.MSZ
XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX new_task_definition success info Optimization iteration 14: objective = 24.58469135802469 YYYY-MM-DDTHH:MM:SS.MSZ
XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX new_task_definition success info Optimization iteration 15: objective = 24.39073302469136 YYYY-MM-DDTHH:MM:SS.MSZ
XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX new_task_definition success info Optimization iteration 16: objective = 24.78566529492456 YYYY-MM-DDTHH:MM:SS.MSZ
XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX new_task_definition success info Optimization iteration 17: objective = 24.8256164266118 YYYY-MM-DDTHH:MM:SS.MSZ
XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX new_task_definition success info Optimization iteration 18: objective = 24.96361796982167 YYYY-MM-DDTHH:MM:SS.MSZ
XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX new_task_definition success info Optimization iteration 19: objective = 24.95444937414266 YYYY-MM-DDTHH:MM:SS.MSZ
XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX new_task_definition success info Optimization iteration 20: objective = 23.86222222222222 YYYY-MM-DDTHH:MM:SS.MSZ
XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX new_task_definition success info Optimization completed, best found objective = 25.0 YYYY-MM-DDTHH:MM:SS.MSZ
XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX new_task_definition success info Saving outputs ["result.output.txt", "optimization", "optimized.input.txt"] YYYY-MM-DDTHH:MM:SS.MSZ
XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX new_task_definition success info Successfully completed task instance 5c7ea111-e094-4a9e-b154-c114587fe0de attempt 1 YYYY-MM-DDTHH:MM:SS.MSZ
Here we can see the optimizer performing 20 iterations, and the objective value for each.
It found the best value of 25 on iteration 4.
You can observe it continuing to search nearby in case a better result can be found.
The optimized custom scenario will produce a few datastreams of interest. We can get the optimized input value:
$ edk stream get CustomOptimized.Quadratic.input.txt -w 08_02_03_optimize_a_custom_scenario
▹▹▹▹▹ Attempting to stream CustomOptimized.Quadratic.input.txt to stdout
4.999999999999999
✔ Download complete
The maximal value of x * (10 - x) occurs at x = 5.
This is exactly half way inside the search range between 0 and 10, so the algorithm stumbles near 5 relatively quickly.
We can get the results from the optimized scenario.
$ edk stream get CustomSimulationResult.Quadratic.output.txt -w 08_02_03_optimize_a_custom_scenario
▹▹▹▹▹ Attempting to stream CustomOptimized.Quadratic.input.txt to stdout
25
✔ Download complete
This result is the maximum output of 25 (which is the same as the objective in this scenario).
Finally, you can see a record of the optimization procedure in a third datastream.
$ edk stream get CustomOptimization.Quadratic -w 08_02_03_optimize_a_custom_scenario
▹▹▹▹▹ Attempting to stream CustomOptimization.Quadratic to stdout
[{"objective":0,"objectives":[0],"values":[{"key":"input.txt","value":"0.0"}]},
{"objective":24.99,"objectives":[24.99],"values":[{"key":"input.txt","value":"5.1"}]},
{"objective":0,"objectives":[0],"values":[{"key":"input.txt","value":"10.0"}]},
{"objective":22.44,"objectives":[22.44],"values":[{"key":"input.txt","value":"3.4000000000000004"}]},
{"objective":25,"objectives":[25],"values":[{"key":"input.txt","value":"5.0"}]},
{"objective":21.99555555555555,"objectives":[21.99555555555555],"values":[{"key":"input.txt","value":"6.733333333333333"}]},
{"objective":24.15972222222222,"objectives":[24.15972222222222],"values":[{"key":"input.txt","value":"5.916666666666667"}]},
{"objective":14.11,"objectives":[14.11],"values":[{"key":"input.txt","value":"1.7000000000000002"}]},
{"objective":7.7775,"objectives":[7.7775],"values":[{"key":"input.txt","value":"0.8500000000000001"}]},
{"objective":20.30555555555556,"objectives":[20.30555555555556],"values":[{"key":"input.txt","value":"2.833333333333334"}]},
{"objective":21.45305555555556,"objectives":[21.45305555555556],"values":[{"key":"input.txt","value":"3.116666666666667"}]},
{"objective":24.86145061728395,"objectives":[24.86145061728395],"values":[{"key":"input.txt","value":"5.372222222222222"}]},
{"objective":24.94425154320988,"objectives":[24.94425154320988],"values":[{"key":"input.txt","value":"5.236111111111111"}]},
{"objective":24.58469135802469,"objectives":[24.58469135802469],"values":[{"key":"input.txt","value":"5.644444444444444"}]},
{"objective":24.39073302469136,"objectives":[24.39073302469136],"values":[{"key":"input.txt","value":"5.780555555555555"}]},
{"objective":24.78566529492456,"objectives":[24.78566529492456],"values":[{"key":"input.txt","value":"5.462962962962963"}]},
{"objective":24.8256164266118,"objectives":[24.8256164266118],"values":[{"key":"input.txt","value":"5.417592592592593"}]},
{"objective":24.96361796982167,"objectives":[24.96361796982167],"values":[{"key":"input.txt","value":"5.19074074074074"}]},
{"objective":24.95444937414266,"objectives":[24.95444937414266],"values":[{"key":"input.txt","value":"5.213425925925925"}]},
{"objective":23.86222222222222,"objectives":[23.86222222222222],"values":[{"key":"input.txt","value":"3.9333333333333336"}]}]
✔ Download complete
In the above we see a record for each iteration, that includes the value used for input.txt.
This record is persisted as a datastream can be used in downstream tasks and layouts. It is only produced once the optimization has completed. The task logs, on the other hand, allow you to observe progress as it is happening. Both records can be useful.
Example Solution
The final solution for this tutorial is available below:
Next Steps
In this tutorial, you have learned how to optimize a custom scenario. This concludes the training material for Elara.