We should support the ability to specify where in the flow of the experiment logic the analyzer and planner service(s) get called. Right now this is mostly motivated by how we're forced to use Athena/educational ARES by running a bunch of 4 experiment campaigns back to back, but as the planner never actually gets called after the final experiment in each campaign, the planner never gets the chance to see that data. More broadly I think this would be valuable in cases where planners may take a significant amount of time to run and/or there are commands that need to be run in order to reset an experiment state for base line.
For a concrete example of how this would be beneficial over the current analysis and planning at the end paradigm, consider a hypothetical process where fitting the planning model takes 5 min, and there is a 20 min purge process required between experiments. With the current behavior ARES would conclude an experiment, call the analyzer, call the planner (5 min), and then start the next experiment, which begins with the purge (20 min).
If instead the the user could specify that the analyzer and planner are called as soon as the data collection is completed within the experiment and the purge step (now as the end of the experiment) would be run in parallel. In this scenario, it would reduce experimental cycle time by 5 min.
We should support the ability to specify where in the flow of the experiment logic the analyzer and planner service(s) get called. Right now this is mostly motivated by how we're forced to use Athena/educational ARES by running a bunch of 4 experiment campaigns back to back, but as the planner never actually gets called after the final experiment in each campaign, the planner never gets the chance to see that data. More broadly I think this would be valuable in cases where planners may take a significant amount of time to run and/or there are commands that need to be run in order to reset an experiment state for base line.
For a concrete example of how this would be beneficial over the current analysis and planning at the end paradigm, consider a hypothetical process where fitting the planning model takes 5 min, and there is a 20 min purge process required between experiments. With the current behavior ARES would conclude an experiment, call the analyzer, call the planner (5 min), and then start the next experiment, which begins with the purge (20 min).
If instead the the user could specify that the analyzer and planner are called as soon as the data collection is completed within the experiment and the purge step (now as the end of the experiment) would be run in parallel. In this scenario, it would reduce experimental cycle time by 5 min.