In running the ~2000 criterion benchmarks currently on stackage, we've run into the problem that some of them are clearly nonlinear.
We are trying to develop a data cleaning step which will throw these out before comparing two different runs of the full benchmark set. Once we find good thresholds, I would like to put some guidelines together to share these with criterion users.
This goes along with other documentation updates, e.g. #92 and #95.
CC @RyanGlScott @vollmerm
In running the ~2000 criterion benchmarks currently on stackage, we've run into the problem that some of them are clearly nonlinear.
We are trying to develop a data cleaning step which will throw these out before comparing two different runs of the full benchmark set. Once we find good thresholds, I would like to put some guidelines together to share these with criterion users.
This goes along with other documentation updates, e.g. #92 and #95.
CC @RyanGlScott @vollmerm