Trainingresults do not match the given Best microAccuracy

[Elantonio]

System Information (please complete the following information):

• Model Builder Version 2.0.0
• Visual Studio Version 2022 community

Describe the bug
start multiclass classification
Evaluate Metric: MicroAccuracy
Available Trainers: SDCA,LBFGS,LGBM,FASTTREE,FASTFOREST
Training time in second: 30

Trainer MicroAccuracy Duration
0 FastTreeOva 0.6467 0.8070
1 FastTreeOva 0.6512 0.3110
2 LbfgsLogisticRegressionOva 0.6146 0.2640
3 FastTreeOva 0.6456 0.4920
4 FastTreeOva 0.6545 0.2540
5 FastTreeOva 0.6390 0.2500
6 LightGbmMulti 0.6478 0.3580
7 SdcaMaximumEntropyMulti 0.5526 0.7970
8 LbfgsMaximumEntropyMulti 0.6401 0.1130
9 LbfgsMaximumEntropyMulti 0.6412 0.1990
10 LbfgsMaximumEntropyMulti 0.6401 0.1010
11 LbfgsLogisticRegressionOva 0.6146 0.1790
12 LightGbmMulti 0.6401 0.4190
13 LbfgsLogisticRegressionOva 0.6146 0.1720
14 FastForestOva 0.6478 0.4520
15 SdcaLogisticRegressionOva 0.5493 2.0300
16 FastTreeOva 0.6501 0.9320
17 SdcaMaximumEntropyMulti 0.3621 0.5490
18 LbfgsMaximumEntropyMulti 0.5570 0.1260
19 FastTreeOva 0.6224 1.4410
20 SdcaLogisticRegressionOva 0.3621 1.1810
21 LbfgsLogisticRegressionOva 0.6146 0.3000
22 LbfgsLogisticRegressionOva 0.6146 0.2330
23 LightGbmMulti 0.6157 1.6110
24 FastForestOva 0.6567 0.3580
25 FastForestOva 0.6523 0.6940
Generate code behind files
the trainingresults comes up with 0.6467 FastTreeOva

To Reproduce
Steps to reproduce the behavior:

1. start dataclassification
2. local (CPU)
3. Choose datafile
4. Train
   5 go back to datafile and exclude 1 feature field
5. Train and see the trainingresults come up with 0.6467 which differs from the maxscore in the score/explored list

Expected behavior
rainingresults comes up with 0.6876

Screenshots

Additional context
Add any other context about the problem here.

[luisquintanilla]

@LittleLittleCloud what impact does this have on the CLI?

[luisquintanilla]

@LittleLittleCloud let's confirm which number is correct and fix it so both CLI and MB provide accurate values.

[LittleLittleCloud]

This issue is because we fail to clear the trial result from previous training. It's a Model-Builder-only issue and we probably want to get it in feb release @luisquintanilla

[luisquintanilla]

@LittleLittleCloud thanks for looking into this issue and fixing it. I think for the most part we're ready to go for February. Let's merge the PR though after review and get this into the March release.

[Elantonio]

Great work! Thnks!

[Elantonio]

Hi Xiaoyun, Sorry to bother you this way. I’m Stuck in using autoML. Do you mow of a non-static Predictor App that can be fed with rows of double[] where the number of doubles is unknown at compile time. And where by setting you can inform the class which column(s) are to be predicted and which columns are categorical. And trains on predicting a value or category (by setting) I’ve been trying to fix this for a few months now but I keep running into problems with saving, predicting, training and determining the importance of columns. You’re the only one with in depth knowledge of AUTOML I know the address from. A last resort actually. I understand when you think this is not your job, so please let me know if so. Best regards Ton de Wilde