Repository navigation
OneVsAllClassifier fits two trainers for binary classification task #3933
Description
System information
- OS version/distro:
- .NET Version (eg., dotnet --info):
Issue
- What did you do?
- What happened?
- What did you expect?
I expected OneVsAll to identify there are only two classes and train a single learner.
Source code / logs
Internally the OneVersusAllTrainer will always instantiate as many binary classifiers as the number of classes. This is inefficient for binary classification, as just a single trainer is needed. I understand that I can just use an out of the box binary classification for this, but it will simplify the usage.
Activity
wschin commentedon Jul 1, 2019
So you're saying that if the number of classes is 2, OneVersusAllTrainer should only produce one binary classifier?
rtanase commentedon Jul 1, 2019
yeah, I don't think it should be necessary to train 2 binary classifiers, one for each of the classes, when a single one would do the trick. Right?
wschin commentedon Jul 1, 2019
From the perspective you pointed out, yes. On the other hand, the model trained by OVA is a list of model, if we follow your suggestion ML.NET will end up with
OutputModel = new Model().OutputModel = new Model[number of classes].It looks a bit wired because the index
iinOutputModel[i]means differently in the two cases above. In other words, it's not a desired strongly typed way to ML.NET's public APIs.Inspired by your comment, if there are only two classes, ML.NET can internally train a binary classifier and copy it to
OutputModel[0]andOutputModel[1].rtanase commentedon Jul 1, 2019
As long as there shouldn't be 2 trainings involved, then it should be fine. The public API should remain same. I wanna avoid having to go through the dataset to figure out how many classes are there to choose between a straight BinaryClassifier or a OneVsRest (which will again go through the dataset to determine number of classes).
If going with training a single model and storing it within OutputModel[0] and OutputModel[1], can those 2 models be exactly the same or does one need to be inverse of the first one?
wschin commentedon Jul 1, 2019
The later one should have
negativeweights of the first one, I assume. For example, ifOutputModel[0]'s weight vector is[1, 2, 3],OutputModel[1]'s weight vector should be[-1, -2, -3]. This PR will expose the trained models to users so that we can check if that assumption is true. If that assumption is true, I'd expect a small change to support your idea.wschin commentedon Jul 2, 2019
Now I feel doing one binary classification is better than copying the weights.