CrossValidation not available for Ranking, TrainTest not suitable for Ranking

[rogancarr]

Ranking does not support cross-validation. This task should support CV as well.

Edited: See comment below.

[rogancarr]

Offline conversation with @TomFinley

Ranking can be done, pretty straightforward. We just need to make sure that we handle GroupId and an option SamplingKeyColumn together. (e.g. Imagine identifying an author by sentence. We'd want to have groups of sentences for ranking, and make sure that the set of authors in a hold-out set were different from the set in the training set)

Also, TrainTest should similarly be modified to support both GroupId and SamplingKeyColumn. Right now, it only allows one SamplingKeyColumn and supposes that the end user understands that GroupId must be a SamplingKeyColumn.

[shauheen]

Thank you for filing this issue, this is a great point and we should definitely implement CV for ranking, we may however need to push this to post March.

[rogancarr]

@shauheen Note that if we want to drop the TrainTestSplit API as it is currently implemented from Ranking, this will be a breaking change post-March.

[oluatte]

Any updates on this? It's been almost a year.

[rosebyte]

Ranking cross-validation is now available through mlContext.Ranking.CrossValidate.