Skip to content

Updated MultiFileSource.Load to fix inconsistent behavior with multiple files - #5003

Merged
mstfbl merged 3 commits into
dotnet:masterfrom
mstfbl:Issue-4982
Apr 8, 2020
Merged

mstfbl merged 3 commits into
dotnet:masterfrom
mstfbl:Issue-4982

Conversation

@mstfbl

@mstfbl mstfbl commented Apr 7, 2020

Copy link
Copy Markdown
Contributor

Fixes #4982

MultiFileSource's Load can now accommodate loading from multiple paths. Also added more test cases for the additional features of MultiFileSource.

Example usages:
textLoader.Load("Data/*");
textLoader.Load("DataFolder/.../*");
textLoader.Load("DataFolder/SubFolder1/*", "DataFolder/SubFolder2/*");
textLoader.Load("DataFolder/SubFolder1/1.csv", "DataFolder/SubFolder2/2.csv");

@mstfbl
mstfbl marked this pull request as ready for review April 7, 2020 01:40
@mstfbl
mstfbl requested a review from a team as a code owner April 7, 2020 01:40
Comment thread test/Microsoft.ML.Core.Tests/UnitTests/FileSource.cs
@frank-dong-ms-zz

frank-dong-ms-zz commented Apr 8, 2020 •

Copy link
Copy Markdown
Contributor

is there any place to describe the correct usage of wildcard of directory like below?
textLoader.Load("DataFolder/.../*"); #Resolved

@mstfbl

mstfbl commented Apr 8, 2020 •

Copy link
Copy Markdown
Contributor Author

@frank-dong-ms A description of the correct usage of wildcards for MultiFileSource doesn't exist in the original codebase. I have added a description on usage of wildcards with MultiFileSource to its XML documentation as a part of this PR. #Resolved

@frank-dong-ms-zz frank-dong-ms-zz left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:shipit:

@mstfbl
mstfbl merged commit 1ea5336 into dotnet:master Apr 8, 2020
@ghost ghost locked as resolved and limited conversation to collaborators Mar 18, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

TextLoader Load From Multiple Files Inconsistent Behavior

2 participants