refactor(proto): migrate CsvSource serde

[buraksenn]

Which issue does this PR close?

• Part of Proto: migrate the simple file scans (CsvSource, JsonSource, ArrowSource, AvroSource) #23516.

Rationale for this change

Part of epic #23494. Moves CsvSource protobuf serialization from the central dispatch into the source implementation.

What changes are included in this PR?

Add protobuf serialization and deserialization to CsvSource, including the CSV format options represented by the existing wire format.

Repoint the live decode arm to CsvSource::try_from_proto and remove the old central encode arm. Keep try_into_csv_scan_physical_plan as a deprecated compatibility wrapper that delegates to the new implementation.

The protobuf wire format remains unchanged.

Are these changes tested?

Yes. Added roundtrip_csv_scan_preserves_format_options, covering header, delimiter, quote, escape, comment, multiline values, and truncated rows.

Are there any user-facing changes?

The existing PhysicalPlanNodeExt method remains available as a deprecated compatibility wrapper. There is no immediate API removal or wire-format change.

[github-actions]

Thank you for opening this pull request!

Reviewer note: cargo-semver-checks reported the current version number is not SemVer-compatible with the changes in this pull request (compared against the base branch).

Details

     Cloning apache/main
    Building datafusion-datasource-csv v54.1.0 (current)
       Built [  48.762s] (current)
     Parsing datafusion-datasource-csv v54.1.0 (current)
      Parsed [   0.012s] (current)
    Building datafusion-datasource-csv v54.1.0 (baseline)
       Built [  43.946s] (baseline)
     Parsing datafusion-datasource-csv v54.1.0 (baseline)
      Parsed [   0.012s] (baseline)
    Checking datafusion-datasource-csv v54.1.0 -> v54.1.0 (no change; assume patch)
     Checked [   0.106s] 223 checks: 223 pass, 30 skip
     Summary no semver update required
    Finished [  94.221s] datafusion-datasource-csv
    Building datafusion-proto v54.1.0 (current)
       Built [  61.018s] (current)
     Parsing datafusion-proto v54.1.0 (current)
      Parsed [   0.019s] (current)
    Building datafusion-proto v54.1.0 (baseline)
       Built [  60.863s] (baseline)
     Parsing datafusion-proto v54.1.0 (baseline)
      Parsed [   0.019s] (baseline)
    Checking datafusion-proto v54.1.0 -> v54.1.0 (no change; assume patch)
     Checked [   0.235s] 223 checks: 222 pass, 1 fail, 0 warn, 30 skip

--- failure trait_method_marked_deprecated: trait method #[deprecated] added ---

Description:
A trait method is now #[deprecated]. Downstream crates will get a compiler warning when using this method.
        ref: https://doc.rust-lang.org/reference/attributes/diagnostics.html#the-deprecated-attribute
       impl: https://github.com/obi1kenobi/cargo-semver-checks/tree/v0.49.0/src/lints/trait_method_marked_deprecated.ron

Failed in:
  method try_into_csv_scan_physical_plan in trait datafusion_proto::physical_plan::PhysicalPlanNodeExt in /home/runner/work/datafusion/datafusion/datafusion/proto/src/physical_plan/mod.rs:1042

     Summary semver requires new minor version: 0 major and 1 minor checks failed
    Finished [ 123.598s] datafusion-proto

[codecov-commenter]

Codecov Report

❌ Patch coverage is 68.60465% with 27 lines in your changes missing coverage. Please review.
✅ Project coverage is 81.05%. Comparing base (e0a05a6) to head (24d07fe).

Files with missing lines	Patch %	Lines
datafusion/datasource-csv/src/source.rs	74.35%	6 Missing and 14 partials ⚠️
datafusion/proto/src/physical_plan/mod.rs	12.50%	7 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main   #24177      +/-   ##
==========================================
- Coverage   81.06%   81.05%   -0.01%     
==========================================
  Files        1107     1106       -1     
  Lines      382191   382209      +18     
  Branches   382191   382209      +18     
==========================================
- Hits       309807   309803       -4     
- Misses      54082    54094      +12     
- Partials    18302    18312      +10     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[adriangb]

Opened #24180 to track the scan options that are silently dropped on the wire (CSV terminator and compression type here, JSON newline_delimited in #24178) — pre-existing, not a blocker for this PR.