fix: normalize emitted MemoryStream batches to declared table schema (#24069)

Ensures batches emitted by MemoryStream conform to its advertised schema
by adapting batches when runtime nested data types have stricter nullability
than the table's declared schema (e.g. accepted by MemTable via Schema::contains).

Also extends datafusion_common::nested_struct::adapt_batch_to_schema with
narrow schema-conformance support for Arrow UnionArray (sparse and dense)
without changing general SQL CAST behavior.

Fixes #24069
Closes #24394

Author: patrickswedish