Add new setting to ignore pagination where it is unsupported in V2.

[Yury-Fridlyand]

Description

Changes how pagination requests with unsupported/incompatible [with pagination] queries are handled.
Adds new setting plugins.query.ignore_unsupported_pagination:

1. true by default
2. when true such requests are performed without pagination
3. when false such requests fall back to legacy engine

Misc changes:

1. Minor logging improvements
2. Minor doc grooming
3. Use pagination in correctness test

Please see team review and discussion in Bit-Quill#318.

The new parameter can be configured by

curl -X PUT localhost:9200/_cluster/settings?pretty -H "Content-Type: application/json" -d '{ "transient": { "plugins.query.ignore_unsupported_pagination": true } }'

Issues Resolved

Fixes #1765, #78

Check List

• New functionality includes testing.
  • All tests pass, including unit test, integration test and doctest
• New functionality has been documented.
  • New functionality has javadoc added
  • New functionality has user manual doc added
• Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 97.30%. Comparing base (6055c7e) to head (35a13e5).
Report is 275 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff            @@
##               main    #1997   +/-   ##
=========================================
  Coverage     97.30%   97.30%           
- Complexity     4622     4623    +1     
=========================================
  Files           407      407           
  Lines         11934    11946   +12     
  Branches        827      828    +1     
=========================================
+ Hits          11612    11624   +12     
  Misses          315      315           
  Partials          7        7           

Flag	Coverage Δ
sql-engine	97.30% <100.00%> (+<0.01%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[Swiddis]

Closing as stale -- reopen if needed