Skip to content

Optimize the listing query on branch 5.6 - #12

Open
eldmark wants to merge 7 commits into
csgt:5.6from
eldmark:feat/query-optimization-5.6
Open

eldmark wants to merge 7 commits into
csgt:5.6from
eldmark:feat/query-optimization-5.6

Conversation

@eldmark

@eldmark eldmark commented Aug 4, 2026 •

Copy link
Copy Markdown
Contributor

What

Brings this branch up to par with branch 5.9:

  1. Listing optimization — ports optimization: move filter and order logic to the query and fix: order by foreign relation.
  2. Column filters — ports feat: search by column, multifilter.
  3. Tests — adds a PHPUnit suite covering the methods that build the listing query.
  4. README — documents the PHP and Laravel versions this branch supports.

Why

data() fetched every row of the table and then filtered, sorted and sliced it
in memory with Collection methods, so the cost of a page grew with the size of
the table. It also ran one extra query per row for multi fields.

How

Search, ordering and pagination now run in the database:

  • Each filter row sent by the view as filters[] becomes a WHERE: a plain
    column with LIKE, a relation column or a multi field through whereHas,
    and a raw expression through whereRaw.
  • Ordering by a relation column uses a correlated subquery
    (getRelationExistenceQuery + orderByRaw) instead of sortBy in memory.
  • Only the requested page is read, with offset/limit.
  • multi fields are eager loaded once per page, removing the N+1.
  • Aliased expressions (col AS alias) are stripped before being used inside
    WHERE/ORDER BY, so filtering or ordering by an aliased column no longer
    produces invalid SQL.
  • The DataTables global search box is replaced by the filter rows, matching the
    behaviour of branch 5.9.

Compatibility

The public API is unchanged: setField/setCampo, setWhere, setOrderBy
and the JSON response shape are the same, and the runtime composer.json
constraints were not touched. The helpers avoid APIs newer than what this
branch already requires (no short list destructuring, no Builder passed to
orderBy), so the branch keeps working on the same Laravel versions as before.

Testing

composer install && vendor/bin/phpunit — 21 tests, 45 assertions, green.

The suite asserts on the SQL and the bindings each method produces: local and
relation filters, multi filters, raw expressions, malformed or empty filters,
alias stripping, correlated-subquery ordering, raw-expression ordering, unknown
relation fallback, and the field/select helpers. It needs Eloquent but never a
database server, so no query is executed and it runs anywhere composer runs.
PHPUnit, illuminate/database and illuminate/routing are require-dev only,
so nothing changes for consumers of the package.

A manual pass over one listing per project is still recommended before tagging
a release, since the suite does not assert against live rows.

Docs

README.md now states the PHP and Laravel versions supported by this branch,
plus a matrix for every branch of the repo. The Laravel bounds are derived from
the framework APIs each branch calls, because the composer.json constraints
were never updated (several branches still declare php >= 5.3.0).

eldmark added 7 commits August 4, 2026 11:14
The data() endpoint used to fetch every row of the table, then filter, sort
and slice it in memory with Collection methods. The cost grew with the size
of the table instead of the size of the page.

Search, ordering and pagination now run in the database:

- Global search is translated into a WHERE ... LIKE clause; relation columns
  are matched with whereHas.
- Ordering by a relation column uses a correlated subquery.
- Only the requested page is fetched with offset/limit.
- multi fields are eager loaded, removing the N+1 query per row.

Aliased expressions (`col AS alias`) are stripped before being used inside
WHERE/ORDER BY, and raw expressions are ordered with orderByRaw so they are
not quoted as identifiers.

Public API and JSON response shape are unchanged.
Ports the per-column filter of branch 5.9 to this branch.

The DataTables global search box is replaced by a list of column/value filter
rows: the view sends them as `filters[]`, and data() turns each one into a
WHERE clause. Relation columns and multi fields are matched with whereHas, and
raw expressions with whereRaw, so nothing is filtered in memory.

Aliased fields (`col AS alias`) are stripped before the column is used inside
the WHERE, so a filter on an aliased column no longer produces invalid SQL.
Adds a PHPUnit suite with one group of tests per method that shapes the
listing query: getSelect, getLocalShowFields, getForeignShowFields,
getShowMultipleFields, getFieldOrder, getFilterColumns, stripAlias,
qualifyRelatedColumn, applyOrderToQuery, applyColumnFilter,
applyFiltersToQuery and downLevel.

The assertions are on the generated SQL and bindings, so the suite needs
Eloquent but never a database server: no query is executed and it runs
anywhere composer runs. illuminate/database, illuminate/routing and phpunit
are require-dev only, so nothing changes for consumers of the package.

Run with `composer install && vendor/bin/phpunit`.
`securefile` was accepted as a field type and rendered in the listing through
`Storage::disk($field['filedisk'])->temporaryUrl()`, but nothing else was in
place for it:

- `Storage` was never imported, so the render call resolved to
  `Csgt\Crud\Storage` and was a fatal error
- `filedisk` was not one of `setCampo()`'s allowed keys, so the field could
  never carry the disk the renderer reads
- `update()` had no branch for the type, so the upload was silently dropped

The facade is now imported, `filedisk` is accepted and required for the type
alongside `filepath`, and `update()` stores the upload with `putFile()` on that
disk, deleting the previous file when replacing it. No other field type changes
behaviour.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant