feat: seatplus:sde-import command + weekly SdeImportJob schedule#677
Conversation
- Add SdeImportCommand: downloads CCP JSONL SDE zip and upserts
categories, groups, types, regions, constellations and systems in
FK-dependency order (500-row chunks, idempotent).
--source option skips the HTTP download for local-file imports.
- Add SdeImportJob: thin ShouldBeUnique queued job that delegates to
Artisan::call('seatplus:sde-import'). timeout=600s.
- Seed Schedules with '0 0 * * 0' (every Sunday midnight) so the
import runs automatically once a week without manual intervention.
- Register SdeImportCommand in EveapiServiceProvider.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
PR Summary
|
Coverage fixes in SdeImportCommand: - Mark fopen(false) and scandir(false) guards with @codeCoverageIgnore (OS-level edge cases, not reasonably testable) - Add test: corrupt ZIP triggers fail() exit code 1 - Add test: empty + non-JSON lines in JSONL are silently skipped - Add test: 501 records triggers mid-stream chunk flush (CHUNK_SIZE=500) - Add test: ZIP with subdirectory exercises removeDirectory() recursion Auto-dispatch on first install: - New migration dispatches SdeImportJob when universe_categories is empty - Idempotent: re-running migrate on existing install is a no-op - Safe in tests: LazilyRefreshDatabase runs migrations after setUp() so Queue::fake() is already active when the migration fires Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Consolidate two migrations into one: the schedule-seeding migration now also dispatches SdeImportJob immediately after seeding the weekly cron. Removes the separate first-install migration and its empty-table guard — migrations only run once, so the unconditional dispatch is correct. Update tests: replace two conditional-dispatch tests with a single test verifying the schedule migration dispatches SdeImportJob. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
FORK_MEM_PER_PROC=107374182400 (100 GB) makes pokio calculate maxByMemory=0, resulting in max(1, min(cpuBased, 0))=1 process. This eliminates the race condition where parallel workers corrupt .temp/v3.php, causing ParseError on the Formats/ci (push) workflow. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
New:
seatplus:sde-importartisan commandDownloads and imports EVE Online static data from the official CCP JSONL SDE.
Usage
What gets imported
Respects FK dependency order:
universe_categoriesuniverse_groups(requires categories)universe_types(requires groups)universe_regionsuniverse_constellations(requires regions)universe_systems(requires constellations)All imports use chunked
upsert()(500 rows per batch). Re-running is safe (idempotent).New:
SdeImportJob(scheduled weekly)ShouldBeUniquequeued job with a 600s timeoutArtisan::call('seatplus:sde-import')schedulestable with0 0 * * 0(every Sunday at midnight)