Repository navigation
Serialization/initial - #31
Open
ViewableGravy wants to merge 10 commits into
Open
ViewableGravy wants to merge 10 commits into
ViewableGravy wants to merge 10 commits into
Conversation
…nd tracking - Introduced `mutate` function to handle nested property changes in components, ensuring proper dirty tracking. - Added `Queue` class for managing serialization commands. - Updated `Serializable` class to `SerializableComponent`, enhancing component serialization capabilities. - Refactored serialization logic into dedicated files for better organization and maintainability. - Implemented tests for serialization diff tracking and structural mutations. - Updated various collider classes to utilize the new serialization system. - Ensured all components now extend from `Component` and use the `SerializableComponent` decorator.
- Updated ESLint and related packages to latest versions. - Added ignore pattern for type files in ESLint configuration. - Enhanced TypeScript path resolution in extend-tsconfigs script. - Refactored tests to use relative imports and improved type handling. - Added tsconfig.json for tests to ensure proper TypeScript settings. - Adjusted paths in engine tsconfig to reflect new directory structure. - Updated Vitest configuration files across various libraries for consistency. - Enabled built-in tsconfig paths support in Vite configuration.
There was a problem hiding this comment.
Pull request overview
Introduces an engine-native serialization + diff/dirty-tracking foundation to support networking-oriented incremental updates, alongside broad component migrations to the new Component / @SerializableComponent model and some build/editor wiring.
Changes:
- Add core serialization infrastructure: serializable metadata registry, dirty queue + diff command model, diff replay, and global engine registration.
- Migrate engine, client, and libs components to
Component+@SerializableComponent+@serializable(...), and update nested mutation sites to usemutate(...). - Add engine UI quick action to download a world snapshot, plus related config/tooling updates.
Reviewed changes
Copilot reviewed 85 out of 87 changed files in this pull request and generated 11 comments.
Show a summary per file
| File | Description |
|---|---|
| vite.config.ts | Switch Vite to built-in tsconfig paths resolution and enable decorator support for esbuild. |
| tsconfig.json | Enable experimental decorators at the root config level. |
| tsconfig.base.json | Enable experimental decorators workspace-wide. |
| src/libs/spatial-contexts/vitest.config.ts | Reorder imports (config housekeeping). |
| src/libs/spatial-contexts/src/portal/portal.component.ts | Migrate Portal to Component + @SerializableComponent + @serializable. |
| src/libs/spatial-contexts/src/components/context-entry-region.ts | Migrate ContextEntryRegion to serializable component model. |
| src/libs/physics/vitest.config.ts | Reorder imports (config housekeeping). |
| src/libs/physics/src/resolve/utils.ts | Use mutate(...) for nested tracked transform writes. |
| src/libs/physics/src/resolve/circle-rect.ts | Use mutate(...) for nested tracked transform writes. |
| src/libs/physics/src/resolve/circle-circle.ts | Use mutate(...) for nested tracked transform writes. |
| src/libs/physics/src/entity/collision-participation.ts | Migrate collision participation metadata to serializable Component with bigint fields. |
| src/libs/physics/src/colliders/rectangle.ts | Migrate RectangleCollider to serializable Component. |
| src/libs/physics/src/colliders/point.ts | Migrate PointCollider to serializable Component. |
| src/libs/physics/src/colliders/compound.ts | Migrate CompoundCollider to serializable Component. |
| src/libs/physics/src/colliders/circle.ts | Migrate CircleCollider to serializable Component. |
| src/libs/fps/vitest.config.ts | Reorder imports (config housekeeping). |
| src/engine/vitest.config.ts | Update Vitest projects list, add GLSL plugin, add engine unregister setup file. |
| src/engine/tsconfig.json | Rewrite TS path aliases to be project-relative for Vite 8 tsconfig paths. |
| src/engine/src/ui/layout/quick-actions/saveWorld.tsx | Add “Save World” quick action button to download a snapshot. |
| src/engine/src/ui/layout/quick-actions/index.ts | Export the new SaveWorld quick action. |
| src/engine/src/ui/layout/index.tsx | Render the new SaveWorld quick action in the editor layout. |
| src/engine/src/ui/components/icons/saveWorldIcon.tsx | Add icon for the SaveWorld quick action. |
| src/engine/src/tests/tsconfig.json | Add a dedicated TS config for engine tests. |
| src/engine/src/tests/setup/unregister-engine.ts | Ensure global engine registration is reset after each test. |
| src/engine/src/tests/serialization/diff.spec.ts | Add tests for dirty tracking + diff queue + replay ordering. |
| src/engine/src/tests/scene/scene.spec.ts | Minor test cleanup (use const). |
| src/engine/src/tests/ecs/world-move.spec.ts | Update test component to new serialization model and add world serialization expectation. |
| src/engine/src/serialization/state.ts | Add registries for serializable fields/components and utilities to materialize serialized values. |
| src/engine/src/serialization/serializableComponent.ts | Implement @SerializableComponent, JSON/binary serialization helpers, and diff tracking hooks. |
| src/engine/src/serialization/serializable.ts | Replace prior serialization base class with tracked accessor decorator implementation. |
| src/engine/src/serialization/queue.ts | Add array-backed queue with peek/drain APIs. |
| src/engine/src/serialization/mutate.ts | Add helper to track nested mutations on serializable fields. |
| src/engine/src/serialization/index.ts | Export new serialization/diff APIs from the serialization entrypoint. |
| src/engine/src/serialization/diff.ts | Add diff command types and apply/serialize/deserialize helpers. |
| src/engine/src/index.ts | Export Component from engine barrel. |
| src/engine/src/ecs/world.ts | Add world serialization, structural mutation tracking hooks, and create-with-id support. |
| src/engine/src/ecs/entity.ts | Add ability to register externally-created entity IDs (for replay). |
| src/engine/src/ecs/component.ts | Introduce base Component with attachment + dirty/version metadata. |
| src/engine/src/core/scene/scene-context.ts | Assign stable world IDs into World instances for diff tracking. |
| src/engine/src/core/render-pipeline/passes/render-world/queue/queue-gizmos.ts | Allow gizmo queueing to fall back to local transform when world transform is absent. |
| src/engine/src/core/index.ts | Export new engine serialization manager from core barrel. |
| src/engine/src/core/global-engine.ts | Add global engine registration/singleton enforcement and fromEngine(...). |
| src/engine/src/core/factory/types.ts | Add serialization config and manual global registration option. |
| src/engine/src/core/factory/index.ts | Register engine globally by default; pass serialization config into EngineClass. |
| src/engine/src/core/engine/index.ts | Add engine.serialization manager. |
| src/engine/src/core/engine-types.ts | Tighten imports and point SceneManager import at scene module. |
| src/engine/src/core/engine-serialization.ts | Implement dirty queue + diff command recording and replay helpers. |
| src/engine/src/core/engine-editor/index.ts | Add world snapshot serialization/stringify/download utilities. |
| src/engine/src/core/context.ts | Update engine lookup to use global registered engine when execution context is missing. |
| src/engine/src/components/transform/transform3d.ts | Migrate Transform3D to serializable Component with post-deserialize hook. |
| src/engine/src/components/transform/transform2d.ts | Migrate Transform2D to serializable Component with post-deserialize hook. |
| src/engine/src/components/transform/shader-transform2d.ts | Make shader transform fields serializable. |
| src/engine/src/components/sprite/sprite.ts | Migrate Sprite to serializable Component. |
| src/engine/src/components/sprite/animated/component.ts | Make AnimatedSprite fields serializable and move defaults into constructor. |
| src/engine/src/components/shape.ts | Migrate Shape to serializable Component. |
| src/engine/src/components/shader-quad.ts | Migrate ShaderQuad to serializable Component. |
| src/engine/src/components/hierarchy/parent.ts | Migrate Parent to serializable Component. |
| src/engine/src/components/editor-hover-highlight.ts | Migrate highlight component to serializable Component. |
| src/engine/src/components/debug.ts | Migrate Debug component to serializable Component. |
| src/engine/src/components/camera.ts | Migrate Camera to serializable Component. |
| src/app/client/src/systems/world/portal/index.ts | Use mutate(...) when teleporting player by writing nested transform state. |
| src/app/client/src/systems/world/player-orbit/index.ts | Use mutate(...) for nested transform writes. |
| src/app/client/src/systems/world/conveyor-movement/index.ts | Use mutate(...) for nested transform writes. |
| src/app/client/src/systems/world/build-mode/components/placeable.ts | Migrate Placeable to serializable Component. |
| src/app/client/src/systems/world/build-mode/components/grid-position.ts | Migrate GridPosition to serializable Component. |
| src/app/client/src/systems/world/build-mode/components/grid-footprint.ts | Migrate GridFootprint to serializable Component. |
| src/app/client/src/systems/core/temp-auto-save/index.ts | Use mutate(...) when restoring player position. |
| src/app/client/src/systems/core/movement/index.ts | Use mutate(...) for nested transform writes. |
| src/app/client/src/render/passes/ApplyContextVisualsPass.ts | Use mutate(...) for nested tint/fill/stroke alpha writes. |
| src/app/client/src/entities/land-claim/component.ts | Migrate LandClaim to serializable Component. |
| src/app/client/src/entities/ghost/utils.ts | Use mutate(...) when updating tint colors for ghost previews. |
| src/app/client/src/entities/ghost/GhostPreviewManager.ts | Use mutate(...) for nested transform writes. |
| src/app/client/src/components/transport-belt-leaf.ts | Migrate TransportBeltLeaf to serializable Component. |
| src/app/client/src/components/render-visibility.ts | Migrate RenderVisibility to serializable Component. |
| src/app/client/src/components/player.ts | Migrate PlayerComponent to serializable Component. |
| src/app/client/src/components/orbit-motion.ts | Migrate OrbitMotion to serializable Component. |
| src/app/client/src/components/inside-context.ts | Migrate InsideContext to serializable Component. |
| src/app/client/src/components/grid-bounds.ts | Migrate GridBounds to serializable Component. |
| src/app/client/src/components/conveyor-belt.ts | Migrate ConveyorBeltComponent to serializable Component and initialize fields in ctor. |
| src/app/client/src/components/context-visual-binding.ts | Migrate ContextVisualBinding to serializable Component. |
| scripts/ts-extend-paths/extend-tsconfigs.mjs | Rewrite generated tsconfig paths to be relative to each tsconfig (posix-safe). |
| package.json | Bump toolchain deps (Vite 8, Nx, eslint, etc.) and add vite-tsconfig-paths. |
| eslint.config.mjs | Ignore generated .types output. |
| docs/architecture/14-NETWORK-DIFF-AND-DIRTY-TRACKING.md | Add architecture/requirements doc for diff + dirty tracking refactor. |
| docs/NETWORK_DIRTY_TRACKING_IMPLEMENTATION_STATUS.md | Add implementation checklist/status tracking doc. |
| AGENTS.md | Update guidance text around using memory/skills. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
You can also share your feedback on Copilot code review. Take the survey.
Comment on lines
+57
to
+61
| shouldTrackMutations(component: Component): boolean { | ||
| return this.#suspendDepth === 0 | ||
| && component.isAttached | ||
| && isSerializableComponentInstance(component); | ||
| } |
Comment on lines
+101
to
+105
| case "add-component": { | ||
| const component = createSerializableComponentInstance(command.componentType); | ||
| applySerializedChanges(component, command.data); | ||
| world.add(command.entityId, component as Component); | ||
| return; |
Comment on lines
+267
to
+270
| for (const [fieldKey, value] of Object.entries(changes)) { | ||
| const current = componentRecord[fieldKey]; | ||
| componentRecord[fieldKey] = materializeSerializedValue(current, value); | ||
| } |
Comment on lines
+14
to
+15
| * is apporpriately serialized and sent to the dirty queue to be processed by the DiffManager. | ||
| * |
Comment on lines
+16
to
+18
| * For primitive values, all serializable fields are already proxied with a setter that marks the field as dirty, | ||
| * so this function is not necessary and actively discourgaged due to the additional overhead. | ||
| * |
Comment on lines
+9
to
+12
| class Marker extends Component { | ||
| @serializable("string") | ||
| public value: string; | ||
|
|
Comment on lines
+105
to
+115
| case "bigint": | ||
| view.setBigInt64( | ||
| offset, | ||
| value === undefined || value === null | ||
| ? 0n | ||
| : typeof value === "bigint" | ||
| ? value | ||
| : BigInt(String(value)), | ||
| true, | ||
| ); | ||
| offset += 8; |
Comment on lines
+121
to
+131
| case "string": { | ||
| const stringValue = String(value ?? ""); | ||
| view.setUint32(offset, stringValue.length, true); | ||
| offset += 4; | ||
|
|
||
| for (let index = 0; index < stringValue.length; index += 1) { | ||
| view.setUint8(offset + index, stringValue.charCodeAt(index)); | ||
| } | ||
|
|
||
| offset += stringValue.length; | ||
| break; |
Comment on lines
1
to
+4
| // Register folder exports | ||
| // Register folder exports | ||
| // Register folder exports | ||
| // Register folder exports |
| - Prefer multiple small scripts for broad workflows. | ||
|
|
||
| **Note: You have access to the memory tool. If any message in the user prompt includes "fact" about the code base, such as "we do X in Y case, so can you please do Z", then X should be stored in the relevant memory. Over time we can retain important information this was.** No newline at end of file | ||
| Agressively use the memory tool during development in this project. Memory is the core way that we keep information across sessions and is therefore extremely important. I would like you to put a priority on skill authoring as well. Memory is good for short term, relevant information (and other information) but I would heavily request that you create skills even when it seems trivial. Skills are extremely useful for the developer and for the agent as they give you awesome context around functions, and patterns in the code base. No newline at end of file |
…se visuals system - Added IndexedDB worker backend for efficient scene state storage and retrieval. - Implemented IndexedDbWorkerInputAdapter and IndexedDbWorkerOutputAdapter for state synchronization. - Introduced constants for storage keys and database names. - Enhanced HouseVisualsSystem to apply visual effects based on spatial contexts. - Refactored persistence logic to utilize IndexedDB instead of local storage. - Added utility functions for reconnecting transport belts in the scene. - Updated serialization manager to handle pending field writes more effectively. - Improved sprite rendering by optimizing alpha handling in the rendering pipeline.
…unctionality - Changed the `@serializable` decorator to `@state` across various components and files. - Updated the `mutate` function to reflect the new state registration message. - Introduced a new `StatePolicy` type to manage field serialization policies. - Enhanced the `SerializableComponent` to support state policies for saving, replicating, and dirty tracking. - Added a `RegisteredQuickActions` component to dynamically render registered quick actions in the UI. - Implemented a clear functionality in the IndexedDB worker backend to manage scene state more effectively.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.