File: /var/www/html/tmpr/../tmpr/../tmpr/../tmpr/..//wowZ/collaboration.md
Goals
- Support real-time co-editing by 3+ users with low latency.
- Persist only incremental changes, not full documents, to reduce storage/bandwidth.
- Handle conflicts automatically with strong consistency guarantees.
- Allow offline edits and seamless re-sync on reconnect.
Architecture
- Client: Canvas/scene editor using a structured document (objects with id , type , props ).
- Collaboration engine: CRDT-based (preferred) or OT-based synchronization.
- Transport: WebSockets for live updates; HTTP for snapshots/history retrieval.
- Storage: Event-sourced patch log + periodic snapshots; assets (images/fonts) in object storage.
- Backend:
- Option A (simpler in PHP stack): Node.js sidecar for WebSockets (e.g., y-websocket with Yjs) + PHP API for auth, snapshots, and history.
- Option B (pure PHP): Ratchet/Swoole WebSocket server + PHP OT/CRDT implementation (more effort).
Collaboration Model
- CRDT (recommended): Use Yjs or Automerge to represent the document as nested maps/lists.
- Pros: Automatic conflict resolution, offline-first, efficient binary updates.
- Cons: Requires a WS server that understands CRDT updates (Node sidecar is typical).
- OT (alternative): ShareDB-style operations against a JSON schema.
- Pros: Mature operational pipeline.
- Cons: More complex server logic; careful transform functions needed.
Data Model
- Design document: JSON-serializable scene graph.
- design = { id, version, objects: Map<id, {type, props, children?}>, meta }
- Assets: Stored separately; objects.props.src holds references.
- Awareness (presence): Separate ephemeral channel for cursors/selection; not persisted.
Change Saving Strategy
- Event sourcing: Append-only log of operations/CRDT updates; periodic snapshotting.
- Patch granularity: Single-object property changes or structural changes.
- Example JSON-Patch ops:
- { op: "add", path: "/objects/abc", value: {...} }
- { op: "replace", path: "/objects/abc/props/x", value: 120 }
- { op: "remove", path: "/objects/abc" }
- Compression/compaction:
- Periodically fold patches into a snapshot (e.g., every N ops or M minutes).
- Coalesce redundant ops (e.g., multiple replace on same path).
- Binary updates (CRDT):
- Store Yjs updates as compact binary; snapshot with encodeStateAsUpdate .
- Save only the delta from the last known state vector.
Backend
- WebSockets:
- Rooms per designId ; broadcast updates to connected participants.
- Rate-limit and batch send to minimize chattiness (e.g., 30–60 Hz or idle debounce).
- Persistence:
- On receiving update: validate auth/ACL → append to design_updates → conditional snapshot.
- Snapshots table: latest materialized state for fast open/restore.
- History:
- Expose endpoints: GET /designs/{id} , GET /designs/{id}/history , POST /designs/{id}/snapshot .
- Access control:
- Roles: owner, editor, viewer; per-team membership (integrate with existing teams ).
- Token-based auth for WS and HTTP; short-lived tokens for WS.
Frontend
- Document state:
- CRDT (Yjs): Y.Doc with Y.Map for objects , Y.Array for z-order; Y.Map for props.
- OT: Client builds operations; server applies and rebroadcasts.
- Awareness:
- y-protocols/awareness (or custom): show cursors, selection, user color/label.
- Change batching:
- Group small changes into a single update; throttle property changes while dragging.
- Undo/redo:
- Client-side stacks driven by operations; safe with CRDT by tagging local intents.
Conflict Handling
- CRDT ensures convergence automatically (e.g., last-writer-wins for scalar props, set semantics for lists).
- For high-level conflicts (e.g., two users deleting/transforming same object), annotate UI (conflict highlights) and offer quick undo.
Performance & Scaling
- Shard WS rooms by designId ; horizontal scale with sticky sessions or a pub/sub (Redis).
- Snapshot cadence tuned to document size/change rate; retain full history for audit or prune beyond retention window.
- Asset uploads via signed URLs; lazy load assets referenced by objects.
- Client CPU: keep object granularity; avoid whole-document updates.
Security
- AuthN/AuthZ on WS connects and update ingestion.
- Validate patches (schema, bounds, allowed ops) server-side.
- Rate-limit abusive clients; audit logs for changes (user, timestamp, op count).
- No secrets in client; use HTTPS/WSS.
Rollout Plan
- Phase 1: Pick collaboration engine (Yjs + y-websocket recommended).
- Phase 2: Define schema for design documents and operations; migrate existing designs to JSON scene graph if needed.
- Phase 3: Implement WS server and PHP endpoints for snapshots/history/auth.
- Phase 4: Integrate client CRDT/awareness; wire presence and UI indicators.
- Phase 5: Persistence layer: event log, snapshotting, compaction; retention policy.
- Phase 6: Access control with teams; invite/share flow.
- Phase 7: Testing: multi-client sessions, offline/reconnect, conflict stress, large documents.
- Phase 8: Observability: metrics on update rates, room sizes, snapshot times; error monitoring.
- Phase 9: Gradual rollout to selected teams; collect feedback; optimize.
Recommended Stack Choices
- Collaboration: Yjs + y-websocket (Node sidecar).
- PHP integration: Use existing PHP app for auth, REST API, and storage; the Node WS server verifies tokens issued by PHP.
- Storage: MySQL for metadata and logs, object storage (S3-compatible/Local) for snapshots and assets.
- Optional: Redis for pub/sub between WS servers; CDN for assets.
Why this works
- CRDTs provide “smart saving” out of the box by emitting minimal deltas, not full documents.
- Event sourcing plus snapshots gives efficient persistence and fast load times.
- Awareness channel improves UX without polluting history with ephemeral state.
- Sidecar WS server avoids retrofitting WebSockets into PHP while keeping your current stack.
If you want, I can draft the concrete data schemas for designs , design_updates , and design_snapshots , and outline the token handshake between PHP and the WebSocket server next.