Skip to content

[WIP] cross-page scope cut-copy-paste #184

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 1 commit into from

Conversation

li-xin-yi
Copy link
Collaborator

@li-xin-yi li-xin-yi commented Dec 27, 2022

Note: Don't merge, this is a draft for tests and ideas

Usage:

like codeNode, click on the button, Ctrl-v to paste, click on pane to confirm, Esc-v to cancel. Others could see nothing until the new nodes are placed.

qq20221226201103_YAKQ7k4D.mp4

The key challenge here is how to handle condition/data races almost everywhere (DB <-> store.pods <-> local nodes <-> yjs nodesMap, all updates propagate in bi-directions and with different unstable delay) when manipulating a bunch of pods at once, which make the entire thing horrible, for examples:

  1. a pod cannot be inserted to the DB if its parent hasn't been inserted completely. (mitigation: initialize the pod in DB with parent ROOT at first time and set the pod dirty to get its parent field updated later)
  2. delete half-transparent/cut nodes not in time, getNode(id) and nodesMap(id) may give different results when creating formal nodes due to the sync speed between front-end and yjs server. (mitigation: use different kits of node id for original pods, temporary pods and formal pods, though they are all the same data, use pasteNode to first update store.pods, and then generateNodes to push nodes to yjs server according to store.pods, DFS in pre-order to make sure the parentNode created as early as possible)
  3. When determine the destination location of the pasting scope, even we call delete operation, the temporary/cut scope may still in nodesMap or nodes, we may regard it as a target scope (Mitigation: ad-hoc rules to avoid dropping into any hidden/transparent scopes)
  4. etc... Maybe there are many other bugs I haven't found, the most terrible thing is that, the bugs caused by condition/data races are hard to reproduce, I came across them occasionally and have to take a lot of time to find any clue. I feel terrible about it. 😫

Tests and feedback are welcome. I will improve the code and performance these days.

@li-xin-yi li-xin-yi marked this pull request as draft December 27, 2022 04:39
@lihebi
Copy link
Collaborator

lihebi commented Jan 3, 2023

The key challenge here is how to handle condition/data races almost everywhere (DB <-> store.pods <-> local nodes <-> yjs nodesMap, all updates propagate in bi-directions

The refactor #189 could hopefully make this cleaner. The key is to make it uni-directional. We operate only on nodesMap, and use the Yjs observer to update store.pods and database when transaction.local === true && !node.data?.clientId.

@lihebi
Copy link
Collaborator

lihebi commented Jan 3, 2023

  1. a pod cannot be inserted to the DB if its parent hasn't been inserted completely. (mitigation: initialize the pod in DB with parent ROOT at first time and set the pod dirty to get its parent field updated later)

This mitigation won't solve it in total, because the deferred "parent update" may appear before the "node insertion" as well (unless we await the node insertion, which seems not ideal).

Instead, I feel we should have a new graphQL API addOrderedPods:

addPod(repoId: String, parent: String, input: PodInput): Boolean
addOrderedPods(repoId: String, parent: String, orderedPods: [PodInput]): Boolean

Then we can do these:

  1. first, generate all the nodes for the scope to be copied => nodes
  2. topological-sort the nodes according to parent-children relation => sortedNodes
  3. push to Yjs peers: sortedNodes.forEach(node=>nodesMap.set(node.id, node)
    3.1. push to zustand store
  4. push to database: fetch(gql’mutation addOrderedPods()’, {variables: orderedNodes})

@lihebi
Copy link
Collaborator

lihebi commented Jan 3, 2023

2. delete half-transparent/cut nodes not in time, getNode(id) and nodesMap(id) may give different results when creating formal nodes due to the sync speed between front-end and yjs server.

I feel there's less to worry about the sync and data race, because Yjs is CRDT, i.e., from Yjs doc:

"Document updates are commutative, associative, and idempotent. This means that you can apply them in any order and multiple times. All clients will sync up when they received all document updates."

So it should be uni-directional technically.

(mitigation: use different kits of node id for original pods, temporary pods and formal pods, though they are all the same data, use pasteNode to first update store.pods, and then generateNodes to push nodes to yjs server according to store.pods, DFS in pre-order to make sure the parentNode created as early as possible)

I feel your use of node.data.clientId is a great way to mark temporary nodes local to a specific user. We just need that to select which data to sync (again something like transaction.local === true && !node.data?.clientId). I think that's sufficient for scope-copying as well.

@lihebi
Copy link
Collaborator

lihebi commented Jan 3, 2023

  1. When determine the destination location of the pasting scope, even we call delete operation, the temporary/cut scope may still in nodesMap or nodes, we may regard it as a target scope (Mitigation: ad-hoc rules to avoid dropping into any hidden/transparent scopes)

The ad-hoc rules sound good. We can add the skipping logic to checkNodesEndLocation.

@li-xin-yi li-xin-yi closed this Feb 20, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants