A new util method proxyWithHistory with history support

[dai-shi]

ref: https://twitter.com/dai_shi/status/1361147616000544769

It would be nice. I don't have any idea yet.

[thelinuxlich]

If we had a way to plug a global listener on the snapshot:

const snapshot = useProxy(state)
const history = []
snapshot.on('global', (newValue, oldValue) => {
    // store the diff between newValue and oldValue in a history array
   history.push(diff(newValue, oldValue))
})

[thelinuxlich]

Maybe even use automerge for efficient storage https://github.com/automerge/automerge

[dai-shi]

Note that snapshot is already efficient. It shares sub-tree unless it's mutated.

[thelinuxlich]

I mean efficient storage of the history, automerge has deltas

[dai-shi]

Is it about serializing? doesn't seem so.
If snapshots are created from a single state, it's almost like deltas.
If you have several state objects to create a single snapshot, then automerge might help.

[thelinuxlich]

no, I mean...if you want history you need to develop some kind of event sourcing with the snapshot...that doesn't need to be within valtio, but the event store needs efficient storage for all the diffs of the state object

[dai-shi]

I'm not sure, but as long as you understand how valtio's snapshot works, I'm fine.

const state = proxy({ a: 1, b: { c: 2 } })
const snap1 = snapshot(state)
state.a++
const snap2 = snapshot(state)
// snap1 !== snap2
// snap1.b === snap2.b

I can't imagine more efficient storage than this.

[thelinuxlich]

I don't think we are talking about the same thing, to provide a history you probably need an immutable sequence of events on the state so you can undo/redo the actions. Snapshot is an efficient way of providing a point in time for the state, so what is missing is a event store maybe having frozen snapshots in the right sequence.

[dai-shi]

Okay, what I was thinking was something about history in redux context.
Never thought about the sequence of events, which is not how valtio works.
I didn't know the word "history" is that confusing. I don't think it's a good idea to call it history.

[dai-shi]

Now #177 is merged, let's revisit this.

@thelinuxlich It should be close to what you meant by event sourcing.

@redbar0n I would still like to hear about the API idea.

@zcaudate If you are interested in working on it, please go ahead.

[zcaudate]

@dai-shi: Hahaha. I don't really know what needs to be done. Reading the post, it seems like the guys here have a fair idea of what they want.

I'm very in favour of an event sourcing model (with history being a side effect) but I think it'd be better to come up with a couple of use cases first.

ie. can I use it for websockets? Like it I wanted to implement a stock ticker, how might that work?

it's also interesting that the proxies don't need react to work so it can be used on the server end as well.

[dai-shi]

can I use it for websockets?

For this, the issue is serialization. For example, how to serialize a symbol.

What we've done in #177 is very primitive, and I'm not sure how it works for history support. If I were to implement history, I'd just use snapshots.

[zcaudate]

In my case, json is good enough representation for data so serialisation is not really an issue.

Where would there be a need to serialise a symbol?

[dai-shi]

It's your use case. I mean from the library perspective, objects are not guaranteed to be JSON serializable. circular structure is not JSON serializable. we have ref() too. oh, and functions. symbols are bad example.
Basically, we'd need to accept custom serialize/deserialize functions for serialization support, and if your use case is enough with JSON, you can pass JSON.stringify/parse.

[dai-shi]

This is how I implement proxyWithHistory.

const proxyWithHistory = (initialValue) => {
  const p = proxy({})
  const history = []
  let index = -1
  let wip
  const undo = () => {
    if (index > 0) {
      p.value = wip = history[--index]
    }
  }
  const redo = () => {
    if (index < history.length - 1) {
      p.value = wip = history[++index]
    }
  }
  subscribe(p, (ops) => {
    if (ops.some((op) => op[1][0] === 'value' && (op[0] !== 'set' || op[2] !== wip))) {
      history.splice(index + 1)
      history.push(snapshot(p).value)
      ++index
    }
  }
  p.undo = undo
  p.redo = redo
  p.value = initialValue
  return p
}

[zcaudate]

it looks pretty sweet with the new subscribe mechanism.

it might also be interesting to take in the methods pushHistory, popHistory and getHistory as arguments so that people can implement their own abstraction - ie, a ring buffer that only saves the last 100 items, etc.

[dai-shi]

We could also make it to have canUndo and canRedo properties.

[zcaudate]

yep. So maybe then proxyWithHistory can go into it's own valtio/history package.

and then the package will also provide methods like:

arrayHistory
circularBufferHistory

and maybe even:

zustandHistory
jotaiHistory

---

btw are the new subscribe changes released on npm? I'd like to start playing around with it.

[zcaudate]

I also took a look at valtio-yjs. that might also be a good target for a history store.

I did some OT previously on the Atlassian Confluence editor (the syncing on the backend). yjs looks good but again, it'd be good to come up with use cases.

[dai-shi]

it's own valtio/history package

I prefer keeping it in valtio/utils. (unless we need an extra dependency)

arrayHistory

proxyWithHistory([]) would work.

circularBufferHistory

Hm, getting your ideas. I guess such extensions are provided by 3rd-party packages valtio-history.

btw are the new subscribe changes released on npm? I'd like to start playing around with it.

No, it's not yet. Please use codesandbox build for now.
https://ci.codesandbox.io/status/pmndrs/valtio/pr/177
See "Local install instructions"

I also took a look at valtio-yjs. that might also be a good target for a history store.

valtio-yjs uses the new subscribe. Note: valtio-* packages are 3rd-party. Haven't thought about history with it. btw, your use case with websocket might fit.

[zcaudate]

I don't think I need yjs at the moment but it's definitely a good library if the need arises.

I'm not sure what you guys are doing at pnmdrs but looking at your libraries, you can may do a collaborative 3d editor or something with it.

[shashtag]

Please explain the use of the wip object?

[dai-shi]

Without wip, it loops infinitely, right?

[shashtag]

Without wip, it loops infinitely, right?

Cool got it, in the subscribe method it prevents saveHistory to prevent the infinite loop. Am I correct?

[shashtag]

Hey, I created proxyWithHistoryWithPersist function for a project. Do you want me to add it?

[dai-shi]

Please create a third-party package, and let's add a link in https://github.com/pmndrs/valtio/blob/main/docs/resources/libraries.mdx.