I'm currently working on a major rewrite and it would be great to start off with great documentation!

It might be interesting to explain new concepts in the field of collaborative editing (e.g. how to attribute content).

Feel free to pm me at: hi@kevinjahns.de

I'd take the slight performance overhead any day if I get guaranteed syncs. Network protocols are not as reliable as you think they are. Detecting random drops of messages is hard. At scale, you are going to appreciate the sync guarantees.

prosemirror-collab doesn't give you offline editing either. Because, guess what - if there is no central server you can't edit the same doc from multiple tabs.

I once had a customer that accidentally deleted part of their database containing Yjs docs. Few of his users noticed, because their docs synced through y-indexeddb.

And it's fun. You can Yjs on anything. There is a company that syncs Ydocs through QR codes.

As a generic collab library, it does a very good job. CRDTs really are a fun thing to use. A lot of people feel that way.

If you want to use something else, that's totally fine! Write an article about how great prosemirror-collab is.

I don't blame you for ProseMirror for being how it is. I'm just offering my feedback for your next editor library.

But I hope that you also can appreciate that mapping prosemirror to a CRDT structure is a very complex thing to do. Your schema implementation and node-splitting behaviors are extremely hard to map to existing conflict resolution libraries.

Other editors, like Quill, map better to CRDT structures.

This is, of course, not your problem. But if you - or any editor author for that matter - ever end up creating another editor library, I'd love to work with you on adaptability to existing conflict resolution libraries. There is a lot to gain from being compatible to the wider ecosystem.

After all, we shouldn't expect our users to set up a tailor made backend for each single collaborative component.

Encrypted editing apps like Proton Mail Docs wouldn't be possible with your solution.

That said, it's not without problems - I acknowledge that. But it's not as bad as you make it sound. You didn't list one concrete case when you had issues with it.

I'm very happy that Nick and I finally found funding to make a rewrite happen. It really did take 6 years to make this happen, because it's hard to find funding for open source projects.

As this article is blowing up now, I want to address a few points.

I, too, feel the need for simplicity over overly complex solutions - and I found it in CRDTs. They beautifully allow me to reason about conflicts - so that my users don't have to. Very few people can design a custom conflict resolution algorithm for an application. Yjs is a general-purpose framework that enables you to make EVERYTHING collaborative. That's the goal.

It's fine if you want to explore different solutions. I don't understand the need to put down one framework in favor of another. It doesn't have to be "OT vs CRDT". Hey, if you found something that works for you - great! But let me tell you that neither solution magically makes everything simpler. There is still a lot to learn.

Different solutions to conflict resolution have different tradeoffs. It's unfortunate that the author of the article attributes all complexity to Yjs. It's just that collaborative editing is a very complex problem and requires a lot of attention to detail. In many regards, Yjs has done very well for the larger ecosystem. In other regards there is room for improvement.

The only thing I acknowledge from the article is the criticism about y-prosemirror "replacing the whole document". Unfortunately, the author extrapolated some false assumptions. This is not a performance issue. y-prosemirror runs at 60 fps even on large documents. It's like arguing React is slow because it replaces the whole document with every edit. We leverage ProseMirror's behavior to do identity checks on the nodes before updating the DOM. However, it's true that this breaks positions for some plugins (e.g. a comment plugin).

Instead of Prosemirror positions, we encourage plugins to use Yjs-based positions, which are more accurate in case of conflicts. Marijn talks about this as well [1]. The collab implementation in Prosemirror does not guarantee that positions always converge. That means, comments could end up in different places for different collaborators. This works in most cases, but in some it doesn't - which is one of the reasons why I prefer CRDTs as a framework to think about conflicts.

But as Nick said, we are currently working on a new y-prosemirror binding that works better with existing plugins.

I'm curious about the section "CRDTs are much, much harder to debug" which ironically talks about how hard prosemirror-collab is to debug. You won't find any such bugs in Yjs. The conflict-resolution algorithm is quite simple and has been battle tested. Before every release, Yjs undergoes extensive fuzz testing for hours in simulated scenarios. I'm very happy to show anyone how to debug a CRDT. It requires some background information, but it ultimately is easier.

To address another unfounded claim by the author: I bet OP $1000 that the GC algorithm in Yjs is correct even in offline-editing scenarios. He won't be able to reproduce the issues he is talking about.

[1]: https://marijnhaverbeke.nl/blog/collaborative-editing-cm.htm...

The time to insert characters is the least interesting property of a CRDT. It doesn't matter to the user whether a character is inserted within .1ms or .000000001ms. No human can type that fast.

It would be much more interesting to benchmark the time it takes to load a document containing X operations. Yjs & Yrs are pretty performant and conservative on memory here because they don't have to build an index (it's a tradeoff that we took consciously).

When benchmarking it is important to measure the right things and interpret the results somehow so that you can give recommendations when to use your algorithm / implementation. Some things can't be fast/low enough (e.g. time to load a document, time to apply updates, memory consumption, ..) other things only need to be adequate (e.g. time to insert a character into a document).

Unfortunately, a lot of academic papers set a bad trend of only measuring one dimension. Yeah, it's really easy to succeed in one dimension (e.g. memory or insertion-time) and it is very nice click-bait. But that doesn't make your CRDT a viable option in practice.

I maintain a set of benchmarks that tests multiple dimensions [1]. I'd love to receive a PR from you.

[1]: https://github.com/dmonad/crdt-benchmarks

Comments URL: https://news.ycombinator.com/item?id=35902557

Points: 1

# Comments: 0

[1]: https://opencollective.com/y-collective

In fact, Yjs - a CRDT implementation- is the most downloaded solution for collaborative applications. Even more than ShareDB (the most popular OT-based solution).

Yjs ~90k/week - https://www.npmjs.com/package/yjs

ShareDB ~13k/week - https://www.npmjs.com/package/yjs

The matrix-crdt works really well. To reduce overloading the Matrix server with many small messages (each single keystroke produces an update message), it stores merged updates in the DAG after a short debounce. The optional WebRTC extension allows you to distribute messages immediately "of the chain", so you don't notice the debounce.

After a time the message-log gets pretty huge. So in matrix-crdt, a random client will eventually store a "snapshots" of the current state in the DAG and removes old entries. This way, new clients don't need to download the huge message-log.

It would be nice if there was a possibility to create a server-component that does the merging.

(Btw, all credit to the above approach goes to Yousef)

Now, there might be a better solution to store CRDT data in the Matrix DAG - the developers probably know best and might be able to expose some hidden API that would make everything even more efficient.

I'm just asking that instead of creating yet another CRDT and integrating it into Matrix, open up this space, provide better APIs, and let others integrate their CRDTs.

> Would it be able to abstract any of the underlying CRDT logic?

Modern shared-editing frameworks don't require you to think about internal logic. They just set some requirements on the ordering of update messages. CRDTs in particular don't care in which order you transmit data, which makes them a very interesting choice in practice.

Instead, I want to encourage you to build an API that others can use to efficiently store shared data. Feel free to ping me if you need input.

- Kevin (Author of Yjs)

Relm [2] even models a 3d world using Yjs. I don't necessarily recommend doing this as 3d applications usually produce a lot more actions per minute than text applications. This required some workarounds and deep knowledge of how Yjs' optimizations work. But it's definitely possible.

[1]: https://blog.kevinjahns.de/are-crdts-suitable-for-shared-edi... [2]: https://www.relm.us/

https://docs.yjs.dev/api/undo-manager

The gist of the below [1] read is that it is impossible for a human to create a document that Yjs can't handle (even in the worst case scenario). But yes, it handles real-world scenarios particularily well.

The concept of "hidden classes" is super old. It has first been implemented in a fork of smalltalk and then became foundational concept of runtime engines for scripting languages. It is implemented in V8, python, ruby, spidermonkey, ..

Yjs does not assume a "real-world scenario" and it is not optimized for any specific runtime engine. It runs fast in any browser. The benchmarks confirm this. [2]

Yjs is being used in practice by several companies (eg Nimbus Notes with >3 million users) for quite some time now. I'm not aware of any performance problems.

[1]: https://blog.kevinjahns.de/are-crdts-suitable-for-shared-edi... [2]: https://github.com/dmonad/crdt-benchmarks

Comments URL: https://news.ycombinator.com/item?id=27475175

Points: 8

# Comments: 0

It is much more than state synchronization.

If there is anything missing, then let's work on it. Yjs is extensible and allows for custom features that others can reuse.

The following article shows that:

https://news.ycombinator.com/item?id=24176455