inkandswitch/peritext

A CRDT for asynchronous rich-text collaboration, where authors can work independently and then merge their changes.

TypeScript

794

294 commits

updated Sep 16, 2022

See the code

See what people are saying

SourceMessageScoreDate

I know it seems that way, but it's actually not 80% of the way to a CRDT because rich text CRDTs are an open research problem. Yjs instead models the document as an XML tree and then attempts to recreate the underlying rich text transaction. This is much, much harder than it looks, and it's…

on Lies I was told about collaborative editing, Part 2: Why we don't use Yjs

0

Mar 16, 2026

Prompting GPT to do rich text editing. Instead of replacing the entire document or selection, we want it to create diffs or operations for the minimal amount of edits as possible. This helps preserve intent better when merging the doc later on with OT/CRDTs. (Of course, you could also ask GPT to…

on Ask HN: What are you working on (September 2024)?

0

Sep 29, 2024

What's great about CRDTs is that they're really good at both real-time and asynchronous collaboration, unlike the operational transform system used by Google Docs. Asynchronous collaboration is Peritext's major motivation [1]: > We interviewed eight people who regularly collaborate professionally…

on Loro's rich text CRDT

0

Jan 23, 2024

This doesn’t appear to support rich text formatting ranges like bold, italic, etc - unless I’m missing something in the API. AFAIK Peritext is still the state of the art in rich text CRDT algorithms https://www.inkandswitch.com/peritext/ I’d love to see this build the rich text stuff from the…

on Cola: A text CRDT for real-time collaborative editing

0

Sep 3, 2023

The work Ink & Switch (unaffiliated) do has been an inspiration to me with regard to local-first and decentralized software: https://www.inkandswitch.com They have a quasi-manifesto on local-first (https://www.inkandswitch.com/local-first/) and have published the best rich text CRDT around,…

on The cloud is a prison. can the local-first software movement set us free?

0

Aug 3, 2023

I believe most do use a DOM like structure for block level elements. For inline styling however they tend to not have separate nested nodes for each style (bold, italic, underline), and are closer to the old HTML4 tag with multiple properties for each style and don't nest them. It makes for much…

on Text Editor Data Structures

0

Jun 13, 2023

Alright, I just added a link to the section of the original Peritext document related to preserving the author's intent: https://www.inkandswitch.com/peritext/#preserving-the-author... The specific definition of user intent in the context of concurrent rich text editing can't be clearly explained…

on CRDT-richtext: Rust implementation of Peritext and Fugue

0

May 18, 2023

The page here has a good overview of Peritext: https://www.inkandswitch.com/peritext/ It's a separate project from the leading academics in CRDTs. It does what is claimed.

on CRDT-richtext: Rust implementation of Peritext and Fugue

0

May 18, 2023

Yes - The BFT problem only matters when you have Byzantine actors. But I think users deserve and expect the system to be reasonably well behaved and predictable in all situations. Anything publically writable, for example, needs BFT resilience. Or any video game. As for the prosemirror problem, I…

on Ask HN: What is new in algorithms and data structures these days?

0

May 10, 2023

Anything CRDT (conflict free replicated datatypes: https://en.wikipedia.org/wiki/Conflict-free_replicated_data_...) related is fun to read up on and play with. Papers and references (page maintained by central academic in the world of CRDTs): https://crdt.tech Group doing research into how they can…

on Ask HN: What is new in algorithms and data structures these days?

0

May 10, 2023

README

Peritext

This is a prototype implementation of Peritext, a CRDT for rich text with inline formatting. The algorithm is described in the following publications:

This repo includes:

  • A Typescript implementation of the core Peritext CRDT algorithm
  • A prototype integration with the Prosemirror editor library
  • An interactive demo UI where you can try out the editor
  • A test suite

Try the editor demo

To see a basic interactive demo where you can type rich text into two editors and periodically sync them:

npm install

npm run start

Code tour

Algorithm code: The main algorithm implementation is in src/peritext.ts. Because the goal of this work is to eventually implement a rich text type in Automerge, we implemented Peritext as an extension to a codebase called Micromerge, which is a simplified implementation of Automerge that has mostly the same behavior but is less performance-optimized.

The essay describes the algorithm in three main parts:

  • Generating operations: happens in changeMark
  • Applying operations: happens in applyAddRemoveMark
  • Producing a document: there are two places this logic is defined. getTextWithFormatting is a "batch" approach which iterates over the internal document metadata and produces a Prosemirror document. There is also a codepath that produces incremental patches representing changes (which is actually what powers the editor demo); these patches get emitted directly while applying the op, within applyAddRemoveMark.

Prosemirror integration: src/bridge.ts contains the code for the integration between the CRDT and the Prosemirror library. There are two main pieces to the integration:

  • Prosemirror to CRDT: when a change happens in the editor, Prosemirror emits a Transaction. We turn that transaction into a list of InputOperation commands for the CRDT, inside the applyProsemirrorTransactionToMicromergeDoc function.
  • CRDT to Prosemirror: when a change happens in the Micromerge CRDT, the CRDT emits a Patch object representing what changed. We turn this into a Prosemirror transaction with the extendProsemirrorTransactionWithMicromergePatch function.

Each direction of this transformation is straightforward, because the external interface of InputOperations and Patches provided by the CRDT closesly matches the Prosemirror Transaction format.

Tests

npm run test will run the manual tests defined in test/micromerge.ts. These tests correspond to many of the specific examples explained in the essay.

You can also run a generative fuzz tester using npm run fuzz. This will randomly generate edit traces and check for convergence.

Build demo artifact for essay

This repo also contains a UI that plays back a preset trace of edit actions, which is included in the Ink & Switch essay about Peritext.

To see that UI, you can run npm run start-essay-demo.

To build an artifact for including in the essay, run npx parcel build src/essay-demo.ts, and then copy the resulting ./dist/essay-demo.js file to content/peritext/static/peritext-demo.js in the essays repo. Also copy over any CSS changes from static/essay-demo.css to content/peritext/static/peritext-styles.css if needed.

inkandswitch/peritext

A CRDT for asynchronous rich-text collaboration, where authors can work independently and then merge their changes.

TypeScript

794

294 commits

updated Sep 16, 2022

See the code

See what people are saying

SourceMessageScoreDate

I know it seems that way, but it's actually not 80% of the way to a CRDT because rich text CRDTs are an open research problem. Yjs instead models the document as an XML tree and then attempts to recreate the underlying rich text transaction. This is much, much harder than it looks, and it's…

on Lies I was told about collaborative editing, Part 2: Why we don't use Yjs

0

Mar 16, 2026

Prompting GPT to do rich text editing. Instead of replacing the entire document or selection, we want it to create diffs or operations for the minimal amount of edits as possible. This helps preserve intent better when merging the doc later on with OT/CRDTs. (Of course, you could also ask GPT to…

on Ask HN: What are you working on (September 2024)?

0

Sep 29, 2024

What's great about CRDTs is that they're really good at both real-time and asynchronous collaboration, unlike the operational transform system used by Google Docs. Asynchronous collaboration is Peritext's major motivation [1]: > We interviewed eight people who regularly collaborate professionally…

on Loro's rich text CRDT

0

Jan 23, 2024

This doesn’t appear to support rich text formatting ranges like bold, italic, etc - unless I’m missing something in the API. AFAIK Peritext is still the state of the art in rich text CRDT algorithms https://www.inkandswitch.com/peritext/ I’d love to see this build the rich text stuff from the…

on Cola: A text CRDT for real-time collaborative editing

0

Sep 3, 2023

The work Ink & Switch (unaffiliated) do has been an inspiration to me with regard to local-first and decentralized software: https://www.inkandswitch.com They have a quasi-manifesto on local-first (https://www.inkandswitch.com/local-first/) and have published the best rich text CRDT around,…

on The cloud is a prison. can the local-first software movement set us free?

0

Aug 3, 2023

I believe most do use a DOM like structure for block level elements. For inline styling however they tend to not have separate nested nodes for each style (bold, italic, underline), and are closer to the old HTML4 tag with multiple properties for each style and don't nest them. It makes for much…

on Text Editor Data Structures

0

Jun 13, 2023

Alright, I just added a link to the section of the original Peritext document related to preserving the author's intent: https://www.inkandswitch.com/peritext/#preserving-the-author... The specific definition of user intent in the context of concurrent rich text editing can't be clearly explained…

on CRDT-richtext: Rust implementation of Peritext and Fugue

0

May 18, 2023

The page here has a good overview of Peritext: https://www.inkandswitch.com/peritext/ It's a separate project from the leading academics in CRDTs. It does what is claimed.

on CRDT-richtext: Rust implementation of Peritext and Fugue

0

May 18, 2023

Yes - The BFT problem only matters when you have Byzantine actors. But I think users deserve and expect the system to be reasonably well behaved and predictable in all situations. Anything publically writable, for example, needs BFT resilience. Or any video game. As for the prosemirror problem, I…

on Ask HN: What is new in algorithms and data structures these days?

0

May 10, 2023

Anything CRDT (conflict free replicated datatypes: https://en.wikipedia.org/wiki/Conflict-free_replicated_data_...) related is fun to read up on and play with. Papers and references (page maintained by central academic in the world of CRDTs): https://crdt.tech Group doing research into how they can…

on Ask HN: What is new in algorithms and data structures these days?

0

May 10, 2023

README

Peritext

This is a prototype implementation of Peritext, a CRDT for rich text with inline formatting. The algorithm is described in the following publications:

This repo includes:

  • A Typescript implementation of the core Peritext CRDT algorithm
  • A prototype integration with the Prosemirror editor library
  • An interactive demo UI where you can try out the editor
  • A test suite

Try the editor demo

To see a basic interactive demo where you can type rich text into two editors and periodically sync them:

npm install

npm run start

Code tour

Algorithm code: The main algorithm implementation is in src/peritext.ts. Because the goal of this work is to eventually implement a rich text type in Automerge, we implemented Peritext as an extension to a codebase called Micromerge, which is a simplified implementation of Automerge that has mostly the same behavior but is less performance-optimized.

The essay describes the algorithm in three main parts:

  • Generating operations: happens in changeMark
  • Applying operations: happens in applyAddRemoveMark
  • Producing a document: there are two places this logic is defined. getTextWithFormatting is a "batch" approach which iterates over the internal document metadata and produces a Prosemirror document. There is also a codepath that produces incremental patches representing changes (which is actually what powers the editor demo); these patches get emitted directly while applying the op, within applyAddRemoveMark.

Prosemirror integration: src/bridge.ts contains the code for the integration between the CRDT and the Prosemirror library. There are two main pieces to the integration:

  • Prosemirror to CRDT: when a change happens in the editor, Prosemirror emits a Transaction. We turn that transaction into a list of InputOperation commands for the CRDT, inside the applyProsemirrorTransactionToMicromergeDoc function.
  • CRDT to Prosemirror: when a change happens in the Micromerge CRDT, the CRDT emits a Patch object representing what changed. We turn this into a Prosemirror transaction with the extendProsemirrorTransactionWithMicromergePatch function.

Each direction of this transformation is straightforward, because the external interface of InputOperations and Patches provided by the CRDT closesly matches the Prosemirror Transaction format.

Tests

npm run test will run the manual tests defined in test/micromerge.ts. These tests correspond to many of the specific examples explained in the essay.

You can also run a generative fuzz tester using npm run fuzz. This will randomly generate edit traces and check for convergence.

Build demo artifact for essay

This repo also contains a UI that plays back a preset trace of edit actions, which is included in the Ink & Switch essay about Peritext.

To see that UI, you can run npm run start-essay-demo.

To build an artifact for including in the essay, run npx parcel build src/essay-demo.ts, and then copy the resulting ./dist/essay-demo.js file to content/peritext/static/peritext-demo.js in the essays repo. Also copy over any CSS changes from static/essay-demo.css to content/peritext/static/peritext-styles.css if needed.

Significant stargazers

(top 24 of 31)

Matteo Collina

11,412 followers · starred Nov 2021

Berker Peksag

957 followers · starred Nov 2023

Jon Sterling

829 followers · starred Sep 2024

Adriano Raiano

533 followers · starred Nov 2021