I Cannot Recommend Etherpad on Sandstorm at This Time

Lately, I’ve been using Etherpad on Sandstorm to track notes and clues in the game Blue Prince. While it started okay, I cannot recommend it at this time for this application. What follows is the specific issues that I’ve encountered.

Note that this may not mean it’s not fit for any purpose, or that these things cannot be fixed (I haven’t figured out how to fix them). But here are the issues I’ve encountered, and then I’ll talk about what I’m doing instead.

To clarify: running version 1.8.18-1.

Issues Encountered

From most to least important. I’ll also comment on the workarounds I tried.

Pads perform poorly with a lot of images

I decided initially to use a single Etherpad grain. After adding about 40 screenshots, the grain began simply failing to load (it would hang indefinitely). I stripped about half the images out; at 20 images, the grain takes 11 seconds to load.

What’s going on here? As best I can tell, Etherpad appears to store pasted images inline in the code (yikes!) instead of as referenceable resources on the host server via URL. One of these images as a data:img/png;base64 is 2 million characters in its data-uri (!!!). The grain reports it’s taking up 697MB to stay loaded. I suspect this is causing a lot of performance problems, including

• freezes in the browser
• frequent disconnects from the server
• occasional desync and data loss (closing the browser or the grain before invisible synchronization completed would just drop the last few edits on the floor).

Workaround: Break the one mega-grain into smaller grains: one for each book I find, one for each major mystery, etc.

Pads can’t shrink

As I noticed, I attempted to address the overload issue by splitting my one grain into two along a reasonable line. So I was surprised to discover that while the new grain was smaller, the previous grain retained the same size. To verify what I was seeing, I attempted to clone the grain and delete everything inside it (which should bring me back down to the 60-ish kb one empty grain takes up). So imagine my surprise when I discovered that it’s still 697MB.

Why? Etherpad saves the entire history of the document. There is a “time slider” feature that lets you rewind state. So purging a document that previously had 40 images in leaves you with a document that still has 40 images in it. To be fair, they don’t appear to directly impact performance of the UI (I suspect the memory counter is the memory consumed on the backend). But I can’t find a way to “compress history” or otherwise actually purge any data in the timeline.

Workaround: (Hat tip to user johnyma22 on Reddit for this recommendation) You can export a grain to docx or PDF and then re-import it to throw away the history. But I don’t know that the total memory footprint of the grain actually impacts performance in the client.

No heading keyboard accelerator

This is a small issue, but: as far as I can tell, there is no keyboard acceleration to select heading level. I don’t like having to go to the mouse to edit a document. Not the end of the world, but a bit irritating.

No search / replace feature

As far as I can tell, Etherpad is a document editor with no search / replace: it relies on the browser to do search, and has no replace feature. This hasn’t been a major issue with Blue Prince, but severely curtails the use cases I have for it.

Chat bug is in the way on mobile UI

On the mobile UI, the “Chat” icon takes up the bottom-right of the UI and you can’t scroll the document content past it (unless you have newlines in the bottom of the doc). Minor issue, does complicate mobile use case.

What To Do Instead?

For my use case, I’m going to migrate my Blue Prince notes to an org file in emacs (using org-roam). This addresses several issues at once:

• emacs supports embedding images
• emacs search is stellar, and org supports some information auto-aggregation and indexing (I can’t wait to pull all these dates into a coherent timeline!).
• Coupled with org-roam, I get the benefits of breaking distinct ideas out into their own files with the added benefits of linking the content together into a distributed map.
• Style is inline in org (“live markup” style).

There’s only one real drawback: I’m pretty sure I can’t just inline-paste an image into an org file (yet…). But I’m willing to take that hit to workaround the other issues at this time.

Suggestions Welcome!

If anyone has had a better experience with Etherpad in Sandstorm for this use case (or a similar one) and has tips to improve the experience, I’m interested! And I don’t want this to be interpreted as dumping on Sandstorm or Etherpad forever; I have both a collection of information on my network configs and an RPG game log I track in Etherpad, and they work fine! But, I’m sad to say, none of these issues are issues I would expect coming from Google Docs and they were the bad kind of surprise.