Last week we spent a couple of days speeding up multi-cursor editing. Specifically, we wanted to take advantage of the batched nature of this operation and edit the buffer's CRDT in a single pass, as opposed to performing a splice for each range. Please, take a look at #82 for all the details.
There is still some work to do in that area to deliver a smooth experience when editing with thousands of cursors, but we are planning to get back to it once we have fleshed out more features.
After demoing Xray to our colleagues, we got a lot of interest in how Xray's CRDT-based approach to buffers might apply to the problem of versioning generally, so we took some time to explore it last week. We were intrigued by the idea of a CRDT-based analog to Git, a kind of operation-oriented version control system that allowed for real-time synchronization among several replicas of the same working tree and persistence of all operations. After spinning our wheels quite a bit, we've concluded that we really need to get clear on the specific problems we might like to solve. They are as follows:
-
Replay: We'd like to allow developers to record a collaboration session and cross-reference their keystrokes to audio, so that it could be replayed later. Assuming people were willing to opt into this, it could provide deep insights into the thought processes behind a given piece of code to future developers. This use case is really all about persisting the operations, and has nothing to do with replicating the entire file tree.
-
Permalinks: Today we have anchors, which automatically track a logical position in a buffer even if in the presence of concurrent and subsequent edits, but these anchors are only valid for the lifetime of the buffer in memory. We'd like to be able to create an anchor that can always be mapped to a logical position at arbitrary points in the future, even thousands of commits later. Again, this has nothing to do with full replication. It's really about indexing the operations we persist and tracking the movement of files over time so that we can always efficiently retrieve a logical position for an anchor.
-
Streaming persistence and code broadcast: Today, code lives on your local machine until you save it, commit it, and push it to the cloud. We want to persist your edit history as it is typed and optionally stream it into the cloud. If your computer spontaneously combusts, your up-to-the-minute edit history is still saved on the server. If you elect for your edits to be public, colleagues or community members could watch your edit stream in real time. This would require full replication if you wanted to allow another party to make edits to the working tree. If the server is just storing your operations, there's really no need to deal with concurrency. It might be cool if someone could come along and edit the server's replica of the work tree and have their edits automatically appear in your replica, but is that actually a good user experience? Real-time collaboration requires tight coordination, so it might be jarring to receive edits from someone you didn't actively invite to your workspace.
-
File-system mirroring for third-party editors: We'd like to allow other editors to use Xray in headless mode as a collaboration engine. In this use case, we'd need to relay edit operations through Xray via specific APIs, but it might be helpful if Xray could mirror the state of a remote project to the local file system. That way, an exiting editor could use its ordinary mechanisms for dealing with local files to interact with the remote workspace, and wouldn't need to perform file system interactions over RPC, which would simplify integration.
I wanted to think through the design implications of these various features early to determine whether any of them had an impact on Xray's core architecture, and after a lot of thinking, my conclusion is that it should be okay to defer these features for now. I had envisioned a single unified design that elegantly addressed all of these features in a single replicated structure, but now we think that that cost of building such a structure probably outweighs its benefits.
For now, we've decided to defer these concerns to until the point that replay, permalinks, or streaming persistence are actually the next most important feature we want to add. Our instinct is that when that time comes, we'll be able to address these features in an additive fashion, and that it doesn't make sense to invest in adding support for them today.
In retrospect, last week was a bit of a distraction. I've done more up-front design thinking for Xray than I ever have for any other project, and it's worked out pretty well overall. But after last week, I think we're approaching diminishing returns for up-front architectural design. We've validated that the current design can be performant and collaborative, and it's seeming like we've struck a nice balance between simplicity and power. Now it's time to return to a more incremental strategy and continually focus on the next-most-important feature until we have a useful editor.
This week, we'll turn our focus to implementing save as well a simple key bindings system, which I wrote about in a previous update. We also plan to clarify our short term roadmap, and we'll post an update about that next week.