October 25, 2021

Engineering round up

Last week was a relatively tough week at the Placemark helm. I've found that sometimes Mercury is in retrograde, or my mood doesn't afford me the ability to laugh at my own mistakes or take a walk when something doesn't work on the first or third try. It's doing a disservice to only write about the problems I feel like I really nailed and not the frustrations in between. So here's a little of that.

DNS & Webflow

I recently ported Placemark's marketing website, and this blog, to Webflow. Before, the marketing site was part of the same Blitz application as Placemark itself, and the Blog was a Jekyll blog with lots in common with my macwright.com's codebase.

I was mostly taking Tyler Tringas's advice on this front, but also following the metrics. Despite everything that Blitz and Next are doing for performance of static marketing sites, they're still overkill, and the old homepage had mediocre performance and SEO. And, speaking of SEO, everything in SEO is uncertain but it seems likely that hosting blog posts like this one on the same domain as the marketing site makes Google more likely to treat them as the same entity.

Webflow is pretty good. The user experience isn't as polished as Figma, but I'm impressed by how well they've implemented some of the CMS-oriented concepts. And the HTML/CSS that Webflow generates is very reasonable. But the porting of everything was tricky: previously the blog was hosted at Netlify and the website at Render. After the swap, the application is hosted by Render at app.placemark.io, and the website is hosted by Webflow at www.placemark.io.

So, I reimplemented the site in Webflow, which only took a few hours, wrote a lot of new documentation, and ported over all the blog posts. Then it was just a matter of adding a new subdomain in Cloudflare, pointing Render to that subdomain, and pointing Webflow to www.

It's never that simple, with DNS and domains. I forgot to update an environment variable in Render that let the main application know which domain it was being hosted on. I tried to proxy app.placemark.io through Cloudflare immediately, which broke Render because Render couldn't confirm that they controlled that domain.

SQL, darned SQL

Placemark is generally using Prisma as an ORM.

I have mixed feelings about this. Not quite about Prisma - Prisma is a high-quality project with an active and intelligent team of developers behind it. But about using an ORM.

I think that development moved quickly away from ORMs after Ruby on Rails started yielding room for Node.js, and then Go and other more modular, performance-oriented tools. This was in part because Ruby on Rails's ActiveRecord can be abused, easily. You can create truly baffling SQL queries by using ActiveRecord without regard to the database underneath.

But it's also because of performance. The absolute best performance that an ORM can offer is no greater than a hand-written query. And very often, ORMs produce lower-quality queries than handwriting.

I hadn't written much SQL at all until I worked at Observable, but there I learned the value of really, really understanding Postgres. Database queries are one of the few things in the web stack that you can reliably guess will be a performance bottleneck and be correct in that assumption. Applications spend a lot of time in the database layer, regardless of what particular database they're using.

So I spent a few years learning how to do more and more with a single query. There are some queries in the Observable codebase that might still remain which will really open your third eye, so to speak. We used CTEs and recursive subqueries and all kinds of magic to produce succinct, thoroughly performant little gems of database magic. And bugs, of course.

Anyway, there is a part of Placemark in which querying and updating the database is the main thing I'm doing, and in which performance of those queries is absolutely central - the collaboration loop. This is the Replicache push endpoint. It matters, a tremendous amount, and relies on some fairly sophisticated assumptions about the database layer. It relies heavily on database transactions and a reliable, monotonic, mutex-like counter.

This is where last week was tough: I've been using Prisma's support for transactions, which is unfortunately a bit fringe. Prisma really would prefer that you implement transactions in your application, but that doesn't quite work for most of the problems I'm tackling. And I ran into a mysterious issue, most likely in Prisma, related to upserting in a transaction, that led to the CPU spinning and the application giving up on future requests. It was frustrating. The leaves are falling in Brooklyn, so it's a good time to go on walks, which I did.

Next up

On the bright side, I've shipped a few important things and am planning some more.

I implemented client-side undo/redo, using unidirectional, mutable undo objects. This was a stark departure from the undo systems I've worked on in the past that had immutability as a core assumption. I'll probably write about this soon, though it's very similar (and directly inspired by) Figma's undo system

The multi-feature-selection state now lets you edit properties of multiple features at once. This pattern has been around for ages in Adobe Illustrator, and more recently in Mapbox Studio and Figma, and I always wanted to implement it. Feels good!

Quite a few different things are vying for the next slot on the implementation conveyor belt, but the most likely is presence and to give more life to the existing multi-player editing. Placemark is in a similar place to Observable at some point in 2020 - multi-player is now deeply embedded in its programming model, but not exposed in many UI elements. And the other thing is a server-side store of versions - another thing that will partly emerge naturally from the principles of the programming model I've set out, but there's a lot of tradeoffs I need to balance for the implementation.