Week of 2023-03-13
Where I point at layer gaps in developer surfaces and invent the term “deep stack engineer”. Because why not.
Layer gaps
I’ve been writing a bit more code lately, and so you’ll notice that some of my stories are gravitating that way. Here’s one about layer gaps.
Layer gaps are when a developer surface layer fails to close fully over the layer below. Another way of saying this is “leaky abstraction”, but I’ll use my own term “layer gaps” to define what it entails in a more nuanced way.
To recap what I’ve written previously about layers, every layer tends to offer a bit of its own opinion about how to best bring value to users. When the layers are able to fully express this opinion, we have no layer gaps in this layer. For example, JavaScript is a gapless layer. It’s a language and a firm opinion about a code execution environment. It might not have features that we would like to have in a language. It might have limits within its execution environment. It might even have frustrating bugs that irritate the dickens out of us.
But at no point of using JavaScript we will suddenly go: “whoa, I am no longer in JavaScript. I fell into some weird gap that I wasn’t previously aware of, and now I am experiencing the lower layer on top of which JavaScript was built”.
To better grasp what a layer gap looks like, we don’t have to go far. Let’s look at TypeScript. TypeScript is a really interesting beast: it’s a layer of a type system that is laid over JavaScript. First off, the type system itself is delicious. It reminds me a bit of the C# type system, and I thoroughly enjoyed learning and using both. However, there’s a gap into which I fell into more than once.
Because the type system is compile-time only, the layer disappears at runtime. It simply doesn’t exist anymore once the code is executed by the underlying JavaScript engine. However, compared to other type systems that I am familiar with, I expect at least some runtime type support.
At least for me, my mental model of a type system includes at least a little bit of ability to reason about types. Like, at least comparing them at runtime. A bit of type reflection might be nice. But because there’s no such thing as TypeScript when the code actually runs, I experience the layer gap.
As developers of layers, we need to remember that if our layer has gaps, our user must not only understand how our layer works, but also how the lower layer works, and have the gaps clearly marked. For if we don’t, we’ll hear frequent screams of anguish as they discover them. A clearly marked gap might look like documentation that helps our developers understand the tradeoffs they are making by using our layer and make the decision to use it on their own terms. It could look like superb tooling that points out the gap as soon as the user gets close to it – and possibly both.
As users of these layers, we need to be ready for every layer to potentially have gaps. We need to invest time upfront to uncover them, and build our practices to fence around the gaps.
I was sharing with my colleagues that using TypeScript is like walking on stilts. I can get really, really good at walking on stilts. I could even learn how to run on stilts and do all kinds of acrobatic tricks while on stills. But I should never forget that I am wearing them. If I do, I may find myself unpleasantly surprised when the ground suddenly hits my face.
Layer gaps aren’t necessarily a terrible thing. They come with tradeoffs, and sometimes these tradeoffs are worth it. For instance, I embraced TypeScript, because I can delegate some of the mental load of reasoning about the data structures to the TypeScript compiler – and it does a pretty good job of it.
I just need to keep remembering that as I am enjoying the benefits of seeing farther and being taller, I am doing this by wearing the stilts.
🔗 https://glazkov.com/2023/03/12/layer-gaps/
Deep Stack Engineer
Riffing on the idea of layer gaps, we can surmise that pretty much every layer we ever get to write code for has gaps. If that’s the case, then anticipating layer gaps in our future can lead to different ways to build teams.
A key insight from the previous essay is that when we work with a layer with gaps, we need to understand both this layer and the layer underneath it. For if we ever fall into the gap, we could use that knowledge of the lower layer to orient and continue toward our intended destination.
Which means that when we hire people to work in a certain stack, we are much better off hiring at least one person who has experience with the stack’s lower layer. These are the people who will lead the team out of the layer gaps. To give our full stack engineers the ability to overcome these gaps, we need at least one deep stack engineer.
A simple rule of thumb: for every part of the stack, hire at least one person who has experience working at the layer below.
For example, if we’re planning to develop our product on top of a Web framework, we must look for someone who deeply understands this framework to join the team. Ideally, this person is a current or former active participant in the framework project.
Approaching this from a slightly different angle and applying the cost of opinion lens, this person will act as the opinion cost estimator for the team. Because they understand the actual intention of the framework, they can help our team minimize the difference of intentions between what we’re engineering in our layer and the intention of the underlying framework. As my good friend Matt wisely said many moons ago, it would help our team “use the platform” rather than waste energy while trying to work around it. Or worse yet, reinvent it.
Note that the experience at the lower layer does not necessarily translate to the experience at the higher layer. I could be a seasoned Web platform engineer, with thousands of lines of rendering engine C++ code under my belt – yet have very little understanding of how Web applications are built.
What we’re looking for in a deep stack engineer is the actual depth: the capacity to span multiple layers, and go up and down these layers with confident ease.
The larger the count of layers they span, the more rare these folks are. It takes a lot of curiosity and experience to get to the level of expert comfort across multiple layers of developer surfaces. Usually, folks tend to nest within one layer and build their careers there. So next time we come across a candidate whose experience spans across two or more, we are apt to pay attention: this might be someone who significantly improves the odds of success in our engineering adventures.