Tasteful debugging

Wednesday August 21, 2024

Let’s put off doing more meaningful work and write down some quick thoughts I have about debugging, since they happen to be on my mind. All of this is just what I feel right now (and is subject to change). But it’s also something I feel strongly about and that plays a substantial part in what I want to do with my life at the moment.

Debugging is useless

When you talk about debugging, anyone who’s written any kind of computer program probably understands what you mean. It’s likely the most relatable and most inevitable part of software development. Everyone debugs code. Most of software development is, in some sense, debugging.

Debugging isn’t really useless, but it does depend on what you’re talking about when you talk about debugging. I would argue that, while many people hold an idealized view of debugging that is actually quite useful and integral to the software development process, most real-world debugging is not that. The bug finding and bug fixing process in most software, particularly production software, is (in my view) pretty meaningless.

The problem isn’t with the idea of debugging. The problem is that we need to make real-world debugging better.

The ideal debugging process that people often talk about when discussing debugging involves a kind of bug where there is some essential misunderstanding about the program requirements, about some related system, or about the program itself that needs to be resolved. In these cases, there is something productive happening. You learn something, you get a bit better, perhaps, and the problem is, if you’ve properly understood things, fixed for good.

This kind of debugging is usually helpful—it’s part of the mistake-feedback-learning loop that can make learning programming so fun and interactive. And this kind of debugging is inevitable and integral to the software development process. No one, no matter how excellent, will produce error-free code within an unfamiliar system. There is always a learning curve.

But most debugging in the real world, in my experience, is not like this. Many misunderstandings that lead to bugs are not about things that are essential, fundamental, or inevitable. Instead, they are results of historical accident. Debugging in real life either:

Both these things suck. This kind of debugging is stupid and uninteresting. I want my solutions to solve the entire problem, from the bottom up. I want my solutions to last. And I want the source of the bug to be something foundational, something fundamental that I misunderstood. I don’t want to deal with arbitrary hacks based on unsound, best-effort assumptions.

But how do we make this all better?

A brief note on assumptions

Bugs are not a natural, fundamental reality of software.

There are a number of assumptions in software engineering that I think a lot of practitioners, especially more experienced ones, hold. Some of these assumptions are, to me, a little frustrating. They constrict what people think is possible. They limit what people demand of their tools and their software. It’s important, at least to me, to not conflate current reality with basic inevitability.

One assumption I wish we would abolish is that bugs are some kind of inevitable, natural law of software artifacts. I think the industry is far too accomodating of bugs—or mistakes, really—especially when it is simultaneously trying to push software into more important parts of our lives(2). Software is, essentially, pure logic. There are no physical realities mandating the presence of bugs. Software can, in theory, be a perfect, perpetual motion machine. That’s part of its appeal.

What we (still) lack

But we also need to be practical. Humans will, inevitably, make mistakes. What we need are tools, abstractions, and techniques to help us make far fewer mistakes about things that matter—ideally, none at all. In a sense, we need ways to make debugging more pleasant and far more effective.

There are multiple directions from which we can approach this. We should try and create as few bugs as possible (during development—i.e. by-construction), we should try and surface bugs as quickly as possible (especially if they make it into production), and we should make it much easier to fix bugs when they surface.

I don’t think I’m saying anything radical here, but I do believe that we need far more radical changes to the way we program in order to really accomplish these things in a way that is actually meaningful. We need to push forward both at a developer-facing level (with new products, startups, whatever) and at a deeper, more foundational research level. We need to stop pretending that we already know how best to program and that all we need are some superficial (and possibly LLM-based) tools to help us move faster in paradigms that we’re already comfortable with. We don’t need easier ways to make the same mistakes.

In particular, I believe we need the following:

There are some things being done along these lines in industry that I do respect a lot. What Antithesis is doing with deterministic testing (which is an extension of the founders’ previous work on FoundationDB’s testing system) is extremely cool and pushes what’s possible in automated testing and debuggability at scale. I like InstantDB’s re-imagining of the database abstraction for client-side collaboration (and also their usage of Clojure!). But we can, and should, go much further. We should try to avoid settling for local maxima.

I’m personally very interested in mathematical approaches to these things. I really like the promise of formal methods and type systems, both intellectually and practically—broadly, I think proofs and formal methods are the closest thing to a silver bullet that we have. I find a lot of satisfaction in correctness-by-construction. I like the idea of clean-slate abstractions and fundamental rethinking of our tools.

I just generally like proofs, rigor, and elegance (and yes, I do think those things go together). Maybe that makes me a bad engineer. Maybe that makes me naive and impractical. I don’t know. Honestly, I don’t really care. I guess it does makes sense that I’m so drawn to research, particularly in programming languages and verification. It’s probably a good thing that that’s the direction I’m currently moving in.

Fin.

One way to sum up is this: debugging is a crucial part of writing software, but it needs to be much more effective. We need a more tasteful debugging experience. Developing simpler, more fitting abstractions at a systemic level, being able to more easily get formal proofs about our software, and surfacing more errors sooner in the development cycle (preferably at the time of construction) are important pieces of making debugging a more productive and useful endeavor.

Generally, we need far fewer lines of code and far stronger guarantees, particularly about correctness and security, from our software. We’re simply not going to get that from hacking at things with currently conventional tools.

Some more food for thought on software quality and verification can be found in Hoare’s “How Did Software Get So Reliable Without Proof?” and in some of Dijkstra’s writing and lectures, particularly “The Humble Programmer.”

Alright, I’ve already spent far, far too long on this wall of text. Time to get back to work.

  1. OK, maybe this is a little too strident. Sometimes you do need to accept your reality, no matter how terrible, if you want to remain sane. But I do stand by my point that you don't need to celebrate it. That just feels like blatant Stockholm syndrome.
  2. To be totally fair though, human intent and execution are both fundamentally imperfect. While we should be less accepting of software bugs, we should still always strive to be kind to ourselves (and more generally, the human part of the process). The answer to all of this is not to punish programmers more for making mistakes. Instead, it's to recognize that our tools should be helping us make fewer mistakes and to demand more of the software artifacts we produce and use.