Test your system design the moment you come up with it

The Overlap #23

Welcome to The Overlap, a biweekly newsletter somewhere between product development & organizational development. It comes out every other Wednesday morning. Although today is an exception!

Today’s bottom line upfront: Don’t design a system for its own sake. Test your system against reality as soon as you come up with it.

You’ve had to revise your work before. Most days, you accept it. But some days, you feel like your hard work went to waste.

  • As a designer, you work hard on designing every screen and component. When you put actual content in our designs, you find that more than half of your screens have to change.

  • As a writer, you outline your essay draft with H2s. You have an idea of what you want to say! When you start writing, you find that what you have to say is different than what you initially thought.

  • As a longtime developer, you prefer building sites with Drupal. Most of your clients are big Drupal sites. You’ve been a part of Drupal’s open source community for over a decade. You’re skeptical of other Content Management Systems like Sanity. But a longtime friend of yours asks you to help them on one of their projects. You find that they’ve used Sanity. You trust your friend’s judgment, so you start a project yourself and read its docs. You fall in love with Schema Types. It’s much easier than Drupal to model content for the specific needs of your project.

You have a mental model for how a system, design, or model should work. Then, reality says “hey, your mental model isn’t entirely accurate.”

When this happens, what will you do? Continue to hold onto your mental model? Or rethink it?

Beliefs are meant to be updated

Adam Grant, an organizational psychologist, says we should rethink often. 

From his latest book, Think Again:

If you can master the art of rethinking, I believe you’ll be better positioned for success at work and happiness in life. Thinking again can help you generate new solutions to old problems and revisit old solutions to new problems. It’s a path to learning more from the people around you and living with fewer regrets. A hallmark of wisdom is knowing when it’s time to abandon some of your most treasured tools—and some of the most cherished parts of your identity.

I think we all agree that an open mind is a good thing. I’m sure you can think of tons of people in your life who should have a more open mind. But we tend to encourage others to change their beliefs before examining our own.

More from Think Again:

We’re swift to recognize when other people need to think again. We question the judgment of experts whenever we seek out a second opinion on a medical diagnosis. Unfortunately, when it comes to our own knowledge and opinions, we often favor feeling right over being right. In everyday life, we make many diagnoses of our own, ranging from whom we hire to whom we marry. We need to develop the habit of forming our own second opinions.

You've read about the overconfidence effect in see the problem before you solve it:

This is the overconfidence effect: our confidence in our own opinions tends to be higher than the accuracy of our own opinions. We’re more confident in our beliefs than we are correct.

We often favor feeling right over being right. Our views make us who we are. When they’re challenged, we feel challenged. From Think Again:

Some psychologists point out that we’re mental misers: we often prefer the ease of hanging on to old views over the difficulty of grappling with new ones. Yet there are also deeper forces behind our resistance to rethinking. Questioning ourselves makes the world more unpredictable. It requires us to admit that the facts may have changed, that what was once right may now be wrong. Reconsidering something we believe deeply can threaten our identities, making it feel as if we’re losing a part of ourselves.

Watch out for premature optimization

DHH, CTO of Basecamp and Heywarns developers to not prematurely optimize when introducing new technology. If you decide to optimize a screen that isn’t going to be used, you’ve wasted your time!

When talking about how he built Hey:

A lot of developers fall into the trap of premature optimization. [...] Start with absolute simplicity. Build the thing with normal forms, normal everything. Then go, 'Oh, it's annoying that subscribers’ edit segment gets reset when I add it. What's the smallest thing I can do to fix that? Oh! I can add a frame.'"

My coworker, Conor, makes a similar point. We tend to slap software onto every problem. Every problem. This tendency accumulates into a rigid system that’s hard to change. From his essay:

We create new features and build on previously solidified business logic and assumptions. As we offer more to our customers, we too make our system more rigid.

This is not unlike any other building project. A subway system, for example, starts with just a few stations. As the city grows, the subway grows with it and the network of dependent stations increase. If we move station A, we also have to fix the connection between station B, C, and D. Older subway stations may not fit the requirements of the newer, bigger city, but oftentimes there is nothing we can do. They have too many dependencies and the traffic above-ground is too busy to stop for the amount of time required to replace them. We’d have too many unhappy commuters (customers), so we’re stuck with what we’ve got.

As Conor is saying, every system designer prematurely optimizes (not just developers). Think about how much time you spend...

  • Organizing your Google Drive folders, only to still find yourself struggling to find that damn spreadsheet

  • Planning the agenda for a big meeting, only to find that your meeting gets derailed. Goodbye perfect agenda.

  • Perfecting your Notion setup.

  • Drafting a product roadmap to share with stakeholders, only to find it irrelevant the next day.

Our brains trick us into thinking structure first, content later. Plan, then do. But the plan doesn’t save us time down the road. The plan demands us to pay interest. And we all hate wasting time! Yet we continue to prematurely optimize.

Prematurely optimized organization design

We prematurely optimize our organizational structures, processes, and policies. And boy, is org debt hard to get rid of.

Organizational debt are the policies and processes your company accumulates. Some examples:

  • You need five levels of approval to make a trivial decision.

  • You need to submit PTO a month in advance.

  • It takes two full months to onboard a third-party consultant or vendor.

From Steve Blank’s Organizational Debt is like Technical Debt But Worse:

While technical debt is an understood problem, it turns out startups also accrue another kind of debt – one that can kill the company even quicker –organizational debt. Organizational debt is all the people/culture compromises made to “just get it done” in the early stages of a startup.

Just when things should be going great, organizational debt can turn a growing company into a chaotic nightmare.

Org debt are overreactions to past mistakes. These mistakes have a 99% chance of never happening again. 

Org debt also tries to prevent worst-case scenarios that sink the company. But that scenario has a 99.99% chance of not happening. 

When there’s technical debt, we pay interest. That interest is time wasted (and therefore, money wasted). That interest is even your developers’ happiness. Do you know any developers who enjoy working in a legacy codebase?

When there’s org debt, we pay interest. That interest is also time wasted (and therefore, money wasted). You wait three weeks for leadership’s approval on your feature idea. Then boom! Your competitor launched the feature before you.

To prevent future technical debt, we improve the design of old codebases.

To prevent org debt, we improve the design of our old structures, processes, and policies.

Test your model the moment you come up with it

Reader, consider this: The very moment you come up with a model, try it on a real situation. Let reality prove it wrong, so you can update your model.

  • Add content to your design to see if your design works.

  • Try your process on a project team to see if it increases shared understanding or autonomy.

  • Make that API call to that third-party software you don’t like before deciding to build your own system.

Product thinkers say get feedback early and often. But what if folks giving feedback aren’t the actual users of the product? Aren’t we just perpetuating echo chambers? And designing a system for its own sake?

Andy Matuschak wrote about how many systems designers don’t test their systems against serious use cases. From effective system design requires insights drawn from serious contexts of use:

That sounds like standard practice: of course, systems have to be evaluated! But most system designers don’t take “serious” seriously. Academics evaluate their systems in artificial settings with artificial data. Startups fool themselves by working with early adopters mostly interested in playing with new technologies. Deep down, such system designers are generally developing a system for its own sake—not because there’s some creative problem they’re desperately trying to solve.

We must test our mental models against reality. And be willing to rethink our mental models after they’re tested.

Every belief you have is a hypothesis

Every belief you have is a hypothesis. An educated guess. 

  • Your commitment to meditating every day is a guess for how you can be less stressed out. 

  • Your effort to build a “community” around your product is a guess for how you can get more active users. 

  • Your proposed org design is a guess for how your company can run on its own. 

Form a hypothesis. Then, disprove it. That is the mindset of a scientist. That also is the mindset of an effective designer, developer, and product manager.

What’s a current belief of yours? 

What information would persuade you to change your current belief?


What I’m Reading

  1. CEOs Don’t Steer (Ribbonfarm)

  2. What does product management have to do with calculus? (Adam Thomas)

  3. remembering that people aren’t problems to solve (Ava Huang)

What I’m Sending

My first outdoor V5!

Got to spend some time in St. George, Utah last weekend.

Also sent Monkey Boy and Y Chromosome. Monkey Boy is my current favorite at Moe’s Valley.

I’m off to Bishop!

See you in two weeks,
–tim