The famous Joel Test is a super quick and fun way to evaluate an engineering organisation by answering 12 simple "yes or no" questions. At the end, you get a score over 12. There is a problem however.
The test is now 22 years old 😱.
Tools and processes to build software have changed a fair bit over the last two decades. What was considered a potential nice to have, may have become an absolute basic. So, I decided to come up with my own, updated test. Make sure to read the original one first.
Behold the David Test.
- Can new joiners set up and run the system locally in less than one day by just reading the docs?
- Are your builds made without any human intervention?
- Do you automatically deploy to production?
- Is the system, to some degree, observable?
- Do you have a technical debt strategy you actually follow?
- Is your team predictable?
- Does the squad (Engineering + PM + Design) refine the backlog together?
- Do engineers have sizeable chunks of time free of meetings?
- Can engineers choose the best tools for the job?
- Does the team collectively own quality and have a testing strategy?
- Do you have a standardized interview process?
- Do you have a continuous discovery culture?
Let's dive a bit deeper.
1. 👩💻 Can new joiners set up and run the system locally in less than one day by just reading the docs?
Companies that do not have a simple and well documented environment setup process suffer all sorts of issues.
When a new engineer joins, they spend forever before they can even think of contributing with a code change. As the process is not documented, they get blocked constantly. Multiple people need to help, as knowledge is spread across. As a result, the new joiner is frustrated from day 1!
A more insidious consequence is the myriad of subtle issues different members of the team experience over time as everyone has environments configured in significantly different ways.
Ideally, a new engineer should follow the documentation and bring up the application locally within 30 minutes max.
2. 🚚 Are your production builds made without any human intervention?
Many companies still rely on developers making production builds on their local machines to ship changes. Not only does this take time, it also introduces a silly amount of risk.
I once caused a major outage after some local changes made it into a build that got deployed 🙈. I am deeply embarrassed but it's very much a real story and the best example I can give to illustrate this point.
Let machines make the builds in reproducible environments with no human intervention at all.
3. 🚢 Do you automatically deploy to production?
When teams deploy to production after every merge to the main branch something magical happens. The size of every deployment becomes smaller and the feedback loop tightens. The consequences are fantastic:
- Changes are smaller, the risk of a change breaking the system is reduced.
- When a breaking change is introduced, it's easier to identify it and fix it.
- The cost of integrating code becomes tiny.
At the very least, teams should routinely deploy to production multiple times a week. Ideally, there should be multiple deployments to production a day, one for every merge to main.
The rule is 15 minutes or bust.
4. 🌡️ Is the system, to some degree, observable?
Observability has many facets and folks often misunderstand what it means. Essentially, can you answer questions about the state of our system and what real users are experiencing just by looking at its outputs?
At the very least, some of the following need to be present.
- Logging
- Monitoring
- Alerting
- Tracing
Tracing is, perhaps, the most advanced, so you get a pass if you don't have it!
5. 🏦 Do you have a technical debt strategy you actually follow?
Too much technical debt can slow you down to a crawl. At the same time, no technical debt at all is also bad! Engineering orgs should have a strategy outlining their approach to technical debt and actually follow it. This often requires strong relationships of trust and alignment with other roles within Product such as Product Managers.
Not having a strategy means that tech debt is tackled in a chaotic, unstructured way, thus limiting the impact. Perhaps, tech debt is not even tackled at all!
6. 📊 Is your team predictable?
Estimates are damn hard but the reality is that businesses need to know approximately when projects will be done, contracts may depend on them! A team that seldom misses deadlines in a sustainable way is one that builds a tremendous amount of trust. It's one of the key indicators of high performing teams.
You don't need to be able to predict everything with high precision. In order to be predictable, you must break down projects into tiny increments. These tiny increments are a lot easier to predict. If you continuously deploy your incomplete increments behind feature flags, the chances of having something releasable when time runs out are much higher.
7. 👌 Does the squad (Engineering + PM + Design) refine the backlog together?
Too often does one see teams where it's just the lead and the PM who refine the backlog. Then, the lead breaks stories into tasks and assigns them to the team. That is a horrendous practice.
- Not everyone will be aligned on what the team is making and why. People will make worse decisions during implementation, resulting in more back and forth or the wrong thing gets built.
- When a diverse set of people who feel safe to contribute refine together, you will inevitably build a better product.
- Individuals will never learn how to break down a larger story into smaller chunks.
- People lose agency over the work they do and become disengaged.
8. 🤔 Do engineers have sizeable chunks of time free of meetings?
Remote work has become more mainstream and many folks no longer need to deal with noisy open plan offices. However, that does not mean they are free to do deep work for large chunks of time because their calendars may be full with a collection of Zoom calls spread across the day.
Everybody is different, but most people are more productive when they can block out time for deep, uninterrupted work.
9. 🛠️ Can engineers choose the best tools for the job?
Being able to get the best tools money can buy is crucial. Engineers are expensive and you don't want to pay them so that they can spend endless hours fighting their tools or doing work that could easily be automated by an off the shelf product.
We can take this forward and say that there isn't a single toolbox to fit them all. People may feel more comfortable with a particular IDE and they should have the freedom to not have to rewire their brain in order to become proficient with another one.
10. 🐛 Does the team collectively own quality and have a testing strategy?
Traditionally, teams that cared about quality simply had a separate QA department. I think that dilutes the ownership of quality in product delivery teams. In my experience, quality is higher when the whole cross-functional squad cares about it and owns it. You may still have a QA role, but they should be embedded within the team.
It certainly helps when a testing strategy exists, it's documented and actually applied. This document should outline the principles for automated and manual testing ans well as figuring out what to test.
11. 💬 Do you have a standardized interview process?
The first step an organisation can take to avoid having a completely broken interview process is to standardize it. An established process everyone goes through allows you to compare candidates more effectively while minimizing bias. It will become slightly easier to increase the diversity of the team, although it won't magically make your team more inclusive.
A standard process is the absolute bare minimum. Do you want to work at a place where only buddies are hired, everyone thinks the same and it's basically a group of techbros?
12. 🔍 Do you have a continuous discovery culture?
The iterative process does not only apply to actually delivering the software, it's extremely relevant within discovery as well. Organisations that continuously learn about users, test for product market fit and analyse the usage of the most recently released feature will always have the upper hand. When folks other than the PM and designer are involved, the team is set up for success. Look at whether, as an engineer, you can actively participate in the following.
- User interviews.
- Design sprints.
- A/B testing.
✅ What is your organisation's score?
12/12 is ideal. 10/12 is ok, it's always good to aspire to do better! Anything below it and you should start making some changes, NOW.
🙌 Thanks for reading!