According to Ilian Iliev, software developers tend to forget to do things they do not have to think about every day, which can cause delays or impact the functionality of the product during a software project. To prevent overlooking something, he suggested starting early with automating deployment, setting up error logging, and using lists and reminders of things that were forgotten previously.
Ilian Iliev spoke about what software developers can do to prevent overlooking things at Dev Challenge Accepted 2023.
People tend to overlook things when developing a new software product or service because they don’t have to think about them on a daily basis, Iliev said. This can cause delays in your software project or it can impact the functionality of your product, or do both, he mentioned. Everything that you forgot during the planning phase means either pushing the deadline or cutting corners and releasing less than promised:
Imagine that two days before release, you suddenly realise that you have totally ignored the fact that there is existing data that is incompatible with the new version of your application and you have missed coming up with a migration plan. That may mean a few days of delay, or weeks of refactoring based on your specific circumstances.
Iliev recalled how one of their servers crashed in the middle of the night as a result of a leap-second handling bug in the OS:
The whole system was operational, except for the part that was processing the customers` orders. The problem was that we found out about it a few days later from a client calling and asking why the system still had not processed their order.
This was a hit both to the company’s reputation and also to our self-esteem as developers, because we had never thought about having 24/7 monitoring of that piece, Iliev said.
One of the things to focus on earlier from a deployment and operations point of view is to automate deployments, Iliev said. You do not need a complex CI/CD pipeline; of course those are awesome, but to start with maybe a simple script that just does all the steps could be enough, he mentioned. Just do not rely on people remembering to execute five different commands in a specific order, even if they are documented. Deployment should be a one-click or command thing, Iliev said.
For operations, availability monitoring and alarming are a must, Iliev argued. This is also true for proper error logging; if something goes wrong with your system you should be able to track down the root cause of it, he added:
Without proper logging, you end up guessing what causes an issue or even worse, not knowing about it until clients start complaining.
Iliev mentioned that saying that something is a bad decision is often interpreted as hostile and also does not bring value to the discussion. He suggested expressing concern that something may not work by pointing out specific reasons, which opens the possibility of identifying possible issues at an early level.
Software developers should not be afraid to ask questions outside of their department, Iliev said. In the end, all of our work is combined together so having a general understanding of all the other aspects is important for the better integration of all pieces in the end product, he added.
Iliev advised teams that when starting a new project, to think about everything that went wrong last time and make sure not to repeat it:
Think about how your product will evolve and how you are going to maintain the system. Keep good communication between the teams and make sure all stakeholders are aware of all the complexities and possible pitfalls.
There will always be things to forget or overlook but the less there are the better the result, he concluded.
InfoQ interviewed Ilian Iliev about how developers can prevent overlooking something.
InfoQ: What’s your advice to teams that are embarking on developing a new system or product?
Ilian Iliev: Using a checklist with topics that you should pay attention to during the process helps a lot. Split it into sections for each phase and use it both as a tool to verify the completion of certain steps, and also as a source of questions that will help you get a better understanding of the project needs. The benefit of those checklists is that they are quick to go over and help you avoid missing steps that will lead to future problems.
InfoQ: Can you give an example of how such a checklist can look?
Iliev: I’ve created a Github project with Project Checklists which contains drafts of the checklists that I use in architecture, implementation, deployment and operation, and evolution and maintenance.
Here are two checklist excerpts to inspire you to create your own:
Implementation:
- Documentation
- Data models documentation - what different attributes represent
- APIs documentation - what the endpoints do, what the expected inputs and outputs are
- Business logic - why certain decisions were made
- Code quality - style guide, linters, review processes
- Versioning
- Stability - unit testing, end-to-end & integration tests, load testing
Deployment & Operation:
- Single command deployment
- Availability monitoring - are you sure your system is running?
- Metrics reporting - how to measure the system performance
- Error logging - operation logs should be easy to access and understand
- Noiseless services - the system should not generate unnecessary noise, e.g. logs, emails, notifications, as it may hide real problems