Efficiency vs. Value
Did you know that in software development, a budget spent on server performance optimisation is an investment that will definitely pay off? In the endless pursuit of new features, we often forget to simply have a look around and look at the product as a whole, at how much the infrastructure meets our expectations and intentions. I’m not talking about whether or not we have the resources to implement these changes, but rather about the picture in our minds of what a perfect project should look like, about an ideal that we would like to approach. This ideal is a comprehensive concept: UI/UX, test coverage, performance, CI/CD, metric collection and analysis, process automation, technical debt reduction and more — and it becomes particularly important when you begin to notice a decline in your product’s performance (web solution, SaaS, server, etc.), when your server is running slow. But is it the time to upgrade your server’s configuration or to optimise your software?
What causes poor performance?
While analysing a server crash, our support team found confirmation of an old hypothesis that had been troubling us for quite some time: suboptimally written SQL queries run very slowly on large amounts of data, with a heavy CPU load on the server. This resulted in a completely unresponsive web interface until the entire queue of SQL queries was no longer being processed.
How to improve performance: some quick wins
As a quick workaround, we used the so-called delayed execution principle, in which resource-hungry legacy functionality is run in the background and the requestor is later notified as soon as the results are ready.
For a while, both the customer and users were left satisfied, as although this did not strictly solve the company’s performance problems, they were given important features which allowed them to generate more income. In fact, because performance improvements demand dedicated development time and customers will by and large hesitate to put feature development on hold, it was actually a respectable trade-off. Moreover, it allowed us to sufficiently ameliorate any performance issues on the surface level. We kept a close watch on their increasing scale to make sure we could keep them under control.
How to improve a slow server response time
So in the end, how did we solve those underlying problems? Well, over time, a plan was concocted to combat our customer’s legacy issues. The plan required comprehensive refactoring to be performed over multiple levels (with layers ranging from ETL to UI and front-end), and we began to take it step by step. You don’t have to try to eat a whole elephant at once, after all.
Server Performance Optimisation: repay your technical debt
Next, the implementation of these practical steps was as follows: first, we spread refactoring tasks across all of our sprints. This was done in small quantities only, so as to never affect our velocity or upset the customer with a lack of business value. The problems in the code were likewise fixed slowly but surely, and though the victories seemed meager at the time, this was only until we compared the system’s performance before and after our intervention.
Test thoroughly after making changes
One of the ideas that significantly improved our lives was to employ containerisation when deploying our customer’s service. This enabled us to verify that our backup policy would work well with the brand new architecture, and that — should disaster strike the customer’s main server — we could deploy the service from scratch in the shortest possible amount of time. Moving on, we also planned trials for the updated deployment process that would prepare us for the possibility of loss of access to the current server. Having developed a detailed plan from account registration with a new hosting provider to a change in DNS, we successfully overhauled this part of the business continuity action plan.
However, our analysis of the server’s resources showed that when utilising the optimised version of the system — in the same conditions as the live production environment — the latest version of the application consumed noticeably fewer resources.
What are the benefits of refactoring code for server performance optimisation?
The most important benefit for the customer is that the system works stably and quickly, and key features are now processed in real time. By gradually adding refactoring and optimisation tasks to our sprints, we were able to significantly improve the performance of the system while still delivering new functionality.
The second thing that made our customer incredibly happy was the new cost of utilising the server.
So, how much does it cost to have your own server?
It’s simples: always compare
Because part of our deployment process required testing with another provider — to ensure that our software solution could be quickly set up wherever — we were able to compare a variety of different offers and packages. What immediately caught our attention was a specific vendor offering basic 24/7 support that was no different to our customer’s current provider. Another serendipitous discovery was in the hardware configuration of the server.
Assess the impact
We quickly realised that we would no longer be spending big on powerful production and staging CPU-optimised servers, as the updated setup not only displayed better performance but called for a much more modest configuration where the use of vCPU would suffice. In numbers, we went from $500 a month ($350 for the production server + $150 for the staging environment) prior to our optimisation to only $48 (or $30 + $18) a month thereafter. The difference in price, as you can see, is roughly 10 times — which the customer couldn’t help but marvel at.
Is technical debt bad for backend optimisation?
Well, if it in any way impacts your solution’s performance, the short answer is yes.
Once again, we are convinced that in any software project, having an action plan and following it is the first step to achieving quality results. An important attribute of this concept is the promotion of taking small steps. Even with tight deadlines and a beckoning backlog, it never hurts to try to put them into practice, as there is no place for technical problems or accumulating technical debt.
So that’s it for our server performance optimisation analysis!