The lack of software testing in production – is it a problem?
In software development, a team, mainly the testers, often does not have access to production. In most cases, access is unavailable due to security rules or the business owner’s rules that have been established. Leading to the problems with post-release integration of development modules and new blocks, sometimes even fatal errors occur in production. In this article, we will discuss why and when it happens and how software testing in production can solve it.
Types of companies and releases in the eyes of the Tester
For a tester, tasks to be solved come in every day, pushed through development. Regardless of the workflow used on the project or in the company as a whole, everything is quite similar. But it’s not easy to test software efficiently if you do not know who it is for and how it will be used. This knowledge fundamentally influences the testing approaches and software testing tools implemented.
What companies can we distinguish?
Outsourcing companies – companies that work for third-party clients. The tester can participate in several projects, and these projects can be completely different – both in terms of the technology stack and the way of use.
Most often product companies handling single projects and attracting the tester, work in close proximity with similar technologies and responsibilities. Consequently, the product is either one, or several, but similar (for example, modules for different CMSs).
А mixed company can function both by means of using similar technologies and principles and by practising other testing methods, depending on the differences between their own products and the outsourcing projects.
Software testing in production for a product company is usually a non-issue, a requirement, even. When it comes to outsourcing companies, however, giving the testing team full access to production can present a problem. This makes software testing in production difficult, if not impossible for some development projects. Let’s see which types of projects are more likely to have testing trouble and why software testing in production is important.
How does your project type affect software testing in production?
You can work in different companies and projects, use various testing tools, approaches, and methods for software testing. As mentioned before, in order to optimize and improve testing as much as possible, whether it is web or mobile application testing, you need to know the app’s possible weak spots, how difficult the post-release troubleshooting will be, and, finally, whom you release the program for as a target audience.
The distribution of priorities is key. As for Facebook, load testing is prioritized over online payments; in e-commerce, payments are prioritized over security testing; payment & billing systems prioritize security testing, while load testing takes a backseat, and so on. Everything hitches on your app’s purpose and the target audience expectations.
Projects can be divided into several different types. What’s important to understand is that the end-user of the product isn’t always the client, and that said client isn’t always loyal enough to be trusted with the end-user production access, but we’ll return to that in just a bit.
So:
- Projects developed by a single company for internal use, with millions of individuals from all around the globe being potential customers (i.e. Youtube, eBay, Facebook, Google). Such products are developed within one interconnected system, which makes software testing in production common practice.
- Projects developed by a single company, customized for every individual client (i.e. website builders, corporate HR systems). Here, every consequent update has to be accompanied by testing in production, since even small customizations to the code may bring on an array of issues. With an increase in testing frequency, the testing itself gets more complex, as more emphasis is being put on carrying out integration tests, system testing, etc. Outsourcing companies have to ask the client’s permission to give the testers production access on any given development project. These requests are denied most of the time.
- Optional modules developed for global systems. The new module is installed into an already active system (i. e. WordPress, Magento and Shopify) by the end-user. While the process of software testing in production is similar to that in point one, here it plays a particularly crucial role. One module’s performance is completely dependent on how smoothly it can run alongside other modules in the system. Therefore, with no access to production, some compatibility issues and bugs can end up being utterly unfixable. Testing in production tends to be rather easy, since the developer is often one of the global system’s users as well.
- Products developed for third-party companies, where the end-user is the client company’s employees (i. e. banking systems), or for companies whose business is not directly related to software, but necessitates the use of it for various side tasks (i. e. car manufacturing). The end-user can be anyone, really – every industry requires software solutions to properly function in the modern world. It is extremely rare for testers to receive any access to production in projects like this.
From the examples given, it becomes clear that projects may also vary on the production’s type and, consequently, on the difficulty of software testing in production.
Each release is a crucial moment; even if your workflow is set up for daily releases – it is still a responsible step. For some reason, it is assumed that testing at this stage has already been completed. Not really! Now the most accountable and meticulous testers enter the game – these users, these guys do not forgive mistakes and start ringing all possible bells – even if there are minor deviations from the expected result. Can we get ahead of them? You can try it by choosing the best time for release: when the product is used the least and the load on it is the lowest possible.
Project inspection from zero to roof
When the construction of the house is completed, it must be handed over to a special сommission, and independent experts will check the quality of the completed building. When testing software, like a construction expert, the tester rises from floor to floor, changing the types and levels of testing. After all, the software release is the same as renting a house, but with one difference. If the construction experts are not able to get on the roof or check the attic, the house won’t be rented or leased, not without a little “there are no guarantees of proper operation of the attic and last floor” note. What this means is possible rain leaks, holes and lack of proper cover, internal communications sticking out, the roof being frankly unsuitable for various weather conditions.
The house roof and production can be compared to one another however a tricky process can occur to make the system more difficult. It is called the Staging – a version of the project, which is the most tested and stable. It is believed that a well-tested project on staging should not cause problems. If you remember the option of project delivery, where the end-user receives a custom project or modules, everything falls into place. Having finally received the keys to the attic, the tester suddenly finds out that it is filled with heavy furniture and houses a sauna, which can cause the collapse of the entire floor in case it rains or the humidity goes slightly overboard.
Keys to software testing in production
Why isn’t it allowed to reach the production? Why does the client deny any access for software developers to production?
- At the very least, they are worried that someone would do something wrong. If you go back to the example of houses, imagine that you stopped by your new apartment in a beautiful modern house, and then one of the builders asks you for the keys to the apartment where you stayed and have already moved your things but offers to wait for him on the street.
- Secret information, especially standard for e-Commerce, where there is much confidential information about customers, suppliers, and partners. Normally, the owner of such a business will do everything to protect himself/herself from leaks of information about his/her project.
How to conduct software testing in production?
On the current project, I encountered a policy of openness and trust; I justified the need to access the project’s production environment and test the site in production immediately after release (after each release!).
- Explain to the client why you need access to the production project.
- Tell who exactly and when will go into the production (ask for written consent from the client).
- Provide a checklist for software testing in production. Most likely, it will include a compressed Smoke test, because the primary goal of this test is to understand that the code stuck and the integration has been successful. You can, of course, play with load testing and security testing, but it all depends on the agreements.
- Use logging tools or screen recording. Software testing in production will not take much time, but the availability of such data will protect you and make the project testing process clear and transparent.
Conclusion
Testing the project after its release can not be called “must-have” in the tester’s activity. In general, in the relationship between the client and the contractor, on the contrary, it is an additional initiative. Software testing in production provides a more stable operation of the project due to timely catching bugs (before users encounter errors). Finally, it improves the relationship between clients and contractors by bringing their cooperation to a higher level of interaction and trust.
