“We don’t have much custom development,” said Pam, the IT manager at a small manufacturing shop in the cosmetics industry, “and we mostly do integrations with our ERP. There is no need for a staging environment.”
“We hire the best developers and our code is fully unit tested,” says Rupmeet, who manages a team of seven developers working on an analytics product in the cloud, “thus we don’t have problems deploying directly in production.”
The above is just a few examples of excuses I have heard from people when asked if they maintain a staging/pre-production environment where they test their updates before pushing changes to production. Can you really get away without staging your changes in an environment that is a close copy of production and simply pushing them to production? The answer is, “It depends.”
It depends how tightly coupled your code/systems are – would a change in one place have a trickle effect in many others?
It depends on the cost of pushing a regression into production – are you risking an outage or loss of data with the change?
It depends on the investment you’re willing to put to pro-actively prevent regressions – both manual tests and development of automation.
It depends on your user base and their threshold for BS – you might be able to pay your regression cost in instalments over time if you don’t need to do extensive testing upfront and handle issues as they come, letting the users experience some pain in the proces.
It depends on your tolerance to technical debt and willingness to pay some of it out over time or all at once.
At the end, the question how much testing is enough before pushing upgrades to production is a question of investment, loss and profit and there is no one size fits all solution. But with the availability of cloud computing and IaaS or PaaS solutions like AWS and Heroku, where you pay what you use and is relatively easy to setup a new environment, there is almost no excuse for not creating one or more staging environments and testing the changes before they go into production.