Automated Builds, Continuous Integrations, DevOps, Oh my!! There are a ton of buzz words surrounding the different methods and ideas for managing hosting environments and deployments. One of the greatly implied concepts that isn’t really discussed very much is the importance of keeping different hosting environments for the same application (ala Dev, Test, UAT, Staging, Production, etc.) separate / isolated. During the development and maintenance processes these different environments can easily be mixed and messed with. This article will explain some details on why they should be kept separated, and how to keep them separated.

How Environments Die

Let’s start with a real world example of a new applications development cycle; starting from app creation to production roll out:

Step 1: The development team creates a new web application and need to deploy it for testing, so they create a UAT environment.

Step 2: There are some new features that need more detailed testing by the developers so they create a new pseudo “DEV” environment for the web application, but use the same database as the UAT environment. “Hey, why create a new database if we don’t need to.”

Step 3: Development is going really well and a release to the customer that they can use needs to be made, so the team creates a new “PROD” (Production) environment with it’s own database.

Step 4: The application hasn’t been rolled out to Production yet, but the PROD environment has been created. The client needs a site to do User Acceptance Testing (UAT) that is different than what the QA Tester on the team is using; which is UAT. So the team creates a UAT2 environment for the client to do User Acceptance Testing that points to the PROD database.

Step 5: All is golden and everything is ready for a Production roll out so the entire web application is rolled out to the PROD environment.

Step 7: The development team hands off the project to the developer maintenance team.

Step 6: The client is testing a new bug fix in UAT2 and changes some data; then confusingly reports that for some reason that change occurred in Production (PROD.) At the same time, a maintenance developer changes code and data in the DEV environment, and the QA Testing confusingly reports the data they relied on for testing just disappeared.

Step 7: After much confusion and many discussions since the original development has since left the project; the developer maintenance team spends a few weeks sorting out the hosting environments so everything is documented.

Result: The result of the above development process is that the environments are a bit comingled. Here’s a breakout of what environments are mixed with what:

• DEV Site with UAT Database
• UAT Site with UAT Database
• UAT2 Site with PROD Database
• PROD Site with PROD Database

Resurrection via Correct Isolation

Implementing correct environment isolation is very important, and the best time to implement it is directly from the start. However, you don’t often have the luxury to starting over with existing systems, so correct isolation is something that needs to be worked towards. No matter where you start, the end desired result is the same. Now, lets take the above “bad” example, and outline how a straightened up, “good” version of it looks like.

It may seems simple, but each environment should be completely isolated from the rest; complete with it’s own database and other resources. Here’s a breakout of what the above “bad” example should look like:

• DEV Site with DEV Database
• UAT Site with UAT Database
• UAT2 Site with UAT2 Database
• PROD Site with PROD Database

Transforming a jumble, mess of “bad” environment isolation isn’t necessarily easy depending on how jumped it is; especially if it’s as bad as the previous example. However, it can definitely be done.

Transforming Bad to Good

Transforming from “bad” isolation to “good”, correct isolation is done by taking a few simple steps in one environment at a time. While implementing correct isolation is done from the bottom up in a brand new system, the transformation of an existing system needs to start at the top down.

The reason transforming an existing system to “good”, correct environment isolation starts at the top is because Production is the highest priority; so that’s where you start. That means disconnecting any environment connected to Production (PROD) resources except the Production environment itself. In the above “bad” example, that means disconnecting UAT2 from PROD. This could be done by taking UAT2 down, or pointing it to the UAT database for the interim.

Point UAT2 to UAT database in the interim? Isn’t that bad too? Yes, but it’s far better than pointing it to PROD. Additionally, by keeping the UAT2 environment up allows for the development, testing and release cycle to continue while the environments are straightened up.

Once UAT2 is disconnected from PROD, the next step is to get UAT2 a proper database and setup it’s own isolation correctly. After that, just move to UAT, then DEV, etc.

Sometimes, the easiest way to transform an existing system to correct isolation is to move the environments. This could be done by taking down the UAT2 environment that is hosted on-premises and moving it into the Cloud; into Microsoft Azure.

How Azure Helps

While Microsoft Azure surely isn’t the only option for implementing correct environment isolation, it definitely can be leveraged to help ensure this isolation. With using features like Azure Web Apps, Azure SQL, Azure Storage and Resource Groups you can logically separate the different environments very easily.

In addition to resource groups, the Shared Access Keys for each Azure resource in addition to possibly using multiple Azure Subscriptions you can tightly control the permissions surrounding the various environments making it difficult for a “rouge” developer to “just” point one environment to another.

A specific feature of Azure Web Apps that helps with this is the ability to override application settings and connection strings from within the Azure Portal. This way you could allow your developers or infrastructure admins to deploy new code to the Azure Web App without letting them even see the appSettings and Connection Strings that particular environment uses.

With Azure SQL, you can grant each developer specific access to each environments database, thus making sure they don’t have an “owner” or “admin” account and connection string they can use to point one environment to another.

There are many other Microsoft Azure features that can be used to help implement a “good”, correct environment isolation Architecture. In the coming weeks, there will be many new articles coming to this site outlining many of these features.

About the Author

Chris Pietschmann

Chris Pietschmann is a Microsoft MVP, HashiCorp Ambassador, and Microsoft Certified Trainer (MCT) with 20+ years of experience designing and building Cloud & Enterprise systems. He has worked with companies of all sizes from startups to large enterprises. He has a passion for technology and sharing what he learns with others to help enable them to learn faster and be more productive.