Automation and Orchestration: What You Think You're Doing is Less Than Half of What You're Really Doing

Customers are always asking about how cloud and cloud technologies can help them in their data centers and of course we have a variety of replies, but one familiar thread revolves around automation and orchestration (A&O).

The value chain

Over the years we've found that one of the main requirements of the cloud (public, private or hybrid) is that most—if not all—of the commodity IT activities (low-level stuff like creating virtual machines, creating firewall rules, etc.) in your data center need to be automated (i.e. translated into a workflow) and then those singular workflows strung together (i.e. orchestrated) into a value chain of events that delivers IT services and business benefits… these are called digital services.

The value chain ensures business benefits are continuously created with all stakeholders and technologies orchestrating together as a system to enable value creation, business innovation, differentiation and speed to market. 

An example of the orchestration of a series of commodity IT activities is the provisioning of a new composite application. This application consists of a collection of digital assets (virtual machines) that represent web, application and database servers with various operating system requirements and infrastructure components such as load balancers and firewalls. 

The outcome of this is a business benefit, whereas a developer can use those assets to create a process, product or service for generating revenue, decreasing costs or for managing existing infrastructure better—the holy trinity of business benefits. 

When you start to look at what it means to automate and orchestrate a process such as the one above, you will start to see what I mean by "what you think you're doing is less than half of what you're really doing." That may be confusing, so let me reset by explaining the general process for turning a series of commodity IT activities (e.g. provision a VM, add a development environment, etc.), into a workflow and by turn, into an orchestration. We'll use the example from above as the basis for the illustration.

From A to Z

The first and foremost thing you need to do before you create any workflow (and orchestration) is that you have to pick a reasonably encapsulated process to model and transform. What I mean by "reasonably encapsulated" is that there are collections of processes—literally thousands of them—going on in your environment right now for "getting stuff done" that can be sorted and lined up.  

Some are simple (i.e. less than 10-15 process steps for changing a firewall rule), while others could be hundreds of steps with multiple organizations required. A reasonably encapsulated process is somewhere on the simple side of the spectrum but not so far over that there is little to no recognizable business benefit resulting from it.

So, once you've picked the process that you want to model (in the world of A&O, modeling is what you do before you get to do anything useful), you then need to analyze all of the processes steps required to get you from "not done" to "done." 

This is where you will find the complexity you didn't know existed. From our example above I can dive into the process steps that you're well aware of, but it makes no sense to do that here. Instead, I'll highlight some areas of the process that you might not have thought about.

What is a "people"?

Aside from standard operating procedures (SOPs), run books and build plans you have for the various IT assets you employ in your environment, there is probably twice that much information needed that is in places not easily reached by a systematic search of your various repositories. 

Those information sources and locations are called "people," and they likely hold over half of the needed process information for building out the assets you use. Automating the process steps that are in those locations is problematic because it is extremely difficult to get an answer to a question we don't yet know how to ask.

Well, I should say "we don't yet know how to ask efficiently" because we do ask similar questions all the time, but in most cases, we ask without context so the folks can seldom answer, at least not completely. If you ask someone how they do their job you will likely get a blank stare for a while before they start in how they arrive at 8:45 AM and get a cup of coffee before they start looking at email, etc.

They answer this way because, without context, people can rarely give a direct or specific answer.  They simply have too many variables to sort through (i.e. what they think you're asking, what they want you to be asking, why you're asking, who you are) for them to give you a coherent, focused answer. 

However, if you give them a list of possible answers in a scenario in which they can relate—when do you commission this type of composite application, based on this list of system activities and tools?—they can absolutely tell you what they do and don't do from the list.

Doors are unlocked with keys

So, context is key to efficiently gaining the right amount of information that is related to the value chain of activities that you are trying to model, but what happens when (and this actually applies to most cases) there is no context in which to frame the question? Well, it is then called observation, either self or external, where all process steps are documented and compiled by actually running through and documenting them. 

Obviously, this is labor intensive and time inefficient, but is unfortunately the reality because probably less than 50 percent of systems are documented or have recorded procedures for how they are defined, created, managed and operated—instead relying on institutional knowledge and processes passed on from person to person.

The process steps in your people's heads—the ones that you don't know about and the ones that you can't get from a systemic search of your repositories—are the ones that will take most of the time automating. This ultimately is my point: what you think you're doing is less than half of what you're really doing, and where a lot of your efforts will be focused initially.

That's not to say that you shouldn't automate and orchestrate your environment—you absolutely should. High velocity delivery of digital services is no longer an option but a necessity. You just need to be aware that this is the reality, so you can then plan for it and not get discouraged.