What is the role of SRE/DevOps in informing the decision to balance technology investment against customer/revenue generating features? In November we hosted the fifth in our series of DevOps Leaders Roundtable events in order to address this question.
The aim of these events is to provide a platform for leaders in DevOps to contribute their experiences of implementing DevOps, and what they've learned. Meeting in a virtual setting, the event was a great success with interesting insights shared between attendees.
Below Co-Host, Grant Smith, Consultant at Zoopla, gives a rundown on the evening.
How should a company choose to invest developer time? How do you choose between building a better deployment pipeline which accelerates all product delivery or building a better search page to increase revenue?
Companies that do this well usually have strong charismatic leaders who push technology transformation work at an executive level. They do this by telling stories. Stories that resonate with their peers that help them understand why such investment is worth it. Story telling isn’t usually part of engineering degree courses and so we’re often ill equipped to create stories that will resonate with our less-technical peers.
One of the problems we face is that in tech the technical services we provide have a very clear link to revenue and costs.
This creates a thought pattern that technical people are linked directly to value whereas other groups such as product or marketing are not linked as strictly to costs and value. This makes it a little easier for other departments to justify experimentation or to spend without as clear a value proposition.
Several large successful organisations, most notably Google, have published guidance about how much should be invested in operations and tooling but it’s usually ignored, until it’s too late.
Digital transformation projects are a clear investment case but this plays into a boom and bust cycle narrative. Company’s wring every penny of value out of their tech until it starts to strangle them in operational problems and other companies, with newer tech beginning to disrupt them. Then there’s a much-begrudged pause in feature delivery and the clock starts ticking and technology has to deliver as much return on investment as it can in a short period of time before the work must be halted because of a change in the market, or pressure from investors stops the work.
Merger and acquisitions provides another opportunity for tech investment, usually shadow-IT. An organisation buys some technology, then realises it’s in a poorer state or much less compatible with the rest of the business than the due diligence suggested. Some investment is liberated to bring it up to scratch and that provides an opportunity to siphon some of this effort into general technology investment, as long as it provides some benefit to the newly acquired technology no one minds too much.
Cloud migration projects present another opportunity for investment. Anyone who has migrated anything to the cloud knows that just moving some virtual machines is a terrible idea and that some investment is needed to modify the solutions to be more cloud-friendly. This can provide an opportunity to build better pipelines, monitoring, and security solutions under the excuse that new solutions are needed for the cloud. There’s usually a second phase to these cloud solutions when organisations realise they can liberate value faster using SAAS and PAAS solutions only available in the cloud.
Cloud migration is usually poorly understood in organisations first embarking on such transformations. Each time a new problem is found it requires more people to be hired to solve these problems. Each one of these problems (and solutions) then requires an investment case. SAAS often doesn’t make a good case from a pure cost perspective because it’s an upfront very obvious cost compared to the alternatives that hide a lot of their costs in salaries.
This then requires something like a time-and-motion study so the real costs of in-house solutions can be compared to SAAS solutions. In other words the solution is often to approach technical investment along the same lines as any business case. In other words tell the story, be clear about the total cost of ownership including operational support.
In order to specify a pipeline, for example, start from the value of what will be achieved with that pipeline and make the case for the costs in the context of the eventual return-on-investment.
Another great example is to create helper applications. By knowing what engineers are spending time in across the company, it’s possible to identify helper applications that could save significant time across the organisation.
Technology services must always be available and performant. This is not something that happens for free. Technology teams invest a great deal of time and money in achieving this. The challenge we face is in translating engineering considerations into business considerations. One mistake we sometimes make is lumping all this work into Tech Debt when in fact the engineering work we need to undertake to manage our tech stack should be in the same backlog as the product work and they should be weighed equally against each other.
Transformation can be approached through strategy, linking business goals to technology goals. This strategy can make best use of commonly accepted best practices like two pizza teams. The mechanism for execution is safe experimentation. Reducing the risk profile of delivery and getting rapid feedback.
Thanks to the DevOpsGroup for their wonderful illustration of the power of stories.
There is a narrative that technology gets involved in technology for its own sake. Whether this is true or not it is a common preconception and it’s one that technology has to work to overcome.
There’s another school of thought that says that technology transformation only really delivers when it goes hand-in-hand with business transformation. This harks back to the old days of DevOps when we used to describe it as a cultural transformation as much as it was a technical transformation. In this case, the business transformation we’re considering is about rapid specification, delivery, learning, and delivering again.
The business then has to be closer to engineering and has to be involved in the benefits and the pain of legacy systems. This new combined team needs to agree the service provided to customers is the most important thing and change activity has to take second place, the quality of service being provided to customers. In this way there is no purely technology transformation and the business experiences the pain of technical debt, obsolescence, and lack of investment as much as engineers do.
In today's market the products we provide to customers are the result of multiple business operations. All are required to create and deliver a successful service. Product, design, engineering, security, etc. are all required together. A significant weakness in any one area will lead to a failure in the product. There needs to be a balanced investment in all aspects of the product.
Clear communication between these business units is key so that all aspects of the business can represent the product success holistically rather than have each business unit campaign against each other for budget. If each of these areas is cutting edge then they all challenge each other and they can all attract the best talent and the products and the business as a whole gets more valuable than the sum of its parts.
To sum up we’ll paraphrase Ben Franklin’s famous quote: “An ounce of delivery is worth a pound of strategy”. Demonstrating what we can achieve with small change earns us the right to tackle larger transformations. One way to do this is via shadow transformations. Then tell the stories of the success of those transformations. We need to tell better solutions but showing is always better than telling.
A final thought: We must strive to be better leaders, that means we need to learn to speak better, we need to prepare good presentations, we need to have better haircuts, we need to invest in ourselves and that will make it easier for people to listen to us when we need to tell them stories of the fantastic promise of technology investment.