Jez Humble on Continuous Delivery and DevOps

In the last year, during the period i have been in France, attending Université du SI, one of the best conferences i've participate in whole life (sponsored by OCTO Technology), i had the privilege of being invited to participate in a very interesting course with Jez Humble,  about Continuous Delivery for DevOps.
After the course, i could make an interview for InfoQ Brazil, but for obvious reasons we translated it to Brazilian Portuguese. And now, here goes the original version. Enjoy!

You wrote a best seller, called Continuous Delivery, What is it all about?

It's about all the stuff that is normally considered somewhat peripheral to software development - configuration, build and deployment management, automated testing, continuous integration, infrastructure and database management. However the discovery that many of us made while working on large agile projects was that this stuff is essential if you want to be able to reliably, repeatably release high quality software.

Actually many of the ideas in the book aren't new. It's really a call to arms to focus on the engineering practices that were always at the heart of agile methodologies, but that somehow got sidelined in many places over the last ten years. Of course several tools like Puppet, Chef, Git and DbDeploy have emerged that didn't exist ten years ago that make much of this stuff much more tractable.

It turns out that the engineering practices - particularly automation of build, test, deployment, and database and infrastructure management - and collaboration between development, testing and operations - are really essential to be able to get high quality software delivered fast. And that, in turn, is central to the ability of organizations to innovate and get new features out of the door quickly and reliably.

What practices do you think are more important?

For developers, you need to start with continuous integration. That doesn't mean installing a CI server and pointing it at your feature branches - it means making sure everyone merges regularly into mainline (and yes, that applies to Git too). In order for CI to be useful, you need automated tests, in particular at the unit and acceptance level. Having a comprehensive, maintainable set of unit tests is almost impossible without doing TDD. Creating maintainable acceptance tests requires close collaboration between developers and testers. Finally, you need to validate every build to ensure it's production ready, which means having the ability to spin up production-like testing environments on demand and run realistic loads against your system. This requires automated provisioning of environments and infrastructure, and automated deployment of your system, including any database changes. All of this needs to be implemented within the context of the deployment pipeline.

These kinds of changes need to be made incrementally, by finding out where the constraints are in your delivery process and removing them one by one. That in turn requires close collaboration between developers, testers and the operations department. More on that later.

We have seen, that some people make some confusion about the term Continuous Delivery, can you explain to us, what's the difference between Continuous Delivery and Continuous Deployment?

Continuous delivery means you can release on demand. I should be able to release the latest good build at the press of a button without having to worry that it's going to break. But when you release is a business decision. Continuous deployment means you actually go ahead and release every good build. Obviously that only really applies in the context of websites or software-as-a-service. But if you can't actually do continuous deployment and release every good build for some reason - say you're working on a user-installed product - you should at the very least be putting every good build into a production-like environment and running realistic acceptance and performance tests against it so that there are no surprises when you come to release it. You can do that on embedded systems, products, pretty much anything that involves software.

Organizations that could in principle do continuous deployment really should aim to. One of the key benefits is organizational. Continuous deployment makes developers care much more about the quality and reliability of what they're doing because if they can push a button to make their changes go live, it connects them with what's going on in production. And it makes the business focus much less on stuffing features into a release when they know they can create some small feature, have it in production in a couple of days' time, and get real feedback on whether it's valuable and whether more effort should be put into it. It is a complete paradigm shift in the way organizations work and they become much more responsive to their customers. Eric Ries writes more about this in his forthcoming book, The Lean Startup

Can you explain to us, the term DevOps, and what that means?

DevOps is kind of an anti-movement, by which I mean it has resisted attempts to pin it down. But fundamentally it's about putting a strong focus on collaboration between everybody involved in software delivery - in particular, developers, testers and operations - and sharing knowledge and techniques between them. Infrastructure-as-code is one example of knowledge sharing - the application of agile techniques such as refactoring and test-driven development to evolving your infrastructure. DevOps is essential to achieve continuous delivery.

Part of the reason people in the movement have resisted (for example) creating a manifesto is to prevent vendors (in particular tool vendors) from appropriating it and trying to own it. Often people are guilty of reaching for tools (even within the movement - myself included) when actually it's the attitude and mindset that makes you successful. That's not to say the tools aren't important - they are. Tools that can't be managed in an automated fashion through an API are especially unpopular within the community.

How do you implement DevOps?

One really good way to start is to get representatives from every part of the organization involved in delivery - development, testing, infrastructure, operations, product management - to get together regularly and have retrospectives to reflect on how to incrementally improve things. In many organizations, you never see people from every department in a meeting together. It just doesn't happen.

Another way to get going is for everyone involved in delivering software celebrate successful releases together. Really implementing DevOps is about change that works up from the people on the ground - it's hard to mandate it. That's one of the reasons senior management has a hard time with it. But it's also the secret sauce - incremental, bottom-up change is usually the most effective and long-lasting.

What skills related to Infrastructure a modern developer should have? And why it is important? Does it make sense, for companies that are not in the cloud?

I think the key element is that infrastructure provisioning and management has to be automated. It should be possible to plug a new server (or workstation) in - power and network - and bring it into the correct state, configure it, and deploy to it in a fully automated fashion. You can do that whether or not you're in the cloud. In fact ThoughtWorks recently did some consulting to automate hardware provisioning and management for one of the cloud vendors. Under the hood, "the cloud" of course runs on real hardware and the same principles apply.


For great companies, that has already an Operation Department, what a developer can do to decrease this gap between devs and ops?

First of all, get to know your operations department. Make friends with them. Invite them along to your retrospectives and showcases. Learn the tools they use. Help them build new ones. Read books like Release It! that talk about how to create production-ready software. Check out the screens they have up that show off production metrics. Get the same screens put up in development rooms (LCD panels are cheap these days). Ask to help out when releases happen. Get involved.


Can you talk about your job at ThoughtWorks?

I'm working on a number of things. On the one hand I am doing a lot of travelling at the moment, going to conferences and talking to our customers. I'm also doing some research and writing, including working on a series of interviews to create some videos about continuous delivery. Finally I'm doing some consulting work to make sure I keep my feet on the ground and stay in touch with the latest developments in the space, and of course to pay my way.

------------------------------------------------

Jez Humble is a Principal at ThoughtWorks Studios, and author of Continuous Delivery, published in Martin Fowler’s Signature Series (Addison Wesley, 2010). He has worked with a variety of platforms and technologies, consulting for non-profits, telecoms, financial services, and online retail companies. His focus is on helping organisations deliver valuable, high-quality software frequently and reliably through implementing effective engineering practices.