As one of the latest catchwords in the world of software, DevOps is a term that has been growing in popularity over the last decade. Is it yet another meaningless jargon which will fade away when the fashion wears off (Blu Ray discs, are you listening?!). Or is it something really useful, practical, helpful and here to stay for generations to come?
To understand what DevOps is, how it works and how it helps the people who use it, we sat down with software developer Dave and operations manager Oprah to understand how they use DevOps in their company, FutureSoft Solutions.
Host: Hello Dave. Hello Oprah. Glad to have you here today. It is a privilege talking to you about a technology as young as DevOps. Please introduce yourselves.
Dave: Hello Tech 101, good day. I am Dave. I work as a software developer at FutureSoft. 12 years already. I saw our company through the economic downturn. I work on mobile apps in Android, web apps and server side using NodeJS and Python. My job is to create all the applications that FutureSoft entrusts me with. These applications help our clients with so many things, most of all in finance.
Oprah: Good day Tech 101. Oprah here. I work as the operations manager at FutureSoft. I have worked here for 8 years. My job is to take all the applications that Dave has finished and deploy them live so that clients can use them. I also watch over our applications so that they run 99% of the time. Our biggest downtime over the last 3 years (since DevOps) has been just 30 minutes.
Host: Awesome friends. Let’s jump straight in. Is DevOps just a fad? Is it useful? If so, how has it been useful to you over the last 3 years?
Dave: I can promise you that DevOps is not a fad. In fact, it is still very young and the software world has yet to adopt it. DevOps is not a tool, it is a methodology. Now you and I know that tools, operating systems and programming languages come and go as technology changes. Methodologies are here to stay.
Remember, when Amazon opened its online shop, people said that e-Commerce was a fad, just like the rest of dot com. Who will trust a seller online? What if they take our money and run with it? What if someone snoops our credit card information? Well, twenty years later, shopping online is as main place as shopping at a mall. Not only do sellers deliver, but you can return merchandise for a money-back guarantee. You can use credit cards safely online. Today e-Commerce is strong and still growing, although e-Commerce websites and apps come and go.
People are apprehensive when a concept is new and they don’t understand. But then it becomes common place and people cannot think of a life without it.
So is the case with DevOps. Today, most companies look at it as a set of fancy tools that don’t achieve anything. That’s because they haven’t understood the concept and realised how well it works. I can tell you that it has saved a lot of time, effort and energy for Oprah and me.
Oprah: He’s right. From being sworn adversaries, Dave and I have become affable colleagues. DevOps has solved a huge conflict and turned it into a balanced work relation.
Host: Really? Was it that bad before DevOps?
Dave: You have no idea. Let’s look at a traditional company with no DevOps. Developers and operations are sitting in seperate silos. The developers are tasked with building new code and new features. They test things on their local computers or local devices and ship a releasable version to the operations. For reasons only known to them, the operations team would take long… really long, at least a week before that version is released in public or before we even get feedback and a list of bugs.
Meanwhile the development team has shifted gears. They are already deep into the feature to be ready for the next week. Then suddenly, the ops come and tell us that things are buggy in the last release and that they must be fixed. The devs are swamped between two versions. Should they fix last week’s bugs or continue working on this week’s features?
This happened every week. After some argument, we’d approach the management who’d reprimand both of us for wasting their time and with a sigh, usually side with one of us. At the end of every meeting, one of us would come out incensed with a big grudge against the other. This just kept building up. Mainly because the devs couldn’t understand what took ops so long to come back with a result of the release.
Oprah: That’s true. What the devs would call a ‘releasable’ version was hardly one for us. On one hand, we had to maintain 99% uptime for our servers with their existing version, while on the other new versions would be given to us at a fast rate by the devs. It would be ‘releasable’ for the dev team, who had only tested it on a development computer or a testing device. We’d have to convert the app given to us into the format needed by the live server / device. We’d have to make some code and configuration changes. But those changes wouldn’t be tested when the developers did their unit testing. So inevitably some changes would break, causing us to delay the release until further adjustments and it would already be 5 days to a week since the devs gave us their version.
The adjustments would go on and on until the app finally ran on a staging server / device. But we’d still have some unexpected errors when the app was deployed to the live server / device. Some errors could be handled by us, so we’d fix them ourselves. But for others, we had no choice but to go to the developers, who were already busy with the current week’s features. You know the rest.
Dave: We’ve had really heated verbal fights in front of the management, whose job was to hire a technical team who’d work together to give their clients what they wanted. Suddenly the management team had to play favourites and solve internal feuds among two tech departments. They’d be really exasperated.
Oprah: One day, one of the executives, a really nice guy, said: “Friends, this can’t go on. We hired you to work for us, not to fight against each other. I have friends at Globewide Solutions across the street. They are able to work together and release a new version every four days. I’ll talk to them and see if they can help us resolve these release day fights. ’cause they don’t seem to have any.”
Dave: Globewide surely charged us a lot for conducting a seminar on a ‘revolution’ called DevOps. We were skeptical. But not after the seminar.
Oprah: That day, something shifted in us and we learnt to work the DevOps way.
Host: Right, this is where our readers get curious. They have waited long enough. What in the world is the DevOps way?
Dave: First, people still keep their jobs with no changes. Developers are still developers and operations people are still operations people. It is just that they adopt some tools along the way that makes their relation a success rather than a conflict. There is a misconception spreading around the world that DevOps is a single team that does both development and operations. Totally NOT TRUE. That would drive people crazy. The only people who can do such a thing are freelancers! On a very small scale.
Oprah: On the contrary, DevOps doesn’t need members to double up roles. Rule #1 is that everything in Ops must be automated. Operations team use a set of tools that make sure that building a finished app, testing it, creating and setting up remote servers / devices for test and production, installing the built app into the remote servers / devices are all done automatically. So is the task of monitoring the health of the deployed apps. But that’s about operations. How does the dev team help? Dave can explain.
Dave: The dev team must learn a few tools that make sure that the afore-mentioned operations tools are kicked off at the right time. The dev team must integrate those tools. Say for example, a developer creates new code. He/she must use a tool that leads to the building and deploying of that code to the live server. All operations are a result of an event that takes place at the developer’s end. Even in the old non-Devops model, the operations team had to get to work because the development team released something. In DevOps, all actions are automated. The operations team makes sure that they write the correct automatically triggered actions.
Host: Can you give me an example of how a developer triggers an automatic operation process?
Dave: Sure. Version control like Git has a feature called a hook. Hooks watch for code additions, changes or deletions. So, when a developer puts a new piece of code into Git, then an appropriate hook triggers and calls a corresponding operations process.
All Oprah needs to do is create a set of automated scripts to work in different scenarios. She doesn’t need to do things manually every time something changes in the code. Building, testing and release are done automatically based on what Oprah has planned and written beforehand.
Host: That’s great. What are the other rules?
Oprah: Dave mentioned rule #2, which is that all automation begins as soon as the developer does something with the code, i.e. either puts in new features or fixes bugs, etc. Rule #3 is that everything is logged and analysed. The results of building a releasable version, the results of the tests run, creation of new servers, deploying an app live, users using the live app, the processing power and memory used by the live machines… everything. I get a daily report of the health of our live servers….. every morning. It’s only from those reports that I can decide a course of action for the day.
I get to look at an array of detailed reports and cool looking dashboards like a spaceship’s dashboard. It’s part of the fun. All the reports are automated because the tools are watching over things for me. 3 years ago, I had no reports and was only reacting based on the crisis of the day. Firefighting — they used to call it.
Host: What else can you give us about DevOps?
Dave: Well, that’s what I would say is the core of DevOps. Those 3 principles. Everything else is about the tools that you use.
Host: Can you name some tools you use for DevOps?
Dave: We don’t use GitHub. We have installed our own Git server. We have configured hooks in that server to run some automated tasks set up on Jenkins by Oprah. I’ll let Oprah take up the discussion from here.
Oprah: So Jenkins is a cool tool on which you can put up a sequence of tasks to be run once a certain event happens. When Dave’s Git server tells us that new code was pushed, we have a task list on Jenkins that builds the code, tests it and uses another tool called Puppet to deploy the code live on the server.
Now what is Puppet? A live server is a machine with its own operating system and other software like e.g. Apache web server, MySQL database, etc. What if you want a tool that doesn’t know which operating system is running on the live server, but can still tell the live server to install something on itself and run commands? That is Puppet. You should read more about it on Puppet’s official website.
Finally, we use a combination of tools like New Relic, Crashlytics, etc to log and generate reports.
With all these tools, code is tested and deployed a mere 30 seconds from the time it is added by the developers to Git. If there are errors, the developers are notified automatically within 30 seconds and I get a bug report mailed to me.
Host: That’s great. So DevOps has made your life easier.
Dave: You know, we were notorious at FutureSoft for our incessant fights around release day. Today, we are able to let the tools take care of things for us while we attend interviews like yours. There is an inside joke at FutureSoft that says Dave + Oprah = DevOps!
DevOps is the perfect solution for managing your software releases and monitoring their health. It is a perfect partnership of development and operations. So seamless!
Host: With DevOps taking so much away form your shoulders, what is the next step in your careers? How will you step up at FutureSoft?
Oprah: Well, a timely question. Both of us thanked FutureSoft for their wonderful support in our careers and resigned. The two of us are starting a DevOps company where we will coach and consult software companies to integrate DevOps in their software development.
Dave: Yes, we are quite excited about it. Our first clients are FutureSoft themselves. Long term contract! Isn’t that great?
Host: That’s great! Congratulations to both of you and thanks for dropping by to tell our readers at Tech 101 about DevOps. Hoping to hear more from you.