Build a pipeline to success with CI/CD

Meet Laura. She’s a popular author and has written 25 books. She’s been writing for 10 years. It takes her about 6 months to roll out a book, from draft to publication. Her books are on an average 150 pages long.

Laura’s books are well-loved in the community, but not all her books are block-busters. 6 of her books are bestsellers. The others have done moderately well. Some of them were just barely successful or not at all.

Here is what some readers had to say in their reviews. “Laura starts her stories well, but the plot thins out towards the end.” “Laura’s ideas are great, but I have to say that her execution sometimes falls short and the story seems insipid.” Laura learns about such feedback only a week or a month after a book is published. She uses the feedback for her next book. But what’s gone is gone.

Wouldn’t it be great if Laura had a bestseller every time she wrote a book? She won’t have to waste 6 months on something people don’t like.

This is like waterfall model in software. A software team may work for several months on a project. When finished, they share it with the client. The client may like the result or they may not.

But then, Laura hires an agent named Mr. Jenkins to help her with her books in future. Mr. Jenkins suggests that she use continuous integration and continuous delivery for her books from now on.

Enter Mr. Jenkins

Mr Jenkins suggests that instead of making people wait until a book is released in the market, Laura should make her book visible to her readers while she is writing it. She should push out her manuscripts to a select group of readers and gather feedback from them. This way her readers are not kept in the dark until the book is released. Laura can tweak the story if some good feedback is received. When everything is exactly as Laura wants and as her readers approve, she can approach a publisher or a self-publishing platform so that her book is published and made available in bookstores, both online and brick-and-mortar.

Continuous Integration

As per Mr. Jenkins’s plan, Laura calls him every time she completes a chapter or is done for the day. Mr. Jenkins picks up the manuscripts from Laura and takes it to her proof-reader. The proof-reader looks for spelling and grammatical mistakes. If he finds mistakes, then he rejects the manuscript and it ends up on Laura’s desk again, along with markings on where changes are required. If that goes well, then Mr. Jenkins takes the manuscript to someone who checks the story for clarity and continuity. If things look off and the story doesn’t add up or look well-tied, this person rejects the manuscript. The above two MUST approve, so that Mr Jenkins can proceed. Once approved, Mr. Jenkins takes the printed sheets of the manuscript to a professional binder. The binder makes a spiral-bound book out of the sheets. On the spine of the book, he includes the words “(manuscript only)”, so that readers don’t mistake the book for a published one.

This way, instead of simply being printed sheets of paper on Laura’s desk, her work has been converted to something that readers can hold in their hands, read and give feedback on.

So, here’s a definition of Continuous Integration in simple English.

Continuous Integration: When changes have been made to a material and the changes are proved to work, then the material is converted into a form that users can use.

Thus, Laura’s manuscript is converted into a spiral-bound book. In software, a program is converted into a form that can be directly loaded into phones, browsers or any devices that a software project is aiming for. The conversion is automated with a software that is responsible for continuous integration.

It is important to note the phrase, “if the changes are proven to work”. The proof-reader and the editor make sure that Laura’s manuscript has error-free English and that the story flows seamlessly. In software, the developers must write tests that will be automatically triggered. The tests verify that the program works as promised. Only if the tests are successful will the program be converted to the final form.

Continuous Delivery

After the binder gives Mr. Jenkins the spiral-bound copy, Mr. Jenkins makes further copies of it and approaches libraries around the city. Laura and Mr. Jenkins have already spoken to these libraries. The libraries have promised to lend space on their shelves for the bound manuscripts. Specific readers who are members of these libraries are notified as soon as new manuscripts of Laura land up on their shelves. These readers are big fans of Laura and would love the opportunity to read her work in progress and offer valuable feedback.

Note that this has given Laura the chance to reach her readers before her book is available in the market. This gives her a chance to make changes and improve her book based on what the readers really want.

We are now ready to define continuous delivery in simple English.

Continuous Delivery: The product created during continuous integration from the latest changes to material is made available in a test environment to potential users.

Just like Laura’s manuscripts are hosted in participating libraries, software is hosted in a test environment for users to try out and give feedback. Staging servers are used for websites and APIs. Instead of uploading to Play Store or iTunes, apps are hosted on a private website or in a sandbox app store so that chosen users can try the latest version before it is released for real world use.

What next?

CI/CD can take several iterations until everyone is happy. That goes for Laura’s book too. When that happens, Mr. Jenkins takes the book to a publisher / printer, who creates paperback, hard-bound and e-book editions that will be sold on bookstores, both Internet and brick-and-mortar. The books sees the light of day in the real world, coming out of its test world.

Deployment: When everyone is happy with a test product, it is polished to be ready for the real world and is then taken to the real world for actual use.

Similarly, mobile apps get released in app store / play store, websites and APIs get uploaded to web servers and car dashboard software gets copied to car dashboards.

Conclusion

Instead of following the outdated waterfall model where creators and users are isolated, where users get to use a product only after creators are done working, CI/CD is a realisation of the AGILE model of creating things, where the creators and users collaborate. The users get to see a creator’s work as often as possible and get to give feedback as soon as possible. The creator can cut his/her losses early if the user doesn’t like his/her work. This helps the creator work towards what the user really wants, before it is too late.

By the way, the name Jenkins is not thought out randomly for Laura’s agent. It is because he indeed works like the software named Jenkins. It is a software tool that can be used to automate continuous integration and deployment whenever a software developer makes changes to his/her program. Alternatives to Jenkins are Travis and AWS CodePipeline.