With the ever-increasing growth of the Internet, it is obvious that all services are being put under immense pressure. Email servers, social networks, file sharing servers and other vital services are seeing fast increasing and voluminous traffic everyday. Based on how a system responds to increasing usage, they are classified as fragile, robust and anti-fragile systems. Here is what you need to know about them.
A real-life example
Take a traditional dinner party where the host serves food. If the party is invite-only and RSVP based, then it is easy to estimate how much food the host should arrange for. But if the party is open to all, then the host has to estimate how many will attend. This depends on how he announces the party. The announcement has to have reached a certain number of people, out of which a percentage decide to attend. Let’s simply assume that the host estimates 100 persons to attend. So he safely orders 120 plates of food from a single caterer.
Fragile
If the number of guests is between 90 to 120, then our host will have had a successful party. Guests will have been happy with their serving sizes. With 90 guests, 30 plates will go waste, which is about 25% of the plates. Our host is okay with that wastage.
But what happens when 140 people attend? There are only 120 plates of food, so the food will have to be shared somehow. The ones that have dinner at the end may not get a few items or may be requested to take smaller portion sizes. Some of them might not get any dinner at all. Thus the quality of service goes down or simply disappears.
What about a party with just 50 people? 70 plates will have gone for waste. The host prepared for a capacity which remained unused, thus leading to cost twice as high as the attendance and massive wastage.
Robust
What if the host tweaks the system a little bit? Instead of telling the caterer to supply 120 plates, he asks for just 40. But, he tells them that he may require additional plates of food in batches of 20 while the party is going on. He promises to tell them well in advance so that they can prepare and deliver on time, without making the party-goers wait for too long.
What if 50 people attend the party? At plate # 21, the host doesn’t take any chances and orders another 20 plates. By the time the plates 21 to 40 are taken, the caterer will have had time to prepare more food. Hopefully, they rush in an additional 20 plates with little or no delay. The host has now ordered 60 plates, i.e. the initial 40 and an additional 20, thus ensuring that only 10 plates are unused.
If 140 persons attend, then the host will keep ordering additional 20 plates whenever he sees that he has just 10 or 20 plates left. At the end, he will have ordered just 10 to 20 plates in excess of 140.
A robust system adds capacity as required. To add capacity, it uses predetermined triggers well in advance before running out of capacity, so that it has time to prepare, thus avoiding a huge drop in the quality of service.
Anti-fragile
What if instead of a party, the host announces a get-together in his house. He invites people to use his house as a venue to talk to others and have a great time. He encourages the use of his large screen TV and his WiFi. But he mentions upfront that he only has limited food and drink, perhaps for just 20 persons. He suggests that people bring their own food, either from home or as takeout on the way. The food can be pooled and there will be enough for everyone, both in quantity and variety.
In such an arrangement, the more the people, the more the quantity of food brought to the table. If people bring their home delicacies, there will be plenty of variety too. It is an arrangement where the more there is, the merrier it is.
Sure, there will be some moochers who’ll just bring fries and chips or nothing at all, while they look forward to a variety of delicacies brought by others. But then, there are people who use exactly this type of occasion to fulfil their desire to serve and care. They will bring plenty for everyone and some more. In the end, there will be plenty of food for the entire attendance.
The three types of systems in software
Readable content
Fragile systems
During the late 90’s and early 2000’s, it was the responsibility of news and content creation websites to produce content, while visitors simply consumed it. With a growing number of people from around the world getting access to the Internet, content producers have to break out of their local region and produce global content. But there is a limit. Geared for local coverage, the websites’ resources are stretched when they need to cover global content. At one point, they can’t continue to deliver hyperlocal content because they simply don’t have the means to.
Despite the local popularity, New York Times & BBC will never be popular in Mumbai. Hindustan Times will never be popular in Copenhagen. These are fragile systems which stay relevant as long as the distance of the reader from the publication’s home location is short.
Robust systems
In came apps like Flipboard and websites like Engadget, Gizmodo and Inhabitat. In addition to some content produced inhouse, the four services mentioned are mainly curators. Instead of deploying reporters all over the globe, they seek content from other sources. Flipboard offers various sources that the user can choose to be part of his/her news feed. There are hundreds of newspaper websites and publications from where they source content.
Instead of deploying reporters around the globe, Engadget, Gizmodo and Inhabitat scour the Internet for good content specific to their niche. Using machine learning, they look for the most shared and highest-rated articles from around the world. Engadget and Gizmodo deliver technology-related articles from North America, Europe, south Asia and south-east Asia. Inhabitat does the same for sustainable and renewable lifestyle.
Anti-fragile systems
Facebook, Twitter, Medium, Instagram and Pinterest thought differently. What if they simply offered a platform where people could write as much as they wished? That’s what we have today. No reporters, no curators, no moderators.
On these 5 platforms, it’s the visitors who produce and share content. You can get hyperlocal content from your friends’ circle or global content by following people from around the world. It’s that simple. On these systems, the more the visitors, the merrier it is.
Media
Fragile
Media service, either through streaming or downloads, makes up a large part of the Internet and is dominated by media companies. Netflix offers movies, Spotify offers music, Amazon and Safari offer books and Coursera offers courses. As with fragile systems, all these services fail at a certain point, either due to scale or due to licenses.
In India, Netflix doesn’t offer Disney animation movies. Those rights have been grabbed by Hotstar. So if I like web series and I like Disney movies, I need to shell out monthly fees to two separate services. Kindle doesn’t offer many Marathi or Tamil books. Coursera has good technology courses, but very few on woodwork. As your needs narrow into certain niches, the existing fragile media companies fall short, as the quality of their content drops or completely disappears.
Robust
A good robust system is Scribd, an online library for readable books and audiobooks. Scribd doesn’t publish books but curates them from Kindle, Nook and even Indie source like Draft2Digital. One doesn’t buy books on Scribd. It is based on a monthly membership. Instead of paying for each book, you get to read as many books every month as you possibly can.
Scribd may not have every book on earth today, but they are always on the lookout for more sources for books and their library will only keep growing.
Anti-fragile
Though illegal for licensed content, the technology of torrents is an extremely powerful setup. In a torrent, individual computers owned by you and me become content sharing machines. When people search for content using a torrent client, all the computers that have that content raise their hands. These providers are called seeders. The torrent client downloads content from the seeders. This is called peer-to-peer sharing. There is no central server for file sharing, nor does it require a single company to set up servers with a lot of bandwidth.
Torrents make it extremely easy to spread quality content throughout the world. While most activity on torrents is unlicensed, pirated and illegal, they are also used for good purposes. E.g. In addition to hosting the latest downloadable version of Ubuntu Linux on their central server, Canonical, the creator of the operating system, also offer a torrent approach. The torrent allows you to download Ubuntu Linux from a computer which already has the downloadable version. Who knows, it’s possible that instead of downloading content from a server halfway around the globe, your Ubuntu Linux download might be coming from your neighbour’s computer.
Torrents become more powerful when more people join and start sharing. Sure, there will be some who simply consume content and don’t give back, thus being termed as leechers. But quite often, people don’t mind leaving their torrent software in the background and thus allowing others to download from them.
For online videos, YouTube, which allows people to post their own content, is obviously popular. So is Udemy for courses. Wikipedia and blogs are popular for local language content.
Summary
I won’t conclude that any of the three systems is better than the other. Each one has its place and purpose. But keep in your mind that traditional fragile systems are not the only solution and that other options exist. Here are short summaries about the three systems we saw.
Fragile: a central location or authority offers a service. The central location is prone to breakage when resources are stretched. The breakage may be failures, unavailability of content or loss of quality.
Robust: pooling of multiple sources that offer the same or similar service. These are managed by one central authority, whose main job is to regulate, distribute and curate. Examples are libraries that curate books from multiple sources and load balancers that distribute traffic among multiple server machines.
Anti-fragile: systems where the participants are both providers and consumers. The more the participants, the more powerful the system. Social media, Wikipedia and torrents are good examples.