Cloud Algorithms

Cloud Algorithms

The term algorithm is currently making a meteoric rise to fame. A geeky term that was previously confined to the world of mathematicians and software engineers is making its way into the mainstream, as people are increasingly recognizing the material impact on society that algorithms are starting to have. Algorithms, that used to be buried away inside of computer program files, used to find the derivative of a slope, or to find the shortest path between two locations, have today expanded to almost all areas of human activity. Algorithms for determining the value of a basketball player based upon a computerized analysis of his performance last season. Algorithms that analyze the incoming customer service calls and rout them to the most appropriate agent. Algorithms determining the likelihood of a convict reoffending, for analyzing insurance claims, for coordinating the nightly maintenance on a mass transit system, for driving cars, identifying symptoms. Algorithms to determine which candidate a company should hire, who should we recommend as a friend on social media or what films, books or music would someone like. And of course, algorithms have taken over financial markets, now making up 70% of trades, as stock markets have become layers upon layers of algorithms. An algorithm is a set of instructions for performing a certain operation. An algorithmic system takes an input and transforms it into a set of operations to create an output. Cooking a loaf of bread may be seen to follow an algorithm, where we take an input such as flour, water, salt and so on and perform a set of operation on them, such as mixing, needing, baking etc. to create an output which is the cooked loaf of bread.

Algorithms

Algorithms are as old a civilization itself – Euclid’s algorithm being one of the first examples dating back some 2300 years – but what we are doing with them today is very different from what they did in the past which was largely strict formal mathematical operations and limited statistical analysis. Algorithms are being transformed from the mechanistic linear form of the past, where we prespecified all the rules, hand-coded them with the end result looking like cogs in a gearbox, to today where algorithms take a more networked form, they are self-organizing and learn from data. These new forms of algorithms take many different names from cognitive systems to artificial intelligence, to machine learning.

Fei-Fei Li of Stanford describes some of the factors involved in this transformation1 “Around 2010, around that time, thanks to the convergence of the maturing of statistical machine learning tools the convergence of big data, brought to us by the internet and by the sensors, and the convergence of computing, the Moore’s law, carried us to much better hardware. These three pillars came together and lifted AI from the in vitro stage into what I call the in vivo stage AI in vivo is where AI is making a real impact to the world it’s just the beginning every single industry… is going through a transformation because of cloud, because of data, because of AI and machine learning and this is what I see as the historical moment, but I also want to say that it is just the beginning.”

These advanced algorithms, unlike the static mechanical models of the past, are adaptive in nature: They may learn as information changes, and as goals and requirements evolve. They may resolve ambiguity and tolerate unpredictability. They may be engineered to feed on dynamic data in real time, or near real time they are amenable to the processing of unstructured data, the processing of millions of parameters and complex patterns. Such as speech recognition, sentiment analysis, face detection, risk assessment, fraud detection, behavioral recommendations. This means these advanced analytical methods are no longer confined to mathematical operations but can handle more unstructured human-like activities such as many basic services.

Cloud Platforms

The idea of a computer is really just an abstract model for systems that store and manipulate data according to a set of instructions called algorithms. The implementation of this model can take many forms. We are used to thinking of it as the personal computer on our desktop, but with the rise of mobile computing, the internet and cloud computing this computing is becoming pervasive but also integrated through these cloud platforms. Cloud computing platforms have been a key innovation over the past decade, although a relatively straightforward idea – of centralizing computing resources within a datacenter and then delivering them as a service over a network – the outcomes of doing this are though extremely impactful. The demand for computation resources does not scale linearly with the size of the data but scales quadratically or cubically with the size of the data and when you are talking about billions of data points that causes problems and we need new computing platforms to mitigate that.

Throughout human history, computing power was a scarce resource, and until the past few years, high-end computer processing and storage offerings were out of the reach of all except the largest of organizations and then at the cost of millions of dollars. However, with the advent of global scale cloud computing, high-end computing is now available to organizations of almost all size at low cost and on-demand. The arrays of billion-dollar scale data centers owned and operated by Amazon, Google, and Microsoft, are now at the fingertips of many. Many of the largest applications on the internet today run on cloud computing infrastructure. Take for example Airbnb, that now coordinates an average of half a million people’s accommodation each night in 65,000 cities with their platform running almost entirely on Amazon Web Services. Likewise, each month Netflix delivers a dillion hours of video streaming globally by running on Amazon cloud. Indeed Amazon’s AWS is so widely used that when it doesn’t work right, the entire internet is in jeopardy.

Services

A basic driver behind many of the recent business disruptions in a wide range of industries is the transformation of computing resources from a scarce to an abundant resource. Combining cloud computing, with advances in algorithms and mobile computing we get machine learning platforms that are able to coordinates and run ever large and more complex service systems. This allows an increasing swath of human activity to be captured by algorithms, which allows it to be split apart, transformed, altered, and recombined. These platforms bring about an ever growing integration between technology and services. As data and information processing become more pervasive and computation becomes embed within virtually all systems, traditional divides are going to become ever more blurred, information technology and socio-economic organization will become ever more integrated and inseparable. As the saying goes, every company will become a technology company and this will fundamentally change the structure and nature of those organizations.

What is happening today is a convergence of these cloud computing platforms, new algorithms and the rise of the services economy. Recent years have seen the emergence of physical products that are digitally networked with other products and with information systems to enable the creation of smart service systems which are coordinated via algorithms. What is happening as we move into the services economy is that products become commoditized, people stop wanting to own things, what they want is to be able to push a button on their smartphone and the thing delivered as-a-service. An app for food services, an app for transport services, an app for accommodation, etc. and of course all these services are delivered on demand via cloud platforms that are coordinated via advanced algorithms.

Services are not like products, whereas products were mass produced, services have to be personalized; products were static once-off purchases, services are processes; products were about things, services are about functionality and value. Service systems are all about the coordination of different components around the end user’s specific needs, to do that you need lots of data, advanced analytics, and cloud computing. We can already see the data-driven services organization in the form of Uber, Alibaba or DiDi Chuxing, which don’t own anything they just use data and advanced analytics within their platform to coordinate resources toward delivering a service. Service companies like DiDi would be impossible without data.

Dematerialization

Dematerialization is one aspect of the information age. Material products become commoditized, data and information are used to strip physical technologies down to there most basic material requirements; value-added shifts to the organization of systems rather than the production and ownership of physical assets. This is seen with the rise of platforms over the past decade, which are really large networks that use data and analytics to optimize systems. As the venture capitalist, Steve Jurvetson put it “[in the past] the thing mattered, now it is all about the software and services layer… reduce the physical thing to its minimalist thing, for a container for software and code and that is what is happening in more and more products and services… the thing that every business makes is becoming a software product, in the long run, everything will cost a dollar a pound for things and what people will pay for and value is the software and services that come around that, it is what makes every product magical… I think what you are seeing as common practice in the IT-centric industries of today, in software and computers, what was the telecom transition of years past will be the case for every industry, the key question is when and in what sequence, some like agriculture and healthcare are in the middle or early phases of that transition but every industry will inevitably compete on how they process information, that’s how they will win or lose and the transition will not be easy for some.”

What will differentiate one company from another is not how fancy their product is, but how seamless and integrated their service system is and this is done through their capacity to master data and analytics. Organizations will become platforms and will compete based on their intelligence, which will be contained in their algorithms and people. In short, the physical technologies of the industrial age are being converted into services and connected to cloud platforms wherein advanced algorithms coordinate them. As Matt Turck of FirstMar puts it succinctly2 “Everything becomes data, your physical activity, traffic, purchases, and the data gets moved to the cloud, but it gets processed and compared with other devices, it is no longer just what you do but what everyone else also does, which keeps making the system smarter and smarter.” This is the essence of the process we are going through today; datafication converting everything into data, cloud platforms for aggregating and running the machine learning for processing it and iterating on that. Through servitization and dematerialization organizations become differentiated based on their data and algorithms as algorithmic systems extend to coordinate more and more spheres of human activity as we move further into the unknown world of the information age.

Citation


1. YouTube. (2018). Past, Present and Future of AI / Machine Learning (Google I/O ’17). [online] Available at: https://www.youtube.com/watch?v=0ueamFGdOpA [Accessed 10 Feb. 2018].

2. YouTube. (2018). Hyperconnectivity. [online] Available at: https://www.youtube.com/watch?v=vutkDpFPQNg&t=109s [Accessed 10 Feb. 2018].

#Algorithms #analytics #CloudComputing

Systems Innovation

  • LinkedIn
  • YouTube
  • Twitter
  • Facebook