The Future Growth of AI and ML

By Rachel Roumeliotis, VP of Data and AI at O’Reilly

We’ve all come to terms with the fact that artificial intelligence (AI) is transforming how businesses operate and how much it can help a business in the long term. Over the past few years, this understanding has driven a spike in companies experimenting and evaluating AI technologies and who are now using it specifically in production deployments.

Of course, when organisations adopt new technologies such as AI and machine learning (ML), they gradually start to consider how new areas could be affected by technology. This can range across multiple sectors, including production and logistics, manufacturing, IT and customer service. Once the use of AI and ML techniques becomes ingrained in how businesses function and in the different ways in which they can be used, organisations will be able to gain new knowledge which will help them to adapt to evolving needs.

By delving into O’Reilly’s learning platform, a variety of information about the different trends and topics tech and business leaders need to know can be discovered. This will allow them to better understand their jobs and will ensure that their businesses continue to thrive. Over the last few months, we have analysed the platform’s user usage and have discovered the most popular and most-searched topics in AI and ML. We’ll be exploring some of the most important finding below which gives us a wider picture of where the state of AI and ML is, and ultimately, where it is headed.

AI outpacing growth in ML

First and foremost, our analysis shone a light on how interest in AI is continuing to grow. When comparing 2018 to 2019, engagement in AI increased by 58% – far outpacing growth in the much larger machine learning topic, which increased only 5% in 2019. When aggregating all AI and ML topics, this accounts for nearly 5% of all usage activity on the platform. While this is just slightly less than high-level, well-established topics like data engineering (8% of usage activity) and data science (5% of usage activity), interest in these topics grew 50% faster than data science. Data engineering actually decreased about 8% over the same time due to declines in engagement with data management topics.

We also discovered early signs that organisations are experimenting with advanced tools and methods. Of our findings, engagement in unsupervised learning content is probably one of the most interesting. In unsupervised learning, an AI algorithm is trained to look for previously undetected patterns in a data set with no pre-existing labels or classification with minimum human supervision or guidance. In 2018, the usage for unsupervised learning topics grew by 53% and by 172% in 2019.

But what’s driving this growth? While the names of its methods (clustering and association) and its applications (neural networks) are familiar, unsupervised learning isn’t as well understood as its supervised learning counterpart, which serves as the default strategy for ML for most people and most use cases. This surge in unsupervised learning activity is likely driven by a lack of familiarity with the term itself, as well as with its uses, benefits, and requirements by more sophisticated users who are faced with use cases not easily addressed with supervised methods. It is also likely that that the visible success of unsupervised learning in neural networks and deep learning has helped our interest, as has the diversity of open source tools, libraries and tutorials, that support unsupervised learning.

A Deep Learning Resurrection

While deep learning cooled slightly in 2019, it still accounted for 22% of all AI and ML usage. We also suspect that its success has helped spur the resurrection of a number of other disused or neglected ideas. The biggest example of this is reinforcement learning. This topic experienced exponential growth, growing over 1,500% since 2017.

Even with engagement rates dropping by 10% in 2019, deep learning itself is one of the most popular ML methods among companies that are evaluating AI, with many companies choosing the technique to support production use cases. It might be that engagement with deep learning topics has plateaued because most people are already actively engaging with the technology, meaning growth could slow down.

Natural language processing is another topic that has showed consistent growth. While its growth rate isn’t huge – it grew by 15% in 2018 and 9% in 2019 – natural language processing accounts for about 12% of all AI and ML usage on our platform. This is around 6x the share of unsupervised learning and 5x the share of reinforcement learning usage, despite the significant growth these two topics have experienced over the last two years.

Not all AI/ML methods are treated equally, however. For example, interest in chatbots seems to be waning, with engagement decreasing by 17% in 2018 and by 34% in 2019. This is likely because chatbots were one of the first application of AI and is probably a reflection of the relative maturity of its application.

The growing engagement in unsupervised learning and reinforcement learning demonstrates that organisations are experimenting with advanced analytics tools and methods. These tools and techniques open up new use cases for businesses to experiment and benefit from, including decision support, interactive games, and real-time retail recommendation engines. We can only imagine that organisations will continue to use AI and ML to solve problems, increase productivity, accelerate processes, and deliver new products and services.

As organisations adopt analytic technologies, they’re discovering more about themselves and their worlds. Adoption of ML, in particular, prompts people at all levels of an organisation to start asking questions that challenge what an organisation thinks it knows about itself. With ML and AI, we’re training machines to surface new objects of knowledge that help us as we learn to ask new, different, and sometimes difficult questions about ourselves. By all indications, we seem to be having some success with this. Who knows what the future holds, but as technologies become smarter, there is no doubt that we will we become more dependent.