A new approach to artificial intelligence, called deep learning, is drawing insights and identifying patterns in huge numbers of data.
Illustration by Kayla Oliver
A sense of excitement is brewing in Silicon Valley's artificial intelligence (AI) community. The race to develop AI is on, and news of major breakthroughs are capturing our imagination and steadily working their way into products we use everyday.
The race is unlike those we’re familiar with. In traditional markets, a company’s intellectual property is its comparative advantage. The formula for Coca-Cola, for instance, is worth millions and has never been made public. But, in the AI community, intellectual property is far less tangible. Keeping AI methodology and technology secret is frowned upon and will earn a disqualification in most deep learning competitions (a popular pastime for enthusiasts and industry experts). Instead, the comparative advantage major tech companies, such as Google, Facebook and Microsoft, have over their peers is the enormous trove of information sitting in their data centres. From these data, they extract useful insights into the human condition and reach ever closer to AI. The process they use to extract those insights is known as deep learning.
Deep learning is a form of AI made of several processing networks of neurons that learn representations of data with multiple levels of complexity. The networks in a deep learning machine, called an intelligent agent, are made up of thousands of smaller nets. Each network responds to unique rules, which vary in complexity and function. They are arranged neatly in layers, with simple rules on the lower layers and more complex rulesets higher up the chain. This forms a hierarchical system of pattern recognition. When thousands of independent networks link together, they create a layered system, which can interpret complex information in much the same way neurons in our brain do. The intelligent agent’s networks then discover patterns in large data sets by using an algorithm to uncover similarities between individual components.
For example, by exposing neural networks to a library of pet photographs labelled as “dog with a toy,” or “cat on grass,” the networks will learn to identify a dog. This is a similar process to a child learning objects’ names. Thoughtfully designed algorithms allow neural networks to discover patterns between objects by training them with huge datasets of unique examples. The accuracy of a neural network can then be improved by increasing the number of the data made available to it.
“Dog with a toy”: Labelling photos such as this one will help deep learning algorithms learn to identify images, in much the same way children learn the names of objects. Andrew Pons/Unsplash (public domain)
Companies such as Google, Facebook and Chinese search giant Baidu have focused their research efforts in recent years on advancing deep learning technologies by strategically hiring the leading thinkers in the field. Some companies are now hiring students working in data science before they even finish their education, causing a brain drain in the academic field. However, research on deep learning is published regularly. By sharing the tools being refined along the way the entire industry gains from any improvement in deep learning technology, no matter what company the ideas originated in.
Artificial intelligence takes on Atari
In 2011, a small British AI company, DeepMind, gave a presentation describing their new platform, which used a variation on popular deep learning models inspired by behaviourist psychology. The engineers at DeepMind created a new neural network, which was trained using a technique known as reinforcement learning. They taught software to optimise algorithmic outputs by maximising rewards. To do this, they chose a system that had predefined rewards: vintage, 1980s Atari games. CEO of DeepMind, Demis Hassabis, described why developing an AI platform with classic games makes sense: “In order to have true thinking machines or cognition, a system … has to figure things out for itself.” The system is given the goal of maximising its game score while being provided with exactly what a human would: access to the controls, and the pixels on the game screen.
The first few games of Breakout were a mess. DeepMind’s artificial player constantly missed the ball as it attempted different, random combinations of moves. Some actions would cause its score to go up. So, as it played more, it learned how to play more effectively. After hours of gameplay and over 500 rounds, the machine was a pro. DeepMind researchers stood by and watched in awe as their creation trounced their best scores. The agent had developed a technique of precisely aiming the ball up the sides of the wall and sitting by while the score shot up. Perhaps the team were better at writing computer programs than playing games—either way, they were impressed. In January of 2015, DeepMind researchers published an article in Nature that demonstrated their artificial agent matching or trouncing professional players’ scores across a set of 49 Atari games.
The DeepMind neural network attempts to play Atari Breakout.
DeepMind’s groundbreaking technology is one of the many success stories of AI research in recent years. The company was acquired by Google in 2014.
Google has been busy applying its deep learning tech in over 100 cases — from bettering its YouTube recommendations, to self-driving cars and automatically generated Gmail replies. This year, Google launched a new product, Google Photos, with one of its standout features being its intelligence capabilities. Google Photos automatically labels pictures using deep learning tech to detect and automatically label objects, friends’ and family’s faces, and landmarks.
Facebook’s 1 billion daily active users witness the effect of the technology in their personally tailored news feeds. Yahoo uses it to intelligently suggest noteworthy photos on the Flickr home page. But, beyond billion-dollar companies improving and launching products, what other potential applications exist for deep learning capabilities?
Tackling medical data
The industries where deep learning have the best prospects for success are those with access to the most data. Where there are huge numbers of data lies untapped potential for deep learning algorithms to uncover patterns, suggest improvements and draw insights. In the medical industry, mountains of data are collected every minute. Fitness trackers, radiology clinics and GP appointments create detailed patient reports, which are then relegated to USB sticks and cabinets at the back of a medical centre.
In 2014, Australian entrepreneur Jeremy Howard founded Enlitic, a medical diagnostics company applying deep learning technology to the vast numbers of data produced in the medical industry every day. Enlitic’s software analyses imaging data from CT scans and x-rays, comparing them against large datasets of existing images, while analysing statistical reports and lab research. The goal of Enlitic is to provide medical care at double the accuracy and speed of traditional options at half the cost. But, the company is young and their technology has not reached a large number of patients yet.
Deep learning has the power to revolutionise the medical industry. Huge sets of images, such as CT scans, can be compared to currently existing lab research. Kyle McDonald/Flickr (CC BY 2.0)
The theoretical basis for deep learning was established over fifty years ago, but when researchers tried to apply mathematical models in the early 80s, the technology wasn’t there. Recently, developments in efficient computer systems, such as Facebook’s Big Sur hardware platform, have allowed vast datasets to be efficiently processed simultaneously.
The intelligent software behind Google Photos was opened up to the public last month online. The AI community’s penchant for sharing knowledge sets it apart from other billion-dollar industries and is undoubtedly one of its strengths. When leading researchers and developers at rival companies share access to each other's tools, while the public can freely experiment with cutting edge technology, something truly unique is occurring.
Music to AI’s ears
Questions remain about the usefulness of this information for the public when individuals lack access to massive data sets and large-scale computing platforms. One sector, which has escaped the interest of deep learning enthusiasts so far, is the music industry. Consider traditional American folk music, popularised by Bob Dylan and the folk revival in the 1960s. Most of the great recording artists in the early 20th century folk movement lacked access to (now ubiquitous) high fidelity audio equipment. And, as such, recordings are typically scratchy and ridden with sonic artefacts. Take this recording of a seminal Woody Guthrie song, This Land is Your Land, plagued by incessant tape hiss:
Woody Guthrie’s song, This is Your Land, has a constant hissing sound running through it. By utilising deep learning algorithms, unwanted elements such as this could be removed from vintage recordings.
Audio restoration is a tricky business. Modern techniques are expensive, must be applied to songs individually, and are beholden to the subjective preference of the listener. A deep learning approach to audio artefact detection and removal could provide a neutrality and effectiveness, which vintage recordings could benefit from. Deep learning pattern recognition tools have achieved high success rates across a variety of applications, which continue to improve over time with enhancements to software and hardware.
Perhaps it’s time deep learning matured beyond its Silicon Valley home to greater things. Companies like Facebook and Google are constantly finding new applications for this technology, paving the way for applications in many unforeseen circumstances. It’s exciting to consider the many possibilities that deep learning could offer, and how it will begin to impact our daily lives going forward. We’re only just beginning to unlock the potential capabilities of this revolutionary tool.
Financial disclosure: Tom Hume is an investor in Capitol Health Limited, a venture capital firm who own a stake in Enlitic.
Edited by Bryonie Scott
