Brief History of Machine Learning

Brief History of Machine Learning

Share on FacebookTweet about this on TwitterShare on Google+Share on LinkedInShare on TumblrPin on PinterestShare on RedditEmail this to someone

1946 – 2017

Before we deep dive into Machine Learning as a due diligence we should know where we have been, before we know where we are going.
As you will see it is not different from any other histories, it has it’s ups and downs, in spite of the current boom and as we go along I’ll let you decide if it’s a hype or a paradigm shift of epic proportions.

The Wikipedia Timeline of machine learning starts at 1763, with the Discovery of The Underpinnings of Bayes’ Theorem, but we’ll start our journey with a more practical 20th-century date, the invention of the computer in 1946.

1946

The Invention of the computer might be as important in human history as the invention of the fire, but AI has a great chance to take its place.
ENICA (Electronic Numerical Integrator and Computer) was the first electronic general-purpose programmable computer, built by the US Army and enhanced by my fellow Hungarian, John von Neumann (Hungarian: Neumann János Lajos), to make it programmable. He wrote programs to verify the feasibility of the H-bomb design, developed by another fellow Hungarian, Edward Teller (Hungarian: Teller Ede)

ENIAC
ENIAC

1950

The press heralded ENIC as a “Giant Brain” and it motivated Alan Turing to design the so-called Turing-Test to detect artificial intelligence. To pass the test, the machine should make a human believe that it is another human being, instead of a computer. It still has not passed.

1952

Arthur Samuel wrote the first computer program that was a learning machine, which learned to play checkers. His algorithms used a heuristic search memory to learn from experience.
By the mid- 1970’s his program was beating capable human players.

1956

At The Dartmouth Workshop Marvin Minsky, John McCarthy, Claude Shannon, and Nathan Rochester proposed the term “Artificial Intelligence”.

1958

Frank Rosenblatt designed the first artificial neural network, the Perceptron, commissioned by the US Office of Naval Research to solve visual recognition tasks.

The New York Times reported the Perceptron to be: “the embryo of an electronic computer that will be able to walk, talk, see, write, reproduce itself and be conscious of its existence”

Mark I Perceptron
Mark I Perceptron

1967

Cover and Hart wrote the Nearest Neighbor algorithm, a huge milestone and the birth of computer pattern recognition.

KNN Classification
KNN Classification

“Winter Is Coming!”

1974 – 1980, The First AI Winter

Research hype led to funds being dried up and caused little to no progress.

1979

Stanford students build Stanford Cart, the first mobile autonomous robot capable of moving around and avoiding obstacles.

Stanford Cart 1979
Stanford Cart 1979

1980 – 1987, The AI Summer

The introduction of the rule-based Expert Systems, which was quickly adopted by corporations and generated renewed interest in Machine Learning.

1981

Explanation-Based Learning (EBL) was introduced. Analyzed training data and created general rules allowing the less important data to be discarded. 

1985 – Neural Network Breakthrough

Researchers (re)discovered the Backpropagation algorithm, this allowed more complex and powerful neural networks with hidden layers to be trained. As a result, it resurrected research and got everyone excited about Neural Nets as the model for the brain. And yet again, hype and pipe dreams went into overdrive.

Neural Network
Neural Network

“Winter Is Coming!”, again

1987 – 1993, Second AI Winter

Neural networks went out of favor (again), due to a lack of good theories and a tendency to overfit.

1990s

A Statistical Approach to Machine Learning, Support Vector Machine (SVM) is the popular new kid on the block. SVM is a good candidate for rigorous mathematical analysis and achieves a state-of-the-art performance.

Support Vector Machine (SVM)
Support Vector Machine (SVM)

1997

IBM’s Deep Blue vs Chess Grandmaster Gary Kaspárov. Machine Wins!

IBM’s Deep Blue vs Chess Grandmaster Gary Kaspárov
IBM’s Deep Blue vs Chess Grandmaster Gary Kaspárov

2006

Present Day, Explosion and Commercial Adoption
Big Data, fast computing, matured Neural net models reinvigorated the interest for Machine Learning. Digital Transformation, Big Data with Machine Learning is incorporated into company processes, products, and services in order to get ahead of the competition. 

Geoffrey Hinton, who has a great Neural Networks for Machine Learning course on Coursera, invented the phrase Deep Learning as a buzzword, or a re-branding of neural networks to explain the new architectures of profound neural networks which are capable of learning much better models.

Geoffrey Hinton - Deep Learning
Geoffrey Hinton – Deep Learning

2011

The IBM Watson computer won the Jeopardy TV show, in which participants answer questions in natural language.

2012

Google’s Jeff Dean and Andrew Ng from Stanford, who has a great Machine Learning course on Coursera, started the Google Brain Project. They used the entire Google infrastructure to develop a deep neural network which detected patterns in images and videos.

Google Brain Project
Google Brain Project

Geoffrey Hinton’s team wins the ImageNet Large Scale Visual Recognition Challenge 2012 (ILSVRC2012) contest, by a large margin, using a Deep Neural Network (DNN) that has reached maturity and getting the attention it deserves.
This lead to the current explosion of Machine Learning based on DNNs.

AlexNet
AlexNet

Google X Laboratory built a DNN, running on a cluster of 16,000 computers, that recognized a cat based on 10 million images from YouTube.

GoogleX Cat with Andrew Ng
GoogleX Cat with Andrew Ng

2013

DeepMind, a British deep learning startup, designed a Deep Reinforcement Learning model to play Atari games that can beat human experts.

Atari Games
Atari Games

2014

Facebook develops DeepFace DNN, which can recognize people like humans do.

Google buys DeepMind.

2015

Amazon introduced Amazon Machine Learning.

Microsoft announced the Distributed Machine Learning Toolkit (DMTK), a framework to solve machine learning problems efficiently on a cluster of GPUs and computers.

Elon Musk and Sam Altman, among others, found the non-profit organization OpenAI, providing it with one billion dollars with the objective of ensuring that artificial intelligence has a positive impact on humanity.

2016

Microsoft releases Computational Network Toolkit (CNTK), its open source deep learning toolkit.

Google’s AlphaGo AI beats professional Go player Lee Se-dol. The algorithm can make creative moves that they had never seen before.

2017

Google announces TensorFlow.

Microsoft releases Microsoft Cognitive Toolkit (previously known as CNTK), its open source deep learning toolkit.

Today, we are experiencing a third explosion in artificial intelligence.
It is driven by the massive amount of data and challenging problems that require entirely new approaches for the effective solutions.
It is creating whole new markets and producing great changes in the strategies of small, medium, and large businesses alike.
The increasing amount of data will keep scientists and researchers busy for a while and they will keep coming back with groundbreaking new ideas till we reach the singularity.
Is the third winter coming? I highly doubt it, why? Technology adoption and tangible results have reached critical mass.

References

 

One Reply to “Brief History of Machine Learning”

Leave a Reply

Your email address will not be published. Required fields are marked *