Supervised v. Unsupervised v. Reinforcement Learning: An Introduction
Machine Learning powers a lot of what we do. Currently there are three main categories of machine learning, supervised learning, unsupervised learning and reinforcement learning.
Let’s talk a bit about them and what differentiates them.
Machine Learning
Machine learning (ML) is the scientific study of algorithms and statistical models that computer systems use to perform a specific task without using explicit instructions.
I am a professional software developer who mainly writes web apps.
When I want a computer to do something, I have to explicitly code for that. If I don’t tell a computer to do something it will never do it.
Machine Learning on the other hand, allows computers to perform tasks that they themselves learn.
You throw data at a computer and it learns patterns from that data. (hopefully)
This has substantial power and also drawbacks. At times computers can surprise or surpass our intellect, but there are lot of tasks computers can’t currently solve.
Broadly speaking, there are three main categories of machine learning:
- Supervised Learning
- Unsupervised Learning
- Reinforcement Learning
Supervised Learning
Our first category is supervised learning. For me personally, this is the easiest category to learn and understand.
In supervised learning, the data that the algorithm trains on has both input and output.
One good example of this is the MNIST Database of Handwritten Digits, the “hello world” of machine learning. This database is a collection of handwritten digits in input and output pairs. The input is the image, and the output is the answer of what digit is in the photo.
When using supervised learning, the algorithm uses the input to make a prediction and compares the prediction against the expected output.
If it’s incorrect, the algorithm will modify itself in some way to make better predictions in the future.
Supervised learning is powerful and simple if you have the data that you need. If you don’t have neatly labeled data, or if your data is poorly labeled, then supervised learning is not going to be as effective.
Unsupervised Learning
Our next category is unsupervised learning. In contrast to supervised learning, unsupervised learning has input but no expected output.
Unsupervised algorithms automatically learning patterns or groupings that exist in the data.
For instance, in a 2012 Google study, researches just threw 10 million YouTube thumbnails at their “brain.” Over the course of three days it began to recognize pictures of cats.
No one told the brain to look for cats or recognize cats. Cats all look rather similar, so the brain remembered that pattern and taught itself how to identify it.
It’s also how a lot of modern NLP models are trained. Just throw lots of text (like the entire English wikipedia) at it and allow it to figure out how to make sense of it.
Unsupervised learning is great when you have a lot of data and a lot of compute power to find patterns in your data.
Reinforcement Learning
Reinforcement learning tries to model problems similarly to the way that humans live our lives. Reinforcement learning learns how to optimize an agents behavior based on the presence or absence of some reward.
For instance in my real life, I go to work because I receive a paycheck every two weeks. The reward (the paycheck) influences my behavior (continuing to be a good employee).
A good example is game playing bots. There’s an environment (the board), the agent takes actions, and receives a reward at the end of the game (win, loss or tie). The agent plays a lot of games to learn what actions lead to better rewards.
One good example of this is DeepMind learning to play Atari games. Raw pixel data was fed into the rl algorithm, it’s reward function was the game score, and it learned how to “play” the game to improve it’s reward.
Deep Learning
One last thing that’s worth mentioning is deep learning. Deep Learning is a sub domain of all of these categories. The label Deep Learning can be applied to any algorithm that specifically uses a multi-layer neural network, a deep network.
For instance, there are numerous ways to solve problems using reinforcement learning. If you’re specifically using a neural network, then you can say you’re using deep reinforcement learning.
If you’re playing around with neural networks, you can usually call it “deep”. But if you’re using some other machine learning algorithm to solve your problem, you wouldn’t call it “deep.”
Final Thoughts
Machine Learning is an exciting field. If you’re just starting out with machine learning, fast.ai has a great free course to help you explain the basics. I’m currently reading Grokking Deep Reinforcement Learning to bump up my reinforcement learning!