Lesson 9: Next Steps

To NLP and beyond...

by CAIS++

Table of Contents

  1. Introduction
  2. Natural Language Processing (NLP)
  3. Reinforcement Learning (RL)
  4. Deep Unsupervised Learning, Generative Models
  5. Decision Trees, Random Forests, Gradient Boosting
  6. Social Network Analysis


If you’ve gone through our previous lessons while retaining most of the information presented, then good news: you’re pretty much ready to start creating some machine learning projects of your own! After getting a grasp on the basics, you’ll find that there’s a whole world of blog posts, tutorials, etc. out there for explaining some of the more advanced topics in machine learning. Our hope is that at this point, you’ll manage to find some topics that interest you, and be able to explore them for yourself.

Below, we’ve compiled a list of some exciting topics within machine learning (and AI in general) that you may want to check out. Note that this is not by any means a comprehensive list of advanced topics in AI/machine learning — this is simply a list of active research areas that we think you should know about, and may hopefully serve as a valuable starting point for branching out into more specific fields.

(As you can see by this Wikipedia article, Outline of Artificial Intelligence, writing a comprehensive post on every sub-field of AI would be quite a feat indeed.)

Natural Language Processing (NLP)

We talked a bit about some natural language processing-related tools when we discussed RNNs in Lesson 7 (e.g. sentence completion, word2vec), but as you probably figured, there are so many different topics in natural language processing out there that it’s hard to give any sort of comprehensive list.

Visualization of word2vec embeddings

Here are a couple more examples of active areas in NLP research, just to name a few:

  • Sentiment Analysis (e.g. Does this Twitter user like my product?)
  • Machine translation (e.g. Google Translate)
  • Automatic Summarization (e.g. Summarizing news articles)
  • Conversational Agents (e.g. Chatbots, Siri)
  • Text Classification (e.g. Spam filtering)
  • Natural Language Generation (e.g. Generating Shakespeare-like text)

Since there are so many different directions to go when it comes to NLP, a good bet with these topics is to first get a general idea of what each one does, then think of how you could apply them via some potential projects. Then, you can look more into the specifics of how they work after you have some general goals in mind.

To give you a concrete example: Dr. Desmond Patton of the Columbia University School of Social Work (and a past CAIS seminar speaker), has used NLP along with some ML-based classifier models to analyze tweets for gang violence prevention.

Additional Resources:

Reinforcement Learning (RL)

Let’s imagine that we’re trying to teach a computer how to play pong. Using supervised learning, what we could do is record a bunch of people successfully playing pong, and then create a training set with the screen pixels as input and the move that the player takes (e.g. move paddle up/down) as output. We could then try to train a machine learning model (likely with the help of ConvNets) based on that data to decide what move it should take depending on the current game situation.

However, there are a couple problems with that: 1. gathering enough training data to make that work would be a huge pain, and 2. if you think about it, that’s not really how people learn in the real world. What we could try instead is to give the computer a reward for each time it wins a game, and then teach it to try to maximize those rewards, thereby training the computer to successively play pong.

This process is essentially what reinforcement learning (RL), a field that combines game theory, machine learning, and even some behavioral psychology, is all about.

Instead of training a computer based on labeled data, RL is about how to train a computer based on rewards that it gets from some environment (e.g. Pong, chess, even Starcraft, etc.). What happens is that over time, the computer learns what optimal action to take (e.g. move up, move down) based on the current state of the environment (e.g. the pixels on the screen, the opponent’s position). This process of taking actions in some environment and racking up rewards (or punishments) is known as a Markov Decision Process (MDP), and forms the basis for much of modern reinforcement learning, as well as much of AI planning.

Visualization of the reinforcement learning process

Reinforcement learning is also how Deepmind, the company behind AlphaGo, works much of its game-playing AI magic!

At Teamcore, to give another example, RL techniques are being used to train simulated wildlife patrollers to find the best routes to find poachers and to combat wildlife crime.

Additional Resources:

Deep Unsupervised Learning, Generative Models

Almost all image-related machine learning models that we’ve seen so far have required some sort of labeled training data in order to learn (e.g. photos of different animals along with their species labels). However, recent advances in generative learning have made it possible to train ML models to do some pretty cool things with images, even without fully labeled training data.

As their name suggests, generative models are capable of generating images that look more-or-less just like something you would see in a real photo. The basic idea is to give a model a lot of unlabeled images, and have it learn to create new images that look like they could belong in that dataset.

Generative adversarial network (GAN) learning to generate images over time

In Generative Adversarial Networks (GANs), this is done by splitting the model into two separate networks: a generative model, which generates the new images, and a discriminator (or classifier), which judges the images to see how well they match something you’d see in the real world (or, since this is machine learning we’re dealing with here, in the training set). The generative network tries to fool the discriminator into thinking that the images it creates are real, while the the discriminator tries to become better at not being fooled. These two networks are then trained simultaneously, eventually resulting in a model that is capable of generating content that could trick even a close observer.

Additional Resources:

Decision Trees, Random Forests, Gradient Boosting

While this isn’t necessarily an area of deep learning, it’d probably be good to be aware of some non-deep learning models that have had success in the machine learning field.

Decision trees are machine learning models that work sort of like flow charts: they learn to classify objects (or predict some numerical output value) by asking a series of questions, and then branching off based on the responses. These trees are then trained from data so that they can make pretty good predictions based on some input features. Unlike neural networks, which often act like black boxes, we can easily peek inside a decision tree to see how it generates its predictions.

Random forests and gradient boosting are two mostly-similar extensions of decision trees. While we won’t get too much into the differences here, the main idea behind these algorithms is to use a sort of voting process to combine a lot of smaller decision trees into one overall strong predictor. (See our write-up on ensemble methods here.) These models are used frequently in Kaggle competitions for regression, classification, etc, so you may run into them a good amount while working on your own projects.

Additional Resources:

Social Network Analysis

No, this isn’t about online social networks like Facebook or Twitter, although social network analysis techniques can definitely be applied to online social networks. Instead, in math and computer science, a social network typically refers to a graph of connections between people: each node represents a person, and each vertex/edge represents some connection between people (e.g. friends, family, etc.).

Social network analysis is all about studying these social networks to solve problems related to their structure. For example, if we wanted to spread some information throughout a group in the most efficient way, which members of the group should we talk to first so that they can spread the information through the rest of the network? In order to answer this question, we’d have to analyze the structure of the network to detect clusters/cliques, “important” (or influential) members, etc.

Example of a social network graph as visualized using Networkx

These social network analysis techniques are frequently used in applications like social work interventions, such as CAIS’s HIV-prevention project, HEALER.