What is an RNN (Recurrent Neural Network) in Deep Learning?

RNN is one of the popular neural networks that is commonly used to solve natural language processing tasks, other neural networks are Feed-Forward Neural Network which is used for regression and classification problems. The last one is a Convolutional Neural Network or CNN which can be used for image classification and object detection.

Table of Contents:

1. What is an RNN?
2. How does an RNN work?
3. 5 Common Use Cases of RNNs

What is an RNN (Recurrent Neural Network)?

RNN stands for Recurrent Neural Network, this is a type of artificial neural network that can process sequential data, recognize patterns and predict the final output.

This Neural Network is called Recurrent because it can repeatedly perform the same task or operation on a sequence of inputs.

An RNN has an internal memory that allows it to remember or memorize the information of the input it received and this helps the system to gain context. Therefore if you have sequential data like a time series, then an RNN will be a good fit to process that data. 

This can not be done by a CNN or Feed-Forward Neural Networks since they can not sort the correlation between previous input to the next input.

Popular products like Google's voice search and Apple's Siri use RNN to process the input from their users and predict the output.

How do RNNs Work?

The logic behind an RNN is to save the output of the particular layer and feed it back to the input in order to predict the output of the layer.

Below is a simple example of how you can convert a Feed-Forward Neural Network into a Recurrent Neural Network (RNN).

On the left side of the image, represent the following:

• X - is the input layer.
• h - is the hidden layer and it holds the information for the previous output and feeds it back to itself.
• y - is the output layer
• A, B, C are the parameters to improve the output of the model.

Here is an example of a fully connected Recurrent Neural Network.

Note: Recurrent Neural Networks share the same weight parameter within each layer of the network while Feed-Forward Networks tend to have different weights across each node.

Now that you understand what a recurrent neural network is, let’s look at the common use case of RNNs.

5 Common Use Cases of Recurrent Neural Networks

1. Machine Translation

RNN can be used to create a deep learning model that can translate a text from the source language into the target language without human intervention. For example, you can translate a text from your local language to the English language.

Resources for Machine Translation 

• https://hackernoon.com/neural-machine-translation-using-open-nmt-for-training-a-translation-model-1129a3a2a2d3
• https://www.youtube.com/watch?v=vI2Y3I-JI2Q

2. Text Generation

RNNs can also be used to create a deep learning model to generate text. A trained model learns the likelihood of occurrence of a word/character based on the previous sequence of words/characters used in the text. You can train a model at the character level, n-gram level, sentence level, or paragraph level.

Resources for Text Generation.

• https://hackernoon.com/generate-stories-using-rnns-pure-mathematics-with-code-cb6f1e967b22
• https://www.tensorflow.org/text/tutorials/text_generation

3. Image Captioning 

Image captioning is the process of generating text that describes the content of a given image. The image's content can show the object and the action of the particular object on the image. For example, in the image below the trained deep learning model by using RNN can describe the image as "A lady with the green coat is reading a book under a tree".

Resources for Image Captioning.

• https://hackernoon.com/summarizing-most-popular-text-to-image-synthesis-methods-with-python-34d3xkn
• https://towardsdatascience.com/a-guide-to-image-captioning-e9fd5517f350

4. Speech Recognition

This is also known as Automatic Speech Recognition (ASR) that can process human speech into a written or text format. Don't confuse speech recognition with voice recognition; speech recognition mainly focuses on transforming voice data into text, while voice recognition identifies the voice of the user.

Products like Alexa, Cortana, Google Assistant and Siri are examples of Speech recognition technologies that are used daily by different users.

Resources for Speech Recognition.

•  https://hackernoon.com/how-to-do-speech-recognition-in-python-bk1234w9
• https://heartbeat.fritz.ai/a-2019-guide-for-automatic-speech-recognition-f1e1129a141c

5. Time-Series Forecasting 

You can use an RNN to create a time series prediction model that predicts the future outcome after being trained from the historical time-stamped data. A good example is in the stock market.

You can use the stock market data to create a machine learning model that can predict the future stock price in the market based on what the model learns from the historical data. This can help investors to make a data-driven decision for their investments.

Resources for Time Series Forecasting.

• https://hackernoon.com/time-series-forecasting-with-tensorflowjs-co3rz32ms
• https://machinelearningmastery.com/time-series-prediction-lstm-recurrent-neural-networks-python-keras/

There is more to cover in Recurrent Neural Networks, I recommend you take one of the following courses to learn more about RNNs.

• https://machinelearningmastery.com/crash-course-recurrent-neural-networks-deep-learning/
• https://www.coursera.org/projects/simple-recurrent-neural-network-keras

If you learned something new or enjoyed reading this article, please share it so that others can see it. Until then, see you in the next post!

You can also find me on Twitter @Davis_McDavid.

And you can read more articles like this here.

Want to keep up to date with all the latest in Machine Learning & Data Science? Subscribe to our newsletter in the footer below.