Convolutional Neural Networks (CNNs) Explained

In this video, I explain what convolutions and convolutional neural networks are, and introduce, in detail, one of the best and most used state-of-the-art CNN architectures in 2020: DenseNet.

Watch the Video Below

If you would like me to cover any other neural network architecture or research paper, please let me know in the comments!

References

• DenseNet paper: https://arxiv.org/pdf/1608.06993.pdf
• DenseNet on GitHub: https://github.com/liuzhuang13/DenseNet

​Follow me for more AI content

• LinkedIn: https://www.linkedin.com/in/whats-ai/
• Twitter: https://twitter.com/Whats_AI
• Facebook: https://www.facebook.com/whats.artifi...
• The best courses in AI: https://www.omologapps.com/whats-ai
• Join Our Discord channel, Learn AI Together: https://discord.gg/learnaitogether

Chapters

0:00​ - Hey! Tap the Thumbs Up button and Subscribe. You'll learn a lot of cool stuff, I promise.

0:18 -​ The Convolutional Neural Networks

0:39 -​ A … convolution?

2:07​ - Training a CNN

2:45​ - The activation function: ReLU

3:20​ - The pooling layers: Max-Pooling

4:40​ - The state-of-the-art CNNs: A quick history

5:23​ - The most promising CNN architecture: DenseNet

8:39​ - Conclusion

Video Transcript

facial recognition targeted ads image

00:03

recognition

00:04

video analysis animali detection these

00:07

are all powerful ai applications

00:09

you must already have heard of at least

00:12

once

00:12

but do you know what they all have in

00:14

common they are all using the same type

00:16

of neural network architecture

00:18

the convolutional neural network they

00:21

are the most used type of neural

00:22

networks

00:23

and the best for any computer vision

00:25

applications

00:26

once you understand these you are ready

00:28

to dive into the field and become an

00:30

expert

00:31

the convolutional neural networks are a

00:33

family of deep neural networks that uses

00:36

mainly convolutions to achieve the task

00:38

expected as the name says convolution is

00:41

the process

00:42

where the original image which is our

00:45

input in a computer vision application

00:47

is convolved using filters that detects

00:50

important small features of an image

00:52

such as edges the network will

00:55

autonomously learn filter's value that

00:57

detect

00:57

important features to match the output

00:59

we want to have

01:01

such as the name of the object in a

01:02

specific image

01:04

sent as input these filters are

01:06

basically squares of size

01:08

3x3 or 5x5 so they can detect the

01:12

direction

01:12

of the edge left right up or down

01:16

just like you can see in this image the

01:18

process of convolution makes a dot

01:20

product between the filter and the

01:22

pixels it faces

01:24

then it goes to the right and does it

01:26

again convolving the whole

01:28

image once it's done these give us the

01:31

output of the first convolution layer

01:33

which is called

01:34

a feature map then we do the same thing

01:37

with another filter

01:38

giving us many filter maps at the end

01:42

which are all sent into the next layer

01:44

as input to produce

01:45

again many other feature maps until it

01:48

reaches the end of the network with

01:50

extremely detailed general information

01:53

about what the image contains there are

01:56

many filters and the numbers inside

01:58

these filters are called the weights

02:00

which are the parameters trained during

02:02

our training phase

02:04

of course the network is not only

02:05

composed of convolutions

02:08

in order to learn we also need to add an

02:10

activation function

02:11

and a pooling layer between each

02:13

convolution layer

02:15

basically these activation functions

02:17

make possible the use of the back

02:19

propagation technique

02:21

which basically calculates the error

02:23

between our guess

02:24

and the real answer we were supposed to

02:26

have

02:27

then propagating this error throughout

02:29

the network

02:30

changing the weights of the filters

02:32

based on this error

02:34

once the propagated error reaches the

02:36

first layer another example is fed to

02:38

the network

02:39

and the whole learning process is

02:41

repeated thus iteratively improving our

02:44

algorithm

02:45

this activation function is responsible

02:48

for determining

02:49

the output of each convolution

02:51

computation and reducing the complexity

02:53

of our network

02:55

the most popular activation function is

02:57

called the real u

02:58

function which stands for rectified

03:00

linear

03:01

unit it puts to zero any negative

03:04

results which are known to be harmful to

03:06

the network

03:07

and keeps positive values the same

03:10

having all these zeros make the network

03:12

much more efficient to train in

03:14

computation time

03:16

since a multiplication with zero will

03:18

always equal

03:19

zero then again to simplify our network

03:22

and reduce the numbers of parameters

03:24

we have the pooling layers typically

03:27

we use a two by two pixels window and

03:30

take the maximum value of this window to

03:32

make the first pixel of our feature map

03:35

this is known as max pooling then we

03:38

repeat this process for the whole

03:39

feature map

03:40

which will reduce the x y dimensions of

03:43

the feature map

03:44

thus reducing the number of parameters

03:46

in the network the deeper we get into it

03:48

this is all done while keeping the most

03:51

important information

03:53

these three layers convolution

03:55

activation and pooling layers can be

03:57

repeated multiple times in a network

03:59

which we call our conf layers making the

04:02

network

04:03

deeper and deeper finally there are the

04:06

fully connected layers that learn a

04:08

non-linear function

04:09

from the last pooling layer's outputs it

04:12

flattens the multi-dimensional

04:14

volume that is resulted from the pooling

04:16

layers into a one-dimensional vector

04:18

with the same amount of total parameters

04:21

then we use this vector in a small fully

04:24

connected neural network

04:25

with one or more layers for image

04:28

classification

04:29

or other purposes resulting in one

04:31

output per image

04:33

such as the class of the object of

04:36

course

04:36

this is the most basic form of

04:38

convolutional neural networks

04:40

there have been many different

04:42

convolutional architectures

04:44

since lenet5 by jan lacun in 1998

04:47

and more recently with the first deep

04:49

learning network

04:50

applied in the most popular object

04:52

recognition competition

04:54

with the progress of the gpus the alex

04:57

net network in 2012

04:59

this competition is the imagenet

05:01

large-scale visual recognition

05:03

competition

05:04

rls vrc where the best object detection

05:07

algorithms were competing every year

05:10

on the biggest computer vision data set

05:12

ever created

05:13

imagenet it exploded right after this

05:16

year

05:17

where new architectures were beating the

05:19

precedent one

05:20

and always performing better until today

05:23

nowadays most state-of-the-art

05:25

architectures perform

05:26

similarly and have some specific use

05:29

cases

05:29

where they are better you can see here a

05:32

quick comparison of the most used

05:34

architectures in 2020

05:37

this is why i will only cover my

05:40

favorite network in this video which is

05:42

the one that yields the best results in

05:44

my researches

05:45

densenet it is also the most interesting

05:48

and promising cnn architecture in my

05:50

opinion please let me know in the

05:53

comments if you would like me to cover

05:55

any other type of network architecture

05:58

the densenet family first appeared in

06:00

2016

06:01

in the paper called densely connected

06:03

convolutional

06:04

networks by facebook ai research

06:07

it is a family because it has many

06:10

versions

06:11

with different depth ranging from 121

06:14

layers

06:15

with 0.8 million parameters

06:18

up to a version with 264

06:22

layers with 15.3 million parameters

06:26

which is smaller than the 101 layers

06:28

deep

06:29

resnet architecture as you can see here

06:32

the densnet architecture uses the same

06:34

concepts of convolutions

06:35

pooling and the relu activation function

06:38

to work

06:39

the important detail and innovation in

06:41

this network architecture

06:42

are the dense blocks here is an example

06:45

of a five-layer dense block in these

06:48

dense blocks

06:49

each layer takes all the preceding

06:51

feature maps as input

06:53

thus helping the training process by

06:56

alleviating the vanishing gradient

06:58

problem

06:59

this vanishing gradient problem appears

07:01

in really deep

07:02

networks where they are so deep that

07:04

when we back propagate the error into

07:06

the network

07:07

this error is reduced at every step and

07:10

eventually becomes

07:11

zero these connections basically allow

07:14

the error to be propagated

07:16

further without being reduced too much

07:19

these connections also encourage feature

07:21

reuse and reduce the numbers of

07:23

parameters

07:24

for the same reason since it's reusing

07:27

previous feature maps information

07:29

instead of generating more parameters

07:31

and therefore

07:32

accessing the network's collective

07:34

knowledge and reducing the chance of

07:36

overfitting

07:37

due to this reduction in total

07:39

parameters

07:40

and as i said this works extremely well

07:43

reducing the number of parameters by

07:45

around 5 times compared to a

07:46

state-of-the-art resnet architecture

07:48

with the same number of layers

07:50

the original dense net family is

07:52

composed of four dense blocks

07:55

with transition layers which do

07:57

convolution

07:58

and pooling as well and a final

08:00

classification layer if we are working

08:02

on an image classification task

08:04

such as the rls vrc competition

08:08

the size of the dense block is the only

08:10

thing changing for

08:12

each version of the densenet family to

08:14

make the network

08:15

deeper of course this was just an

08:18

introduction to the convolutional

08:19

neural networks and more precisely the

08:22

dense net architecture

08:23

i strongly invite you to further read

08:25

about these architectures if you want to

08:27

make a well thought choice for your

08:29

application

08:30

the paper and github links for densenet

08:32

are in the description of the video

08:34

please let me know if you would like me

08:36

to cover any other architecture

08:39

please leave a like if you went this far

08:41

in the video

08:42

and since there are over 90 of you guys

08:44

watching that are not subscribed yet

08:46

consider subscribing to the channel to

08:48

not miss any further news clearly

08:50

explained

08:51

thank you for watching

08:55

[Music]