List

“The ever accelerating progress of technology … gives the appearance of approaching some essential singularity in the history of the race beyond which human affairs, as we know them, could not continue.” – John von Neumann

Like John von Neumann once said, we have reached a point of singularity where technological development is incomprehensibly rapid and complicated. Now, we are able to communicate with a non-existing artificial assistant in our phones and computers that can understand our speech; build self-driving cars and algorithms that recognize faces, objects like us! All of these fascinating technologies developed very rapidly when we start to use deep neural networks.

A neuron is a cell that creates chemical and electrical signals to communicate with other neurons to enable us to react to our environment. For example, when you are reading this post, the light waves coming from the screen excites your optical nerves at the back of your eye, and the neurons there transmit information to your visual cortex where earlier neurons there recognize edges of these weird patterns on the screen and later neurons there recognize that these weird patterns are letters and their combinations have a meaning. Under the inspiration of such a fascinating information processing that our brains perform in seconds, artificial neural network (ANN) technology was built. 

ANNs are working “conceptually” similar to neurons in the human nervous system. Human neurons (Figure 1, A) receive input from dendrites. This input can be an electrical or chemical signal due to an environmental stimulus such as light. If the input signal is strong enough, the neuron becomes activated and the signal is transmitted through the axon and arrives at terminal axons where it triggers various chemical and electrical activities and returns an output that will be used as an input by the next neuron. Likewise, artificial neurons (Figure 1, B) receives input that we give to them (e.g., x: cat or dog image), performs operations inside (e.g., a function of f(x) ), and then returns an output (e.g., y: detection of whether the input is a cat or dog image).

However, humans have approximately 100 billion neurons and these neurons are highly connected. Therefore, our fascinating ability to learn and interact with the environment is not the result of one neuron; instead, it is the result of a network of neurons that works in an organized way to perform such complex information processing. Correspondingly, by using ANN structures, a machine learning technique called deep learning was developed to make machines process information with deep neural connected layers like humans. This network structure was called deep neural networks (DNN). Human neural networks (Figure 1, C) activate in different patterns in order to process various environmental stimuli. For example, the neural network in our primary visual cortex (V1) processes low-level visual features of the stimuli such as edges and lines; whereas the neural network in our inferior temporal gyrus (IT) processes higher-level visual features such as shape.  One of the DNN types, Convolutional Neural Network (CNN) works in a similar manner too! In CNNs, there is an input and output layer and at least one hidden layer between them. Each layer is formed by several neurons and functions based on a feature hierarchy like our brain. For example, assume that we want to create a CNN model (Figure 1, D) that will detect whether a given image belongs to a cat or dog. In this case, the input layer will receive images of cats and dogs; then, neurons in the first layers of the model will detect edges whereas the later layers will detect shapes; and like human neural networks, the output layer will return a result such as detection of the image class (e.g., ‘this is a dog image’).

 

Figure 1. (A) A human neuron. (B) An artificial neuron. (C) Human neural network composed of 2 connected neurons. (D)Artificial deep neural network with one input, 2 hidden, 1 output layers

 

References:

https://en.wikipedia.org/wiki/Deep_learning

https://www.techopedia.com/definition/32902/deep-neural-network

https://skymind.com/wiki/neural-network

https://en.wikipedia.org/wiki/Neuron

https://www.medicalnewstoday.com/articles/320289.php

10 Responses to “A Glance at Deep Neural Networks”

  1. Anonym1

    Organized, fluent and good summary of AI. Now, I have a clue about ANN and DNN. Waiting for next blog..

    • btugcegurbuz

      Thanks!

  2. Serap

    Very clear, explanatory and succesful, Let’s see the next one.

    • btugcegurbuz

      Thank you!

  3. Alper YILMAZ

    Stephen Hawking (1942-2018) said, “AI will be able to redesign itself at an ever-increasing rate. Humans, who are limited by slow biological evolution, couldn’t complete and could be superseded by AI.”

    We still have a lot to learn from the human brain. Thank you for your effective and fluent article with their biological counterparts. We are waiting next blog.

    Alper YILMAZ
    CRYPTTECH AI Labs

    • btugcegurbuz

      Thanks for including great quote from Hawkings and your ideas!

  4. Actually

    Fun summary to read but actually, deep learning is not that similar to biological computation.
    First of all, each biological neuron is very hard to model since it works with spikes (AC model).
    On the other hand, each artificial neuron is just a summation and a non-linearity function and has no memory (DC model).
    The only resemblance with neuroscience I could see is the hierarchical feature representation in CNN’s which is not made of artificial neurons but rather 2D filters!
    So if you just put artificial neurons in layers in a fully connected manner, they would not learn hierarchical visual features like the V1 and V2. The artificial neurons are just good at classifying which can be likened to V4 neurons.

    • btugcegurbuz

      Thank you so much for making this point clear!
      Of course they are not the same working mechanisms! With this blog post, I wanted to demonstrate a “conceptual” resemblance of human brain and deep neural networks. In my following blog posts, I will explain how deep neural networks work in more detail. Please stay updated! 🙂

  5. схемы

    I’d like to thank you for the efforts you’ve put in writing this website. I am hoping to check out the same high-grade content by you in the future as well. In fact, your creative writing abilities has encouraged me to get my own site now ;)| а

  6. Sean

    Awesome post! Keep up the great work! 🙂

Leave a Reply

Your email address will not be published. Required fields are marked *