Cover Story


I think people need to understand that deep learning is making a lot of things, behind-the-scenes, much better. Deep learning is already working in Google search, and in image search; it allows you to image search a term like “hug.” Geoffey Hinton

Deep Learning

Throughout the entire expanse of history, humans have been known to copy ideas from nature’s great creations and make artificial versions of them to benefit in daily lives. Some modelled planes after birds so we could fl, - some modelled helicopters after dragonflies so we could
Deep Learning
hover in the air flawlessly, whereas some have been modelling injections after mosquito proboscis for painless extraction of blood, and so on. Computers, as we all have been well aware, are also derived versions of human brain that aid us in numerous kinds of work. However in reality, typical computers come nowhere close to the computational complexity of our brain which comprises of billions of interconnected neurons to form what goes by the name of a “biological” neural network. Each individual neuron is capable of generating electrical impulses that propagate throughout the network and enable thinking and decision making. Over the years, the scientists have been making efforts to make technology smarter by creating artificial networks modelled after the biological neural network. Termed as artificial neural networks, these systems are a set of hardware and/or software, patterned after the operation of actual neurons.

Neural networks take a different approach to problem solving compared to conventional computers. Conventional computers use an algorithmic approach, i.e. they follow a set of instructions for problem solving. Unless the specific steps that the computer needs to follow are known, it cannot solve the given problem. That limits the problem solving capability of conventional computers to problems that we already understand and know how to solve. Now imagine how much more useful modern day computers would be, if they could do things that we don’t exactly know how to do.

Neural networks process information in a similar way the human brain does. The network is composed of a large number of highly interconnected processing elements (known as neurons), working in parallel to solve a specific problem. Neural networks are adaptive. They learn by example, and cannot be programmed to perform a specific task. The examples need to be selected carefully, otherwise useful time is wasted, or even worse, the network might be functioning incorrectly. On the other hand, conventional computers use a cognitive approach to problem solving; i.e. the way the problem is to solved must be known and stated in small unambiguous instructions, that are then converted to a high level language program and then into machine code understandable by the computer. These machines are totally predictable. In case anything goes wrong, it would be due to a software or a hardware fault.

Artificial Intelligence
What is a Neural Network?

A perceptron takes several binary inputs and produces a single binary output. Each input has a corresponding real number expressing the importance of the respective input to the output. The neuron’s output, 0 or 1, is determined by whether the weighted sum Σjwjxj is less than or greater than some threshold value. That is how a perceptron works!

For the purpose of imagination, consider the perceptron to be a device that makes decisions by weighing up evidence. Suppose you’re in the market and you like shopping. You love jeans and you try to decide whether to buy a pair or not. There might be three factors contributing to your decision making.

  1. Is it your favorite color?
  2. Is there a discount available?
  3. Do you have enough money?

We can represent these three factor by corresponding binary variables x1, x2 and x3. x1 = 1, if it is your favorite color; 0 otherwise. x2 = 1, is there is a discount available; 0 otherwise. x3 = 1, if you have enough money; 0 otherwise.

Now, suppose you really like your favorite color jeans but you would never buy it if you do not have enough money. This can be done by setting w1 = 4 , w2 = 2 and w3 = 8. The large value of w3 indicates that your budget is much more important to you than your favorite color or the availability of discount.

Figure illustrating a perceptron
Figure illustrating a network of perceptrons.

Suppose, you choose a threshold of 7 for the perceptron. With these choices, you have implemented a model which outputs 1 whenever you have enough money available and 0 otherwise. Your favorite color and availability of discount do not matter in this model. By varying the values of weights and threshold, different models can be achieved. This was a very simple example illustrating how a perceptron can weigh up different kinds of evidence in order to make decisions. The right figure illustrates what might seem a complex network of perceptrons. The first column/layer of perceptrons makes three very simple decisions, by weighing the inputs. Each of the perceptrons in second layer makes a more complex decision by weighing up the results from the first layer.

Similarly, the third layer perceptron weights the outputs of the second layer and makes even more complex decisions. This is how a multi-layer network of perceptrons can make quite complex decisions.

The problem with perceptrons is that a small change in the weights of any single perceptron in the network can sometimes cause the output of that perceptron to completely flip. This change may cause the behaviour of the rest of the network to change in some complicated way. This behaviour is undesirable in practical problem solving. This problem can be overcome by another type of artificial neuron called a sigmoid neuron. Sigmoid neurons are similar to perceptrons, but modified so that small changes in their weights cause only a small change in their output.

Convolutional Neural Networks
Convolutional Neural Networks are similar to ordinary Neural Networks which have learnable weights but they make an explicit assumption that the inputs are images. This makes some complex machine object recognition, image captioining, creating artworks, etc, easier. Regular Neural Nets don’t scale well to full images as the number of learnable parameters (weights) increase drastically for current computers to handle. The primary purpose of Convolution in case of a ConvNet is to extract features from the input image.

Imagine a greyscale image to be a 2D (say 256×256) matrix with pixel values ranging from 0 to 255. Now, consider a comparatively smaller matrix (say 3×3) called a ‘filter’. This filter is slided over the whole image while making some calculations to create a ‘Feature Map’. These filters act as feature detectors from the original input image. Different values of the filter matrix will produce different Feature Maps for the same input image. In practice, a CNN learns the values of these filters on its own during the training process (some parameters need to be specified such as number of filters, filter size, etc. before the training process). The more number of filters we have, the more image features get extracted and the better our network becomes at recognizing patterns in unseen images.

DeepDream is a computer vision program created by Google which uses a convolutional neural network to find and enhance features in an image; resulting in dreamlike output images.

Recurrent Neural Networks
An important problem in deep learning is to take into account the information of past or the computation that the network has already performed. Recurrent networks make this possible by using loops that allow information to be carried across neurons while reading some new information/input. However, modelling a RNN is a bit more complicated than other type of networks.

RNNs have been widely used with Natural Language Processing to perform linguistic magic. It is no longer impossible to generate your own Shakespearean poems that are almost indistinguishable from originals. All it requires is a large set of original Shakespeare poems and a fast computer!

Open AI

OpenAI is a non-profit AI research company that aims to carefully promote and develop friendly AI to benefit humanity as a whole. Some scientists, such as Stephen Hawking believe that if an AI gains an ability to redesign itself at an ever-increasing rate, it could lead to human extinction. OpenAI states that AI “should be an extension of individual human wills and, in the spirit of liberty, as broadly and evenly distributed as possible...”. Elon Musk acknowledges that “there is always some risk that in actually trying to advance (friendly) AI we may create the thing we are concerned about”; nonetheless, the best defense is “to empower as many people as possible to have AI. If everyone has AI powers, then there’s not any one person or a small set of individuals who can have AI superpower.”

Unavailability of relevent datasets in the past was a primary limitation in the breakthrough of Deep Learning. Today, the problem of face recignition can be regarded as solved; as social media sites have provided enough data for training and testing. Companies working on building self driving cars keep gathering data everyday so it can be used in near future. Data gathering is the new coal mining these days!

Another factor which limits deep learning performance is computational power. A small Neural Network can be trained on a simple CPU but training Deep Neural Nets with GBs of data is almost impossible for an average computer user. It is made possible only by GPU computing. GPUs have high processing speed and enough memory for training Deep Nets. Though for some tasks a cluster of multiple GPUs is used for weeks to get appreciable results.

Self Driving Cars are slowly approaching full automation. In the past, self-driving vehicles, such as the ones competing in the DARPA challenge, have relied on manually-coded algorithms to control the vehicle. With deep learning, it has now become possible for a car to navigate freely after only a thousand miles of training

The Impending Revolution
Electricity revolutionized the way we live. Bulbs replaced candles in every house, a fridge appeared in every kitchen and kids started talking about TVs. Our progress in mining gave us enough materials to mass produce cars, trains, ships, etc, and also the machinery to automate the process. Our softwares now enable us to communicate with our loved ones in any part of the world instantly; also we can see their faces! We’ve come so far; from living in caves to flying in air over the oceans. Some of us were just beginning to think that our progress was reaching to a point of saturation and it was then Artificial Intelligence came along. The field possesses a potential to again revolutionize the way we have been living for quite a while. It is expected that machines will soon start replacing humans at their tasks. But what about the humans? Although that day is not near but it isn’t much far either. Autonomous cars have started eliminating drivers. Machines have started beating us at video games. When they cross a threshold of intelligence, they would start outperforming us at our own jobs. What will we do then? The revolution seems inevitable and all we can do is to prepare ourselves for the future. But the question remains , “How”.