Dr. Quoc Viet Le is a research scientist at Google Brain known for his path-breaking work on deep neural networks (DNN). He is especially famous for his Ph.D work in image processing under Andrew Ng, one of the pioneers of the DNN revolution. Le’s and Ng’s work demonstrated how computers could be used to learn complicated features and patterns in a way similar to how the mammalian brain learns, with better performance than earlier neural network technology. One of their first breakthroughs was demonstrating the training of a large neural network to detect cats from YouTube videos.
This revolutionized the interest in DNNs, and got the current giants of the computer industry such as Google, Facebook and Microsoft in a race to incorporate AI techniques into their software. Recently, Google announced a cloud platform for machine learning to encourage people into this area. DNNs have now become a buzzword among tech enthusiasts. They perform effectively in tasks such as image processing, handwriting recognition and game-playing, and are being explored for solutions to other problems such as self-driving cars, robotics, medical diagnosis and environmental and social problems.
Quoc Le was listed as one of the top tech innovators under 35 in the MIT tech review. At EmtechAsia, we asked Quoc Le a few questions about his take on neural networks, its development, philosophy, challenges and future role in enabling or threatening humanity.
In part 1 of our interview, we ask Le on the inspirations behind the development of neural networks and its various applications (Read part 2 here).
Q: During your development of deep neural networks, were you able to draw inspiration from detailed knowledge of the brains working through neuro-scientific findings ? How inspired were you by these insights in developing your techniques ?
Le: Actually, the brain is so complicated that we don’t know much about it. We know somewhat about the `hardware structure’, but not much about the `software’ and how it works. It is hard to find out much about these things, because you would have to cut brains and let animals/humans die in the process. So that makes this learning complicated.
DNNs are a first order approximation of what seems to happen in the brain. What we do in our field of simulating the brain (DNNs) is to mimic some of the hardware architecture. We learnt the brain is hierarchical, and that neurons are organised into layers with different functionalities, so these were nuances of the `hardware architecture’ that we could mimic in developing DNNs. But other than that, the structure of the brain is just an inspiration to DNN development.
Q: Do you have any shocking examples where AI has outclassed humans in pattern recognition tasks?
Le: One is image recognition. A researcher at Stanford, Dr. Andrej Karpathy was dealing with the problem of sifting through images and labeling them. He realized that in a head-to-head comparison, the machine learning algorithm was not far away from, or sometimes even better than, humans.
In face recognition, a lot of progress has been made. Handwriting, I would consider to be a (computer science) problem that is now solved, unless its really bad handwriting! But these are only particular narrow areas where AI outclasses humans because scientists have been working a lot on them and were able to make progress. We don’t have something like a general algorithm that can outdo humans in many categories.
Q: What about other areas like Robotics? Is it hard to train machines to move and balance things ?
Le: Yes, its hard. My friend at Berkeley University in Silicon valley used DNNs to train a robotic arm to grasp objects – move back and forth and stuff like that. He had some early success. I’d say this field is a good investment, but a long term one.
Hard to speculate how big a range of other activities DNN can be used for, but healthcare for one will greatly benefit from AI. Also, Smart Transportation. Right now, human drivers still cause accidents. But if an AI can help it recognize objects, routes and threats, it can help drive better.
Q: How can AI be used to revolutionize healthcare? Can you explain ?
Le: An example is medical diagnosis. For example, in my home country, Vietnam, I never had access to a good doctor in my youth. But now, we can train AIs to take over the task of a good doctor, or help doctors. I imagine it can be an algorithm on the phone, for an example, though we don’t have one like it yet. Your phone could be used to monitor your body status in terms of measurable quantities – your temperature, heart beat, pulse, weight, color of skin etc. Then, an AI could use this data to diagnose that maybe you have a a particular sickness or condition, a cold or a flu or some skin disease.
For this, the AI must be trained to learn from the best experience and based on previous labelled records. These labelled records would involve cases where a human doctor examined the data and classified it into some diagnosis – `these symptoms mean the patient has a cold’, `this indicates so-and-so disease’, etc. The AI can learn to do the same from these records.
As an extension, it would be great if we can do this using unlabeled data as well, which is called unsupervised training (More on unsupervised training in part two of this interview).
Q: What about cyber security ?
Le: Yes, many use AI for cyber security. However, being such a tough task, AIs cannot manage or take over cyber security fully by themselves yet. What companies currently do is to hire security experts, code up a bunch of decisions for security applications, and then use a small AI on top of it to select or enable decisions. As far as I know, that is the only work being done there, and that is not so much of AI. But that’s second hand knowledge, I’m not an expert on it.
Q: A more quirky question for you.
Technology has been progressing rapidly in the last decade. With the current advances we are having in synthetic biology, scientists are exploring how you may be able to grow everything in a lab. Coupling this with deep learning, in next 50 years or so do you think there’s a possibility we could do something like download our brain into a hard disk, transplant it, and continue to live on again ?
Le: Well, it’s not so easy. Not in the next 10 years I think, but I don’t know. The thing is, technologies face a lot of surprises in the course of their development. The rapid improvement in accuracy that was achieved by DNNs in image processing took us by surprise. After that, people anticipated similar progress in tasks like robots being able to collect objects, but now we are seeing that such a thing is still far away. So while DNNs have made breakthroughs in some areas, extrapolating the success likewise to other areas is non-trivial. We have to wait for the tech to catch up in other aspects.