When David Marr at MIT moved into computer vision, he generated a lot of excitement, but he hit up against the problem of knowledge representation he had no good representations for knowledge in his vision systems. – Marvin Minsky
If you just have a single problem to solve, then fine, go ahead and use a neural network. But if you want to do science and understand how to choose architectures, or how to go to a new problem, you have to understand what different architectures can and cannot do. – Marvin Minsky