The huge triumph of [deep learning] has been figuring out that as long as you can pose a problem in a differentiable way and you can obtain a sufficient amount of data, you can efficiently tackle it with a function approximator that can be optimized with first order methods - from that, flows everything

The huge triumph of [deep learning] has been figuring out that as long as you can pose a problem in a differentiable way and you can obtain a sufficient amount of data, you can efficiently tackle it with a function approximator that can be optimized with first order methods - from that, flows everything. — Great concise description of what deep learning actually is from Fede_V on Hacker News (e.g., not a catch-all big-data-will-save-the-world thing).