Deep Learning

Deep learning is a subset of machine learning that uses multi-layer neural networks to learn hierarchical representations from large datasets.

Deep learning stacks many layers of simple processors so the model can learn complex patterns from lots of data.

Deep networks stack nonlinear layers to learn hierarchical features. Training typically uses backpropagation and large-scale compute (GPUs/TPUs). Architectures include CNNs for vision and Transformers for language and multimodal tasks.

Deep learning unlocked modern computer vision, speech, and large language models. Buyers need to know when a product depends on deep models versus classical ML.

• Image and speech recognition
• Foundation model training
• Representation learning for search and recommendations

• Handles unstructured data well
• Reduces some manual feature engineering
• Transfers across tasks via pretraining

• Data and compute hungry
• Harder to interpret than some classical models
• Sensitive to distribution shift and adversarial inputs

• Deep learning is not required for every tabular problem
• Larger depth does not guarantee better real-world outcomes

Is an LLM always deep learning?

Modern LLMs are deep neural networks, typically Transformers. Classical ML models are not deep learning.

• NIST AI RMF 1.0

• LeCun, Bengio & Hinton — Deep learning (Nature, 2015)