# Deep Learning Deep Learning is a subset of [[Machine Learning (ML)]] that uses [[Neural Networks (NNs)]] with many layers (hence "deep") to learn hierarchical representations of data. Each layer learns increasingly abstract features—from edges to shapes to objects in image recognition, or from characters to words to meaning in language. Deep learning powers modern AI breakthroughs including image recognition, speech synthesis, and [[Large Language Models (LLMs)]]. The field emerged from decades of neural network research but took off after 2012 when [[Geoffrey Hinton]]'s AlexNet dramatically won the ImageNet competition. Key enablers were massive datasets, GPU computing, and algorithmic improvements like dropout and batch normalization. Deep learning has since achieved superhuman performance on many tasks and forms the foundation of [[Generative AI (Gen AI)]]. ## Why Deep Learning Works | Factor | Contribution | |--------|--------------| | **Big Data** | Internet-scale datasets for training | | **GPU Computing** | Parallel processing for matrix operations | | **Algorithms** | Dropout, batch norm, residual connections | | **Architectures** | CNNs, RNNs, Transformers | | **Frameworks** | TensorFlow, PyTorch simplify development | ## Key Architectures | Architecture | Year | Use Case | |--------------|------|----------| | **CNN** | 1989/2012 | Image recognition | | **RNN** | 1986 | Sequential data | | **LSTM** | 1997 | Long sequences, memory | | **GAN** | 2014 | Image generation | | **Transformer** | 2017 | NLP, vision, everything | | **Diffusion** | 2020 | Image generation (Stable Diffusion) | ## Deep Learning vs Traditional ML | Aspect | Traditional ML | Deep Learning | |--------|----------------|---------------| | Feature engineering | Manual | Automatic | | Data requirements | Moderate | Large | | Interpretability | Higher | Lower ("black box") | | Compute requirements | Lower | Higher (GPUs) | | Performance ceiling | Limited | Higher on complex tasks | ## Introduction to Deep Learning - [[Neural Networks (NNs)]] - [[Why Deep Learning takes off]] - [[Unsupervised Learning]] - [[Binary Classification]] - [[Deep Learning Notation]] - [[Activation Functions]] - [[Sigmoid Function]] - [[Logistic Regression]] - [[Backpropagation]] - [[Convolutional Neural Networks (CNNs)]] - [[Transformers]] ## References - https://en.wikipedia.org/wiki/Deep_learning - Goodfellow et al. *Deep Learning* (2016) - https://www.deeplearning.ai ## Related - [[Neural Networks and Deep Learning]] - [[Generative AI (Gen AI)]] - [[Machine Learning (ML)]] - [[Geoffrey Hinton]] - [[Yann LeCun]] - [[Yoshua Bengio]] - [[Computer Vision (CV)]] - [[Natural Language Processing (NLP)]]