# Natural Language Processing (NLP) Natural Language Processing (NLP) is a field of [[Artificial Intelligence (AI)]] focused on enabling computers to understand, interpret, and generate human language. It combines linguistics, computer science, and [[Machine Learning (ML)]] to bridge the gap between human communication and computer understanding. NLP powers applications from search engines and voice assistants to machine translation and chatbots. The field has evolved dramatically: from rule-based systems (1950s-1980s), to statistical methods (1990s-2000s), to [[Deep Learning]] approaches (2010s), to the current era of [[Large Language Models (LLMs)]] like GPT and BERT. The 2017 Transformer architecture revolutionized NLP, enabling models to process entire texts in parallel and capture long-range dependencies, leading to unprecedented performance on language tasks. ## Core Tasks | Task | Description | Example | |------|-------------|---------| | **Tokenization** | Split text into units | "Hello world" → ["Hello", "world"] | | **Part-of-Speech Tagging** | Label word types | "The cat sat" → [DET, NOUN, VERB] | | **Named Entity Recognition** | Identify entities | "Apple in Cupertino" → [ORG, LOC] | | **Sentiment Analysis** | Detect emotion/opinion | "Great product!" → Positive | | **Machine Translation** | Convert between languages | "Hello" → "Bonjour" | | **Question Answering** | Answer questions from text | Extract answers from documents | | **Text Generation** | Create new text | Autocomplete, story writing | | **Summarization** | Condense text | Long article → key points | ## Evolution of NLP | Era | Period | Approach | |-----|--------|----------| | **Rule-based** | 1950s-1980s | Hand-written grammars and rules | | **Statistical** | 1990s-2000s | Probabilistic models, n-grams | | **Neural** | 2013-2017 | Word embeddings, RNNs, LSTMs | | **Transformer** | 2017-present | Attention mechanisms, BERT, GPT | | **LLMs** | 2020-present | Massive pretrained models | ## Key Architectures | Architecture | Year | Innovation | |--------------|------|------------| | **Word2Vec** | 2013 | Word embeddings from context | | **Seq2Seq** | 2014 | Encoder-decoder for translation | | **Attention** | 2015 | Focus on relevant parts of input | | **Transformer** | 2017 | Self-attention, parallelizable | | **BERT** | 2018 | Bidirectional pre-training | | **GPT** | 2018-24 | Generative pre-training, scaling | | **T5** | 2019 | Text-to-text framework | ## NLP Pipeline ``` Raw Text → Tokenization → Preprocessing → Embeddings ↓ Model (Transformer) ↓ Task Output ``` ## Challenges - **Ambiguity**: "I saw the man with the telescope" - **Context**: Meaning depends on surrounding text - **Commonsense**: Understanding implied knowledge - **Sarcasm/Irony**: "Great, another meeting" - **Low-resource languages**: Limited training data - **Bias**: Models reflect training data biases ## Applications | Application | Examples | |-------------|----------| | **Search** | Google, Bing semantic search | | **Assistants** | Siri, Alexa, Google Assistant | | **Translation** | Google Translate, DeepL | | **Writing tools** | Grammarly, autocomplete | | **Chatbots** | Customer service, ChatGPT | | **Content moderation** | Spam detection, toxicity | ## References - https://en.wikipedia.org/wiki/Natural_language_processing - https://nlp.stanford.edu - Jurafsky & Martin. *Speech and Language Processing* ## Related - [[Artificial Intelligence (AI)]] - [[Machine Learning (ML)]] - [[Deep Learning]] - [[Large Language Models (LLMs)]] - [[Transformers]]