# Connectionism Connectionism is an approach to cognitive science that models mental processes using artificial neural networks—interconnected units that process information in parallel through weighted connections rather than explicit symbolic rules. The approach emerged in the 1980s with Rumelhart and McClelland's *Parallel Distributed Processing* (PDP) volumes, offering an alternative to the [[Computational Theory of Mind]] and [[Jerry Fodor]]'s [[Language of Thought]] hypothesis. While classical cognitive science treats thinking as symbol manipulation (like a digital computer), connectionism treats it as pattern activation across distributed networks (more like a brain). [[Paul Churchland]] and [[Patricia Churchland]] championed connectionism philosophically, arguing it better captures how brains actually work. Key features include: learning through weight adjustment (backpropagation), graceful degradation (partial damage doesn't cause total failure), and pattern completion (recognizing wholes from parts). Critics like Fodor and Pylyshyn argued connectionism can't explain the systematicity and productivity of thought—that anyone who can think "John loves Mary" can think "Mary loves John." The debate between symbolic and connectionist approaches anticipated the modern deep learning revolution, where neural networks have achieved remarkable success while still facing questions about whether they truly "understand." ## Connectionism vs Classical Computation ``` ┌─────────────────────────────────────────────────────────────┐ │ CONNECTIONISM vs SYMBOLIC AI │ ├─────────────────────────────────────────────────────────────┤ │ │ │ SYMBOLIC (Classical) CONNECTIONIST (PDP) │ │ ┌─────────────────────┐ ┌─────────────────────┐ │ │ │ │ │ ○───○───○ │ │ │ │ IF X THEN Y │ │ /│\ /│\ /│\ │ │ │ │ LOVES(JOHN,MARY) │ │ ○─○─○─○─○─○─○ │ │ │ │ │ │ \│/ \│/ \│/ │ │ │ │ Explicit rules │ │ ○───○───○ │ │ │ │ Discrete symbols │ │ │ │ │ │ Serial processing │ │ Distributed │ │ │ └─────────────────────┘ │ Parallel │ │ │ │ Weighted │ │ │ └─────────────────────┘ │ │ │ │ Knowledge in: RULES Knowledge in: WEIGHTS │ │ Processing: STEP-BY-STEP Processing: SIMULTANEOUS │ │ Learning: RULE INDUCTION Learning: WEIGHT ADJUST │ │ Failure: CATASTROPHIC Failure: GRACEFUL │ │ │ └─────────────────────────────────────────────────────────────┘ ``` ## Key Features | Feature | Description | |---------|-------------| | **Distributed representation** | Concepts spread across many units | | **Parallel processing** | Many computations simultaneously | | **Learning from examples** | Adjust weights via backpropagation | | **Graceful degradation** | Partial damage → partial impairment | | **Pattern completion** | Recognize wholes from parts | | **Similarity-based generalization** | Similar inputs → similar outputs | ## Core Concepts | Concept | Description | |---------|-------------| | **Units/Nodes** | Simple processing elements | | **Connections** | Links between units | | **Weights** | Strength of connections | | **Activation** | Level of unit activity | | **Layers** | Input, hidden, output units | | **Backpropagation** | Learning algorithm adjusting weights | ## Historical Development | Year | Development | |------|-------------| | 1943 | McCulloch & Pitts neural model | | 1958 | Rosenblatt's Perceptron | | 1969 | Minsky & Papert critique (AI winter) | | 1982 | Hopfield networks | | 1986 | PDP books (Rumelhart & McClelland) | | 1986 | Backpropagation popularized | | 2012+ | Deep learning revolution | ## Key Figures | Person | Contribution | |--------|--------------| | David Rumelhart | PDP, backpropagation | | James McClelland | PDP, language models | | [[Paul Churchland]] | Philosophical defense | | [[Patricia Churchland]] | Neurophilosophical support | | Geoffrey Hinton | Backpropagation, deep learning | | Yann LeCun | Convolutional networks | ## Connectionism vs Language of Thought | Issue | [[Language of Thought]] | Connectionism | |-------|------------------------|---------------| | **Representations** | Symbolic, discrete | Distributed, continuous | | **Rules** | Explicit, manipulated | Implicit in weights | | **Compositionality** | Built-in | Emergent (problematic?) | | **Systematicity** | Explained | Challenged to explain | | **Brain-like** | Less | More | | **Learning** | Hypothesis testing | Gradient descent | ## The Systematicity Challenge (Fodor & Pylyshyn) | Objection | Description | |-----------|-------------| | **Systematicity** | If you can think aRb, you can think bRa | | **Productivity** | Infinite thoughts from finite resources | | **Compositionality** | Meaning from parts + structure | | **Inference** | Logical relations require structure | ## Connectionist Responses | Response | Approach | |----------|----------| | **Tensor product** | Implement structure in vectors | | **LISA model** | Structured connectionist reasoning | | **Emergent structure** | Structure from training | | **Eliminativism** | Maybe systematicity is overstated | ## Strengths and Weaknesses | Strengths | Weaknesses | |-----------|------------| | Brain-like architecture | [[Symbol Grounding Problem]] remains | | Learns from examples | Needs lots of training data | | Graceful degradation | Hard to interpret ("black box") | | Pattern recognition | Systematicity challenges | | Handles noise well | Catastrophic forgetting | ## Modern Relevance: Deep Learning | Connection | Description | |------------|-------------| | **Deep networks** | Many hidden layers | | **Transformers** | Attention mechanisms | | **LLMs** | Language models (GPT, etc.) | | **Renewed debates** | Do neural nets understand? | ## References - Rumelhart, D. & McClelland, J. *Parallel Distributed Processing* (1986) - Fodor, J. & Pylyshyn, Z. "Connectionism and Cognitive Architecture" (1988) - Churchland, Paul. *The Engine of Reason, the Seat of the Soul* (1995) - https://plato.stanford.edu/entries/connectionism/ ## Related - [[Paul Churchland]] - [[Patricia Churchland]] - [[Jerry Fodor]] - [[Language of Thought]] - [[Computational Theory of Mind]] - [[Symbol Grounding Problem]] - [[Artificial Intelligence]] - [[Deep Learning]] - [[Neural Networks (NNs)]] - [[Philosophy of Mind]]