1.2  What is Artificial Intelligence?

Artificial Intelligence (AI) is a field of computer science and engineering focused on creating systems that can perform tasks typically requiring human intelligence. From an engineering perspective, AI is about building computational systems that can perceive, reason, learn, and act in complex environments.

1.2.1 Defining AI

Defining AI precisely is challenging because intelligence itself is multifaceted. (Russell and Norvig, 2020, ch. 1) propose examining AI through four contrasting dimensions:

1.2.1.1 Four Perspectives on AI

1. Systems that think like humans (Cognitive Modeling)

• Goal: Automate cognitive functions like learning, memory, reasoning
• Requires understanding human cognition through psychology and neuroscience
• Leads to cognitive science, not necessarily practical AI systems
• Example: Cognitive architectures like SOAR and ACT-R

2. Systems that think rationally (Symbolic AI)

• Goal: Build systems based on formal logic and rational thought
• Emphasizes correct reasoning independent of human cognition
• Approach: Use formal logic to represent knowledge and derive conclusions
• Limitation: Tractability—real-world problems often exceed computational capabilities
• Legacy: Expert systems, knowledge-based approaches

3. Systems that act like humans (The Turing Test)

• Goal: Achieve indistinguishable behavior from humans
• Alan Turing's seminal 1950 paper proposed this as the ultimate test of intelligence
• Requires: Natural language processing, knowledge representation, reasoning, learning
• Limitation: Behavioral equivalence ≠ true understanding

4. Systems that act rationally (Rational Agent Framework)

• Goal: Design agents that perform optimally given their knowledge and computational constraints
• An agent is anything that perceives its environment via sensors and acts via actuators
• Follows the rational agent framework: act to maximize an expected performance measure
• Most practical definition for engineering AI systems
• Leads to: Machine learning, planning algorithms, decision theory

The Rational Agent Perspective dominates modern AI engineering because it's pragmatic—it focuses on achieving goals rather than mimicking human cognition or implementing formal logic. We design agents that perceive, reason, and act in environments to accomplish measurable objectives.

1.2.1.2 The Learning Perspective

Contemporary AI emphasizes systems that can learn from experience and data rather than being explicitly programmed. Machine learning has become the dominant paradigm because:

• Real-world environments are too complex for hand-coded rules
• Data is increasingly available to train systems
• Learning allows systems to adapt and improve over time

1.2.2 Brief History of AI

1.2.2.1 Emergence (1950s)

• 1950: Alan Turing publishes "Computing Machinery and Intelligence," proposing the Turing Test as a measure of machine intelligence
• 1956: Dartmouth Conference marks the birth of AI as an academic discipline. Optimism abounds; early researchers believe human-level AI is achievable within a generation

1.2.2.2 Symbolic AI Era (1960s-1970s)

• Early success: General Problem Solver (GPS), SHRDLU, early natural language systems
• Assumption: Intelligence can be reduced to symbol manipulation and formal logic
• Emphasis: Knowledge representation, rule-based systems, logical inference
• Limitation: Combinatorial explosion—problems become intractable as complexity grows

1.2.2.3 AI Winter I (Late 1970s)

• Limitations of symbolic reasoning exposed
• Computational limitations prevent scaling to real problems
• Funding and interest decline

1.2.2.4 Expert Systems (1980s-early 1990s)

• Renaissance: Knowledge-based systems (e.g., MYCIN for medical diagnosis) show commercial promise
• Approach: Encode expert knowledge as rules and heuristics
• Success: Domain-specific systems outperform humans in narrow tasks
• Limitation: Brittleness—systems fail when encountering situations outside their training domain

1.2.2.5 AI Winter II (Late 1980s-1990s)

• Expert system limitations become apparent
• Expert knowledge is difficult to acquire and maintain
• General AI progress stalls; market enthusiasm collapses

1.2.2.6 Probabilistic and Statistical AI (1990s-2000s)

• Paradigm shift: Move from symbolic logic to probability theory and statistics
• Key developments: Bayesian networks, hidden Markov models, support vector machines
• Why it works: Real-world environments involve uncertainty; probabilistic reasoning is more robust
• Success: Spam filtering, speech recognition, information retrieval

1.2.2.7 Machine Learning Dominance (2000s-2010s)

• Data revolution: Internet-scale data becomes available
• Computational power: GPUs enable training of large models
• Key algorithm: Backpropagation rediscovered and applied to deep neural networks
• Success: ImageNet competition (2012) shows deep learning dramatically outperforms traditional methods

1.2.2.8 Deep Learning Era (2010s-present)

• Breakthroughs: Deep convolutional networks, recurrent networks, transformers
• Applications: Computer vision, natural language processing, game-playing, autonomous systems
• Current limitation: Deep learning requires enormous datasets and computational resources; interpretability remains challenging
• Trend: Integration of learning with classical AI (combining deep learning with symbolic reasoning, planning, and knowledge)

1.2.3 AI in Engineering

In engineering contexts, AI serves several key functions:

1. Optimization: Finding optimal designs and control strategies
2. Prediction: Forecasting system behavior and failures
3. Classification: Identifying patterns in sensor data
4. Control: Adaptive and intelligent control systems
5. Design: Generative design and automated engineering

1.2.4 Types of AI Systems

1.2.4.1 By Scope: Narrow vs General

Narrow AI (Weak AI)

• Designed for specific, well-defined tasks
• Examples: Image classification, machine translation, chess engines, recommendation systems
• Achieves or exceeds human performance in narrow domains
• Current state of practical AI technology

General AI (Strong AI)

• Hypothetical systems with human-level intelligence across all domains
• Could transfer learning and reasoning across diverse tasks
• Could understand context and meaning in the way humans do
• Remains a long-term research goal; significant theoretical and practical challenges remain

1.2.4.2 By Agent Architecture

(Russell and Norvig, 2020, ch. 2) classify agent types by their internal architecture:

1. Reflex Agents (Reactive)

• Decision: Current percepts → Actions (no memory)
• Example: Thermostat, simple robot controller
• Limitation: Cannot handle partially observable environments; no planning ahead

2. Reflex Agents with State (Simple Memory)

• Decision: Current percepts + Internal state → Actions
• Maintains memory of past events to track hidden aspects of the environment
• Example: Robot that remembers obstacles it has encountered
• Limitation: Still reactive; no planning or reasoning about future

3. Goal-Based Agents

• Decision: Percepts + Goals → Actions
• Considers future consequences of actions
• Example: GPS route planner, game-playing AI
• Advantages: Can select actions strategically; handles multiple possible futures
• Limitation: Often computationally expensive; must search through action sequences

4. Utility-Based Agents

• Decision: Percepts + Utility function → Actions
• Maximizes a performance measure (utility) rather than just reaching a goal
• Handles uncertainty and tradeoffs between competing objectives
• Example: Autonomous vehicle balancing safety, efficiency, and passenger comfort
• Most general and powerful architecture

5. Learning Agents

• Feedback loop: Perform actions → Observe results → Learn and improve
• Components: Performance element (acting), learning element (improving), critic (feedback)
• Example: AlphaGo, recommender systems, adaptive control systems
• Advantage: Improves performance over time without being explicitly programmed

1.2.5 AI and Related Fields

AI is inherently interdisciplinary, drawing from:

• Computer Science: Algorithms, data structures, computational complexity
• Mathematics: Logic, probability theory, optimization, linear algebra
• Engineering: System design, control theory, signal processing
• Neuroscience/Psychology: Understanding cognition and learning
• Philosophy: Epistemology, ethics, the nature of intelligence
• Linguistics: Natural language understanding and generation
• Economics: Decision theory, game theory, mechanism design

1.2.6 The Complexity of Real-World Environments

Real-world environments present challenges that simple agents cannot handle:

• Partially observable: Agents don't have complete information about the environment's state
• Stochastic: Actions have uncertain outcomes; the environment itself is non-deterministic
• Dynamic: The environment changes while the agent is deciding what to do
• Continuous: State and action spaces may be continuous (infinite possibilities)
• Multi-agent: Multiple independent agents interact, creating game-theoretic scenarios
• Unknown: Agents often must learn about their environment as they interact with it

For example, a self-driving car must handle a partially observable, stochastic, dynamic, continuous, multi-agent environment where the stakes are high and mistakes are costly.

1.2.7 Key Challenges in Engineering AI

Building practical AI systems requires addressing:

1. Data Requirements

• Most modern AI systems are data-hungry
• Quality matters: garbage in, garbage out
• Scarcity in specialized domains; abundance in others
• Privacy and ethical concerns with large datasets

2. Computational Resources

• Training large models (especially deep neural networks) is expensive
• Requires specialized hardware (GPUs, TPUs)
• Inference speed matters for real-time applications
• Energy consumption is a growing concern

3. Robustness and Reliability

• Systems must handle novel situations gracefully
• Adversarial examples: small perturbations can cause failures
• Domain shift: performance degrades when test environments differ from training
• Safety-critical applications require formal guarantees

4. Interpretability and Explainability

• "Black box" systems make decisions without human understanding
• Why did the algorithm reject a loan application?
• Medical and legal applications demand interpretable decisions
• Debugging deep learning systems is challenging

5. Ethical and Social Issues

• Bias: Algorithms can perpetuate or amplify societal biases
• Fairness: How should we balance competing notions of fairness?
• Accountability: Who is responsible when an AI system causes harm?
• Transparency: Should training data and model details be public?
• Autonomy: How much decision-making should we delegate to machines?

1.2.8 Why Study AI as an Engineer?

As an engineer, studying AI provides:

Practical Impact

• Solve complex problems in robotics, autonomous systems, optimization, sensing
• Build intelligent systems that adapt and improve over time
• Create value in domains from healthcare to manufacturing to scientific discovery

Theoretical Foundation

• Understand the fundamental principles underlying intelligence and decision-making
• Learn formal frameworks (logic, probability, optimization) applicable beyond AI
• Grasp the limits of what's computationally possible

Career Opportunities

• AI is one of the fastest-growing fields in technology
• Demand exceeds supply for engineers with AI expertise
• Opportunities span every engineering discipline: mechanical, electrical, civil, software, chemical, aerospace
• Academic and industrial research pathways both strong

Responsible Development

• As AI becomes more powerful and pervasive, engineers have a responsibility to understand its implications
• Ability to design systems that are robust, fair, interpretable, and aligned with human values
• Understanding both capabilities and limitations prevents over-promising and engineering failures

1.2.9 The Rational Agent Framework: Our Roadmap

Throughout this course, we adopt the rational agent framework from (Russell and Norvig, 2020, ch. 2) as our primary lens:

An agent perceives its environment through sensors and acts through actuators to maximize a performance measure given its knowledge and computational constraints.

This framework unifies the diverse subfields of AI:

1. Perception: How do agents sense their environment?
   • Computer vision, natural language processing, signal processing
2. Representation: How do agents represent knowledge about their environment?
   • Logic, Bayesian networks, embeddings and learned representations
3. Reasoning and Planning: How do agents decide what to do?
   • Search, logical inference, probabilistic inference, reinforcement learning
4. Learning: How do agents improve through experience?
   • Machine learning, deep learning, reinforcement learning
5. Action: How do agents act on their decisions?
   • Control systems, planning, execution monitoring

We'll build rigorous foundations in each of these areas, progressing from mathematical fundamentals through modern deep learning techniques and practical applications. By the end, you'll understand both how to engineer AI systems and their inherent limitations.