This section follows the definitions and framing in Russell and Norvig, 2020, chapter 2, and connects them to engineering design decisions. The core idea is that we design agents by specifying their environments and then choosing the simplest program that behaves rationally within those constraints.
An agent is anything that perceives its environment through sensors and acts upon that environment through actuators.
This definition is deliberately broad. A thermostat and a warehouse robot both qualify as agents (see definition 1.1); the difference is the richness of their percepts and actions.
A percept is the agent's immediate sensory input at a given time.
A percept sequence is the complete history of everything an agent has perceived so far.
The percept sequence matters because rational action depends on history, not just the most recent snapshot (see definition 1.3).
The agent function is a mapping from percept sequences to actions. It defines what the agent does, independent of how it is implemented.
An agent program is a concrete implementation of the agent function on a particular agent architecture (hardware and software). We further require that the agent program takes as input only the current percept, not the full percept sequence.
The second requirement implies that the agent program must maintain internal state to access the full percept sequence if needed. This is a practical constraint for implementation.
In engineering practice, we often start with the agent program because it is implementable, but we should still be able to describe the underlying agent function it approximates (see definition 1.4).
A performance measure defines what counts as success for the agent in its task environment, over time.
A rational agent chooses the action that is expected to maximize its performance measure, given the percept sequence and any built-in knowledge.
Rationality is always evaluated relative to the performance measure (see definition 1.6) and the information available to the agent.
Autonomy refers to the degree to which an agent's behavior depends on its own experience rather than on built-in knowledge of the environment.
Russell and Norvig emphasize describing the task environment before designing an agent program.
A task environment is the problem setting in which an agent operates, specified by the performance measure, environment, actuators, and sensors.
PEAS stands for Performance measure, Environment, Actuators, and Sensors. It is a structured way to specify a task environment.
PEAS is the bridge between abstract definitions and implementable designs.
Design implication: if any PEAS component is vague, the agent's behavior will be underspecified and difficult to evaluate.
Task environment properties tell you what kind of agent program is feasible. The agent's environment is only one element of the task environment, alongside the performance measure, actuators, and sensors (see definition 1.9).
A task environment is fully observable if the agent's sensors give it complete access to the state; if it has incomplete access, it is partially observable; otherwise it is unobservable.
Example: In chess, the full board is visible, so the task environment is fully observable. In a warehouse, a robot only sees nearby aisles, so the task environment is partially observable; if sensors are down, it can be effectively unobservable.
A task environment is deterministic if the next state is fully determined by the current state and action; otherwise it is stochastic.
Example: In a deterministic grid world, an agent moves to its intended cell every time. In a stochastic grid world, an agent sometimes slips into an unintended neighboring cell.
In an episodic task environment, each decision is independent; in a sequential task environment, current actions affect future decisions.
Example: Image classification is usually episodic because each image is independent. Route planning is sequential because a wrong turn changes what actions are possible later.
A task environment is static if it does not change while the agent deliberates; it is dynamic if it does.
Example: A crossword puzzle is static because the clues do not change while you think. Driving in traffic is dynamic because other vehicles move while you decide.
A task environment is continuous if between any pair of states, moments, percepts, or actions there are infinitely many intermediate states, moments, actions, or percepts; otherwise it is discrete.
Example: A robot arm moving in space operates in a continuous task environment. Checkers is discrete because between any two positions a piece can occupy, there are finitely many other positions.
Together, these categories determine what the agent can reasonably represent and compute. They are design constraints that shape algorithms, memory, and testing strategies.
These definitions tell us what an agent is; design tells us how to build one. The workflow below aligns directly with the definitions:
For example, a fully observable, deterministic, episodic task often admits a simple reflex agent. A partially observable, sequential, dynamic task usually requires internal state, planning, or learning. Rationality does not mean perfection; it means the best expected outcome given the percept sequence, built-in knowledge, and computational limits.
Russell and Norvig describe agent structure as the combination of an agent program and an agent architecture, with the program implementing the agent function in a concrete way Russell and Norvig, 2020, § 2.4. The subsections below summarize the main agent types and design tradeoffs.
An agent program takes current percepts as input and produces an action as output. One may be tempted to structure the program as a lookup table that maps percept sequences to actions. However, this is impractical for all but the simplest discrete environments due to combinatorial explosion: the number of possible percept sequences grows at least exponentially with time.
This implies that agent programs require a structure and strategy for handling the massive space of possible percept sequences. Common strategies include the following structures.
Simple reflex agents select actions based only on the current percept, using condition-action rules. They ignore history and do not maintain internal state.
For instance, a thermostat that turns the heater on if the temperature is below a threshold and off otherwise is a simple reflex agent.
These agents can be rational in fully observable, episodic environments where the right action depends only on the present state. They fail in partially observable or dynamic settings because they cannot infer hidden state or remember previous events.
Model-based reflex agents augment reflex rules with an internal state that summarizes the percept history. They use a model of how the world evolves to update that state over time.
For example, a robot vacuum that keeps track of which areas have been cleaned and which remain dirty uses internal state to inform its actions beyond the current sensor readings.
This architecture handles partial observability by maintaining beliefs about the current state. The quality of the agent depends on the accuracy of its model and state update rules.
Goal-based agents choose actions that achieve a specified goal. This requires search or planning: the agent evaluates possible action sequences and selects one that leads to a goal state.
For instance, a soccer-playing robot that plans a sequence of moves to score a goal is a goal-based agent.
Goals make behavior more flexible than reflex rules, but they increase computational cost. The same environment may admit many goal-achieving actions, and the agent needs a strategy for selecting among them.
Utility-based agents go beyond goals by ranking outcomes with a utility function. This supports tradeoffs among competing objectives, such as speed versus energy or accuracy versus cost.
For example, an autonomous car that balances safety, speed, and passenger comfort uses a utility function to evaluate different driving strategies.
In uncertain environments, a rational utility-based agent chooses the action that maximizes expected utility, which requires estimating outcome probabilities and utilities.
The concept of alignment in AI safety relates closely to utility-based agents: ensuring that the agent's utility function reflects human values and desired outcomes.
Learning agents improve over time by adapting their behavior based on experience. Russell and Norvig describe four components:
Learning is not a separate agent type; it is a capability that can be added to reflex, model-based, goal-based, or utility-based agents.
There are three common ways to represent state in agent programs:
The choice of representation affects the agent's ability to reason, plan, and learn. More complex representations enable sophisticated behavior but require more computation and memory.