In Chapter 1, we defined agents using the PEAS framework (Performance, Environment, Actuators, Sensors) and classified task environments by their properties. These abstractions help us think about what an agent should do. This chapter focuses on how to find good actions: problem-solving through search, planning, and constraint satisfaction.
The bridge from agent design to problem-solving is problem formulation: converting a task environment into a mathematical model that algorithms can operate on. Done well, formulation clarifies the problem and enables efficient solutions. Done poorly, it leads to intractable computation or incorrect behavior.
Recall from Chapter 1 that a task environment is specified by PEAS. The environment component describes the world the agent operates in, but at a high level—"a warehouse with racks, robots, and charging stations." To solve problems algorithmically, we need a more precise representation.
First, we define states.
A state is a complete description of the world at a given moment, capturing all relevant information for decision-making and from which future states can be derived.
For example, in the warehouse environment, a state might include:
Next, we distinguish the state space from the search problem that operates over it.
The state space \(S\) is the set of all possible states the world can be in. A solution is a sequence of actions that transitions from an initial state to a goal state.
A search problem over a state space specifies:
Depending on the transitioin model, this formulation can be deterministic or nondeterministic. However, full observability is assumed (i.e., the agent knows the current state). Later chapters relax these assumptions.
Consider the warehouse robot from Chapter 1. A simple state-space model for navigation:
This model abstracts away low-level control (motor commands, localization uncertainty) and focuses on discrete decisions. The abstraction is justified if the robot can reliably execute these actions.
The state-space model operationalizes components of PEAS:
The state-space model is a design artifact derived from the PEAS specification. Different state-space formulations of the same task environment can lead to dramatically different computational complexity.
Not all problems require the same granularity. Engineers use different abstractions depending on the task:
A path is a sequence of states connected by actions: \(s_0 \xrightarrow{a_0} s_1 \xrightarrow{a_1} \cdots \xrightarrow{a_{n-1}} s_n\). The path cost is \[\sum_{i=0}^{n-1} c(s_i, a_i, s_{i+1}).\]
Paths are appropriate when:
Example: Finding the shortest route from the charging station to a pickup location in the warehouse is a path-finding problem. Algorithms like \(A^*\) and Dijkstra's operate on paths.
A plan is a sequence of actions (sometimes with partial ordering) that achieves a goal, possibly abstracting away the exact states visited. Plans often include preconditions and effects for each action.
Plans are appropriate when:
Example: A high-level plan for the warehouse robot might be:
This plan doesn't specify exact grid positions, leaving room for different paths depending on runtime conditions (e.g., obstacle avoidance).
A schedule is an assignment of tasks to resources (agents, machines) with temporal or resource constraints. Schedules typically optimize objectives like makespan (total time) or resource utilization.
Schedules are appropriate when:
Example: Assigning five delivery jobs to three warehouse robots, respecting battery limits and time windows, is a scheduling problem. Constraint Satisfaction Problems (CSPs) are a common tool for scheduling.
| Abstraction | Focus | Typical Algorithms | Example |
|---|---|---|---|
| Path | State-to-state trajectories | \(A^*\), Dijkstra, BFS | Grid navigation |
| Plan | Action sequences with logic | STRIPS, GraphPlan | Pick-and-place tasks |
| Schedule | Resource allocation over time | CSP, optimization | Multi-robot coordination |
Many engineering problems combine these: use scheduling to assign jobs, planning to decompose tasks, and path-finding for low-level execution.
Problem formulation requires clarity on what counts as success and what makes one solution better than another.
A goal test defines when the problem is solved. It can be:
In the warehouse, the goal might be "all packages delivered" rather than a single state, since different end positions are acceptable.
Constraints are hard requirements that solutions must satisfy:
Constraints can be encoded in the transition model (e.g., blocking illegal transitions), in the goal test (reject solutions violating constraints), or as penalties in the cost function (soft constraints).
The performance measure (from PEAS) quantifies solution quality. The path cost function operationalizes this for search algorithms. Key considerations:
The warehouse navigation problem admits different cost models depending on objectives. Let's compare three formulations.
Cost function: \(c(s, a, s') = t(a)\), where \(t(a)\) is the time to execute action \(a\).
Optimal solution: Shortest path in terms of time. Ignores energy use and wear-and-tear.
Cost function: \(c(s, a, s') = E(a, s)\), where \(E(a, s)\) is the energy consumed by action \(a\) in state \(s\).
Optimal solution: May take longer paths to avoid carrying loads far. Useful for battery-limited systems.
Cost function: \(c(s, a, s') = w_t \cdot t(a) + w_s \cdot \text{risk}(s')\), where \(w_t, w_s\) are weights and \(\text{risk}(s')\) is a safety penalty.
Optimal solution: Balances speed and safety. Tuning weights \(w_t, w_s\) reflects design priorities (e.g., \(w_s \gg w_t\) for safety-critical systems).
For a path \(\pi = (s_0, a_0, s_1, a_1, \ldots, s_n)\), the total cost is:
\[ C(\pi) = \sum_{i=0}^{n-1} c(s_i, a_i, s_{i+1}) \] (2.1)
Key properties:
Example calculation: A path with 10 moves (1 unit each), 1 pick (2 units), 1 drop (2 units), and 2 high-traffic penalties (5 units each) has total cost:
\[ C = 10 \cdot 1 + 1 \cdot 2 + 1 \cdot 2 + 2 \cdot 5 = 24 \text{ units} \]
Real warehouse systems have thousands of states. Solving problems at full fidelity is often intractable. Engineers use hierarchical abstraction:
Each level uses a different state-space model with appropriate granularity. Solutions at one level constrain or guide the next.
For example:
Decomposition reduces complexity but requires careful interface design to ensure consistency across levels.
Problem formulation bridges abstract task environments and concrete algorithms:
With a well-formulated problem, we can now apply search algorithms to find solutions. The next section introduces uninformed and informed search, starting with uniform-cost search and progressing to \(A^*\).
Comment 2.1 Designing Cost Functions
Cost function design is an engineering decision with system-level consequences. A poorly chosen cost leads to brittle or unsafe behavior. Always validate cost models against real system performance.