Optimal Modified Feedback Strategies in LQ Games under Control Imperfections

Analyzed how execution errors in one player's feedback Nash strategy affect the other player's performance, and developed a disturbance-aware compensation law for finite-horizon LQ games.

Key Result In a simulated two-cart system, compensated feedback reduced Player 1's cost by 10.7% and integrated squared error by 41.3% relative to uncompensated feedback under the same actuator lag.

Overview

Feedback Nash equilibrium provides state-dependent strategies for interacting agents, but typically assumes that computed inputs are executed exactly. In practice, actuator dynamics can introduce deviations between intended and applied inputs. Because players interact through shared dynamics, one player's execution error can also affect the other player's cost.

This work characterizes that effect and develops a compensated feedback strategy that uses measurable execution deviations to mitigate the resulting performance loss.

Two-cart spring–damper example, all three cases on a synchronized time axis from the same initial condition: nominal feedback Nash (top), Player 2 actuator lag with reference feedback (middle), and the same lag with Player 1's compensated feedback (bottom).

Key results

10.7%

lower Player 1 cost

41.3%

lower Player 1 integrated squared error

Both relative to uncompensated reference feedback (REF) with the same Player 2 actuator lag, in a finite-horizon LQ two-cart spring–damper numerical example.

Problem formulation

Two players interact over a finite horizon in a discrete-time linear–quadratic (LQ) dynamic game, each minimizing its own quadratic cost. Their feedback Nash policies are computed assuming nominal input execution.

Player 2's applied input, however, deviates from its nominal Nash command; in the benchmark, because of first-order actuator lag. The deviation propagates through the shared closed-loop dynamics, so Player 1's realized state trajectory and objective change as well, even though Player 1 executes its own policy exactly.

Two-cart spring–damper system at its initial condition: Player 2 (orange cart, input u2) and Player 1 (blue cart, input u1) are coupled by a spring and a damper and start at offsets from the origin. — The two-cart benchmark: each cart is one player, and the carts are coupled by a spring and a damper, so each player's input affects the other's motion.

Contributions

Sensitivity analysis. Derived a first-order characterization of the effect of Player 2's execution deviations on the joint closed-loop state and Player 1's objective.
Deviation-aware compensation. Developed a causal compensated-feedback strategy that combines state feedback with a feedforward term based on measured execution deviations. The controller is obtained through an augmented finite-horizon LQ formulation using a Riccati recursion:

u_1,k = −K_k x_k − L_k w_k

where x_k is the joint state, w_k is the measured execution deviation of Player 2, and the gains K_k, L_k come from the augmented Riccati recursion.
Theoretical analysis. Established that the compensated policy is optimal within the specified causal affine policy class, which contains the uncompensated reference feedback, and a local improvement guarantee: under the paper's assumptions, for sufficiently small deviations it performs no worse than the uncompensated feedback to second order.

Numerical evaluation

All three cases start from the same initial condition over the same horizon (N = 40 steps, ΔT = 0.015 s); Player 2's actuator lag has time constant τ = 0.8 s.

FNE: Both players execute their nominal feedback Nash policies exactly.
REF: Player 2 experiences actuator lag; Player 1 keeps its nominal feedback law.
CF: Player 2 experiences the same lag; Player 1 uses the compensated feedback.

Finite-horizon results for the two-cart example
Metric	FNE	REF	CF
Player 1 cost	325.31	492.40	439.67
Player 1 integrated squared error	0.1655	0.4566	0.2681
Player 2 cost	330.45	492.11	556.40

Player 2's actuator lag raises Player 1's cost from 325.31 under FNE to 492.40 under REF.
Compensated feedback reduces Player 1's cost from 492.40 to 439.67, a 10.7% reduction relative to REF, and Player 1's integrated squared error from 0.4566 to 0.2681, a 41.3% reduction.
Measured against the FNE benchmark, compensation recovers about 31.6% of the additional Player 1 cost introduced by the lag.
Player 2's cost increases under CF (556.40 vs. 492.11), so this is an improvement for the compensating player, not a Pareto improvement.

Skills

Dynamic Games
Robust/Optimal Control
Interaction Modeling
Riccati Methods
Simulation