Tagged “value-iteration”

3 articles.

14 September 20265 min readReinforcement Learning

The Parameter Nobody Chooses

The living reward in a grid world is written down once and never discussed, and the optimal policy is a step function of it: eight thresholds between -3 and 0, each flipping exactly one square. The textbook value of -0.04 sits 0.0048 away from the one that decides whether the agent takes the shortcut past the pit, and above -0.0221, when steps are nearly free, the optimal move in one corner is to walk into a wall on purpose.

Artificial Intelligence

9 September 20269 min readSequential Decisions and Reinforcement Learning

Acting When You Cannot See the State

What changes when an agent gets noisy percepts instead of its state: the belief state that replaces it and the filtering update that maintains it, the exact reduction of a POMDP to an MDP over beliefs, the piecewise-linear convex value function that makes the reduction computable in principle, and the measured reasons it is not computable in practice - with the fixed point of belief, the alpha vectors and the value function all computed rather than asserted.

Artificial IntelligenceProbability

27 August 20269 min readReinforcement Learning

Markov Decision Processes

How to plan when actions do not reliably do what you intend: states, transition models, rewards and discounting, the Bellman equation, and value iteration worked numerically to its fixed point.

Reinforcement LearningProbabilityArtificial Intelligence