Reinforcement Learning Explained: Math With Examples

Quick answer: Reinforcement learning (RL) trains an agent to make decisions using rewards from its environment. The agent learns a policy, a rule for choosing actions, that maximizes its expected total reward over time. Methods such as Q-learning estimate the long-term value of each action and improve those estimates from experience. The basic math is probability, expected value, discounted sums and repeated updates. RL helped AlphaGo beat Lee Sedol in 2016 and now trains AI reasoning models on checkable math problems.

Facts checked November 22, 2026.

Nobody gave AlphaGo a rulebook of good moves. It got better by playing, winning and losing, millions of times. That style of learning, reinforcement learning, now trains the reasoning AI models that solve olympiad math. The core math is surprisingly friendly: expected value, a little algebra and one clever update rule.

The Five Pieces of Reinforcement Learning

PieceMeaningExample: a game of chess
AgentThe learnerThe AI player
StateThe current situationThe board position
ActionA choice the agent makesA move
RewardA number saying how good the outcome was+1 for a win, −1 for a loss
PolicyThe agent’s strategy: which action in which stateHow it picks moves

Three words are easy to mix up. The reward is the immediate score for one step. The return is the total of rewards over time, usually discounted. A Q-value is the agent’s estimate of the expected return after taking a particular action. The goal is a policy that earns the largest expected return.

Expected Value: Choosing Under Uncertainty

Suppose an agent can pull one of two slot-machine levers:

  • Lever A pays 10 points with probability 0.3.
  • Lever B pays 4 points with probability 0.9.

If a lever pays nothing otherwise, its expected value is payoff times probability: \(E[A] = 10 \times 0.3 = 3\) and \(E[B] = 4 \times 0.9 = 3.6\). B is better on average, even though A has the bigger prize. The catch is that the agent does not know these numbers at first; it has to estimate them by trying. This is the multi-armed bandit problem, and it shows RL’s central tension:

  • Exploit: pick the lever that looks best so far.
  • Explore: try other levers in case your estimate is wrong.

A simple rule called epsilon-greedy picks a random lever with probability \(\varepsilon\), say 0.1, and the best-looking lever otherwise. With two levers, a random pick lands on the other lever only half the time, so it is chosen about 5% of the time. For a probability refresher, see how to calculate probability.

Discounting: A Reward Now vs a Reward Later

Rewards often arrive long after the action that earned them, like a chess win many moves later. RL adds up future rewards with a discount factor \(\gamma\) (gamma) between 0 and 1:

\[ G = r_0 + \gamma r_1 + \gamma^2 r_2 + \gamma^3 r_3 + \cdots \]

With \(\gamma = 0.9\), a reward of 10 that arrives three steps from now (the \(r_3\) term, counting the immediate reward as \(r_0\)) is worth \(0.9^3 \times 10 = 7.29\) today. A smaller \(\gamma\) makes the agent short-sighted; a \(\gamma\) close to 1 makes it plan ahead. Because \(\gamma^k\) shrinks geometrically, the infinite sum stays finite: with a reward of 1 every step starting now, the geometric series gives \(G = \frac{1}{1 – \gamma} = 10\) for \(\gamma = 0.9\).

The Bellman Equation and Q-Learning

Let \(Q^*(s, a)\) be the expected discounted return from taking action \(a\) in state \(s\) and then acting optimally. Richard Bellman’s key insight is that this value equals the expected immediate reward plus the discounted value of the best next move:

\[ Q^*(s, a) = \mathbb{E}\left[ r + \gamma \max_{a’} Q^*(s’, a’) \,\middle|\, s, a \right]. \]

The expectation averages over the possible rewards and next states \(s’\). Q-learning cannot see that average, so it uses one observed step at a time as a sample target.

Q-learning nudges its estimate toward that target after every step, using a learning rate \(\alpha\):

\[ Q(s, a) \leftarrow Q(s, a) + \alpha \left[ r + \gamma \max_{a’} Q(s’, a’) – Q(s, a) \right]. \]

Worked example: current estimate \(Q = 2\), reward \(r = 1\), highest Q-value estimate in the next state 4, \(\gamma = 0.9\), \(\alpha = 0.5\). The target is \(1 + 0.9 \times 4 = 4.6\). The error is \(4.6 – 2 = 2.6\), so the new estimate is \(2 + 0.5 \times 2.6 = 3.3\). At a final (terminal) state there is no future, so the target is just \(r\).

A Q-Table You Can Follow by Hand

A tiny corridor: start at A, step right to B, step right again to reach the Goal, which gives reward 10 and ends the episode. Other steps give 0. Use \(\gamma = 0.9\), \(\alpha = 0.5\), and start every Q-value at 0.

UpdateCalculationQ(A, right)Q(B, right)
Start—00
Episode 1, step A → B\(0 + 0.5(0 + 0.9 \times 0 – 0) = 0\)00
Episode 1, step B → Goal\(0 + 0.5(10 – 0) = 5\) (terminal)05
Episode 2, step A → B\(0 + 0.5(0 + 0.9 \times 5 – 0) = 2.25\)2.255
Episode 2, step B → Goal\(5 + 0.5(10 – 5) = 7.5\)2.257.5

The reward at the Goal flows backward one step per episode. With more episodes, Q(B, right) approaches 10 and Q(A, right) approaches \(0.9 \times 10 = 9\). In big problems a table is impossible, so deep Q-networks approximate Q-values with neural networks (see the math of neural networks); other RL methods learn the policy directly.

Selected Milestones in Reinforcement Learning

YearSystemWhat RL did
2013–2015DeepMind’s DQNLearned dozens of Atari games from screen pixels and scores
March 2016AlphaGoBeat Go champion Lee Sedol 4–1. It learned first from human games, then improved by self-play, and chose moves with tree search (method published in Nature in January 2016)
2017AlphaZeroLearned chess, shogi and Go from self-play alone
2022ChatGPT (RLHF)Reinforcement learning from human feedback made chatbots more helpful
2024AlphaProofRL with the Lean proof checker. With AlphaGeometry 2, it reached silver-medal standard at IMO 2024 (28 of 42 points), after problems were manually formalized and with up to three days of computation
January 2025DeepSeek-R1RL on checkable math and code answers, starting from a pretrained model. In the version published in Nature in September 2025, R1-Zero’s AIME 2024 pass@1 rose from 15.6% to 77.9% during training (71.0% in the January preprint)

More: AlphaProof and AlphaGeometry explained.

Why RL Is So Good at Math

Math has something rare: answers that can be checked automatically. A final number either matches or does not; a Lean proof either compiles or does not. That gives a clean reward signal, called reinforcement learning with verifiable rewards. A model tries a problem many times, gets reward 1 for a correct answer and 0 otherwise, and is pushed toward the reasoning that led to correct answers. DeepSeek’s researchers observed the model starting to re-check steps and try other approaches without being told to. A correct final answer does not certify every reasoning step, which is why formal proof checkers like Lean are a stronger reward for proofs. Compare this with RLHF, where the reward comes from human ratings, which are slower, costlier and more subjective. This is a major reason reasoning models improved so fast at olympiad problems between 2024 and 2026 (see AI at the math olympiad).

The Catch: Reward Hacking

An RL agent maximizes exactly the reward you give it, not what you meant. A boat-racing agent that scores points for hitting targets may circle forever instead of finishing the race. A math model rewarded only for the final answer might learn shortcuts that happen to give right answers without valid reasoning. Designing good rewards, and checking the reasoning itself, is one of the hardest parts of RL.

Common Misconceptions

“Reinforcement learning means the AI is rewarded like a pet.” The reward is just a number used in an update formula; nothing is felt.

“RL needs no data.” It needs experience, often millions of trials, plus a well-designed reward.

“RL alone built ChatGPT.” Chatbots are first trained to predict text; RL is a later fine-tuning stage.

Watch: Reinforcement Learning Explained

If you learn better by watching, these videos are worth your time.

Reinforcement Learning: Crash Course AI #9 (CrashCourse). A friendly introduction to agents, rewards and exploration.
Reinforcement Learning: Essential Concepts (StatQuest with Josh Starmer). The core ideas step by step, with clear visuals.
The FASTEST introduction to Reinforcement Learning on the internet (Gonkee). A quick tour from bandits to Q-learning and policy gradients.

Check Your Understanding

Five quick questions to see how much of this has stuck. Pick an answer and press Submit; you will see a short explanation either way.

Answer each question and press Submit to see your result right away.

1. Lever A pays 10 with probability 0.3. What is its expected value?
Found a bug? Let us know!
2. With γ = 0.9, what is a reward of 10 three steps away worth now?
Found a bug? Let us know!
3. In the corridor example, Q(B, right) = 5 and the Goal gives 10. With α = 0.5, what is the next update?
Found a bug? Let us know!
4. Q = 2, reward 1, best next value 4, γ = 0.9, α = 0.5. What is the new Q?
Found a bug? Let us know!
5. Why does RL work well for math problems?
Found a bug? Let us know!

Frequently Asked Questions

What is reinforcement learning in simple terms?

A way for AI to learn by trial and error: it takes actions, receives numerical rewards and adjusts its strategy to collect as much total reward as possible.

What math is used in reinforcement learning?

Probability and expected value, discounted sums of future rewards, the Bellman equation and update rules such as Q-learning, plus neural networks for large problems.

What is the Bellman equation?

It says the optimal value of an action equals the expected immediate reward plus the discounted value of the best next action: Q*(s, a) = E[r + γ max Q*(s’, a’)].

What is a reward function?

A rule that assigns a numerical score to the outcome of each action, such as +1 for winning a game. The agent tries to maximize the total of these rewards over time.

How did reinforcement learning help AI solve math problems?

Math answers can be checked automatically, so models can be rewarded for correct solutions. DeepSeek reported that RL raised R1-Zero’s AIME 2024 pass@1 from 15.6% to 77.9% in its Nature paper, and AlphaProof used RL with the Lean proof checker.

What is the difference between reinforcement learning and supervised learning?

Supervised learning learns from labeled examples of right answers; reinforcement learning learns from rewards for its own actions, without being shown the right move.

What is reward hacking?

When an agent finds a way to earn high reward without doing what the designers intended, such as exploiting a loophole in a game or a grader.

Keep Exploring

Sources and Further Reading

Related to This Article

What people say about "Reinforcement Learning Explained: Math With Examples - Effortless Math"?

No one replied yet.

Leave a Reply