Reinforcement Learning Explained: Math With Examples
Quick answer: Reinforcement learning (RL) trains an agent to make decisions using rewards from its environment. The agent learns a policy, a rule for choosing actions, that maximizes its expected total reward over time. Methods such as Q-learning estimate the long-term value of each action and improve those estimates from experience. The basic math is probability, expected value, discounted sums and repeated updates. RL helped AlphaGo beat Lee Sedol in 2016 and now trains AI reasoning models on checkable math problems.
Facts checked November 22, 2026.
Nobody gave AlphaGo a rulebook of good moves. It got better by playing, winning and losing, millions of times. That style of learning, reinforcement learning, now trains the reasoning AI models that solve olympiad math. The core math is surprisingly friendly: expected value, a little algebra and one clever update rule.
The Five Pieces of Reinforcement Learning
| Piece | Meaning | Example: a game of chess |
|---|---|---|
| Agent | The learner | The AI player |
| State | The current situation | The board position |
| Action | A choice the agent makes | A move |
| Reward | A number saying how good the outcome was | +1 for a win, −1 for a loss |
| Policy | The agent’s strategy: which action in which state | How it picks moves |
Three words are easy to mix up. The reward is the immediate score for one step. The return is the total of rewards over time, usually discounted. A Q-value is the agent’s estimate of the expected return after taking a particular action. The goal is a policy that earns the largest expected return.
Expected Value: Choosing Under Uncertainty
Suppose an agent can pull one of two slot-machine levers:
- Lever A pays 10 points with probability 0.3.
- Lever B pays 4 points with probability 0.9.
If a lever pays nothing otherwise, its expected value is payoff times probability: \(E[A] = 10 \times 0.3 = 3\) and \(E[B] = 4 \times 0.9 = 3.6\). B is better on average, even though A has the bigger prize. The catch is that the agent does not know these numbers at first; it has to estimate them by trying. This is the multi-armed bandit problem, and it shows RL’s central tension:
- Exploit: pick the lever that looks best so far.
- Explore: try other levers in case your estimate is wrong.
A simple rule called epsilon-greedy picks a random lever with probability \(\varepsilon\), say 0.1, and the best-looking lever otherwise. With two levers, a random pick lands on the other lever only half the time, so it is chosen about 5% of the time. For a probability refresher, see how to calculate probability.
Discounting: A Reward Now vs a Reward Later
Rewards often arrive long after the action that earned them, like a chess win many moves later. RL adds up future rewards with a discount factor \(\gamma\) (gamma) between 0 and 1:
\[ G = r_0 + \gamma r_1 + \gamma^2 r_2 + \gamma^3 r_3 + \cdots \]With \(\gamma = 0.9\), a reward of 10 that arrives three steps from now (the \(r_3\) term, counting the immediate reward as \(r_0\)) is worth \(0.9^3 \times 10 = 7.29\) today. A smaller \(\gamma\) makes the agent short-sighted; a \(\gamma\) close to 1 makes it plan ahead. Because \(\gamma^k\) shrinks geometrically, the infinite sum stays finite: with a reward of 1 every step starting now, the geometric series gives \(G = \frac{1}{1 – \gamma} = 10\) for \(\gamma = 0.9\).
The Bellman Equation and Q-Learning
Let \(Q^*(s, a)\) be the expected discounted return from taking action \(a\) in state \(s\) and then acting optimally. Richard Bellman’s key insight is that this value equals the expected immediate reward plus the discounted value of the best next move:
\[ Q^*(s, a) = \mathbb{E}\left[ r + \gamma \max_{a’} Q^*(s’, a’) \,\middle|\, s, a \right]. \]The expectation averages over the possible rewards and next states \(s’\). Q-learning cannot see that average, so it uses one observed step at a time as a sample target.
Q-learning nudges its estimate toward that target after every step, using a learning rate \(\alpha\):
\[ Q(s, a) \leftarrow Q(s, a) + \alpha \left[ r + \gamma \max_{a’} Q(s’, a’) – Q(s, a) \right]. \]Worked example: current estimate \(Q = 2\), reward \(r = 1\), highest Q-value estimate in the next state 4, \(\gamma = 0.9\), \(\alpha = 0.5\). The target is \(1 + 0.9 \times 4 = 4.6\). The error is \(4.6 – 2 = 2.6\), so the new estimate is \(2 + 0.5 \times 2.6 = 3.3\). At a final (terminal) state there is no future, so the target is just \(r\).
A Q-Table You Can Follow by Hand
A tiny corridor: start at A, step right to B, step right again to reach the Goal, which gives reward 10 and ends the episode. Other steps give 0. Use \(\gamma = 0.9\), \(\alpha = 0.5\), and start every Q-value at 0.
| Update | Calculation | Q(A, right) | Q(B, right) |
|---|---|---|---|
| Start | — | 0 | 0 |
| Episode 1, step A → B | \(0 + 0.5(0 + 0.9 \times 0 – 0) = 0\) | 0 | 0 |
| Episode 1, step B → Goal | \(0 + 0.5(10 – 0) = 5\) (terminal) | 0 | 5 |
| Episode 2, step A → B | \(0 + 0.5(0 + 0.9 \times 5 – 0) = 2.25\) | 2.25 | 5 |
| Episode 2, step B → Goal | \(5 + 0.5(10 – 5) = 7.5\) | 2.25 | 7.5 |
The reward at the Goal flows backward one step per episode. With more episodes, Q(B, right) approaches 10 and Q(A, right) approaches \(0.9 \times 10 = 9\). In big problems a table is impossible, so deep Q-networks approximate Q-values with neural networks (see the math of neural networks); other RL methods learn the policy directly.
Selected Milestones in Reinforcement Learning
| Year | System | What RL did |
|---|---|---|
| 2013–2015 | DeepMind’s DQN | Learned dozens of Atari games from screen pixels and scores |
| March 2016 | AlphaGo | Beat Go champion Lee Sedol 4–1. It learned first from human games, then improved by self-play, and chose moves with tree search (method published in Nature in January 2016) |
| 2017 | AlphaZero | Learned chess, shogi and Go from self-play alone |
| 2022 | ChatGPT (RLHF) | Reinforcement learning from human feedback made chatbots more helpful |
| 2024 | AlphaProof | RL with the Lean proof checker. With AlphaGeometry 2, it reached silver-medal standard at IMO 2024 (28 of 42 points), after problems were manually formalized and with up to three days of computation |
| January 2025 | DeepSeek-R1 | RL on checkable math and code answers, starting from a pretrained model. In the version published in Nature in September 2025, R1-Zero’s AIME 2024 pass@1 rose from 15.6% to 77.9% during training (71.0% in the January preprint) |
More: AlphaProof and AlphaGeometry explained.
Why RL Is So Good at Math
Math has something rare: answers that can be checked automatically. A final number either matches or does not; a Lean proof either compiles or does not. That gives a clean reward signal, called reinforcement learning with verifiable rewards. A model tries a problem many times, gets reward 1 for a correct answer and 0 otherwise, and is pushed toward the reasoning that led to correct answers. DeepSeek’s researchers observed the model starting to re-check steps and try other approaches without being told to. A correct final answer does not certify every reasoning step, which is why formal proof checkers like Lean are a stronger reward for proofs. Compare this with RLHF, where the reward comes from human ratings, which are slower, costlier and more subjective. This is a major reason reasoning models improved so fast at olympiad problems between 2024 and 2026 (see AI at the math olympiad).
The Catch: Reward Hacking
An RL agent maximizes exactly the reward you give it, not what you meant. A boat-racing agent that scores points for hitting targets may circle forever instead of finishing the race. A math model rewarded only for the final answer might learn shortcuts that happen to give right answers without valid reasoning. Designing good rewards, and checking the reasoning itself, is one of the hardest parts of RL.
Common Misconceptions
“Reinforcement learning means the AI is rewarded like a pet.” The reward is just a number used in an update formula; nothing is felt.
“RL needs no data.” It needs experience, often millions of trials, plus a well-designed reward.
“RL alone built ChatGPT.” Chatbots are first trained to predict text; RL is a later fine-tuning stage.
Watch: Reinforcement Learning Explained
If you learn better by watching, these videos are worth your time.
Check Your Understanding
Five quick questions to see how much of this has stuck. Pick an answer and press Submit; you will see a short explanation either way.
Answer each question and press Submit to see your result right away.
Frequently Asked Questions
What is reinforcement learning in simple terms?
A way for AI to learn by trial and error: it takes actions, receives numerical rewards and adjusts its strategy to collect as much total reward as possible.
What math is used in reinforcement learning?
Probability and expected value, discounted sums of future rewards, the Bellman equation and update rules such as Q-learning, plus neural networks for large problems.
What is the Bellman equation?
It says the optimal value of an action equals the expected immediate reward plus the discounted value of the best next action: Q*(s, a) = E[r + γ max Q*(s’, a’)].
What is a reward function?
A rule that assigns a numerical score to the outcome of each action, such as +1 for winning a game. The agent tries to maximize the total of these rewards over time.
How did reinforcement learning help AI solve math problems?
Math answers can be checked automatically, so models can be rewarded for correct solutions. DeepSeek reported that RL raised R1-Zero’s AIME 2024 pass@1 from 15.6% to 77.9% in its Nature paper, and AlphaProof used RL with the Lean proof checker.
What is the difference between reinforcement learning and supervised learning?
Supervised learning learns from labeled examples of right answers; reinforcement learning learns from rewards for its own actions, without being shown the right move.
What is reward hacking?
When an agent finds a way to earn high reward without doing what the designers intended, such as exploiting a loophole in a game or a grader.
Keep Exploring
- AlphaProof and AlphaGeometry 2 Explained
- What Is a Neural Network? The Math, Step by Step
- How AI Predicts the Next Word: Probability Explained
Sources and Further Reading
- Sutton and Barto, Reinforcement Learning: An Introduction (2nd ed., MIT Press)
- Silver et al., Mastering the game of Go with deep neural networks and tree search (Nature, 2016)
- Silver et al., Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm (arXiv 1712.01815)
- Ouyang et al., Training language models to follow instructions with human feedback (arXiv 2203.02155)
- DeepSeek-AI, DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning (arXiv 2501.12948; Nature, Sept 2025)
- Google DeepMind: AI achieves silver-medal standard solving International Mathematical Olympiad problems (July 2024)
Related to This Article
More math articles
- Why Teaching Math At Home Is Beneficial For Your Kids?
- Mootness
- Free Grade 7 English Worksheets for Michigan Students
- How to Graph Transformation on the Coordinate Plane: Rotation?
- How to Do Multiple Ways of Fractions Decomposition
- Free Grade 7 English Worksheets for North Dakota Students
- How to Memorize Multiplication Tables: 5 Fun Methods
- Factoring Trinomials Worksheet with Answers
- The Ultimate Praxis Algebra 1 (5162) Course (+FREE Worksheets)
- Agency capture


What people say about "Reinforcement Learning Explained: Math With Examples - Effortless Math"?
No one replied yet.