Blog 1: The Candy Store Dilemma - How Our Mind Makes Choices

Scene 1: The Child in the Candy Store

Imagine a little child standing in front of a candy store. On the left, there's a shiny red lollipop waiting to be unwrapped right now. On the right, there's the promise of a big chocolate bar - but it will only be available tomorrow.

What should the child do? Grab the lollipop and enjoy the instant sugar rush, or hold back and wait patiently for the richer chocolate reward tomorrow?

This tiny decision may feel trivial, but hidden behind it is a universal question: Why do humans often choose short-term pleasures over long-term benefits?

We face this same dilemma every day:

• Should I eat that extra slice of cake now, or stick to my diet for future health?
• Should I binge-watch Netflix till 3 AM, or sleep early to feel fresh tomorrow?
• Should I procrastinate today and finish the work later, or get it done right now for peace of mind tomorrow?

This constant tug-of-war between now vs later is where the world of decision-making under uncertainty begins.

Scene 2: Decisions Are Not Just Random

Psychologists and AI researchers both agree - decisions are not just impulsive flips of a coin. There's an underlying structure in the way we choose.

Our brains evaluate:

1. Current state (Am I hungry? Am I tired? Am I stressed?)
2. Available actions (Should I eat, wait, scroll Instagram, or study?)
3. Rewards (Instant pleasure now vs better outcomes later).
4. Uncertainty (Will I really get that chocolate tomorrow? What if it's gone?).

This structure is mathematically captured in something called a Markov Decision Process (MDP).

Scene 3: Breaking Down the MDP

An MDP is like a blueprint for decision-making. It has four main ingredients:

1. States (S) The situation you are in right now.
   • Example: Hungry, Not hungry, Craving sugar, Feeling sleepy.
2. Actions (A) The possible moves you can make.
   • Example: Buy the lollipop, Wait for chocolate, Go home without candy.
3. Rewards (R) The feedback you get for your choice.
   • Example: Lollipop → quick sugar rush (reward = +5).
   • Chocolate tomorrow → bigger, richer taste (reward = +10).
4. Transition Probabilities (P) The uncertainty of moving from one state to another after taking an action.
   • Example: If I wait, what's the probability that the chocolate will actually be available tomorrow? (Maybe only 70%).
   • If I buy the lollipop now, tomorrow I'll definitely be in a "no chocolate" state (100% probability).

Mathematically, an MDP is written as a 4-tuple:

$$\text{MDP} = (S, A, R, P)$$

Where:

• \(S\) = Set of all possible states.
• \(A\) = Set of all possible actions.
• \(R(s,a)\) = Reward function, the payoff of action $a$ in state $s$.
• \(P(s'|s,a)\) = Transition probability, i.e., the likelihood of reaching a new state $s'$ from state $s$ after taking action $a$.

Scene 4: Expected Rewards - The Child's Calculation

Now here's the hidden calculation the child's brain (and ours!) makes subconsciously:

$$\text{Expected Reward} = \sum_{s'} P(s'|s,a) \times R(s,a)$$

Translated to the candy store:

• Lollipop Now:
  • Reward = +5 (guaranteed).
  • Probability = 100%.
  • Expected Reward = \(1.0 \times 5 = 5\).
• Chocolate Tomorrow:
  • Reward = +10 (bigger).
  • Probability = 70% (maybe it's sold out tomorrow).
  • Expected Reward = \(0.7 \times 10 = 7\).

So mathematically, the chocolate looks better because its expected reward = 7, compared to 5 for the lollipop.

But psychology is rarely that simple…

Scene 5: The Psychology Twist - Why We Don't Always Wait

If the chocolate is mathematically "better," why do so many children (and adults) still grab the lollipop right away?

That's where instant vs delayed gratification comes in. Our brains are wired to discount future rewards. The longer we have to wait, the less valuable that future reward feels.

This is why people smoke despite knowing health risks. Why students procrastinate even though they know it will cause stress tomorrow. Why investors sell early instead of waiting for bigger returns.

Our decision-making is not purely rational. It's emotional, time-biased, and influenced by uncertainty.

Scene 6: Wrapping Up - A Glimpse of What's Next

The candy store dilemma teaches us a powerful truth: our brains are like little MDP solvers, weighing states, actions, rewards, and probabilities - but often skewed by our impatience.

In this blog, we've scratched the surface of how MDP explains everyday decisions. In the next part, we'll dive into one of the most famous psychology experiments ever conducted:

The Marshmallow Test - where children faced the ultimate test of patience, and researchers unlocked deep insights into human self-control and success in life.

Takeaway: Next time you choose Netflix over sleep, or junk food over diet, remember - your brain just solved an MDP, and you picked the lollipop.