Euler's Number: e ≈ 2.71828

What it is, where it comes from, and why it appears everywhere in AI.

What is e?

A mathematical constant (like π). The base of natural growth.

e = 2.71828182845904523536...
Irrational (never ends, never repeats)

Where It Comes From — Compound Interest

Question: What happens if you compound $1 at 100% interest continuously?

Compound once (n=1):      (1 + 1/1)¹   = 2.000
Compound twice (n=2):     (1 + 1/2)²   = 2.250
Compound 4 times (n=4):   (1 + 1/4)⁴  = 2.441
Compound 12 times (n=12): (1 + 1/12)¹² = 2.613
Compound 365 times:       (1 + 1/365)³⁶⁵ = 2.715
Compound ∞ times:         → e = 2.71828...

e = lim(n→∞) (1 + 1/n)ⁿ

Step by Step: n=2 (compound twice)

Split 100% into 2 equal parts → 50% each period

Start:           $1.00
After 6 months:  $1.00 × 1.50 = $1.50  (50% of $1.00)
After 12 months: $1.50 × 1.50 = $2.25  (50% of $1.50!)

Extra $0.25 = interest earned ON interest
This is compounding!

n=1: 100% of $1.00 = $2.00
n=2: 50% twice, but second 50% applies to $1.50 = $2.25

Where "Compound Twice" Is Actually Used

Semi-annual ("compound twice") compounding isn't just a textbook idea — it's a real financial standard, especially in bond markets:

• US Treasuries (T-Notes, T-Bonds): the world's largest bond market pays interest twice a year (every 6 months). Yield-to-maturity (YTM) is computed on a compound-twice basis — the global standard spec for bond valuation.
• Corporate bonds & overseas products: many pay coupons every 3 or 6 months; reinvested coupons naturally compound twice (semi-annual) or 4× (quarterly).
• Deposit/loan rate comparison (APR vs EAR): the same nominal rate produces different real returns depending on how often it compounds.

Effective annual rate (EAR) for a 10% nominal rate:

Compounding            Effective rate   Interest on $10,000
Annually    (n=1)      10.00%           $1,000.0
Twice       (n=2)      10.25%           $1,025.0   ← semi-annual
Monthly     (n=12)     10.47%           $1,047.0

Everyday savings usually compound monthly; bond markets and global finance use semi-annual ("compound twice") as the standard.

Why It Converges (Doesn't Go to Infinity)

Extra gain from each additional split DECREASES:

n=1 → n=2:     +$0.250
n=2 → n=4:     +$0.191  ← less
n=4 → n=12:    +$0.172  ← even less
n=12 → n=365:  +$0.102  ← smaller
n=365 → n=∞:   +$0.003  ← tiny

More splits help, but each additional split helps LESS.
Total converges to e = 2.71828, never reaches 3.

The intuition: why does the extra gain shrink?

As you split more, the interest arrives sooner, but each period's rate becomes much smaller — and the rate shrinks faster than the added frequency helps.

• n=1 → n=2: at 6 months you get a big $0.50 bonus early, which then earns 50% over the remaining 6 months → a large extra 0.50 × 0.50 = $0.25.
• n=2 → n=4: at 3 months you only receive $0.25, and multiplying that small amount by a small 25% rate produces only tiny additional interest.

In short: the "time for interest to make interest" grows, but the "seed principal that makes new interest" gets too small. This is the same idea as diminishing marginal returns in economics — which is exactly why the total is capped at e.

A Little History: Bernoulli & Euler

In 1683, Swiss mathematician Jacob Bernoulli studied this exact "$1 at 100%, split n times" problem and noticed the total didn't blow up to infinity — it settled near 2.718. That was humanity's first glimpse of e.

As n → ∞, two forces fight: the inside (1 + 1/n) → 1 (and 1 to any power is 1), while the exponent n → ∞ (repeated multiplication grows). The tug-of-war balances at 2.71828...

Later, Leonhard Euler recognized its importance, named it e, proved it irrational, and gave the series:

e = 1 + 1/1! + 1/2! + 1/3! + 1/4! + ... ≈ 2.71828

Why "natural"? Though discovered through money, e governs all natural growth and decay — bacterial reproduction, radioactive decay, population growth — because nature grows continuously, and continuous compounding's limit is e.

The MAGIC Property of e

e^x is the only function whose derivative equals itself:

f(x)  = eˣ
f'(x) = eˣ    ← same!

No other base does this:
  f(x) = 2ˣ  → f'(x) = 0.693 × 2ˣ  (not itself)
  f(x) = 3ˣ  → f'(x) = 1.099 × 3ˣ  (not itself)
  f(x) = eˣ  → f'(x) = 1.000 × eˣ  (exactly itself!)

e is the unique base where value = slope at every point.

Where e is Used in AI

Each formula below uses e for a specific reason:

• Sigmoid — σ(x) = 1 / (1 + e−x)
  Smooth 0 → 1 squashing, clean derivative.
• Softmax — exi / Σj exj
  Turns scores into probabilities (always positive, sums to 1).
• Cross-entropy loss — −ln(ŷ)
  Natural penalty for wrong predictions (log base e).
• Normal distribution — e−x²/2
  The bell-curve shape.
• Learning-rate decay — lr0 × e−decay·t
  Smooth decay toward zero, never exactly zero.

Softmax Example

Scores: [2.0, 1.0, 0.1]

e²∙⁰ = 7.389
e¹∙⁰ = 2.718
e⁰∙¹ = 1.105
Sum = 11.212

Probabilities:
  7.389 / 11.212 = 0.659 (66%)
  2.718 / 11.212 = 0.242 (24%)
  1.105 / 11.212 = 0.099 (10%)
  Sum = 1.000 ✓

Why eˣ?
  - Always positive (valid probability)
  - Amplifies differences (sharp predictions)
  - Clean derivative for backpropagation

Why e Makes AI Math Clean

With e:    d/dx(eˣ) = eˣ           (simple!)
With 2:    d/dx(2ˣ) = 0.693 × 2ˣ  (ugly constant)

With ln:   d/dx(ln x) = 1/x        (simple!)
With log₂: d/dx(log₂ x) = 1/(x·0.693) (ugly!)

Since AI = gradient descent = millions of derivatives,
using e eliminates ugly constants everywhere.

Bonus: Exponential Growth & "Hwang's Law"

The same e-driven exponential growth shows up in hardware, not just interest:

• Moore's Law: transistor count doubles roughly every 18–24 months.
• Hwang's Law (Hwang Chang-gyu, 2002): memory density doubles every year.
• Huang's Law (Jensen Huang, 2020): AI chip performance more than doubles every 2 years — and even this looks conservative next to today's AI compute growth.

The link to e: like compound interest (interest breeds interest), technology breeds technology — this year's better chip designs next year's stronger chip. "Growth proportional to current size" is exponential growth, whose exact mathematical language is e.

Doubling time (Rule of 72): for continuous growth, the time to double satisfies rate (r) × time (t) ≈ ln(2) ≈ 0.693 (natural log, base e). Finance turns this into the "Rule of 72"; chip engineering uses it to predict generational cycles.

Bonus: Exponential Growth & AI

Does AI's development speed relate to e? Yes — directly. AI progress is one of the steepest exponential curves in tech history, and any exponential is naturally written with base e: a quantity growing at rate r follows value(t) = value₀ × e^(r·t).

How fast is AI actually growing?

Trend                         Doubling time      Source
Moore's Law (transistors)     ~24 months         classic baseline
AI training compute (deep-    ~6 months          Epoch AI / deep-learning era
  learning era, 2012+)
AI training compute (peak)    ~3.4 months        OpenAI "AI and Compute"
LLM "capability density"      ~3.5 months        Nature Machine Intelligence 2025

Since 2012, training compute grew ~300,000x.
(A 2-year doubling would give only ~7x over the same span.)
That's roughly 0.5-1.0 orders of magnitude PER YEAR.

For comparison: doubling every ~3.4 months means about 3.5 doublings per year, i.e. roughly 2^3.5 ≈ 11× annually — far faster than Moore's Law.

Why the connection to e is more than a formula

The deep reason is feedback compounding, the same engine behind interest:

Compound interest:  interest earns interest       → capped by e
Chips (Hwang's Law): chips design better chips     → exponential
AI:                  data → better models → better
                     data/tools → even better models  → exponential

Better models help generate better training data, write better code, and design better chips — which train the next, stronger models. "Growth proportional to how much you already have" is precisely the property that produces e^(r·t), and it's why the natural log (base e) is the right tool to measure AI "doubling times."

The catch: diminishing returns (again)

Just like extra compounding splits help less and less, AI scaling laws show diminishing returns: each 10× more compute buys a smaller accuracy gain. Pure brute-force scaling alone would eventually demand unrealistic compute. Sustained exponential progress therefore depends on efficiency doubling (better algorithms + hardware) keeping pace — the same tug-of-war between "more frequency" and "smaller marginal gain" that caps compound interest at e.

Money compounds and converges to e. AI compounds through a data→model→data feedback loop — explosively fast (months, not years, per doubling), but still shadowed by the same diminishing-returns math that defines e.

Summary

Why e exists:     Natural limit of continuous compounding
Why it's special: d/dx(eˣ) = eˣ (derivative = itself)
Why AI uses it:
  1. Clean derivatives → fast backpropagation
  2. Always positive → valid for probabilities
  3. Amplifies differences → sharp predictions
  4. Natural growth/decay modeling

Same math, three worlds:
  Money   → interest breeds interest
  Chips   → technology breeds technology (Hwang's Law)
  AI      → data breeds performance, performance breeds smarter AI