Market data fundamentals

Every strategy starts with market data — a time series of prices. Quant Buffet's lab uses daily adjusted close for liquid ETFs, loaded through load_daily_prices() and cached on disk so re-runs are fast and reproducible.

Core vocabulary

What a panel actually looks like

A panel is just a spreadsheet with dates down the side and tickers across the top. Illustrative shape:

Notice the NaN cells. XLC did not exist before June 2018, so no amount of data cleaning can invent a price for it in 2015. This is not a defect in the data — it is a real constraint on any strategy that wants to trade communication-services stocks over a long history.

Reading a price panel in code

# Inside make_on_day — prices is already loaded for your ASSETS
cols = [c for c in ASSETS if c in prices.columns]
close = prices[cols]

daily_return = close.pct_change()                       # simple returns
sma_200 = close.rolling(200, min_periods=200).mean()    # trend filter
mom_12m = close.pct_change(252)                         # 12-month momentum
vol_20d = daily_return.rolling(20).std() * (252 ** 0.5) # annualised vol

# Never start trading before every input exists for every asset
ready = close.dropna().index[0] if not close.dropna().empty else None

The three biases that ruin beginner backtests

1. Survivorship bias

If you build a universe from *today's* winners, you have quietly deleted every failure. Enron, Lehman Brothers, Washington Mutual, Kodak and Bear Stearns were all respectable S&P 500 members before they were not. A strategy tested only on companies that survived to 2026 inherits a tailwind that no real investor ever received. Broad ETFs like SPY reduce this problem because the index itself replaces failing members — the ETF survives even when its constituents do not.

2. Look-ahead bias

This is the most common beginner error and it is nearly invisible. Any statistic computed over the *whole* sample leaks the future into the past:

# WRONG — full-sample mean includes prices from the future
z = (close - close.mean()) / close.std()

# RIGHT — only data available up to each date
roll_mean = close.rolling(60).mean()
roll_std = close.rolling(60).std()
z = (close - roll_mean) / roll_std

The wrong version routinely produces Sharpe ratios above 3 and equity curves that look like a straight line. If your first backtest looks that good, suspect leakage before celebrating.

3. Corporate actions and unadjusted prices

Apple split 4-for-1 in August 2020; Nvidia split 10-for-1 in June 2024; Tesla split 3-for-1 in August 2022. On raw price data, each of those looks like a catastrophic overnight crash of 75%, 90%, and 67% — a momentum strategy on unadjusted prices would sell every one of them at exactly the wrong moment. Dividends matter just as much in aggregate: SPY distributes roughly 1.3–2% per year, and ignoring that over a 20-year test is the difference between compounding at about 8% and about 6.5% — on $10,000 that is roughly $46,600 versus $35,200.

Real example: when the price series is not the asset

Two whitelisted commodity ETFs are famous cautionary tales, and both are worth loading in the IDE precisely because they misbehave.

History start dates you will actually hit