Statistics / Methods

Noise is not a bug

Illustrative mathematical study for Noise is not a bug

Illustrative mathematical surface · Research note

A simulation-first introduction to sampling variability, false discoveries, and the value of a good null model. This illustrative research note sets out an approach, its assumptions, and the questions left open.

Go deeper

Research
repositories

$19 / month

Get the notebooks, modeling code, and supporting datasets behind the videos as they become available.

  • Full research archive
  • New published projects
  • Reproducible experiments

Membership opens when the first repository files are ready. No payments are being collected.

Recurring monthly subscription · Cancel anytime
Repository files: Coming soon

Randomness deserves a baseline

A pattern can appear meaningful because we looked at enough possibilities. Simulation provides a practical way to ask how often an equally interesting pattern appears when the proposed mechanism is absent.

Build a null model

Choose which properties to preserve. Randomly shuffling observations destroys time dependence, which may make it a poor null for a financial series. Block resampling can preserve some local structure, though it introduces its own assumptions.

import numpy as np
rng = np.random.default_rng(42)
null_returns = rng.normal(0, 1, size=(1000, 252))
max_sample_mean = null_returns.mean(axis=1).max()

This synthetic example illustrates selection over many random series. It is not a simulation of realistic asset returns.

Methodology

Specify the test statistic first. Run the full selection procedure on each simulated dataset, including the search over variants. Compare the selected real statistic against selected null statistics, rather than against a single untouched benchmark.

What this teaches

The more opportunities we give randomness to impress us, the more carefully we should evaluate the resulting winner. Report uncertainty and the number of alternatives considered.

Limitations

A null simulation is only as relevant as its assumptions. Heavy tails, nonstationarity, and dependence deserve explicit sensitivity checks. This note reports no significance test on real market data.

Explore all notes