Resources

Vanity Analysis Documentation

From methodology to research.

Vanity Analysis is designed to transform a trader’s methodology into a structured research specification and evaluate it within a simulated research environment. This documentation explains how that process works, section by section, and — just as importantly — where its limits are. Vanity is substantially more than a conventional backtester, and understanding why is the fastest way to read its reports well.

01 — Describe your methodology

You begin by describing how you trade, in plain language. There is no scripting language to learn and no code to write. A description can include the parts that define a setup:

  • the setup and its context;
  • the sequence of events that must occur;
  • the confirmation you wait for;
  • the entry;
  • the stop and the target;
  • the risk per trade;
  • the market, session, and timeframe you trade.

Vanity is built to understand the relationships and the order in a description, not merely to match keywords. “A sweep, then displacement, then a return into the imbalance” is a sequence, and it is read as one.

02 — Strategy interpretation

Vanity identifies the trading concepts in your description and the relationships between them, and assembles them into an ordered specification. Consider a sequence such as:

liquidity sweep → displacement → imbalance → retracement → entry

Detecting those five concepts somewhere in the data would not faithfully represent the strategy. What makes it this strategy is that they occur in order, each conditioned on the one before it. Vanity preserves that order and those conditions, so the trade that fires is the trade you described — not a coincidence of five ideas appearing on the same bar.

03 — Modelability: two kinds of confidence

Vanity distinguishes two questions that are easy to conflate, and reports them separately.

Parser confidence

How confidently Vanity understood your description.

Simulation confidence

How confidently Vanity can reproduce the described behaviour using the market information it holds and the modelling concepts it has validated.

These are independent. A strategy can be understood clearly and still carry limited simulation confidence — for example, when part of it depends on information the platform does not have, or on discretion that cannot be fully specified. Keeping the two apart is what lets Vanity be honest about the difference between “we understood you” and “we can faithfully model this.”

04 — When Vanity refuses a strategy

Vanity may decline to analyse a strategy. This is a deliberate research safeguard, not a limitation to apologise for. A strategy may be refused when it contains, for example:

  • insufficient information to define a testable setup;
  • a requirement for data the platform does not hold;
  • rules that contradict one another;
  • a condition that cannot be modelled against price;
  • ambiguity too great to resolve responsibly.

In these cases Vanity does not invent the missing assumptions simply to produce a number. Refusing to model what cannot be modelled honestly is a feature: it is what keeps the results you do receive trustworthy.

05 — The research environment

A single historical path can tell you what happened once. It cannot tell you everything that could have happened. This is where Vanity goes beyond a conventional backtest.

Vanity uses historical market data to characterise the behaviour of the market being researched — its volatility, its return characteristics, the shape of its distributions, and how these have shifted across historical regimes. Those measured characteristics are then used to construct simulated market environments conditioned on that historical behaviour. Intraday structure is built from a daily framework combined with a measured intraday shape carrying real clock times, which is what allows a session filter to be an actual filter.

To be precise about what this is and is not: these are simulated environments conditioned on measured historical market behaviour. They are not replays of real historical markets, they do not predict future markets, and they are not an unlimited set of alternate realities. They are a principled way to examine a strategy across many plausible paths that retain the statistical character of the market.

Evaluating a strategy across such an environment lets the research examine:

  • how the strategy behaves when the sequence of outcomes changes;
  • how it behaves under different volatility and regime conditions;
  • how robust the equity curve is to different market paths;
  • how sensitive performance is to a single, particular historical sequence.

06 — Simulation

The structured strategy specification is evaluated bar by bar within the research environment. Each trade is executed against generated price, and an execution cost — representing spread, commission, and slippage — is applied to every trade as a fraction of the risk distance, scaled to the strategy and to the market era being simulated. This is a modelled cost designed to keep results honest; it is not a claim of universal tick-by-tick execution. Where a strategy leaves direction, a stop, or a target unspecified, Vanity models a defensible default and discloses that it did so in the report.

07 — Statistical research

The report brings together the measures that matter, and explains what each is telling you:

  • Net return — how the account changed over the research period.
  • Win rate — the share of decisive trades that were profitable.
  • Profit factor — gross profit divided by gross loss; above 1 means the wins outweighed the losses.
  • Expectancy — the average result per trade, in units of risk (R).
  • Drawdown — the depth and duration of declines from an equity peak.
  • Trade distribution — how outcomes are spread, and whether results depend on a few outliers.
  • Monthly performance — consistency across the calendar.
  • Equity curve — how capital developed over time.
  • Risk of ruin — the modelled chance of losing the account under the tested assumptions.
  • Probability of profit — the share of simulated paths that ended profitable.
  • Monte Carlo outcomes — the distribution of results when the order of trades is varied (see below).
  • Confidence — parser and simulation confidence, kept distinct.
  • Regime behaviour — how the strategy fared across different market conditions.

08 — Monte Carlo

A single historical equity curve can be misleading: it is one ordering of outcomes among many that were equally possible. Monte Carlo analysis re-samples and re-orders the realised results to study the distribution of outcomes a strategy could plausibly produce, rather than the one path it happened to take. It is a probabilistic research and stress tool. It does not predict the future; it characterises the range and shape of what the evidence supports.

09 — The verdict

Vanity’s verdict reflects the whole research picture — consistency, robustness, risk, sample depth, and uncertainty — rather than a single metric. The current verdicts are:

  • Statistically Viable — the evidence supports greater confidence in the methodology under the research assumptions.
  • Marginal Edge — there are signs of an edge, but it is slight or not yet well supported.
  • Requires Optimisation — the methodology shows potentially useful characteristics but needs refinement or further research.
  • No Reliable Edge — the available research does not provide sufficient evidence of a reliable edge.

These are research conclusions about a strategy. They are not investment recommendations.

10 — Data coverage and limitations

Being exact about data is part of Vanity’s credibility. The platform supports six instruments, calibrated at three levels of directness:

  • Directly calibrated from their own historical datasets: NQ (Nasdaq 100 futures, up to ~26 years), GC (gold futures, up to ~50 years), and CL (crude oil futures, up to ~25 years).
  • Derived: MNQ is calibrated from NQ — the micro contract tracks the same underlying index with the same statistical character.
  • Analogue: ES and MES (S&P 500 futures and the micro) do not have their own dataset. Their regime and session structure are taken from NQ with volatility rescaled to the S&P’s long-run level, up to ~23 years. These figures are indicative rather than measured, and are labelled as such.

Other characteristics and limitations you should keep in mind:

  • Available historical depth differs by instrument, as noted above, and the usable depth for very fine timeframes is further bounded by the constraints below.
  • Intraday behaviour is built from a daily framework combined with a measured intraday shape; it is not a stored tick-by-tick recording of every historical day.
  • Session filtering applies where the timeframe carries clock times. A one-day bar spans the whole session, so a session filter cannot be applied at daily resolution, and Vanity says so when that happens.
  • When a description does not specify direction, Vanity models a defensible assumption and discloses it.
  • Market holidays and exchange closures are not individually modelled as calendar events.
  • The simulation runs in your browser. Very long histories at very fine resolutions are limited by practical memory and time; the platform caps depth accordingly rather than producing an unreliable result.

11 — Reading the report

The report is organised so that it can be read as a whole:

  • Executive verdict — the headline research conclusion.
  • Rating and confidence — the research rating, evidence strength, and stability.
  • Performance — the core return and outcome statistics.
  • Equity curve — how capital developed.
  • Drawdown — the depth and persistence of declines.
  • Monte Carlo — the distribution of possible outcomes.
  • Risk — drawdown, risk of ruin, and related measures.
  • Calendar / monthly performance — consistency over time.
  • Trade history — the individual simulated trades that produced the result.
  • Commentary — a written interpretation of the evidence.

The most reliable way to read a Vanity report is as a body of evidence. A single strong metric rarely settles the question; the picture across return, risk, distribution, and stability is what warrants confidence — or withholds it.

12 — Important limitations

  • Historical and simulated results are not guarantees of future performance.
  • Markets change, and a strategy that performed under past conditions may not perform under future ones.
  • Results depend on the simulation’s modelling assumptions and on the data available.
  • Not all discretionary behaviour can be perfectly reproduced.
  • Information the platform does not hold may prevent a strategy from being researched.
  • Because the simulation runs in your browser, there are practical limits on the depth and resolution of any single analysis.
  • Analogue instruments are calibrated indirectly and are not identical to directly calibrated ones.
  • Past behaviour, however thoroughly measured, does not guarantee future behaviour.

None of this diminishes the research. Being explicit about what a result rests on is what makes it worth acting on. For the legal terms governing use of the platform, see the Terms of Service; for how we handle information, see the Privacy Policy.

Back to home