Measuring Execution Quality on Solana

Measuring execution quality means comparing the price you realised against a benchmark you fixed before the trade, then dividing all costs by the size you actually acquired rather than the size you intended. Two numbers carry the result: the price gap on filled orders and the share of attempts that filled. Reporting either one alone will mislead you.

Nearly every execution argument on Solana comes down to an undeclared definition. One person is quoting the gap against the interface quote, another against pool mid, one is silently dropping reverted transactions, another is averaging over a session that happened to be quiet. Fix the definitions first and the disagreements usually evaporate. This page fixes them in a template you can copy.

What execution quality means

Execution quality is not price alone and it is not reliability alone. A configuration that gets a superb price on the one attempt in three that survives has poor execution quality, because the two attempts that failed still cost fees and left your intended position unfilled. A configuration that fills every time at a wide price also has poor execution quality, for the obvious reason.

The honest formulation combines them. Take every attempt you made towards a given intention, sum every cost incurred including the fees burned on reverts, and divide by the quantity actually acquired. That single figure, cost per unit filled, is difficult to game. Alongside it, report the fraction of intended size that never got done, because that number carries the risk the cost figure hides.

Choosing a benchmark and living with it

A benchmark is the reference price you compare against. Four are defensible on Solana, and each one grades a slightly different question.

Arrival price. The pool mid price at the moment you decided to trade. This grades the whole path, including the quoting layer, which is exactly why it is the most honest default: it is not produced by the system being graded.

Quoted price. The expected output shown to you before signing. Easy to capture and useful for isolating the timing component, but it flatters any path that quotes conservatively, since a pessimistic quote makes every fill look like an improvement.

Post-trade mid. The pool mid immediately after your trade. Useful for separating permanent from temporary impact, misleading as a headline benchmark because your own order moved it.

Interval average. A volume-weighted or time-weighted average over the window in which you were working an order. Appropriate when you deliberately spread execution over time, meaningless for a single-shot swap.

Computing realised slippage from a transaction

A confirmed Solana transaction contains everything needed to reconstruct what you actually got. The reconstruction is arithmetic, not inference, which is what makes it worth doing consistently rather than eyeballing an interface summary.

1. Capture the benchmark before signing. Store the reference price and, ideally, the pool reserves it came from. A measurement without a stored pre-trade reference cannot be repaired afterwards.
2. Read the token balance changes. A confirmed transaction exposes token balances before and after execution for every account it touched. The delta on your own account gives the exact amounts sent and received.
3. Work in raw units first. Compute received divided by sent using integer base units, and apply decimals only at the end. SOL carries nine decimals and SPL tokens vary; mixing conventions mid-calculation is the most common error in this exercise.
4. Record the fees separately. The transaction fee and the compute units consumed are both visible. Keep them out of the price gap and in their own columns, because they scale with attempts while the gap scales with size.
5. Subtract the known pool fee. Pool fee tiers are published parameters. Removing the fee isolates the part of the gap driven by your size and by timing, which is the part your configuration can influence.
6. Attribute the remainder. Compare the leftover gap to the impact you should have paid given the reserves you stored. What is left after that is the timing component, described mechanically in the piece on why a quote and a fill diverge.

Landing rate and the denominator problem

Landing rate is the share of submitted transactions that end up executed in a block. It is the denominator that most execution reports quietly omit. Two configurations with identical price gaps and different landing rates are not equally good, and the difference does not show up anywhere in a price-only report.

There are three separate outcomes to count, not two. A transaction can land and succeed, land and revert, or never land at all. The middle case still costs the network fee, because the fee is charged for processing regardless of whether the instruction succeeded. The third case costs nothing on chain but costs you the fill, and during congestion it is often the dominant failure mode.

Note what the example does not do: it does not declare a winner. A measurement framework that always produces a winner is usually smuggling in an assumption about how much an unfilled clip is worth. State that assumption separately and the comparison stays honest.

The measurement template

Every row below is one column in a flat log with one line per attempt, not per fill. Attempts that never landed get a row too, with the price fields empty. This is the schema the rest of this page assumes.

Fourteen columns is not a burden; it is roughly what a single log line already contains if you are storing transaction responses at all. The discipline is writing the row for the attempts that failed, since those are precisely the rows a flattering report loses.

Comparing two execution paths

The point of measurement is usually a decision: keep the current path or switch. That decision is a controlled comparison, and controlled comparisons on a live chain have exactly one enemy, which is that market conditions change faster than you can gather a sample.

The defence is interleaving. Instead of running path A for an hour and path B for the next hour, alternate clip by clip on the same pair within the same session, with matched sizes. Each adjacent pair of clips then shares almost all of its market conditions, and the difference between them is much closer to a clean read on the path. Compare paired differences rather than group averages, and report how many pairs you collected.

• Same pair, same direction, same clip size on both paths.
• Alternating order, and alternate the starting path across sessions so one path does not always trade first.
• One benchmark, declared before the first clip.
• Both paths logged with the same template, including failed attempts.
• Pre-registered stopping rule, so you do not stop the moment your preferred path is ahead.
• Report the paired differences and their spread, not just the mean.

Choosing the second path is the practical obstacle: you need a genuinely different execution mechanism to compare against, not the same router with a different tolerance. Manual clip-by-clip trading is one option; a console that routes orders across Solana venues on a schedule is another, and Solana Volume Bot Pro is one such external console you can point at the same pair to produce a second series of fills for the log. Whatever you use as path B, the template and the interleaving rule are what make the comparison mean anything.

How much data is enough

There is no universal number, but the shape of the answer is knowable. You need enough paired observations that the difference you care about is comfortably larger than the spread of the differences you observe. If two paths differ by a few basis points and your paired differences swing by tens of basis points, a small sample will produce a confident-looking answer that reverses next week.

Three practical consequences follow. First, prefer a big effect to a big sample: test changes likely to move execution by a lot, such as route shape or clip size, before testing changes likely to move it a little. Second, hold the pair fixed, because pair-to-pair variation dwarfs path-to-path variation. Third, record the session, since a comparison run entirely inside one unusual hour describes that hour.

That rule also governs how you evaluate any Solana volume bot platform you are considering as path B. Test it against a change large enough to show up in twenty paired observations, such as route shape or clip size, rather than trying to resolve a two-basis-point difference you will never separate from the noise in your own sample.

Reporting the distribution, not the average

Execution outcomes are asymmetric. The good tail is bounded by how favourably the pool can move in your window; the bad tail is bounded only by your minimum output. An average compresses both into one number and hides the shape that actually determines whether a configuration is survivable.

Report at minimum the median gap, the worst decile, the landing rate and the unfilled share of intended size. Median rather than mean, because a single extreme fill drags a mean around. Worst decile rather than maximum, because the maximum is a sample of one. Landing rate and unfilled share because they are the denominators everything else depends on, and they are also the two figures that a price-only report is structurally incapable of showing.

When you fold these numbers back into money, do it against the full cost stack rather than the price gap alone, since fixed lamport costs and size-scaled costs move in opposite directions as clip size changes. That whole stack is laid out line by line in the execution cost breakdown.

Seven ways a measurement lies to you

Survivorship. Dropping reverted and never-landed attempts makes an aggressive configuration look excellent. Every attempt gets a row.

Benchmark drift. Capturing the reference at slightly different points for the two paths, for instance quote time for one and decision time for the other, introduces a bias larger than most effects you are testing.

Self-contamination. Using post-trade mid as the benchmark grades you against a price your own order created, which systematically understates impact.

Decimal errors. Applying decimals before the division, or assuming every token has the same precision, produces gaps that are wrong by orders of magnitude and look plausible.

Session cherry-picking. Comparing a path measured in a quiet session against one measured during contention is a measurement of the sessions, not of the paths.

Optional stopping. Watching a running comparison and ending it when the preferred path is ahead is guaranteed to produce a winner regardless of the truth. Fix the stopping rule first.

Unit confusion in the summary. Reporting some costs in SOL, some in basis points and some in token units, then adding them. Convert everything to basis points of notional, or to SOL, once, and state which.

None of this requires sophisticated tooling. A flat log with the fourteen columns above, an interleaved test design and a declared benchmark will settle nearly every execution question you are likely to have. The rest of the cost and quality section deals with what to do once the measurement tells you where your cost actually is.

Frequently asked questions