Prebuilt Datasets

Data

Similar to bundles, a few prebuilt datasets are available off-the-shelf. They make it possible to analyze execution quality and liquidity across venues without building a replay pipeline, even for teams new to Solana's programming model.

More specifically, these datasets can answer questions like: where a venue's quotes diverge from what aggregators see, how deep each pool actually is at a given size, and which swaps a venue lost and at what price.

Dataset	Description	Download Path
Onchain Depths	Liquidity depth for each venue, computed directly from onchain pool state.	datasets/depth/onchain/v2/
Aggregator Depths	Liquidity depth for the same venues as quoted by Jupiter Metis.	datasets/depth/diagnostic/v2/
Executed + Hypothetical Fills	Fills that were executed, alongside fills each venue would have received had it won the swap.	datasets/estimated-fills/v2/

Schemas

All datasets are stored as Parquet, partitioned by date, market, and venue. The path pattern for each dataset is in each section.

Token amounts are shown in their base units (e.g. SOL is 9 decimals, BTC is 8, USDC is 6).

Onchain Depths

datasets/depth/onchain/{slot|oracle}/v<version_number>/date=<YYYY-MM-DD>/market={token_pair}/venue={venue}/part-<start_slot>-<end_slot>.parquet

Column	Note
slot
input_token	ticker (e.g. SOL)
output_token
in_amount
out_amount
price_impact_bps	rate compared against rung 0
trigger	by grid: always once per slot by event: can be intra-slot or once every few slots

Aggregator Depths

datasets/depth/router/slot/v<version_number>/date=<YYYY-MM-DD>/market={token_pair}/venue={venue}/part-<start_slot>-<end_slot>.parquet

Column	Note
slot
router_slot	Jupiter Metis' ingest slot when it responded (this may lag behind slot)
pool	address for the pool
input_token	ticker (e.g. SOL)
output_token
in_amount
out_amount
usd_value	in_amount in USD
quote_error	why Metis wasn't reachable
error	why Metis returned no price (e.g. pool didn't have enough liquidity)

Executed + Hypothetical Fills

datasets/estimated-fills/v1/date=<YYYY-MM-DD>/market={token_pair}/part-<start_slot>-<end_slot>.parquet

Column	Description
slot
block_time	timestamp
original_signature	signature of the original swap
tx_index	where in the block the transaction was positioned
ix_index	where in the transaction the instruction was positioned
swap_mode	exact in: input amount is fixed and output varies (e.g. sell exactly 10 USDC) exact out: output amount is fixed and input varies (e.g. receive exactly 1 SOL)
input_mint	ticker (e.g. SOL)
output_mint
amount	for the original swap
l1_fill	for the original swap
l1_program	address of the DEX that filled the original swap
l1_venue	human-readable name of the DEX
input_amount	matches l1_fill if swap_mode is exact out matches amount if swap_mode is exact in
output_amount	matches l1_fill if swap_mode is matches amount if swap_mode is
original_failed	whether the original swap succeeded or failed
venue
estimate_fill	simulated venue's version of l1_fill
replay_input_amount	simulated venue's input_amount
replay_output_amount	simulated venue's output_amount
outcome	result of the simulated swap either filled, reverted, unattributed, unsimulatable
category	revert reason for simulated swap

Access

Datasets are hosted on S3 as Parquet files. Request credentials from the authentication URL below, then access the bucket directly, as shown in the example code.

	URL
Dataset Authentication	https://simulator.termina.technology/datasets/credentials
S3 Bucket	s3://data.termina.technology

example_access.py

import boto3
import requests

API_KEY = ""
AUTH_URL = "https://simulator.termina.technology/datasets/credentials"
DATA_URL = "data.termina.technology"

# get access credentials
creds = requests.post(AUTH_URL, headers={"X-API-Key": API_KEY}).json()
print(f"creds: {creds.keys()}\n")

# set up an authenticated s3 client
s3 = boto3.client(
    "s3",
    aws_access_key_id=creds["access_key_id"],
    aws_secret_access_key=creds["secret_access_key"],
    aws_session_token=creds["session_token"],
    region_name=creds["region"],
)

# list every parquet file in the datasets
resp = s3.list_objects_v2(Bucket=DATA_URL, Prefix="datasets/")
files = [obj["Key"] for obj in resp.get("Contents", [])]
print(files)