Your first dataset¶
This notebook builds a plotsim dataset from scratch — no bundled template. You will:
- Describe a domain (a coffee chain's customers) with a single
create(...)call. - Generate CSV files and a
manifest.jsonground-truth payload. - See the trajectory-first contract in action: pick one entity, read its trajectory from the manifest, and watch a metric column track the trajectory position month by month.
- Try three single-cell tweaks (correlated metrics, a custom archetype, a holdout split) that each map to a User Guide page when you want depth.
Time-to-first-output: ~10 seconds. The point isn't volume — it's seeing the engine's core promise (every metric value derives from one shared trajectory position) made executable.
Setup¶
If you opened this in Binder or Colab the install already ran. Locally, pip install plotsim matplotlib is enough.
import json
from pathlib import Path
import matplotlib.pyplot as plt
import pandas as pd
import plotsim
print(f"plotsim {plotsim.__version__}")
Build a config¶
Imagine a small coffee chain with three customer segments — loyal regulars who grow, occasional drop-ins who plateau, and a churning cohort whose visits decline. We will measure visit frequency, average ticket size, and a loyalty score, over twelve months.
plotsim.create(...) takes the user-facing shape directly. Three required pieces:
metrics— what you measure. Each metric carries atype(score,amount,count,index) and apolarity(positiveornegative). Range is required foramountandindex.segments— who you simulate. Each segment names anarchetype(the trajectory shape) and acountof entities.window— the time span, given as(start, end, granularity).
Everything else (schema, dims, facts, events) is auto-generated when omitted.
cfg = plotsim.create(
about="Coffee chain customers",
unit="customer",
window=("2024-01", "2024-12", "monthly"),
seed=42,
metrics=[
{"name": "visits", "type": "count", "polarity": "positive"},
{"name": "avg_ticket", "type": "amount", "polarity": "positive", "range": [3, 25]},
{"name": "loyalty_score", "type": "score", "polarity": "positive"},
],
segments=[
{"name": "regulars", "count": 30, "archetype": "accelerating"},
{"name": "occasional", "count": 20, "archetype": "flat"},
{"name": "churning", "count": 15, "archetype": "decline"},
],
)
print(f"entities: {len(cfg.entities)}")
print(f"periods: {cfg.time_window.period_count()}")
print(f"tables: {[t.name for t in cfg.tables]}")
Generate and write¶
plotsim.run(cfg, output_dir) runs the full pipeline in one call — generate, validate, build the ground-truth manifest, and write everything to disk. The returned path is the resolved output directory.
If you don't need the manifest, plotsim.generate_tables(cfg) returns just the {table_name: DataFrame} dict. plotsim.run is the default for tutorials because the manifest is the load-bearing artefact that makes the trajectory-first contract reproducible downstream.
out_dir = Path("output_first")
plotsim.run(cfg, out_dir)
for path in sorted(out_dir.iterdir()):
print(f" {path.name:30s} {path.stat().st_size:>10,d} bytes")
Take a quick look at the auto-generated fact table — one row per customer per month with the three metric columns.
fct = pd.read_csv(out_dir / "fct_customer.csv")
print(f"rows: {len(fct):,} columns: {list(fct.columns)}\n")
fct.head()
The trajectory-first moment¶
plotsim's core invariant: for every (entity, period), the engine computes one trajectory position in [0, 1] from the entity's archetype, then derives every metric value from that single position. If the trajectory dips at month 7, every positive-polarity metric for that entity also dips at month 7 — not because of post-hoc correlation, but because they read from the same source.
You can verify this directly. The manifest records each sampled entity's trajectory array. Pick one entity, overlay its trajectory position against a metric column, and watch them move together.
manifest = json.loads((out_dir / "manifest.json").read_text())
entity_id = manifest["trajectory_samples"][0]["entity"]
entity_samples = sorted(
(s for s in manifest["trajectory_samples"] if s["entity"] == entity_id),
key=lambda s: s["period_index"],
)
positions = [s["position"] for s in entity_samples]
entity_rows = fct[fct["customer_id"] == entity_id].sort_values("date_key")
metric_values = entity_rows["visits"].to_numpy()
fig, ax_left = plt.subplots(figsize=(8, 4))
ax_right = ax_left.twinx()
ax_left.plot(positions, marker="o", color="#1f77b4", label="trajectory position")
ax_left.set_ylabel("trajectory position (0–1)", color="#1f77b4")
ax_left.set_xlabel("period index")
ax_right.plot(metric_values, marker="s", color="#d62728", label="visits")
ax_right.set_ylabel("visits", color="#d62728")
plt.title(f"Entity {entity_id} — visits tracks the trajectory")
fig.tight_layout()
plt.show()
The two lines move in lockstep. That is the contract. Every metric, every entity, every period — same shape, same source.
When you change a segment's archetype, every metric for entities in that segment reshapes accordingly. When you set seed=42, every run reproduces byte-for-byte.
Three single-cell tweaks¶
Each of the next three cells reuses the same base config and adds one feature. The point is not to teach the feature in depth — that's what the User Guide is for. The point is to show how the surface area extends.
1. Correlate two metrics¶
Declare that avg_ticket is tightly linked to loyalty_score. The engine adds a Cholesky copula layer that pulls the two metrics' residuals together at sampling time. → see user guide: connecting metrics.
cfg_corr = plotsim.create(
about="Coffee chain customers",
unit="customer",
window=("2024-01", "2024-12", "monthly"),
seed=42,
metrics=[
{"name": "visits", "type": "count", "polarity": "positive"},
{"name": "avg_ticket", "type": "amount", "polarity": "positive", "range": [3, 25]},
{"name": "loyalty_score", "type": "score", "polarity": "positive"},
],
segments=[
{"name": "regulars", "count": 30, "archetype": "accelerating"},
{"name": "occasional", "count": 20, "archetype": "flat"},
{"name": "churning", "count": 15, "archetype": "decline"},
],
connections=["avg_ticket 0.7 loyalty_score"],
)
tables_corr = plotsim.generate_tables(cfg_corr)
fct_corr = tables_corr["fct_customer"]
print(
f"observed Pearson(avg_ticket, loyalty_score) = {fct_corr[['avg_ticket', 'loyalty_score']].corr().iloc[0, 1]:.2f}"
)
2. A custom archetype with timing¶
Archetype names are composable. flat > growth @ 5 means "plateau for the first five periods, then sigmoid up". Use > to chain shapes and @ N to mark inflection periods. → see user guide: shaping metrics.
cfg_arch = plotsim.create(
about="Coffee chain customers",
unit="customer",
window=("2024-01", "2024-12", "monthly"),
seed=42,
metrics=[
{"name": "loyalty_score", "type": "score", "polarity": "positive"},
],
segments=[
{"name": "slow_burn", "count": 10, "archetype": "flat > growth @ 5"},
],
)
tables_arch = plotsim.generate_tables(cfg_arch)
fct_arch = tables_arch["fct_customer"]
one_entity = fct_arch["customer_id"].iloc[0]
print("archetype 'flat > growth @ 5' — one entity's loyalty_score per period:")
print(
fct_arch[fct_arch["customer_id"] == one_entity][["date_key", "loyalty_score"]].to_string(
index=False
)
)
3. Holdout split¶
Reserve the last three months for model validation. The engine writes <fact>_train.csv and <fact>_holdout.csv next to the full fact table, and the manifest records the cutoff. → see user guide: running experiments.
cfg_holdout = plotsim.create(
about="Coffee chain customers",
unit="customer",
window=("2024-01", "2024-12", "monthly"),
seed=42,
metrics=[
{"name": "visits", "type": "count", "polarity": "positive"},
{"name": "loyalty_score", "type": "score", "polarity": "positive"},
],
segments=[
{"name": "regulars", "count": 30, "archetype": "accelerating"},
{"name": "occasional", "count": 20, "archetype": "flat"},
],
holdout={"target": "loyalty_score", "periods": 3},
)
out_holdout = Path("output_holdout")
plotsim.run(cfg_holdout, out_holdout)
print("files written:", sorted(p.name for p in out_holdout.iterdir() if p.suffix == ".csv"))
Where to go next¶
You have built a dataset, watched a metric track its trajectory, and seen three feature surfaces. The User Guide is organised around the decisions you make next:
- Shaping metrics — pick distributions and archetypes that match your domain.
- Connecting metrics — correlations, causal lag, seasonality.
- Designing tables — when the auto-schema isn't enough.
- Filling columns — every column source type with examples.
- Adding realism — noise presets, heavy-tailed families, quality issues.
- Running experiments — A/B tests, holdout splits, entity features.
- Output and scaling — parquet, partitioning, cell budgets.
For full end-to-end use cases, see the credit risk modeling and retail analytics warehouse tutorials.