Data science¶

Recipes for ML training data, feature engineering, evaluation, and causal-inference exercises. Every recipe ships with a ground-truth answer in the manifest. For mental-model context, see How plotsim works.

Flat per-entity feature table¶

Set entity_features: true and the engine writes _entity_features.<ext> — one row per entity with per-metric aggregates (_mean, _std, _slope, _first, _last, _peak_period) plus archetype and final_trajectory_position ground-truth labels.

PythonYAML

from plotsim import create, generate_tables, write_tables

config = create(
    about="Churn prediction features",
    unit="customer",
    window=("2024-01", "2024-12", "monthly"),
    entity_features=True,
    metrics=[
        {"name": "engagement", "type": "score", "polarity": "positive"},
        {"name": "churn_risk", "type": "score", "polarity": "negative"},
    ],
    segments=[
        {"name": "active",   "count": 100, "archetype": "growth"},
        {"name": "churning", "count": 80,  "archetype": "decline"},
    ],
)
write_tables(generate_tables(config), config, output_dir="./data")

about: Churn prediction features
unit: customer
window: { start: "2024-01", end: "2024-12", every: monthly }
entity_features: true

metrics:
  - { name: engagement, type: score, polarity: positive }
  - { name: churn_risk, type: score, polarity: negative }

segments:
  - { name: active,   count: 100, archetype: growth }
  - { name: churning, count: 80,  archetype: decline }

On disk: _entity_features.csv with columns entity_id, engagement_mean, engagement_std, engagement_slope, ..., archetype, final_trajectory_position.

Learn more: Running experiments → Entity features.

Narrow the feature set or strip labels¶

Use the dict form to pick specific metrics or remove the ground-truth columns:

PythonYAML

entity_features={
    "metrics":        ["engagement"],
    "include_labels": False,
}

entity_features:
  metrics: [engagement]
  include_labels: false

Train / holdout split¶

The engine slices every per_entity_per_period fact at cutoff = n_periods - holdout_periods and emits <fct>_train and <fct>_holdout file pairs alongside the unsplit fact.

PythonYAML

config = create(
    about="Credit risk training",
    unit="customer",
    window=("2024-01", "2024-12", "monthly"),
    holdout={"target": "default_risk", "periods": 3},
    metrics=[
        {"name": "credit_score", "type": "amount", "polarity": "positive", "range": [300, 850]},
        {"name": "default_risk", "type": "score",  "polarity": "negative"},
    ],
    segments=[{"name": "applicants", "count": 500, "archetype": "growth"}],
)

about: Credit risk training
unit: customer
window: { start: "2024-01", end: "2024-12", every: monthly }

holdout:
  target:  default_risk
  periods: 3

metrics:
  - { name: credit_score, type: amount, polarity: positive, range: [300, 850] }
  - { name: default_risk, type: score,  polarity: negative }

segments:
  - { name: applicants, count: 500, archetype: growth }

On disk: fct_<name>.csv, fct_<name>_train.csv, fct_<name>_holdout.csv per per-entity-per-period fact. manifest.json records the cutoff period in the holdout section.

Learn more: Running experiments → Holdout split.

A/B treatment with a known lift¶

Carve a segment into a treatment arm with a known lift_log_odds and score your inference against the manifest's treatment_cohorts section.

PythonYAML

segments=[
    {
        "name":      "campaign_recipients",
        "count":     200,
        "archetype": "flat",
        "treatment": {
            "fraction":      0.5,
            "lift_log_odds": 0.45,
            "start_period":  3,
        },
    },
]

segments:
  - name: campaign_recipients
    count: 200
    archetype: flat
    treatment:
      fraction:      0.5
      lift_log_odds: 0.45
      start_period:  3

On disk: each entity carries a treatment_group label; manifest.treatment_cohorts records every per-entity assignment plus per-cohort summaries. The pre-treatment window (periods < start_period) has identical trajectory positions across arms — the AC for "pre-treatment baseline is identical."

Learn more: Running experiments → Treatment / control.

Per-metric ATE recovery exercise¶

Add target_metric to narrow the lift to a single named metric. Every other metric on the treatment-arm entity is byte-identical to its control-arm draw. The recovery exercise: can your inference correctly identify which metric was actually treated?

PythonYAML

segments=[
    {
        "name":      "email_test",
        "count":     200,
        "archetype": "flat",
        "treatment": {
            "fraction":      0.5,
            "lift_log_odds": 0.45,
            "start_period":  3,
            "target_metric": "open_rate",
        },
    },
]

segments:
  - name: email_test
    count: 200
    archetype: flat
    treatment:
      fraction:      0.5
      lift_log_odds: 0.45
      start_period:  3
      target_metric: open_rate

Caveat: when the targeted metric is correlated with another, some lift propagates through the Cholesky copula. The residual is bounded but non-zero — expected, not a bug.

Learn more: Running experiments → Per-metric treatment.

Cohort-evolution exercises (arrival shapes)¶

A step arrival models cohort cuts; linear arrivals model organic back-loading; explicit arrivals let you pin per-entity start periods for golden fixtures.

PythonYAML

segments=[
    {
        "name": "cohorts",
        "count": 100,
        "archetype": "flat",
        "arrival": {
            "kind":   "step",
            "blocks": [
                {"period": 0,  "fraction": 0.5},
                {"period": 6,  "fraction": 0.3},
                {"period": 12, "fraction": 0.2},
            ],
        },
    },
]

segments:
  - name: cohorts
    count: 100
    archetype: flat
    arrival:
      kind: step
      blocks:
        - { period: 0,  fraction: 0.5 }
        - { period: 6,  fraction: 0.3 }
        - { period: 12, fraction: 0.2 }

On disk: cells before each entity's start_period have NaN trajectory positions; fact rows for those cells are stripped. The fact table contains only the periods each entity was actually active.

Learn more: Running experiments → Arrival distributions.

Heavy-tailed noise for robust-modeling teaching¶

Switch the additive noise family to student_t to give models a heavy-tailed distribution to handle:

PythonYAML

noise={
    "gaussian_sigma":     0.05,
    "outlier_rate":       0.01,
    "mcar_rate":          0.0,
    "noise_family":       "student_t",
    "degrees_of_freedom": 4.0,
}

noise:
  gaussian_sigma:     0.05
  outlier_rate:       0.01
  mcar_rate:          0.0
  noise_family:       student_t
  degrees_of_freedom: 4.0

student_t requires degrees_of_freedom ≥ 1.0. laplace is the other heavy-tailed option (no degrees parameter).

Learn more: Adding realism → Heavy-tailed families.

Heteroscedastic noise¶

Larger trajectory positions get larger absolute gaussian noise — realistic when "big accounts vary more":

PythonYAML

noise={
    "gaussian_sigma":         0.04,
    "outlier_rate":           0.005,
    "mcar_rate":              0.0,
    "scale_with_trajectory":  True,
}

noise:
  gaussian_sigma:        0.04
  outlier_rate:          0.005
  mcar_rate:             0.0
  scale_with_trajectory: true

Learn more: Adding realism → Heteroscedastic noise.

Recover declared correlations from the data¶

Every declared correlation pair gets an OLS regression record in the manifest under regression_pairs_global (and a per-archetype variant under regression_pairs_by_archetype). β + intercept in both directions, plus r_squared and per-direction residual variances.

The test loop:

Run the dataset.
Read manifest.json → regression_pairs_global.
Compute pair-wise OLS β on your output tables.
Assert they match within tolerance.

Learn more: Reference: Manifest schema → regression_pairs.

Decompose variance against archetype + segment¶

variance_partitions and variance_partitions_by_segment carry a three-level nested-ANOVA decomposition per metric: between-archetype, within-entity, and residual time-series variance.

For each metric: - ss_between — variance attributable to archetype (or, in the by-segment variant, segments within an archetype). - ss_within_entity — entity-to-entity dispersion within the same group. - ss_residual — within-entity time-series variance. - fraction_* fields normalize to ss_total.

Use these as the answer key when teaching variance decomposition or multi-level modeling.

Learn more: Reference: Manifest schema → variance_partitions.

Per-archetype GP kernel fits¶

gp_kernel_fits carries an RBF Gaussian-process fit per archetype (plus per-entity fits for entities with overrides). Hyperparameters in their natural scale: length_scale (in units of period indices), signal_variance, noise_variance.

The optimizer (L-BFGS-B in log-hyperparameter space) gracefully returns converged=False for flat trajectories (variance below 1e-12), trajectories with fewer than three finite training points, or Cholesky failures — failed fits never abort the manifest build.

Learn more: Reference: Manifest schema → gp_kernel_fits.

Seasonal decomposition¶

seasonal_decomposition (always emitted) records the deterministic seasonal layer applied at metric generation:

seasonal_factors[period] — global per-period strength array.
metric_seasonal_sensitivities[metric] — per-metric multiplier.
entity_seasonal_sensitivities[entity] — per-entity multiplier.

Effective seasonal lift at cell (entity, period, metric) = seasonal_factors[period] * metric_sens[metric] * entity_sens[entity] — exactly as the engine applied it.

Learn more: Reference: Manifest schema → seasonal_decomposition.

Train a baseline with the manifest as the answer key¶

End-to-end loop:

Python

import json
import pandas as pd
from pathlib import Path
from plotsim import load_template, generate_tables, write_tables

config = load_template("saas")
tables = generate_tables(config)
write_tables(tables, config, output_dir="./data")

# Train a baseline on the entity-features table
# (saas template doesn't set entity_features by default —
# extend the template before this step if you need it)

# Read the manifest for ground truth
manifest = json.loads(Path("./data/manifest.json").read_text())
archetypes = {a["entity_id"]: a["archetype"] for a in manifest["archetype_assignments"]}

# Score your model against archetypes
preds = pd.read_csv("./data/_entity_features.csv")
preds["true_archetype"] = preds["entity_id"].map(archetypes)
accuracy = (preds["predicted_archetype"] == preds["true_archetype"]).mean()

The manifest's other ground-truth sections — treatment_cohorts, quality_injections, source_entity_mappings, scd_events — score whichever inference task your exercise centers on.

Learn more: How plotsim works → What you get on disk.