ameau01/synthesized-cloud-optimization-recommendations

Synthesized Cloud-Optimization Recommendations

18 scenarios that pair cloud telemetry with a hand-crafted optimization recommendation. Use them to train models or to evaluate AI agents.

Summary

Each scenario has multi-tier telemetry, a Terraform file describing the deployed infrastructure, and a gold-standard recommendation.

The dataset is built around a simple input-output mapping. The input is telemetry plus the infrastructure. The output is an optimization recommendation that says what to change and what the impact will be.

The dataset is synthesized. Telemetry was generated procedurally to match each scenario's narrative. Gold recommendations were hand-crafted and verified.

The dataset uses AWS vocabulary throughout. Instance types, service names, and field names match AWS. This makes the scenarios concrete instead of vendor-neutral.

Folder layout

README.md                                # this file
LICENSE                                  # MIT
EVAL.md                                  # what eval.py checks
eval.py                                  # Floor sanity check (smoke test)
sample_predictions.json                  # worked example of submission shape
scenarios_summary.jsonl                  # one row per scenario (viewer table)
scenarios/
  01/
    metadata.json                        # scenario summary + fixtures
    main.tf                              # Terraform for the infra
    compute_telemetry.json               # CPU, memory, latency
    database_telemetry.json              # query rates, pool stats
    cache_telemetry.json                 # hit rate, eviction
    network_telemetry.json               # bandwidth, packet loss
    correlation_evidence.json            # cross-tier correlations
    handcrafted_recommendation.json      # the gold answer
  02/
    ...

Each scenario covers a different optimization situation. Some are single-tier (only compute is wrong). Some span tiers (database problem that surfaces in compute). Some are no-action cases. Two are diagnostic deferral cases. One asks for an SLA review instead of an infra change.

The summary table (scenarios_summary.jsonl)

The Hugging Face Dataset Viewer renders scenarios_summary.jsonl as a browsable table. Each row is one scenario and includes the headline fields from that scenario's metadata and gold recommendation.

The summary is for discovery only. The full inputs (telemetry, Terraform, correlation evidence) live in scenarios/NN/. Always train or evaluate on the full files, not on the summary.

Columns in the summary table:

Some scenarios have negative savings_monthly_usd. That is expected. For those scenarios the right action increases cost to fix a performance or reliability problem (for example, adding a read replica).

Schema

Scenario inputs

Each scenarios/NN/ folder has these files.

Recommendation shape

{
  "scenario_id": "01",
  "finding_type": "issue_found",
  "specific_change": "...",
  "primary_tier": "compute",
  "secondary_tier": null,
  "action_category": "rightsizing",
  "conclusion": { ... },
  "evidence": {
    "telemetry_observations": [ ... ],
    "infrastructure_context": [ ... ],
    "correlation_observations": [ ... ]
  },
  "reasoning": "...",
  "projected_state": { ... },
  "cost_impact": { ... },
  "risk_assessment": { ... }
}

Allowed values

• —finding_type: issue_found, no_issue_found, diagnostic_deferral, insufficient_data
• —primary_tier: compute, database, cache, network, deferred, or null
• —secondary_tier: same set as primary_tier
• —action_category: rightsizing, scaling_policy_change, query_cache_optimization, cache_capacity_adjustment, pool_sizing, replica_adjustment, load_balancer_reconfiguration, network_topology_change, sla_review, or null

The deferred tier sentinel is used in diagnostic-deferral scenarios where the agent explicitly cannot pick a tier yet (scenarios 15 and 17). insufficient_data is reserved for future scenarios where the dataset is too sparse to support any finding; no current scenario uses it.

Scenario coverage

How to use it

You can use this dataset two ways.

Train or fine-tune. Treat each scenario's telemetry plus metadata as input. Use the handcrafted_recommendation.json as the target output.

Evaluate AI agents. Run your agent on the scenario inputs. Compare its output to the hand-crafted recommendation in that scenario's folder.

Quick sanity check

python eval.py --predictions sample_predictions.json

This runs the bundled Floor sanity check. It confirms your predictions parse, have the required fields, and use allowed category values. It does NOT score recommendation quality. See EVAL.md for what is checked.

Prediction shape

See sample_predictions.json for a worked example. Required fields per prediction: scenario_id, finding_type, specific_change, primary_tier, action_category. Optional but useful for deeper scoring: secondary_tier, reasoning, evidence, projected_state, cost_impact, risk_assessment.

How to score beyond the Floor check

The dataset ships gold answers and a Floor sanity check. It does not ship a quality scorer. Beyond the Floor check, the scoring method is up to you. Common options:

• —Exact match on the enum fields (finding_type, primary_tier, action_category).
• —Keyword or substring checks on specific_change.
• —Semantic similarity on the prose fields.
• —A custom rubric per scenario, comparing prediction fields against the matching handcrafted_recommendation.json.

Intended uses

• —Train or fine-tune a model that maps cloud telemetry to an optimization recommendation.
• —Evaluate AI agents on cloud-optimization reasoning.
• —Compare single-shot vs orchestrated agent designs.

License

MIT. See LICENSE.

Citation

@misc{synthesized_cloud_optimization_recommendations_2026,
  title = {Synthesized Cloud-Optimization Recommendations},
  author = {Alexander Meau},
  year = {2026},
  version = {1.0.0}
}