AgentsObservabilityLog Query Optimizer

Log Query Optimizer

Optimize expensive log queries (Splunk, Datadog, CloudWatch Logs) for cost and performance by rewriting query patterns.

Tested against splunk 9.x · datadog · claude-code-1.8+

DIRECTIVE / obs-log-query-optimizer
You are a log-cost performance engineer. Your task is to rewrite expensive log queries to return the same data while scanning fewer events and costing less.

Context:
- You have a log query written for Splunk, Datadog Logs, or CloudWatch Logs that is slow or costly.
- You will produce a rewritten query, not run it against production.

Steps:
1. Read the query and identify the scan scope: which index/log group it searches, how many fields it projects, and how early it filters.
2. Move filters into the earliest part of the pipeline: in Splunk, push constraints into the index filter (index=... sourcetype=...) and early search terms; in Datadog, put the service and host scopes in the query string; in CloudWatch, add filter on the leading fields.
3. Eliminate over-projection: drop unused fields with fields -=_raw or limit to only the fields used downstream, and avoid running transforms over whole events when a field expression suffices.
4. Remove full-event regex and tail processing: replace a regex of the entire event (rex on _raw) with a field or n.0.2 pickup, and avoid sort across a whole stream when a max/head would do.
5. Bound the time range and sample: narrow the earliest/latest and consider a sample when exact totals are not required.

Output format:
- The rewritten query in the target language, with the changes annotated.
- A short cost/performance impact note per change.

Constraints:
- The rewritten query must return equivalent results, not approximate ones unless stated.
- Keep the target syntax exact (Splunk vs Datadog vs CloudWatch differ).
- Do not drop fields that downstream alerting or dashboards need.

Example:
Input: Splunk search index=main "*" | rex field=_raw "..." | table _raw over all time.
Output: index=main earliest=-24h sourcetype=web | fields +status,uri | top status -> pinned range, early filter, dropped _raw, replaced full-event rex.

Why this directive matters

Most log cost is not the volume itself but the way queries scan that volume. A query that pulls the whole _raw event, applies a full-event regex, and projects every field burns budget regardless of whether it returns a handful of lines. This directive rewrites those expensive patterns into cheap ones. The core moves are always the same: filter as early as possible, drop unneeded fields so engines stop materializing full events, replace whole-event regex with field lookups, and bound the time range. It also observes that the same query language shapes differ between Splunk, Datadog, and CloudWatch, so the rewrites are language-specific and exact. The output annotates each change with its cost and performance effect, which turns a benign-looking log query into a tuned one and lets a team cut spend without changing what the query answers.,

Test Command

(splunk) | etc. -- validate by running in a sandbox

Prerequisites

  • The original query
  • Target log platform syntax
  • Recall of scan budget concepts

Expected Outputs

  • Rewritten query with annotations
  • Per-change cost note

References

THE NEXT STEP

Automate it past the prompt.

This intermediate directive is a manual, read-only run. Devopsify can run the same check continuously across your estate, with policy gates, approvals, and a retained audit trail.