Because a perfectly written prompt is useless if it solves the wrong problem.

I've spent a considerable amount of time building backend systems, working with AI coding agents, and experimenting with prompt engineering frameworks. One pattern keeps showing up: we spend too much effort telling AI how to execute a task and not enough effort making sure it understands why the task exists in the first place.

A prompt can be beautifully structured. It can have a role, detailed context, carefully defined instructions, examples, constraints, and a specific output format.

And it can still produce the wrong result.

Not because the model misunderstood the instructions.

Because the instructions were aimed at the wrong problem.

That distinction became the starting point for my latest open-source project: CoMPaSS — an Analysis-First Prompt Framework for AI Agents.

CoMPaSS introduces a deliberate step before prompt construction: understanding the user's actual intention, identifying the desired outcome, and choosing the simplest reliable path to get there.

It then structures the resulting prompt around explicit objectives, relevant context, carefully selected optional components, operational boundaries, and verifiable success criteria.

The framework is available on GitHub, packaged as an Agent Skill, and released under the MIT License.

Repository: YourAKShaw/CoMPaSS-prompt-framework

Agent Skill: Install via skills.sh

One important distinction upfront: CoMPaSS is currently a research-grade, unbenchmarked proposal. Its design is informed by published research and existing prompting practices, but I am not claiming that it outperforms established frameworks. The repository includes an evaluation plan specifically designed to test that hypothesis.

This article explains the engineering problem behind CoMPaSS, how the framework works, why I made certain design decisions, and where I believe it can be useful.

The Problem: AI Can Execute Instructions Without Understanding the Objective

Consider a relatively simple engineering request:

Our checkout service has a flaky test. Fix it.

An AI coding agent could interpret that request in several ways.

It might add a retry mechanism. It might increase a timeout. It might isolate the test, modify an assertion, or skip an unreliable check.

Some of those changes might make the CI pipeline green.

But what was the engineer actually trying to accomplish?

Probably not just a green status indicator.

The real objective is to restore confidence in the test suite without hiding a defect that could affect production behavior.

A flaky test might expose a race condition, shared mutable state, nondeterministic execution, or an actual bug in the application.

If the agent eliminates the symptom while preserving the underlying defect, the task has technically been completed according to one interpretation.

Operationally, however, it has failed.

This is the problem I wanted CoMPaSS to address.

The literal action requested by a user is not always equivalent to the outcome they actually need.

The distinction matters even more when AI systems are trusted with consequential decisions or autonomous execution.

The same problem appears outside engineering

Imagine a founder asking:

Write a prompt to help me choose between PostgreSQL and ClickHouse.

A conventional prompt might compare performance characteristics, scalability, operational complexity, and costs.

That sounds reasonable.

But the founder's real objective might be to make analytics dashboards fast enough without increasing infrastructure complexity beyond what their team can maintain.

A database migration is only one possible solution.

Query optimization, indexing, pre-aggregation, materialized views, or other improvements might solve the actual problem at substantially lower cost.

If the prompt begins with the assumption that a migration must be evaluated as the central objective, it may overlook the most valuable alternative.

The prompt has already narrowed the solution space before the analysis begins.

That is not primarily a formatting problem.

It is an intention-discovery problem.

And solving it requires more than adding another instruction to the prompt.

Why I Built Another Prompt Framework

There are already several established approaches to organizing prompts.

ART, RTF, CO-STAR, RISEN, CRISPE, and other techniques help structure instructions around roles, tasks, context, expected outputs, and constraints.

These approaches are useful.

I have also developed the ART Prompt Framework, which focuses on giving prompt authors a lightweight way to express a role, request, and terms.

But a limitation emerges when the task itself is ambiguous.

Most structural frameworks begin once the author has already decided what the prompt should accomplish.

They help express the objective clearly.

They do not necessarily establish whether the objective was correctly identified.

CoMPaSS moves one step earlier in the process.

Before assembling the prompt, it asks four questions:

What is the user actually doing?

Why are they doing it?

What deliverable or decision needs to exist at the end?

What is the cheapest reliable path to that outcome?

These questions sound obvious.

In practice, they are easy to skip when a capable model can immediately generate something that looks convincing.

That is especially dangerous with AI agents.

An ordinary conversational response can be reviewed before anything happens.

An agent might edit files, execute commands, interact with tools, or produce artifacts that influence business decisions.

A wrong assumption at the beginning can propagate through every subsequent action.

CoMPaSS is my attempt to make that initial interpretation explicit, inspectable, and correctable.

Not by introducing a huge procedural template.

By establishing a small number of rules that determine when structure is necessary and when it is just overhead.

Introducing CoMPaSS

CoMPaSS stands for:

There is an intentional distinction in the lettering.

Uppercase components are mandatory for prompts that pass the framework's complexity gate. Lowercase components are optional and must justify their inclusion.

This is not a decorative naming convention.

It is a design constraint.

I wanted CoMPaSS to resist a common tendency in prompt engineering: adding more instructions simply because they can be added.

A prompt does not automatically improve because it contains more sections.

Every section consumes attention, tokens, and potentially reasoning resources.

The objective is not to maximize prompt complexity.

The objective is to include the minimum structure required to make the intended outcome clear and verifiable.

Let's examine each component.

C — Comprehend: Understand Before You Instruct

This is the foundation of the framework.

Before writing the actual prompt, CoMPaSS performs a use-case analysis.

That analysis captures six fields:

Field — Question

Action: What is the user doing right now?
Intention: Why are they doing it, and what outcome do they want?
Deliverable / Decision: What must exist when the task is finished?
Efficient path: What is the simplest reliable route, including what can be skipped?
Tier: How much structure does the task justify?
Intention confidence: How confident are we about the user's intention, and what is the strongest alternative interpretation?

The first four establish the nature of the task.

The final two determine whether and how the framework should proceed.

Why intention confidence matters

Suppose a user asks:

Create a strategy for our AI agency.

What does "strategy" mean here?

A positioning strategy? A customer acquisition plan? An operating model? A pricing strategy? A comprehensive business strategy?

Choosing one interpretation without acknowledging uncertainty might produce an impressive document that addresses the wrong need.

CoMPaSS introduces an ambiguity gate.

High confidence: Proceed with the identified intention.

Medium confidence: Proceed, but make the assumed intention explicit in the Mission.

Low confidence: Ask up to three targeted clarifying questions before constructing the prompt.

This is important because an intention-analysis stage creates its own risk.

If the analysis is wrong, the resulting prompt can reinforce that mistake more strongly than a loosely specified request would.

The solution is not to pretend that intention can always be inferred correctly.

It is to make uncertainty visible.

The analysis card is not the final prompt

Another deliberate decision: the Comprehend analysis is a working artifact, not a section blindly pasted into the resulting prompt.

Information is incorporated into the components where it belongs.

The intention informs the Mission.

The deliverable informs the Mission and Success spec.

The efficient path determines which context and optional steps are necessary.

That keeps the final prompt focused.

CoMPaSS analyzes before constructing, but it does not require every intermediate thought to become additional prompt content.

The Tier Gate: Sometimes the Best Framework Is No Framework

This is one of my favorite design decisions in CoMPaSS.

The framework has explicit permission to stop itself.

There are three tiers:

Tier — Task type — Appropriate response

Tier 0: Trivial question or simple request — Answer directly or use a minimal prompt
Tier 1: Consequential single-turn deliverable or decision — Apply the CoMPaSS construction process
Tier 2: Multi-step or agentic execution — Apply CoMPaSS with relevant execution and verification controls

Consider this question:

What's the default PostgreSQL port?

The answer is 5432.

There is no reason to construct a seven-component prompt, perform an elaborate intention analysis, or introduce a Success spec to answer that question.

Doing so would be absurd.

Yet prompt frameworks often encourage exactly this sort of unnecessary ceremony when users apply them indiscriminately.

For simple tasks, I would rather use a plain instruction or a lightweight framework such as ART.

CoMPaSS should be reserved for situations where getting the intention wrong has a meaningful cost.

Good engineering is not about using the most sophisticated abstraction. It is about choosing the simplest abstraction that solves the problem.

Prompt engineering should follow the same principle.

o — order of operations: Sequence Only When It Matters

The first optional component is the order of operations.

This defines the required sequence of work.

But CoMPaSS does not include it automatically.

It appears only when one of three conditions applies:

The task requires a particular sequence.

A non-reasoning model needs decomposition for a multi-step reasoning task.

An agent requires checkpoints between phases.

Consider the flaky checkout test again.

Here, order genuinely matters.

The agent should reproduce the failure before changing code, identify the source of nondeterminism, apply a root-cause fix, and verify the result.

That sequence protects against superficial fixes.

In contrast, a prompt to draft a short internal announcement probably does not need an elaborate procedure.

Specifying five arbitrary writing steps would add complexity without changing the acceptance criteria.

I also wanted to avoid forcing reasoning models through unnecessary step-by-step instructions.

A required workflow is different from prescribing an artificial thought process.

CoMPaSS focuses on the former.

The rule is straightforward:

If the order of execution is important to correctness, specify it. Otherwise, let the model handle the task without procedural micromanagement.

M — Mission: Define What Success Is Supposed to Achieve

The Mission is the first section of the constructed prompt.

It answers two questions:

What must be produced or decided? And why does that outcome matter?

A weak mission might say:

Compare PostgreSQL and ClickHouse and recommend the better database.

A more useful mission would say:

Determine the lowest-cost, operationally sustainable way to meet our analytics dashboard latency requirements, considering PostgreSQL optimization and ClickHouse migration as possible solutions.

The second version does not assume that migration is the objective.

It makes the business outcome primary.

This distinction is central to CoMPaSS.

An agent should not simply optimize for the most literal interpretation of an instruction when the real objective is clear from context.

At the same time, it should not invent an alternative intention and silently substitute it.

That is why medium-confidence interpretations are explicitly marked as assumptions.

A good Mission is compact, outcome-oriented, and transparent.

If it takes several paragraphs to explain what the agent is supposed to accomplish, the requirement probably needs further clarification.

P — Provisions: Context Is Useful Until It Becomes Noise

Context engineering has become an important part of building effective AI systems.

But more context is not inherently better context.

A large repository dump, a long conversation history, or an enormous collection of unrelated documentation can introduce distractions and increase cost.

CoMPaSS approaches context as a relevance problem.

The Provisions component supplies the smallest set of information that meaningfully changes the result.

For a coding agent, this might include:

Relevant repository paths and entry points.

Test commands and execution environment.

Known failure symptoms and CI logs.

Architecture constraints or invariants.

Available tools and their limitations.

The framework prefers pointers to information over indiscriminate duplication of information.

For example, rather than embedding an entire codebase into a prompt, provide the paths that allow the agent to retrieve the relevant files when needed.

This is especially practical for terminal-based coding agents, where context consumption and execution costs are meaningful operational concerns.

Why there is no default "world-class expert" persona

Another design choice is that CoMPaSS does not automatically begin with instructions such as:

You are a world-class software engineer with 30 years of experience.

A role or perspective can be useful when it changes writing style, framing, or professional viewpoint.

But a grandiose persona should not be treated as a substitute for evidence, domain knowledge, or verification.

If perspective is relevant, CoMPaSS allows a concise perspective instruction.

If it isn't, the prompt skips it.

I would rather give an agent the correct repository path, constraints, and tests than spend tokens declaring how brilliant it is supposed to be.

a — anchors: Examples Should Have a Job

Examples can be useful when a desired result is difficult to specify with prose alone.

This is particularly true for strict output formats, fine-grained classifications, or distinctive writing styles.

CoMPaSS treats examples as optional anchors.

They are included when:

The expected format, labels, or voice are easier to demonstrate than describe.

The task requires fine-grained classification.

The target model vendor recommends examples by default.

For few-shot tasks, the framework recommends a small, diverse set of examples, typically three to five.

For a fixed output format, one clearly labeled template may be sufficient.

But examples have a cost.

They can cause the model to imitate incidental details, repeat assumptions, or anchor its solution to a narrow pattern.

For reasoning models, CoMPaSS generally starts without anchors unless there is a concrete reason to include them.

This is an important distinction.

Examples should clarify a requirement, not silently dictate the answer.

S — Scope: Define the Boundaries Before Execution

The first S stands for Scope.

This is where the prompt defines what the model is allowed, expected, and not expected to do.

In practical engineering work, scope can determine whether an agent produces a maintainable solution or creates unnecessary work.

Imagine asking an agent to resolve a flaky test.

Without clear boundaries, it might rewrite unrelated modules, introduce new dependencies, weaken assertions, or modify production behavior without sufficient justification.

A useful Scope section can specify:

Which parts of the system are in scope.

Which changes are explicitly out of scope.

What shortcuts are prohibited.

What time, cost, or length constraints apply.

When the agent should ask questions versus proceed with stated assumptions.

For the flaky-test example, an important constraint would be:

Do not silence the failure through retries, skips, or weakened assertions unless the evidence establishes that such a change is appropriate.

That single boundary can materially change the direction of the task.

Scope also establishes an assumption policy.

If required information is missing, should the agent ask, use a placeholder, make a reversible assumption, or stop?

Different tasks require different answers.

CoMPaSS makes that policy explicit rather than leaving it to chance.

S — Success Spec: A Prompt Is Incomplete Without a Definition of Done

This is the second major foundation of CoMPaSS.

Every constructed CoMPaSS prompt must define what a successful result looks like.

Not in vague terms.

In observable, checkable terms.

The Success spec has two parts.

1. Output shape

What should the final result look like?

For example:

A Markdown decision memo with specified sections.

A pull request containing the root-cause fix.

A structured report containing evidence and recommendations.

A defined JSON schema for a machine-consumed response.

When an API supports structured output enforcement, CoMPaSS recommends using that mechanism instead of relying exclusively on prose instructions.

2. Verification and acceptance checks

How can we determine whether the output satisfies the task?

For a coding agent, this might include:

The failing test can be reproduced before the fix.

The suspected root cause is explained.

The test passes repeatedly after the change.

The relevant regression suite passes.

Any unverified behavior or remaining risk is disclosed.

These checks are more useful than instructions such as "make sure your answer is accurate" or "produce a high-quality solution."

Those phrases express desirable qualities.

They do not define how those qualities will be established.

A Success spec gives both the agent and the reviewer a concrete contract.

It also creates a better feedback loop.

When the result fails, we can identify whether the problem lies in the Mission, Provisions, Scope, optional procedure, or acceptance checks.

That is more productive than continuously appending instructions to the end of an increasingly complicated prompt.

A task is not complete because the model says it is complete. It is complete when the agreed acceptance criteria have been satisfied—or when the remaining gaps have been reported.

Putting It Together: A Real Engineering Example

Let's return to the flaky checkout test.

The original request is:

The checkout-service CI keeps failing on test_apply_discount at random. Fix it.

Here is a condensed CoMPaSS-style prompt illustrating how the components work together.

Step 1: Comprehend the requirement

Field — Interpretation

Action: Resolve an intermittently failing checkout-service test
Intention: Restore trust in CI without masking a production defect
Deliverable: A root-cause fix with evidence and a change summary
Efficient path: Reproduce, isolate, fix, verify, report
Tier: 2 — multi-step agentic task
Intention confidence: High; a weaker alternative is merely making CI green

The analysis determines that this task needs an order of operations because the execution sequence affects correctness.

An anchor may also be useful if the team has a strict pull request format.

Step 2: Construct the prompt

<mission>
Identify and fix the root cause of the intermittent
test_apply_discount failure in checkout-service.

The objective is to restore confidence in CI without
masking a possible defect in production code.
</mission>

<provisions>
Repository: [REPOSITORY_PATH]
Service: services/checkout/
Relevant test: tests/test_discounts.py

Test command: [TEST_COMMAND]
Full suite command: [FULL_SUITE_COMMAND]
CI failure logs: [CI_LOG_LOCATION]

Inspect relevant files as needed instead of loading
the entire repository into context.
</provisions>

<order_of_operations>
1. Reproduce the intermittent failure and record evidence.
2. Investigate the source of nondeterminism.
3. Identify whether the defect is in test code,
   production code, or the test environment.
4. Apply a targeted root-cause fix.
5. Run repeated verification and relevant regression tests.
6. Summarize the changes and remaining risks.
</order_of_operations>

<scope>
- Preserve the intended behavior of the checkout service.
- Do not hide failures through skips or weakened assertions.
- Avoid unrelated refactoring.
- Use only evidence available from the repository,
  environment, and test results.
- Report uncertainty rather than inventing a diagnosis.
</scope>

<success_spec>
Deliver:
- Root-cause explanation
- Summary of changed files
- Evidence of the original failure
- Verification commands and observed results
- Remaining risks or unverified assumptions

Acceptance checks:
1. The fix addresses the identified failure mechanism.
2. Repeated test runs provide evidence of stability.
3. Relevant regression checks have been executed.
4. Any failed or unexecuted checks are explicitly reported.
</success_spec>

Notice what this prompt does not contain.

There is no exaggerated persona.

There are no examples inserted merely to satisfy a template.

There is no requirement to generate an enormous reasoning transcript.

Every included section has a responsibility.

The Mission captures the actual outcome.

The Provisions identify the working environment.

The order of operations protects the investigation process.

The Scope prevents unacceptable shortcuts.

The Success spec establishes what the agent must demonstrate.

That is the design philosophy behind CoMPaSS.

The framework's repository contains a fuller version of this example, alongside decision-making, communication, and founder-oriented use cases.

Where CoMPaSS Fits in Real AI Workflows

I did not build CoMPaSS as a universal replacement for every prompting technique.

Its value proposition is narrower and more deliberate.

1. Briefing coding agents and subagents

When delegating software engineering tasks, the most expensive failure is not always incorrect syntax.

It is an agent confidently implementing a solution that violates the intended behavior or creates unnecessary complexity.

CoMPaSS can help define the mission, operational boundaries, required checkpoints, and acceptance criteria before execution.

This is relevant to tools such as Claude Code, Codex, and other agentic development environments.

2. Architecture and technical decisions

Should a team adopt microservices?

Should an existing backend move to a different database?

Should a workflow use autonomous agents or conventional deterministic automation?

These decisions require more than a comparison of fashionable technologies.

They need clear success criteria, realistic constraints, alternative options, and evidence.

CoMPaSS can structure the decision prompt around the business or engineering outcome instead of the technology named in the original question.

3. Founder and business decisions

Founders frequently operate with incomplete information.

Requests such as "help me create an ICP," "design a go-to-market strategy," or "decide which offer to prioritize" can produce elaborate answers built on unexamined assumptions.

CoMPaSS encourages explicit intention analysis, evidence-versus-hypothesis distinctions, and clearly defined deliverables.

For a founder, this matters because a persuasive answer is not necessarily a useful decision.

4. Consequential writing and communication

Sometimes the actual objective behind a writing request is not simply to produce text.

It might be to explain a difficult organizational change, establish expectations, reduce confusion, or support a decision.

Understanding that intention changes the resulting communication.

A technically correct announcement can still fail if it ignores the concerns of the people receiving it.

5. Reviewing existing prompts

CoMPaSS can also evaluate prompts that have already been written.

Instead of simply asking whether a prompt is detailed enough, the review can identify missing objectives, unsupported assumptions, unnecessary optional sections, unclear boundaries, and unverifiable outputs.

This gives prompt reviews a more systematic structure.

CoMPaSS vs. ART: Why I Maintain Both

I have already released the ART Prompt Framework.

So why build CoMPaSS as a separate project?

Because they address different levels of complexity.

ART stands for Act as, Request, Terms.

It provides a lightweight structure for expressing what a model should do and under which conditions.

When the objective is already understood, that can be sufficient.

CoMPaSS is designed for situations where the objective itself requires analysis.

Dimension — ART — CoMPaSS

Primary purpose: Express a clear task efficiently — Discover intention and construct a consequential prompt
Starting assumption: The objective is sufficiently understood — The intended outcome may need interpretation
Complexity: Lightweight — Adaptive, with tier and ambiguity gates
Context treatment: As required by the task — Explicit minimum relevant context
Optional execution steps: Not a dedicated component — Included only when triggers apply
Verification: Can be expressed through Terms — Mandatory Success spec
Best fit: Straightforward instructions — Agent briefs, complex decisions, high-stakes deliverables

I do not believe every prompt needs CoMPaSS.

I also do not believe a small prompt framework becomes obsolete simply because a more elaborate one exists.

The appropriate abstraction depends on the problem.

Sometimes ART is enough.

Sometimes the request itself is enough.

And sometimes the most important part of prompt engineering happens before the first instruction is written.

That is where CoMPaSS belongs.

Making It Practical: CoMPaSS as an Agent Skill

A framework that exists only as an article or diagram has limited practical value.

I wanted CoMPaSS to be something an AI agent could actually use.

The repository therefore packages the technique as an Agent Skill with a structured entry point, reference documentation, reusable templates, examples, helper scripts, validation tools, and tests.

The primary skill lives at:

skills/compass-prompt-framework/SKILL.md

The skill instructs an agent when to use CoMPaSS, when to skip it, how to analyze intention, which components to include, and how to construct and review prompts.

The framework itself is model-agnostic.

The packaged skill provides a practical way to bring that process into supported AI development workflows.

Install through the Skills CLI

For supported agents:

npx skills add YourAKShaw/CoMPaSS-prompt-framework

Install in Claude Code

For Claude Code versions supporting the one-step marketplace installation:

/plugin install compass-prompt-framework --marketplace YourAKShaw/CoMPaSS-prompt-framework

For earlier supported versions, the repository documents the marketplace registration process:

/plugin marketplace add YourAKShaw/CoMPaSS-prompt-framework
/plugin install compass-prompt-framework@compass-prompt-framework

Install as a personal Claude Code skill

git clone https://github.com/YourAKShaw/CoMPaSS-prompt-framework.git

mkdir -p ~/.claude/skills

cp -R \
  CoMPaSS-prompt-framework/skills/compass-prompt-framework \
  ~/.claude/skills/

For project-level usage, the skill can instead be placed under .claude/skills/.

Once installed, the skill can be invoked explicitly:

/compass-prompt-framework

The installation and host requirements are documented in the repository. Host-specific activation should be verified separately; packaging validation is not proof that every supported environment behaves identically.

The Engineering Behind the Repository

I wanted the implementation to reflect the same engineering principles that motivated the framework.

Clear responsibilities.

Minimal unnecessary dependencies.

Explicit validation.

Reproducible checks.

And documentation that can be inspected rather than simply trusted.

The repository separates the agent-facing skill from the broader development and research materials.

Its structure includes:

CoMPaSS-prompt-framework/
├── skills/
│   └── compass-prompt-framework/
│       ├── SKILL.md
│       ├── references/
│       ├── assets/
│       ├── scripts/
│       └── tests/
├── docs/
├── .claude-plugin/
├── .github/
├── CHANGELOG.md
├── SECURITY.md
└── LICENSE

The skill contains a documented procedure and worked examples.

The references explain the individual components, their inclusion rules, their limitations, and the research used to inform the design.

The assets include templates and decision aids.

The scripts provide optional tooling for drafting analysis cards, scaffolding prompts, checking prompt structure, and supporting evaluation runs.

Lightweight TypeScript helpers

The repository includes zero-dependency TypeScript utilities designed to run directly with Node.js 22.18 or later.

For example:

node skills/compass-prompt-framework/scripts/analyze-requirement.ts \
  "Should we migrate our analytics workload to ClickHouse?"

This drafts an analysis card from the requirement.

A prompt scaffold can be generated with:

node skills/compass-prompt-framework/scripts/scaffold-prompt.ts \
  "Fix the flaky checkout-service test."

An existing prompt can be structurally checked with:

node skills/compass-prompt-framework/scripts/validate-prompt.ts \
  prompt.md

These helpers are intentionally modest.

They use heuristics to assist the workflow.

They do not magically determine the true intention behind a request.

And a structurally valid prompt is not automatically a successful prompt.

That distinction matters.

Validation of the framework's structure is not validation of the framework's effectiveness.

Repository checks

From the project root, the documented development checks can be run with:

npm ci
npm run check

The checks cover package integrity, reference consistency, tests, and linting, with the documented Node.js and Python requirements.

They help ensure that the skill is packaged consistently and its supporting materials remain synchronized.

They do not establish that CoMPaSS produces better model outputs.

That requires a different kind of evaluation.

The Research Question: Does Analysis-First Prompting Actually Help?

This is where I want to be particularly precise.

CoMPaSS is built around a hypothesis:

For consequential tasks, explicitly identifying the user's intention before constructing the prompt may improve alignment between the generated result and the outcome the user actually needs.

That is plausible.

It is not yet experimentally established for this framework.

A useful prompt framework should not be evaluated merely by whether its structure feels logical or whether its examples look impressive.

It should be tested against alternatives.

The repository includes a pre-registered evaluation design intended to do exactly that.

The proposed evaluation

The plan covers approximately 150 real-world requests across communication, analytical decisions, and agentic coding.

Each request is paired with a hidden statement of the requester's true intention.

The evaluation compares several conditions:

Plain, unstructured requests.

ART.

RTF.

CO-STAR.

RISEN.

Full CoMPaSS.

CoMPaSS variations with selected components removed.

It also calls for reasoning and non-reasoning models across multiple vendors, with meaning-preserving prompt layout variations.

The primary outcome is intention-match rate.

In other words: did the generated result actually serve the objective the requester intended?

Additional measurements include task success, Success-spec compliance, token consumption, latency, clarification behavior, and sensitivity to formatting changes.

For tasks with automatic metrics, the plan includes an optimized-prompt baseline using techniques such as DSPy or GEPA.

That last point is important.

If a task has a reliable automatic evaluation metric, programmatic prompt optimization may be more effective than manually designing a framework.

I do not want CoMPaSS to compete with a better optimization method simply because I created it.

I want it to be useful where its trade-offs make sense.

What would count as evidence?

The primary hypothesis focuses on whether CoMPaSS improves intention-match rate over ART and RTF on Tier 1 and Tier 2 tasks.

The proposed analysis uses paired uncertainty intervals and reports token costs alongside the outcome.

If the measurements do not support the hypothesis, that is a result worth publishing.

If CoMPaSS helps with some classes of tasks but introduces too much overhead for others, that is also useful.

If a simpler approach performs just as well, I want to know.

A framework should earn its complexity through measurable results, not through the confidence of its author.

At the time of this release, those benchmark results do not exist.

The evaluation plan is published so that the hypothesis can be tested openly rather than assumed.

You can read it here:

CoMPaSS Evaluation Plan

The Trade-Offs I Don't Want to Hide

CoMPaSS introduces additional structure.

That structure comes at a cost.

1. More analysis means more time and tokens

The Comprehend stage requires effort.

If an agent performs it, that may increase inference costs. If a human performs it, that increases preparation time.

For a consequential task, that investment may be justified.

For a trivial task, it almost certainly isn't.

That is why the Tier 0 gate is part of the design.

2. Intention analysis can be wrong

This is the most significant conceptual risk.

A model can misinterpret the user's intention and then construct a highly persuasive prompt around that mistake.

The ambiguity gate reduces this risk by requiring clarification when confidence is low and explicit assumptions when confidence is medium.

It does not eliminate the risk.

3. Structure is not a substitute for information

A detailed prompt cannot compensate for missing requirements, unreliable data, inaccessible tools, or an invalid premise.

CoMPaSS can expose missing information.

It cannot manufacture trustworthy evidence.

4. The structure may behave differently across models

Different models respond differently to instructions, examples, delimiters, and output formatting.

The default CoMPaSS section order was selected for clarity and maintainability.

It is not proven to be the optimal ordering for every model.

5. Existing techniques already solve parts of this problem

The components themselves are not entirely new.

Objectives, context, constraints, examples, and verification criteria have appeared in prompting practices for years.

The proposed contribution of CoMPaSS is their organization around a mandatory intention-analysis stage, explicit complexity and ambiguity gates, conditional optional components, and a mandatory definition of done.

That is the combination I want to evaluate.

There is no reason to pretend that every ingredient was invented from scratch.

What I Want CoMPaSS to Become

I see CoMPaSS as part of a broader effort to make AI-assisted engineering more disciplined.

AI agents are becoming capable of operating across larger codebases, using external tools, and handling increasingly complex workflows.

But capability without a clearly understood objective creates a different class of risk.

An agent can execute a long sequence of steps flawlessly and still deliver something the user never actually needed.

As we give agents more responsibility, the quality of task specification becomes increasingly important.

Not just instructions.

Not just tool permissions.

Not just context windows.

But explicit intentions, operational boundaries, and evidence that the result satisfies the requirement.

My longer-term interest is in systems where agents can receive ambiguous requests, identify the relevant objective, choose an appropriate execution path, and report their outcomes against verifiable criteria.

CoMPaSS is one experiment in that direction.

There is more to investigate.

How accurately can different models infer intention?

When does asking clarifying questions improve outcomes enough to justify the interruption?

Which components actually contribute to task success?

How much additional token cost is acceptable for consequential tasks?

Can intention-analysis errors be detected before they influence downstream execution?

Those are engineering and research questions, not marketing claims.

And they are the questions I want this project to help explore.

Open Source, Reproducible, and Open to Criticism

CoMPaSS is released under the MIT License.

The repository includes the skill implementation, its supporting research document, examples, helper scripts, tests, development documentation, and an evaluation plan.

I especially welcome contributions that challenge the framework's assumptions.

A real task where CoMPaSS misidentifies intention is useful.

An example where the framework introduces unnecessary complexity is useful.

An evaluation showing that ART or a plain request performs just as well is useful.

So is a proposed rule change backed by evidence.

I would rather improve a small framework through credible criticism than preserve an elaborate framework that cannot justify its existence.

If you're experimenting with AI agents, prompt engineering, context engineering, or model evaluations, the repository is open for exploration and contributions.

GitHub: YourAKShaw/CoMPaSS-prompt-framework

Installation: CoMPaSS on skills.sh

Research paper: Read the CoMPaSS Paper

Evaluation: Read the Evaluation Plan

Final Thoughts

The more time I spend building AI systems, the more I think we need to distinguish between making models produce better-looking outputs and making those outputs genuinely useful.

These are not the same problem.

A polished answer can be wrong.

A successful tool invocation can advance the wrong objective.

A passing test can conceal a broken system.

And a perfectly structured prompt can fail before the model generates a single token.

CoMPaSS starts from that observation.

It asks us to identify the intention before constructing the instruction, include only the structure the task justifies, and define success in terms that can actually be checked.

Whether this approach produces measurable improvements remains an open research question.

The implementation and evaluation plan are now public.

My guiding principle is simple: before teaching an AI agent how to complete a task, make sure we understand what completing the right task actually means.

That is the idea behind CoMPaSS.

And that is the question I want to keep testing.

Built and maintained by Ayush Kumar Shaw, founder-engineer working on backend systems, AI agents, developer tooling, and practical automation.

CoMPaSS is open source under the MIT License. Contributions, reproducible evaluations, and constructive criticism are welcome.