How GitHub Copilot, Claude Sonnet 4.5, and a 444-credit refactor led me to question whether AI credits are measuring software understanding rather than software generation.

---

Why This Investigation Started 🚀

On June 1, 2026, GitHub Copilot transitioned from its request-based pricing model to usage-based billing using AI Credits. Instead of charging primarily for the number of premium requests, Copilot now measures actual consumption based on model usage and token volume. For many developers, this was more than a pricing change. It was the first time we were given visibility into the economics of AI-assisted software development.

Like many engineers, my initial reaction was practical rather than philosophical.

• How many credits was I consuming?
• Would I need a larger plan?
• Was Claude Sonnet 4.5 significantly more expensive than other models?

At the time, I was actively developing a healthcare platform written in Go, backed by PostgreSQL and Neo4j. The codebase had evolved into a fairly sophisticated system containing authentication flows, recommendation engines, patient-context reasoning, explainability layers, graph traversals, and a growing collection of business rules.

AI assistance had become part of my daily workflow. GitHub Copilot was no longer just autocomplete. It was reviewing code, generating migrations, debugging workflows, writing tests, and proposing refactors.

Then one morning I opened the Copilot dashboard and noticed something alarming 😳. I had consumed 821 AI credits in a single day. That works out to roughly $8.21 for one day of development, which felt steep given that I was not working on a major feature or refactor.

"GitHub is extorting power users."

After that initial emotional reaction subsided, curiosity took over 🤔.
Engineers are naturally suspicious of numbers that don't align with their mental model. Whether it's CPU utilization, query latency, memory consumption, or build times, an unexpected metric usually indicates that some underlying system is behaving differently than expected.

I assumed the explanation would be straightforward.

• Perhaps Claude Sonnet 4.5 was simply expensive.
• Perhaps agent mode was inefficient.
• Perhaps I was unknowingly triggering costly workflows.

What followed was more interesting than I expected. The more I examined where the credits were going, the less this looked like a billing problem and the more it looked like an information problem, and eventually an architecture problem.

The surprising realization was that Copilot seemed to spend far more effort understanding my system than generating code for it. That observation became the starting point for everything that follows.

---

Context 🧩

This entire experiment was conducted using:

Component	Details
AI Tool	GitHub Copilot
Model	Claude Sonnet 4.5
IDE	VS Code
Codebase	Production healthcare platform
Language	Go
Database	PostgreSQL
Graph Database	Neo4j
Architecture	Growing monolith with multiple reasoning layers

This is not a toy repository. The system contains:

Capability
User registration and authentication
Recommendation engines
Patient context processing
Health scoring logic
Explainability layers
PostgreSQL persistence
Neo4j graph reasoning
Multi-step workflows

As the system evolved, so did its complexity.

---

What I Initially Assumed 🧠

My initial assumption was straightforward and, in hindsight, entirely reasonable.

I expected AI credit consumption to correlate primarily with the amount of work being delegated to the model. A large feature spanning multiple files, database migrations, tests, and business workflows should naturally consume more credits than a small bug fix or a localized refactor. In other words, I assumed that AI cost would scale roughly with implementation effort.

This mental model closely mirrors how engineers estimate work. Larger changes typically require more design, more coding, more testing, and therefore more time. It seemed reasonable to expect that AI systems would exhibit similar economics.

The observations that followed challenged that assumption almost immediately.

As I began tracking individual tasks, I discovered cases where relatively simple refactors consumed hundreds of credits, while substantially larger feature implementations consumed only a fraction of that amount. The discrepancy was large enough that it could not be explained by implementation effort alone.

Something else was contributing to the cost.

The investigation that followed would eventually reveal that generating code was often the least expensive part of the process. (agent load) The real cost appeared to lie in the work required before code generation could even begin.

---

Discovery #1: The Cost of Finding the Answer 🔍

The first meaningful clue emerged from an unexpected source: GitHub Copilot's context breakdown. Like many developers, I initially assumed that prompt quality was the primary determinant of AI efficiency. If the model was consuming too many credits, perhaps my prompts were too verbose, too vague, or simply poorly structured.

Then I looked at the actual context composition being sent to the model.

Context Source	Percentage
Files	31.7%
Messages	9.4%
Tool Results	4.7%
User Instructions	3.5%

The numbers were surprising 😮.

Over the previous few days I had spent considerable effort refining prompts, experimenting with different instructions, and trying to communicate requirements more clearly. Yet the telemetry revealed that prompt content represented only a tiny fraction of the total context window. Repository files, on the other hand, dominated the context.

The implication was difficult to ignore.

The model was spending significantly more effort understanding the codebase than understanding my instructions.That was the first moment where my original mental model began to break down.

Experiment 🧪

To test this hypothesis, I started reducing the amount of architectural discovery required by the agent.

Instead of issuing broad requests such as:

Implement patient context support.

I began explicitly identifying the likely area of interest:

Relevant Files:
- scoring.go
- recommendation_engine.go

Implement patient context support.

The change was deceptively simple.

• I was not providing additional requirements.
• I was not changing the model.
• I was not altering the implementation itself.

I was merely reducing the amount of searching the agent needed to perform before it could begin useful work.

The effects were immediate ⚡.

• File context usage dropped noticeably
• Repository exploration decreased
• Credit consumption improved
• Responses became more focused

The model had not suddenly become more intelligent. It simply had less discovery work to perform.

Observation 👀

This led to the first significant insight of the investigation:

The expensive part was not generating the answer.

The expensive part was finding where the answer lived.

In retrospect, this makes intuitive sense.

Before an AI agent can modify a system, it must first build a mental model of that system. The larger the search space, the greater the amount of context that must be acquired before implementation can begin. This was the first hint that AI credits might be measuring something more interesting than code generation.

They appeared to be measuring understanding.At this point I was still convinced the problem was pricing.The evidence was beginning to suggest otherwise. I initially assumed the solution was simply declaring that GitHub was charging too much Unfortunately, AI credits appear to be immune to this strategy 😅.

Much like Michael Scott discovering bankruptcy !

---

Discovery #2: Feature Size Does Not Predict Cost 📉

Up until this point, I still believed that AI credits were broadly correlated with the amount of work being performed. Discovery #1 had shown that context acquisition mattered, but I assumed implementation effort would remain the dominant factor. A larger feature should still cost more than a smaller one.

The next set of observations challenged that assumption directly.

The data suggested that feature size was a surprisingly poor predictor of AI cost.

Example A: Login Service Refactor 🛠️

One of the most expensive tasks I observed during the investigation was a relatively straightforward refactor of the login service.

Task	Credits
Login Service Refactor	444

At first glance, nothing about this task appeared unusual 🤷.

• The business requirements were clear.
• The implementation itself was not particularly complex.
• The scope seemed reasonably contained.

Yet the AI consumed 444 credits completing the work. At the time, I simply assumed this was an expensive outlier. I would later discover that it was actually an important clue.

Example B: Forgot Password Feature 🔐

A few days later I delegated the implementation of an entirely new forgot-password workflow. Unlike the refactor, this was a genuine feature enhancement touching multiple parts of the system.

The implementation included:

Component	Included
HTML Template Changes	✅
JavaScript UI Logic	✅
PostgreSQL Migration	✅
OTP Persistence	✅
SMTP Integration	✅
API Endpoint #1	✅
API Endpoint #2	✅
Password Hashing	✅
Validation Rules	✅
OTP Cleanup Logic	✅
Testing	✅

The implementation required changes across:

• Frontend templates
• Backend APIs
• Database schema
• Email infrastructure
• Validation logic
• Test workflows

In other words, this was the sort of task I would normally expect to be significantly more expensive than a localized refactor.

It wasn't 🙃.

Task	Credits
Forgot Password Feature	69

The entire feature cost 69 credits. The login refactor had cost 444 credits. The discrepancy was too large to ignore.

Why This Was Interesting 🤨

At this point, the relationship between effort and cost no longer appeared linear. If AI credits were primarily measuring implementation effort, the numbers should have looked very different.

Instead, the opposite appeared to be happening.

The larger task was dramatically cheaper than the smaller one. The obvious question became:

What was the model actually spending credits on?

The answer seemed increasingly clear.

• The forgot-password feature had been specified in extraordinary detail.
• The business logic was documented.
• The implementation steps were documented.
• The acceptance criteria were documented.
• The testing strategy was documented.
• The environment details were documented.

Very little discovery was required. The model did not need to determine what to build.

It merely needed to build it.

The login refactor, on the other hand, required significantly more exploration and interpretation before implementation could begin.

Observation 💡

This was the moment my original theory collapsed.The technical size of a task was not a reliable predictor of cost. The amount of understanding required before implementation began appeared to be a far stronger predictor.

For the first time, I began to suspect that AI credits were measuring something closer to cognitive effort than implementation effort.

This was the moment the investigation became genuinely strange 🤯. A complete forgot-password feature cost 69 credits. A login refactor cost 444. The data appeared to be arguing with me. As every engineer knows, arguing with production telemetry rarely ends well.

---

Discovery #3: Specification Quality Matters More Than Prompt Length 📋

At the beginning of this investigation, I interacted with Copilot in the same way many developers interact with AI systems. The conversation was informal, iterative, and intentionally conversational. Typical prompts looked something like:

The login flow seems wrong.
Can you investigate?

These interactions felt natural / expresive 🙂

The model would explore files, ask follow-up questions, inspect related code, formulate hypotheses, and gradually work its way toward a solution. The process often resembled onboarding a new engineer to an unfamiliar subsystem. Over time I began experimenting with a different approach.

Instead of conversational prompts

I started supplying what looked much closer to engineering specifications. A typical request would include:

Handler:
#sym:getUserStatusHandler

Modification:
...

Justification:
...

Test:
...

Validation:
...

The difference was subtle but important.

• I was no longer describing a problem.
• I was describing a solution boundary.

The model no longer needed to determine:

• Where the change belonged
• Why the change was required
• What success looked like
• How the implementation should be verified

That information was already present.

Two Issues Resolved ✅

One of the first experiments using this format resulted in:

Metric	Result
Issues Solved	2
Credits Consumed	153.56
Credits per Issue	~77

The results were encouraging but not yet conclusive.

Then came a more surprising observation.

Critical Bug Fix 🐞

A production issue involving database logic, code changes, build validation, and testing was resolved using a similarly structured specification.

Metric	Result
Credits Consumed	13

Thirteen credits.!!! 😲

At this point, a pattern was beginning to emerge.

The AI was spending dramatically less effort on discovery and dramatically more effort on execution. The distinction may sound subtle, but it is fundamental.

In the earlier conversational workflow, the model was effectively acting as both architect and implementer. It needed to understand the problem, construct a mental model, identify the likely solution, and only then begin implementation.

In the specification-driven workflow, much of that reasoning had already been completed by the human.

The model could focus primarily on execution.

Why This Matters 🎯

As engineers gain experience, they tend to spend less time writing code and more time constructing mental models. Architects rarely operate at the level of individual functions. They operate at the level of systems, constraints, interfaces, and outcomes.

Looking back, the structured prompts were effective because they encoded that mental model directly into the request.

The AI was no longer being asked:

"What should we do?"

It was being told:

"Here is what we need to do. Please implement it."

That distinction turned out to be remarkably important.

Observation 🧭

This led to one of the strongest conclusions of the investigation:

The cheapest AI task was not the smallest task.

It was the best specified task.

Or stated differently:

The more thinking that occurred before the prompt was written, the less thinking the AI needed to perform after receiving it. - cognitive load in building the mental model was being transferred from the AI to the human.And less thinking, quite often, meant fewer credits.

---

Discovery #4: Tool Orchestration Is Surprisingly Expensive ⚙️

One of the more surprising observations emerged from tasks involving Git operations, pull requests, deployments, and CI/CD workflows. Unlike the previous examples, these tasks involved relatively little code generation. Most of the implementation work had already been completed.

The remaining activities consisted primarily of:

• Git operations
• Commit creation
• Push workflows
• Pull request management
• Deployment orchestration
• CI/CD pipeline validation

One such task consumed:

Task	Credits
Git / Deployment Workflow	88.3

For comparison:

Task	Credits
Critical Bug Fix	13

At first glance, this seemed counterintuitive 🤨.
How could a deployment workflow consume significantly more credits than resolving a production bug?
The answer became clearer after examining the actual interaction pattern.

More Than Just Git 🌿

The task was not simply:

git add
git commit
git push

The workflow involved multiple rounds of observation, interpretation, and debugging.

After deployment, CI/CD pipeline failures required investigation. Build logs were inspected, pipeline outputs were copied into the conversation, hypotheses were evaluated, corrective actions were proposed, and the process repeated until the deployment succeeded.

The interaction pattern looked more like:

Observe
↓
Interpret
↓
Hypothesize
↓
Modify
↓
Deploy
↓
Observe Again

rather than:

Implement
↓
Done

This distinction turned out to be important.

Tool Orchestration Cost 💸

In earlier examples, the model spent most of its effort implementing well-defined requirements.
In deployment and CI/CD scenarios, the model spent much of its effort coordinating tools and interpreting outputs.
The AI was repeatedly:

• Running commands
• Reading logs
• Interpreting failures
• Proposing fixes
• Running additional commands
• Validating outcomes

The implementation effort itself was relatively small.
The coordination effort was not.

An Alternative Interpretation 🔁

There is another possible explanation worth acknowledging.

• Much of the CI/CD investigation was conversational in nature.
• Pipeline logs were pasted into the chat.
• Failure modes were discussed.
• Potential causes were explored and eliminated.

This resembles the earlier pattern observed with loosely specified prompts, where the AI must spend significant effort constructing an understanding of the problem before it can propose a solution.

Viewed through that lens, the high credit consumption may not have been caused by Git operations themselves.
It may have been caused by the discovery process surrounding the deployment.

The distinction is subtle but important.

A deployment task may involve very little implementation work while simultaneously requiring substantial investigation, interpretation, and iterative reasoning.

Observation 🧪

This led to another refinement of the emerging model.

AI cost appeared to be influenced not only by implementation effort and context acquisition, but also by the amount of coordination and discovery required during execution.

In other words:

Building a solution is often cheaper than figuring out why the solution did not work.

Any engineer who has spent an afternoon debugging a CI/CD pipeline will likely find that observation familiar 😵.

---

Discovery #5: Modularization Helps 🧱

By this point, a pattern had started to emerge.

The most expensive tasks consistently shared a common characteristic: They required the AI to build a large mental model before useful work could begin.

The relationship appeared straightforward:

Better Architecture
        ↓
Less Context Acquisition
        ↓
Lower AI Cost

At first, this felt like speculation 🤔. Then I realized something uncomfortable.

The same parts of the system that were expensive for the AI to understand were also becoming difficult for me to reason about. The codebase had evolved significantly over time.Like many successful projects, it had accumulated responsibilities faster than it had accumulated boundaries.

The symptoms were familiar:

• Large packages
• Growing responsibilities
• Cross-cutting concerns
• Increasing cognitive load
• Expanding dependency graphs

Most importantly, the repository was beginning to challenge my own mental model.

There were moments where I already knew what needed to change, but still found myself spending time rediscovering where those changes belonged.

That observation turned out to be important.

If the system was difficult for an experienced developer to navigate, perhaps it should not be surprising that an AI agent experienced similar challenges.

The Experiment 🧬

Rather than optimizing prompts further, I decided to attack the problem from a different angle.

I began modularizing parts of the repository.

• The goal was not primarily to reduce AI credits.
• The goal was to improve navigability.
• Responsibilities were separated into smaller domains.
• Package boundaries became more explicit.
• Related functionality was grouped together.

The resulting structure provided clearer signals regarding where changes should occur.

Initial Results 📊

The experiment is still ongoing, but the early observations were encouraging.

Change	Result
Clearer package boundaries	Improved navigation
Smaller domains	Reduced exploration
Reduced context span	8-10 credit savings on comparable tasks

The savings themselves were not dramatic.

In fact, if the story ended there, the results might appear underwhelming. What interested me more was the consistency of the effect. The AI appeared to spend less effort searching and more effort implementing. The change was modest but directionally correct.

Why This Matters 🧠

One of the easiest mistakes to make when evaluating AI costs is to focus exclusively on prompt engineering. Prompt quality certainly matters. However, prompt engineering assumes the AI already knows where to look. Architecture determines whether it can find the answer efficiently in the first place. In hindsight, the repository structure was acting as a form of documentation. Clear package boundaries communicated intent. Smaller domains reduced ambiguity. Well-defined responsibilities constrained the search space. The AI required less context because the architecture itself provided stronger clues.

Observation 🔎

This remains one of the weaker conclusions in the investigation because the sample size is still small.I cannot confidently claim that modularization reduces AI costs by a specific percentage.

What I can say is this:

The same architectural changes that improved my ability to reason about the system also appeared to improve the AI's ability to reason about it.

That observation feels significant.
Because it suggests that AI optimization and software architecture optimization may not be separate activities at all. They may simply be different perspectives on the same underlying problem:

How much understanding is required before a safe change can be made?

At this point the billing dashboard had evolved from a financial instrument into an architectural critic. The first system that complained about my package boundaries wasn't a developer.
It wasn't an intern.
It wasn't a code reviewer.
It was an AI agent armed with a credit meter.
I was not emotionally prepared for that feedback channel 😅.

---

The Bigger Realization 🤯

As the investigation progressed, I noticed something unexpected happening 👀.

The question I was trying to answer had quietly changed. When I began, my focus was almost entirely economic. I wanted to understand why certain tasks consumed more AI credits than others and whether there were practical ways to reduce those costs. The problem appeared to be a billing problem.

By the end of the investigation, I was asking a very different question.

How much understanding is required before useful work can begin?

That may sound like a subtle distinction, but it fundamentally changes the nature of the problem.The first question is concerned with cost. The second is concerned with comprehension.One asks how much an AI agent charges to perform a task. The other asks how much information an AI agent must acquire before it can perform that task safely.The observations throughout this article repeatedly pointed in the same direction.The largest costs rarely appeared during implementation. They appeared during discovery.

The AI consumed credits while locating relevant code, constructing a mental model of the system, understanding business requirements, interpreting deployment failures, identifying boundaries, and determining where a change should occur.

Only after that process was complete did implementation begin.This realization reframed the entire investigation.The billing dashboard was not merely exposing the economics of AI-assisted development. It was exposing the economics of understanding.Viewed from that perspective, the observations began to make much more sense.

• Large, well-specified features were often inexpensive because little discovery was required.
• Small, ambiguous changes were often expensive because substantial discovery was required.
• The dominant variable was not always the amount of code being generated.
• It was the amount of understanding that had to be acquired before code generation could begin.

And that is a very different kind of problem. It is not primarily a billing problem. It is an architecture problem.

---

Context Economics 💰

The mental model that emerged looks something like this:

AI Cost
=
Context Discovery
+
Requirement Discovery
+
Architecture Discovery
+
Tool Orchestration
+
Implementation

Most discussions focus only on implementation. My observations suggest implementation is often the cheapest component.

The expensive part is understanding.

---

A New Architectural Metric? 📐

Traditional software engineering gives us:

Metric	Measures
Latency	Runtime Performance
Memory Usage	Resource Consumption
Cyclomatic Complexity	Code Complexity
Test Coverage	Testing Surface

AI agents may be exposing something new.

Metric	Measures
Context Acquisition Cost	Architectural Navigability

This remains a hypothesis. But it is an interesting one.

---

The Human Parallel 👥

One observation kept repeating itself.The parts of the codebase that were expensive for AI to understand were also expensive for me to reason about.

When I found myself thinking:

"This monolith is becoming difficult to hold in my head."

the AI seemed to agree. The difference was that the AI left a receipt 🧾.

---

The Architect's Perspective 🏗️

As engineers progress, they naturally operate at higher levels of abstraction.

A developer may think about:

• Functions
• Classes
• Queries

An architect thinks about:

• Systems
• Boundaries
• Interactions
• Evolution

The scarce resource is no longer typing speed. It is attention. This may explain why architects naturally delegate implementation tasks to AI agents. Not because implementation is unimportant. Because their attention creates more value when focused on system-level concerns.

---

Task	Credits
Critical Bug Fix	13
Migration + Build + Test	32
Forgot Password Feature	69
Git / Deployment Workflow	88
Two Structured Issues	154
Login Service Refactor	444

Final Thought 🧾

I started this journey investigating GitHub Copilot billing. I expected to learn about pricing, instead, I learned something about architecture. The billing dashboard became a profiler.

• Not for CPU.
• Not for memory.
• Not for latency.
• For understanding.

The first system that complained about my architecture was not:

• a customer
• a developer
• a tester
• an intern

It was my AI billing dashboard.And that may be the most unexpected software architecture feedback mechanism I have encountered in years.

• Perhaps AI credits are not merely measuring computation.
• Perhaps they are measuring the amount of understanding required before computation can begin.

If that turns out to be true, we may have accidentally discovered a new way to think about software architecture in the age of AI.