Rubble / Blog
How Can Software Engineering Learn from History in the AI Era?
When we are about to remove an interface-conversion layer that looks redundant, we usually start by finding out when it was introduced.
Git can find the corresponding commit and show which files changed. We may also be able to find the associated requirement and tests. But the code can still look strange: if there is already an internal data structure, why maintain another representation that is almost the same?
The comment says, "for compatibility with older versions."
Which older versions? Are they still in use? Was the layer added to support clients that had already been released, or because some tool temporarily did not support the new format? Were other approaches considered? If we remove this layer directly, what exactly will break?
The answers to these questions can directly change the decision we make today.
Suppose the design record from that time says that the conversion layer was introduced to protect three types of legacy clients: A, B, and C. We can now confirm that A and B have been retired, but we cannot find enough evidence to tell whether C is still in use.
We now know the history, but we still cannot simply delete the code.
History does not provide the answer here. It only narrows a vague question into one fact that genuinely needs to be established today:
Does C still need this compatibility?
In the earlier discussion of how an implementation earns the right to enter a system, we distinguished between an implementation that works and a change that can be accepted. The engineering system needs to understand the impact, obtain the corresponding evidence, and send a decision back for reassessment when new results invalidate the original judgment.
Following that question further leads to another difficulty.
Even if software state, changes, and verification results are all preserved well, we may still not know why the system was changed in that particular way, how that path was found, or whether what happened in the past can help us make the next judgment.
This problem did not begin with AI.
What is different is that as coding agents participate in more development work, much of the exploration that used to be scattered across people's memory, logs, and different tools now has a chance to be preserved as a more continuous process.
The question then becomes:
What kind of history can actually become useful engineering experience later?
Version History Does Not Preserve the Whole History
When one version follows another, it is easy to think of software history as the accumulation of changes over time.
That accumulation is real.
For development work, the easiest thing to understand is the current point in time.
After implementing a requirement, the first questions are usually: Is the code correct? Do the tests pass? Is the requirement complete? Can this commit be merged?
All of these questions matter. They ask whether the software is sound at this particular point.
But software does not stop at that point.
Code merged today will be changed again tomorrow. A dependency will continue to be upgraded. An interface will gain new callers. A test suite will continue helping future engineers understand the original behavior. Countless implementations, fixes, migrations, and refactorings continue over time until they form the software system we see today.
So when we extend our view from one point to a span of time, the questions change as well.
We begin asking: Is this dependency properly isolated? Are the tests verifying behavior, or have they become too tightly coupled to the current implementation? Will this design be easy to change later? Will a relationship introduced today cause an ordinary change several months from now to require modifications across many places?
A small decision may cost almost nothing at the time, yet gradually become a constraint that the system can no longer ignore after subsequent changes accumulate around it.
In this sense, the system software engineering faces today is the result of countless past acts of programming, design, decision-making, migration, compromise, and change. Changes accepted today settle into history, and that history becomes part of the conditions tomorrow's changes have to deal with.
But looking only at this accumulation over time still does not show the whole history.
Suppose an internal module is about to expose an interface to external systems.
Exposing the internal data structure directly is usually the easiest implementation. Another option is to add a conversion layer between the internal structure and the external interface so that the two use different representations. A third option is not to expose the interface yet and first solve the problem within a smaller scope.
At the time, each option may have had good reasons behind it.
┌─ Expose the internal structure directly
Current state ──────┼─ Create an independent external interface ──→ New state
└─ Do not expose the interface yetOnly one of these paths is eventually taken.
Knowing which path was taken is useful, but it is not enough. When we face the same interface, dependency, or architectural boundary again, what we really need to know is why that path looked more appropriate at the time, and whether those conditions still hold today.
Knowing why a past choice was made is not about inventing an explanation after the fact. It lets us judge more accurately which reasons still apply, which have disappeared, and what conditions our new choice will leave for the future.
If we look only at the implementation that was eventually merged and the version diff, we can usually tell where the system ended up. It is much harder to recover which alternatives were still available at the time, and why some of them were not pursued.
More importantly, the choice changed more than the code at that moment.
If internal fields become part of a public interface directly, external clients may begin depending on the meaning of those fields. When the internal model needs to change later, the team also has to consider client compatibility.
Adding a conversion layer increases current implementation and maintenance cost, but it may allow the internal structure to continue evolving independently.
Both approaches may work today and even pass the same functional tests, but they leave different conditions for the next change.
Software history therefore cannot be understood simply as the accumulation of a number of changes.
Every choice is made under particular conditions, and every choice changes the conditions for later choices.
One way to make this easier to see is through "point, line, and plane," all viewed in terms of state change.
A point is one specific state.
In development, it first corresponds to the version, change, and result we are dealing with now: Is the implementation correct? Do the tests pass? Can the current state be accepted?
A line is the actual evolution of these states over time.
S0 → S1 → S2 → S3Implementation and change accumulate one after another, and this is how software gradually becomes what it is today. Once we look along this line, we care not only about whether each individual point works, but also whether the system remains understandable, changeable, and able to continue evolving after all of these changes.
But around every state, other choices were also possible.
S2a
/
S0 → S1 ── S2b
\
S2cOnly one of those paths is actually taken.
We can think of these possible directions as a "plane." This is not intended as a precisely defined state-space model. It is simply a reminder that knowing what was done in the past and what result it produced is not enough.
We also need to know what conditions existed at the time, why one option seemed more appropriate, which paths were abandoned, and which choices changed the space in which the system could continue evolving.
A timeline can tell us when changes happened and in what order. It cannot explain all of this by itself.
How Today's Choice Becomes Tomorrow's Condition
When discussing candidate changes and their long-term consequences, the previous article distinguished between change load and commitment cost.
Change load is about which existing relationships a modification has to deal with now. Commitment cost is about which relationships will need to be maintained, kept compatible, migrated, or eventually ended after the change is accepted.
Viewed historically, these two concepts connect the past, present, and future.
Past choices
│
▼
Form today's state, relationships, and commitments
│
▼
Become conditions today's change has to deal with
│
▼
Make a new choice today
│
▼
Change how the system can continue evolving in the futureData, dependencies, and commitments formed in the past become part of today's change load.
A change accepted today then changes the conditions for future changes.
But this is not simply an accumulation of cost.
A stable interface may constrain arbitrary changes to internal semantics while allowing more modules to evolve independently. A migration creates current cost but may also end a long-term commitment. A compatibility layer adds another implementation layer, but may preserve the option of running old and new systems side by side for some period of time.
A choice may close some paths and open others.
Engineering is not about preserving the maximum number of options. It is about deciding which constraints are worth accepting in exchange for which capabilities.
This also gives us another way to look at technical debt and refactoring.
Allowing one module to read another module's internal state directly may save only a few lines of code today. As more callers copy the same approach, replacing the lower-level module eventually requires several higher-level systems to change together.
The problem is not merely that maintenance cost rises little by little.
Changes that were once independent now require many parts of the system to move together.
Conversely, a refactoring may add no functionality, improve no performance, and temporarily increase the amount of code, yet still reestablish boundaries that let several parts of the system change independently again.
Its value may become visible only when later changes arrive.
History is not the same thing as technical debt either.
An interface used by customers for years, or a format storing important data, may be a normal commitment created by a perfectly reasonable decision. Software that continues providing value can hardly avoid forming relationships like these.
Even if maintaining two interface representations later turns out to have substantial cost, that does not automatically mean the original decision was wrong.
The team may have known from the beginning that two representations would need to be maintained and explicitly accepted that cost in exchange for isolation between the internal implementation and external protocol.
What later facts can really tell us is how large the actual cost turned out to be, whether it matched the original expectation, and whether the property the decision was intended to protect was actually protected.
This also means that if we later need to understand the choice again, a final record saying only "we chose this design" is usually not enough.
We also need to know what cost was accepted, what the team intended to protect, and what later facts should cause the decision to be reconsidered.
Software Changes, and So Does Our Understanding of It
So far, we have mainly discussed how software itself forms a history.
But if a past choice was valid only under the conditions that existed at the time, and the software continues changing, then years later, knowing only what happened to the system is still not enough.
We also need to know:
Why did we think those conditions mattered at the time?
And:
Did later facts change that understanding?
Software development therefore contains two histories that affect each other.
One is how the software and its engineering state change.
S0 → S1 → S2 → S3The other is how our understanding of those states and relationships changes.
M0 → M1 → M2 → M3The two do not always move together.
The software itself may remain unchanged while an investigation changes how we understand it.
Conversely, once our understanding changes, we may update declarations, engineering boundaries, or the implementation itself, changing the software state in turn.
This is why preserving only the final version makes it difficult to preserve engineering experience completely.
The final version tells us what the system eventually became. It rarely preserves the full process by which we gradually came to understand why it should become that way.
The Final Successful Path Is Still Not the Whole Story
Design documents, ADRs, reviews, and decision records can preserve some of the reasoning.
But some understanding disappears during exploration before the final solution is ever reached.
In the three-stage GCC bootstrap described in the previous article, we asked AI to start from an empty directory, find the source, dependencies, tools, and system conditions, and gradually establish engineering units and their relationships.
The process took more than a day.
It did not begin with a complete map and then simply follow it.
The actual process was more like exploring a maze: discover a missing condition, add it, continue, and then find that a new execution result shows the original route or boundary does not hold. Go back, revise the relationships, and try another path.
Initial understanding
│
├─ Try one path → Failure → Inspect the result
│ │
│ └─ Revise the earlier understanding
│
├─ Try another path → Discover a new condition → Continue adjusting
│
└─ Establish executable relationships → Recheck in other environmentsEventually, the same engineering relationships could reproduce the three-stage bootstrap in other environments.
The final correct build relationships obviously matter. Future execution should not continue depending on the memory of the original agent.
But the final build relationships do not automatically preserve the process through which they were found.
If we keep only the final successful declaration, we know how to build it next time. We may no longer know why another path originally looked as though it should work, which failure exposed an omission, or what fact forced the original boundary to change.
Failure itself is not experience.
A failure may come from the approach itself, but it may also come from a version mismatch, a missing condition, an incorrect invocation, or a temporary fault.
What matters is:
Why did this failure change the earlier judgment?
Experience does not come only from failure either.
If an important relationship is independently verified again and again at different times and in different environments, the judgment may remain unchanged, but we gain more reason to continue trusting it.
The fact that GCC could complete the same three-stage bootstrap in other environments is an example.
Exploration can overturn an earlier understanding, or it can make that understanding more reliable.
Agents Make the Exploration Process Easier to Connect
Software engineers have always gone through this kind of exploration.
Terminals, IDEs, CI systems, issues, code reviews, and build logs have also preserved large amounts of digital traces for a long time.
The problem is that these traces are usually scattered across different places.
Reconstructing why a difficult investigation moved from approach A to approach B may require piecing together shell history, logs, commits, review discussions, and the engineer's own memory after the fact.
Coding agents do not make software development recordable for the first time.
The more practical change is that many steps within one task can increasingly be recorded and connected as one continuous process:
Read context
↓
Search and inspect
↓
Modify
↓
Invoke tools
↓
Observe results
↓
Revert or adjust
↓
Verify againTraces that used to be scattered are therefore easier to preserve as they happen and associate with the task and engineering state at that time.
But saving the entire agent conversation still does not mean that we have captured engineering experience.
First, we need to distinguish how an agent "thinks" from the engineering trail we can actually observe.
How a model internally arrives at a judgment is not an engineering fact that can be completely and reliably recorded. Reasons explicitly stated by an agent may also be wrong, and a fluent explanation generated afterward cannot replace the operations that were actually performed and the results that were actually observed.
What is more reliable from an engineering perspective is:
What it was given, what it actually read, which assumptions it explicitly proposed, which operations it performed, what results it observed, and how those results changed what happened next.
The fact that a tool actually executed a command is something we can verify.
The agent's judgment that "this failure was caused by the toolchain" remains an explanation that requires further verification.
The two do not become the same kind of fact simply because they appear in the same record.
Likewise, old logs, issues, or agent records retrieved into a new task are still only material for the current judgment. They do not gain new authority to issue instructions or execute actions merely because they have entered a new agent context.
What Really Needs to Be Preserved Is the Basis for a Judgment
Saving one long agent conversation by itself may still be of little use several years later.
The record says:
This version is not supported.
Which version?
This cannot be changed.
Because of which constraint?
Approach A has already failed.
Under which environment, dependencies, and task objective did it fail?
What is worth associating with an engineering state, then, is not just a conclusion, but the conditions and evidence that supported that judgment at the time.
Engineering state at the time
│
└─ Task and objective
│
├─ Candidate approaches
├─ Known constraints
├─ Actual observations
├─ Evidence supporting the judgment
└─ Final choiceWhat we are discussing here is engineering basis that can be stated explicitly and checked later.
We do not need to pretend that we can completely reconstruct how a person or model internally arrived at a decision.
There is a natural correspondence here with the engineering state discussed in the first three articles.
An engineering result is formed from conditions such as source code, toolchains, configuration, and environment. When conditions that actually affect the result change, the old result may need to be reestablished.
Engineering judgments work similarly.
If an important condition supporting an earlier judgment changes, if new facts weaken the original evidence, or if important evidence appears that was previously unknown, the judgment needs to be reassessed.
Engineering result
│
└─ Depends on a set of engineering inputs
│
└─ Key input changes → Result needs to be reconsidered
Engineering judgment
│
└─ Has a set of conditions and supporting evidence
│
└─ Key basis changes → Judgment needs to be reassessedThe two obviously cannot use exactly the same algorithm.
Within controlled relationships, a build system can mechanically determine which results become invalid when an input changes.
Whether a customer is still worth supporting today, or whether a maintenance cost remains acceptable, often requires new business facts and human judgment.
But at least one principle is the same:
Do not separate a conclusion from the conditions that made it valid.
Returning to the compatibility layer from the beginning, we can now at least know whom the original decision protected and which fact is still unconfirmed today.
But that is still only the basis left by the past.
One question remains:
Did what happened later change how we understand the original judgment?
Later Results Need to Feed Back into the Earlier Judgment
Even after we record context, exploration, and decisions, experience does not form automatically.
A judgment that looked reasonable at the time may later encounter new facts.
History begins to become experience when what happens afterward can feed back into the original judgment.
How an Environment Setting Later Gained Another Meaning
A build environment explicitly requires an English locale.
The original reason is straightforward: some build scripts parse tool output. If a tool produces different text depending on the machine's language environment, string matching in those scripts may fail.
So the team already has a clear piece of experience:
Locale affects tool messages, so the build environment must not depend on the machine's default setting.
Later, a build completes normally.
The tool output is in English, the script's checks behave correctly, and the artifact is produced successfully.
A few days later, however, the test team discovers that a string-sorting result differs from before.
The issue crosses several teams and departments and takes a long time to investigate. The source code, build process, and directly related logic all appear normal. Eventually, the problem is traced back to the same environmental condition that seemed to have already been handled: one side used the POSIX C locale, while the other used en_US.
The problem is that locale is not simply a switch for "which language to use."
In POSIX locales, tool messages and string collation are separate concerns, normally controlled by LC_MESSAGES and LC_COLLATE respectively. Two environments can both produce English messages without using the same collation rules.
This incident did not invalidate the original experience.
The build scripts really did need stable tool output.
What changed was our understanding of this environmental condition.
Earlier understanding:
locale
│
└─ Affects tool messages
│
└─ Affects script decisions
Later understanding:
locale
├─ Affects tool messages
│ └─ Affects script decisions
│
└─ Also affects collation rules
└─ Affects program or test resultsThe original rule was not wrong.
We simply thought we understood the input when we had understood only one of its effects.
This case has something in common with the compatibility layer from the beginning.
A later fact does not necessarily prove that an earlier judgment was wrong.
Sometimes the original judgment really was based on a mistaken understanding.
Sometimes the original judgment was entirely reasonable at the time, but the conditions that made it valid later disappeared.
And sometimes, as in the locale case, the original judgment remains valid but we later discover an important effect that nobody knew about at the time.
Later results therefore should not be used simply to put the past on trial.
What we need to compare is:
Which conditions did we think mattered at the time? What cost did we accept? What did we hope to gain? And how do the things that happened later relate to those judgments?
Once we know more later, we can change how we understand the past today. But we should not rewrite history into:
We knew the real reason all along.
What was observed, believed, and attempted at the time should remain part of the record.
New evidence and the way it changed that understanding should be linked back as well.
What we really need to preserve is the process by which an incomplete judgment gradually became more accurate.
What Experience Improves First May Not Be the Answer
This gives "learning from history" a more practical meaning.
After the locale investigation, the team does not merely learn:
The last sorting problem was caused by locale.
A more useful lesson is:
When the same strings sort differently across environments, locale and collation rules should be among the conditions checked early.
That does not necessarily tell us the answer to the next problem. But it changes where the next investigation begins.
Past experience may also suggest several directions at once.
Similar problems may previously have come from a Unicode library version, a base image, or some other environmental difference.
The useful response is not to mechanically repeat the exact sequence from last time. It is to use what we see today to choose an early check that can distinguish between these possible causes.
An experienced engineer does not necessarily know the answer sooner.
Often, they simply know sooner:
What should I check first to eliminate a large class of wrong explanations quickly?
So the first thing experience improves may not be the answer.
More often, its value is:
Helping the next investigation obtain genuinely discriminating evidence earlier.
If past records have been associated reliably enough with the corresponding engineering objects and states, then when a new task touches them again, a person or agent can first retrieve the relevant earlier decisions and exploration, and compare what is different between then and now.
If a build path failed in the past only because of a tool version that has since been retired, "it failed before" should not remain a permanent prohibition today.
What experience really changes is where the next judgment starts. It does not provide the next conclusion in advance.
Experience Should Not Remain in History Forever
But when an understanding has been repeatedly verified and becomes stable enough that we no longer need to reconsider it every time, should it still remain only as a piece of history that must be reread?
Usually not.
Knowledge that can be executed explicitly can move into dependency declarations, environment constraints, tests, diagnostic tools, or other automated checks.
The exploration history can continue to explain:
Why this rule exists, what it protects, and which changed conditions would make it worth reassessing.
Even after experience has become an automatically enforced rule, it is better not to sever that connection completely.
Otherwise, several years later we may end up with another familiar problem:
CI has always required this setting, but nobody knows why anymore.
Automation can reduce repeated judgment. It should not recreate a tradition whose reason has been lost.
At the same time, the fact that something can be recorded does not mean everything should be preserved forever.
One agent task may produce large amounts of searches, temporary attempts, repeated output, and intermediate information that loses value quickly.
If all of it is permanently fed into future context, new engineers and agents instead have to spend more time deciding what still matters.
Secrets, sensitive data, and permission boundaries do not disappear simply because these records may be useful later.
Long-term records should focus first on the points that genuinely changed important judgments:
- Which important assumption was overturned by evidence;
- which important verification made a key judgment more trustworthy;
- which failure changed the direction of an investigation;
- which engineering boundary was changed as a result;
- which cost was explicitly accepted;
- which condition determines whether an old path still applies today.
Knowledge that has become stable can move into executable mechanisms.
Records that are no longer meaningful should also be allowed to be archived or closed.
Preserving history itself must not become another maintenance burden that grows without limit.
The goal is never to preserve more.
What matters is that when we face a similar change later, we can more accurately know why something was done in the past, which conditions still exist, and what is most worth checking next.
How Software Engineering Can Really Learn from Its Own History
In the past, much exploration was understood and compressed by people into personal experience.
An engineer who has maintained a product for many years gradually learns which areas cannot be understood from code alone, which problems are more likely to come from the environment, which convenient-looking interface will become difficult to retire, and when continuing to patch something is no longer as useful as redrawing the boundary.
These judgments are valuable, but much of them is difficult to inherit completely.
Documentation, reviews, and organizational processes have always tried to solve this problem, but they usually preserve only the part that has already been selected and organized.
The new opportunity in the AI era is not that AI can decide which experience will remain correct forever.
What has changed is that more of the exploration once scattered across code, logs, tools, and individual understanding can now be naturally connected within a single task.
State, context, tool operations, failures, reversals, changes in judgment, and actual results can begin to form relationships while the work is happening instead of depending on the person who performed it to reconstruct everything from memory afterward.
But recording is only the first step.
For history to genuinely become experience, later results need to be able to come back and challenge or support the earlier judgment. We need to know under which conditions a conclusion holds. We need to reassess it when those conditions change materially instead of inheriting it mechanically. And knowledge that has become sufficiently stable needs to continue moving into engineering mechanisms that can actually be executed.
Only then does the past become more than something we preserved.
It changes where the next exploration begins.
Several years later, when we again prepare to remove the interface-conversion layer from the beginning of this article, the ideal starting point should no longer be only a commit and a comment saying "for compatibility with older versions."
We should be able to know whom the original decision was protecting, why that path was chosen, what cost was explicitly accepted, what happened afterward, and which conditions are still unconfirmed today.
The past will not decide what we should do now.
A piece of history becomes engineering experience when it tells the next judgment what the earlier conclusion depended on, which changes are worth reassessing, and what to check first when facing new uncertainty.
About the author
Chai Feng is a Senior Technical Consultant and Team Coach at Odd-e, with over 20 years of experience in software and hardware R&D and engineering management. His work focuses on modern software development, software architecture, large-scale software engineering, build and CI/CD systems, and AI-assisted development.