Rubble / Blog
What Is Build Engineering Solving in the AI Era?
Suppose a product family uses several processors, different memory capacities, and multiple peripheral hardware configurations. Most of the code is shared, but the toolchains, configuration options, and final artifacts are not exactly the same. Now a shared module needs to be changed. AI can quickly understand the interfaces, generate the code, and add tests, reducing what once took days to perhaps tens of minutes.
But for the build team, the work has only just begun: Which products are affected by this change? Can older models still be built? Do artifacts produced by different toolchains still meet the same acceptance requirements? Can the new dependency be used on every hardware platform? Did the local build succeed because everything was declared correctly, or because some package happened to be installed on the developer's machine?
AI lowers the cost of getting an implementation, but it does not automatically answer these questions. The easier implementations are to produce, the more the engineering system needs the ability to determine which changes can enter the system, what builds and rebuilds they will trigger, and how they should be verified. Functional, safety, and runtime impacts beyond the build still require judgment from other engineering functions.
Implementations Can Be Rewritten; System Relationships Cannot Be Reset as Easily
AI has reduced not only the cost of writing the first version of some code, but also the cost of changing it, throwing it away, and trying again. Asking it to use a different implementation, remove a dependency, or preserve an old interface often takes only a few additional requirements. Code that has not yet entered the system increasingly looks like an intermediate artifact for exploring a problem, rather than something that inherently represents an engineering commitment already made.
But that low cost exists only while the change has not become deeply embedded in the system.
Once an implementation enters the system, it gradually forms relationships with other components, interfaces, data, toolchains, runtime environments, and delivery requirements. These constraints do not divide neatly by industry. Communications equipment has explicit boundaries such as CPU architectures, memory specifications, vendor SDKs, and product models, but it also has to evolve APIs, protocols, and runtime environments over time. Large Internet systems may not have the same fixed combinations of hardware products, but they are still constrained by base images, language runtimes, database schemas, public APIs, and message protocols. Once many services depend on them, many of these become boundaries that cannot be changed casually either.
So AI may quickly rewrite a shared module or modify a service, but the cost no longer comes only from the implementation itself. A change to a low-level component may affect dozens of hardware combinations, or propagate through shared libraries, interfaces, and service dependencies into a large number of software systems. Different industries have different kinds of constraints, rates of change, and verification methods. The common problem is how to handle changes within these existing relationships. Code can be rewritten quickly. The relationships already built into the system cannot simply be erased with it. Database formats, public interfaces, hardware protocols, compiler toolchains, test assets, and product maintenance commitments all impose their own compatibility, rebuild, and verification costs on later changes.
There is another asymmetry that is easy to miss: the cost of regenerating an implementation has fallen, but the cost of understanding it may not have. AI can rewrite a module quickly, but reviewers still need to understand what implicit behavior changed, why a new abstraction was introduced, whether it adds new failure modes, and whether other product configurations are affected.
The faster generation gets, the larger the share of total cost that understanding, verification, and integration may take.
Why Verification Does Not Naturally Get Cheaper at the Same Rate
If AI can generate code, it can of course also review code, write tests, and analyze impact. So can generation and verification become cheaper together?
AI can indeed reduce some analysis and verification costs. But finishing an implementation and formally accepting a change are not the same thing. For AI, finishing an implementation usually means finding a path that satisfies the current requirements. For an engineering system, accepting that change also requires enough evidence for the important failure modes.
For example, an embedded driver compiling successfully on the default development board, with all tests passing, does not mean it is ready for the entire product family. Some models may use a different compiler version. Some may have tighter memory limits. Other configurations may enable different interrupt, concurrency, or low-power paths. These differences may not normally appear in the local code context available to AI, but they may determine whether the final artifact can actually be used.
This does not mean every product combination must be fully verified after every change. Engineering still needs to choose the level of verification based on the impact scope and the consequences of failure. A change with limited impact that is easy to detect and recover from can use lighter verification. A change involving a shared low-level module, a hardware interface, or a safety boundary needs stronger evidence.
AI can participate in implementation and verification, but the fact that the same AI agent generated both the code and the tests does not make those tests independent evidence. The agent may have carried the same mistaken assumption through both stages.
So AI can reduce the cost of verification, but it does not remove the responsibility for verification. The rate at which implementations can be generated and the rate at which an engineering system can accept changes remain constrained by different conditions.
The Real Bottleneck Is How Changes Are Handled
AI will not necessarily make every industry release software faster. In automotive systems, communications equipment, industrial control, and large legacy systems, product qualification and validation processes may continue to move at much the same pace. What changes is that more implementations and modifications can reach the entry point of the engineering system at lower cost.
If a large software project uses multiple coding agents at the same time, they can modify different modules very quickly. Taken individually, each change may be perfectly reasonable. Taken together, they may increase the number of dependencies, widen the range of toolchain versions, introduce redundant abstractions, or keep expanding the build and test matrix. The stronger code generation becomes, the more easily the verification queue can become the bottleneck.
So an engineering system needs more than faster pipelines. It also needs the ability to keep handling changes as they arrive. I will call this capability change handling here. It is not about processing more modifications faster. It means understanding the impact of a change, establishing evidence proportionate to the risk, and reaching explicit disposition decisions such as accepting, rejecting, deferring, narrowing the scope, or redesigning the change.
For example, AI may introduce a new package into a shared module, but that package may support only the toolchains used by some products. The reasonable response is not necessarily for the build team to make every platform work with it. It may instead be to require a redesign so that the new dependency is limited to the part of the system that actually needs it.
Stopping an unsuitable change in time is also an engineering capability. Otherwise, what appears to be faster delivery is simply turning time saved today into complexity that has to be paid for later.
Change handling should not become another throughput metric either. The amount of code generated, the number of merge requests created, and the number of builds executed do not by themselves show that engineering efficiency has improved. What matters is whether the engineering system can make clear decisions about these changes without diluting the evidence, blurring responsibility, or allowing complexity to grow without control.
Engineering Trade-offs Begin with Boundaries
To decide whether a change should be accepted, adjusted, or rejected, you first need to know which conditions can be traded off and which boundaries cannot be crossed.
Technical choices are rarely better in every direction. Improving performance may increase complexity. Expanding compatibility increases verification cost. Stricter isolation may reduce developer convenience. Engineering is not about making every metric as good as possible at the same time. It is about finding an appropriate balance within real constraints.
But balance does not mean everything is negotiable. Trade-offs only happen in the space that remains after the boundaries that cannot be crossed are clear.
In embedded and automotive systems, product-side resource budgets, real-time requirements, and safety requirements cannot be violated because of delivery pressure. On the build side, established execution isolation, toolchain ranges, and dependency constraints cannot simply be disabled to shorten build times. Pure software systems have non-negotiable boundaries as well, such as externally committed API compatibility, data formats, runtime platform requirements, and compliance constraints. The exact form of these boundaries varies by product, but once they become explicit product or engineering constraints, they can no longer simply be traded against development schedule.
Within those boundaries, many choices still need to be balanced. Should every hardware specification be verified after every change, or should the necessary set of builds and tests first be selected from the impact scope? Should a rarely used old toolchain continue to be supported, or should it be frozen on a maintenance branch? Should the system use more caching for speed, or reduce cache sharing for stronger isolation?
This is the more accurate meaning of "engineering is the art of compromise." Compromise is not an excuse to lower standards. It means deciding, within conditions that cannot be violated, what to protect, what to give up, and accepting the corresponding cost.
Risk acceptance authority has boundaries too. A project cannot redefine legal, regulatory, contractual, or safety requirements as ordinary technical choices through an internal approval simply because the schedule is tight. Having authority over the project plan does not mean having authority to waive external obligations.
Build Engineering Must Control What Can Affect an Artifact
As implementations arrive faster, it becomes difficult to keep handling growing change pressure by relying on engineers to remember product differences, inspect environments manually, and maintain pipelines one item at a time. Engineering requirements that are already explicit need to become repeatable system capabilities wherever possible.
Build Engineering here means more than running a compile command. It is the build and verification system around artifact production. It includes dependencies, toolchains, execution environments, testing, packaging, artifact provenance, and gating. Its concern is how a code change becomes a deliverable artifact, and whether that process can be explained and verified. Broader business decisions and governance of production operations are outside the scope of Build Engineering discussed here.
Doing this well requires solving at least two problems at the same time.
The first is to state clearly which factors are allowed to affect an artifact. Source code, third-party dependencies, compilers, system software, build parameters, hardware specifications, and product configurations can all change the final result. For a team with multiple product families, "the same source code" is far from enough to determine an artifact. Target hardware, feature options, and toolchains are inputs too.
The second is to ensure that, within a clearly defined build trust boundary, only those declared and authorized inputs can take part in the build. Platforms, hardware, and services that the build must still depend on outside that boundary should be listed explicitly as trust assumptions. No matter how complete the dependency declarations are, if build scripts can still read arbitrary files from a developer machine, invoke undeclared tools, or access the external network, the actual result can still be changed by accidental properties of the environment.
A common example is a developer machine that already has a vendor SDK installed. The build script finds it through the default search path, so local builds keep succeeding. Move the build to another machine or into CI, and it suddenly fails. A more troublesome case is when both environments build successfully, but find different SDK versions and produce different artifacts.
One limitation matters here: build systems are good at computing build and rebuild impact. When an input changes, they can determine which components and which product configurations need their artifacts rebuilt, and, under verification rules established in advance, which related checks need to run again. Functional behavior, safety risk, and runtime impact still cannot be determined from the build graph alone.
These problems cannot be solved simply by reminding developers to "remember to declare dependencies." The execution environment must actually constrain which files, tools, and network resources a build can read. If a build needs an undeclared input, it should fail and expose the missing dependency instead of quietly using whatever happens to exist on the current machine.
One way to think about these two conditions is "make it explicit" and "enforce it." The first states which inputs are allowed to affect an artifact. The second prevents undeclared inputs from affecting the build result. Declarations without enforcement still rely on discipline. Isolation without clear declarations makes it difficult to explain why the build is able to succeed.
Downstream Engineering Needs a Stable Artifact First
A common practice in the past was for the development team to build one copy of the software for its own testing, for the test team to rebuild from the same code, and then for the release environment to build it once again before deployment. On the surface, all three builds used the same code. In practice, they may have gone through three different toolchains, configurations, and system environments. What developers tested, what the test team passed, and what was finally deployed were not necessarily the same software artifact.
Even if a team standardizes its development environment, it is difficult to keep that environment completely identical over time. A tool upgrade, a configuration change, or one additional package installed on a machine can all change the build result. Unchanged source code does not mean an unchanged artifact.
Build Engineering therefore has to solve determinism first: the same inputs should produce the same output. Beyond that, a release artifact intended for delivery should have a stable, verifiable identity. In many cases, the same artifact can move through testing, security checks, release, and deployment. If it is rebuilt at different stages, the team must be able to show that the same inputs were used and the same output was produced. The fact that the source code is the same is not enough to treat the results as the same software artifact.
This is the foundation for everything that follows. The test team needs to know that the artifact it verified is the artifact that will actually be delivered. Security and compliance checks need to be bound to a specific artifact. The release system needs to record its provenance accurately. Only then can operations reliably deploy, roll back, and trace it. If the artifact itself does not have a stable identity, the evidence and controls that follow no longer have a stable object to attach to.
Build Engineering therefore sits in a bridging position. On one side, it takes the source code, dependencies, and configurations produced by development and turns them into stable artifacts. On the other, it connects to quality verification, software supply chain security, product qualification, release and deployment, and operations. It does not replace those control planes, but it gives them a common foundation to work from.
Not Every Requirement Can Become a Build Rule
A stable artifact gives downstream verification a stable object, but it does not mean every engineering property can be established automatically by the build system.
Build Engineering can turn many requirements into machine-enforced controls: dependency versions, supported toolchain ranges, access permissions, artifact size, API compatibility checks, and required tests. But not every engineering boundary can be verified independently by the build system.
Whether a communications device can run reliably over long periods under a particular load may require real hardware and long-running tests. Whether a control system degrades correctly under abnormal conditions may require a test bench or system-level validation. Some safety requirements also depend on deployment methods and runtime procedures.
A mature engineering system should not pretend that every requirement can eventually be encoded into a pipeline if only the policy language is expressive enough. What Build Engineering needs to do is turn the parts that can be machine-enforced into controls that are difficult to bypass, while making explicit which conditions still depend on human assessment, specialized testing, or evidence from the runtime environment.
The fact that something cannot be verified automatically should itself be a visible engineering fact. It should not remain hidden in the experience of a senior engineer.
The Smarter the Agent, the More Important Capability Boundaries Become
Agents make execution boundaries more important, not less.
Suppose an agent discovers a missing header file during a build. To finish the task quickly, it may download a package from the network. It may instead adjust the search path and use a version already installed on the host machine. From the agent's point of view, it has solved the build problem. From an engineering point of view, it may have bypassed the dependency declaration and introduced an environmental condition that cannot be reproduced reliably.
Agents also read repository documentation, issue records, and test logs. These sources may contain useful information, but they may also contain incorrect or even malicious instructions. Even if the final code looks reasonable, that is not enough to trust that the entire execution process used only allowed inputs.
Agents should therefore be treated like other highly capable execution subjects: with least privilege, input isolation, and behavior auditing. The fact that an agent can explain what it did does not mean that its explanation can replace system evidence. The smarter it appears, the less we can rely on it simply choosing to respect the boundaries.
AI capabilities broaden the range of work that automation can perform, but they also amplify the impact of mistakes. The more capable the system becomes, the more its permissions and execution boundaries need to be controlled by systems outside the model.
Build Systems Provide Evidence, but They Do Not Make Decisions for the Organization
Even if dependency declarations are complete, the execution environment is hermetic, and every build and test passes, that still does not show that an engineering choice itself is necessarily wise. The build system is responsible for executing verification requirements that have already been established and recording what has been completed and what is still missing. What evidence is sufficient still needs to be judged by the relevant engineering teams based on the product's intended use and risk.
For example, after upgrading a shared toolchain, the build system can calculate which products need new artifacts and run the corresponding verification. It can record the old and new results. It may also discover that some older hardware no longer passes verification. But whether to stop maintaining those old models, broaden the verification scope, or accept some residual risk is not something the build system can decide on its own.
"Acceptance" is not a technical truth computed automatically by a pipeline. The relevant specialists first need to explain the technical consequences of the change. Then a responsible party with the appropriate authority has to make the decision based on the intended use, applicable environment, key assumptions, residual risk, and available evidence. The same artifact may be acceptable for an internal experiment while still lacking the evidence required for a formal product release.
Conversely, the build system should not merely be a tool for helping a project find some way to pass. When a constraint cannot be enforced, verification costs turn out to be far higher than expected, or new results invalidate an earlier assumption, the engineering process should force the original decision back into assessment.
For example, impact analysis may show that an apparently routine upgrade to a shared component requires dozens of hardware combinations to be verified again. The right response is not necessarily to add more machines, shorten the tests, or request an exception. It may instead be to reconsider the scope of the upgrade, split the change, or defer migration for some products.
Decisions create constraints, and the engineering system enforces and verifies those constraints. New evidence produced by execution may in turn force those decisions back into assessment. This feedback capability matters. Without real backpressure and the authority to block a change, a pipeline is only an automated procedure, not an engineering control.
What Build Engineers Are Building
Mature Build Engineering has long been about more than maintaining Makefiles, shell scripts, and CI configurations. Dependency graphs, incremental builds, cache consistency, toolchain management, cross-platform compatibility, reproducibility, and software supply chain security are all problems build teams have been dealing with for years.
AI does not suddenly turn build engineers from script writers into constraint managers. A more accurate description is that AI will reduce a large amount of local, manual, experience-driven work while amplifying the systemic responsibilities Build Engineering already carries.
It is important here to distinguish the people from the system. Build engineers are not the control plane. Build Engineering is a key technical control plane in an automated software production system. It is not the whole of software governance. Through dependency relationships, execution isolation, permission rules, testing requirements, release gating, and evidence recording, it turns the parts of architectural, security, and quality requirements that can be automatically enforced into controls in the software production process.
Build engineers design, maintain, and evolve this mechanism. They translate the parts of safety, architecture, product maintenance, and organizational risk requirements that can be engineered into rules that cannot easily be bypassed. For the parts that cannot be enforced or verified automatically, they make the gaps explicit.
The value of build engineers therefore does not come from personally carrying out every build, or from continually copying another pipeline for every new product. It comes from building an engineering system that can keep handling product differences, environmental changes, and large-scale automated changes over time.
In the past, some dependencies and product differences could still be maintained through team experience. As AI makes implementations and modifications appear faster, this reliance on human memory and ad hoc handling will become increasingly difficult to sustain. Build Engineering matters not because scripts are becoming harder to write, but because the entire software production process increasingly needs explicit, stable controls that cannot be easily bypassed.
Conclusion
AI can produce changes faster than before, but it cannot automatically make those changes acceptable. Code being easy to generate does not mean its impact is easy to understand. An implementation being easy to rewrite does not mean a system change is easy to undo. An agent being able to check its own work does not make those checks independent evidence.
The goal of engineering is not to stop change, nor is it to build a system that never fails. It is to keep known risks under explicit control, make unknown failures harder to spread and easier to expose and trace, and make sure the organization can still recover, learn, and continue evolving after an incident.
For build teams, what really needs to be built in the AI era is not more scripts maintained by hand, but an engineering system that can keep answering these questions:
What can affect an artifact? Which product and hardware configuration artifacts need to be rebuilt after a change? Have the required verifications been completed? Which conditions still cannot be verified automatically? When the evidence no longer supports an earlier decision, can the system actually stop the related change from moving forward?
The easier implementations are to produce, the clearer the conditions for accepting them should be. The stronger automation becomes, the less boundaries, evidence, and the authority to block a change can depend on ad hoc judgment.
About the author
Chai Feng is a Senior Technical Consultant and Team Coach at Odd-e, with over 20 years of experience in software and hardware R&D and engineering management. His work focuses on modern software development, software architecture, large-scale software engineering, build and CI/CD systems, and AI-assisted development.