AI can make an engineer faster without making product development faster.
That was the tension at the center of my conversation with Marc Osofsky, CEO of Jama Software. Requirements can be written faster. Code can be generated faster. Reviews can be automated. Yet none of this proves that a product reaches the market sooner.
I summarized the problem in one sentence:
“Productivity is local but real velocity is systemic.”
Michael Jastram
Marc had seen the same effect in software. Developers produce more code with AI, but the extra output moves into testing and other downstream stages.
“Product velocity in companies has actually actually decreased.”
Marc Osofsky
AI has not failed in that situation. It has done exactly what it promised. It has accelerated a task and shifted the bottleneck.
Measure the product, not the AI
A metric such as requirements generated per hour shows whether one activity became faster. It says little about product velocity.
Marc described Jama’s internal measurement as follows: They look at features delivered over time and then at adoption as a proxy for value. He also shared results from Spec-Driven Development at Jama.
“We proved out at a project level that we could go five times faster.”
Marc Osofsky
He later said that Jama had reached a 2.5 times increase at team level. These are Jama’s internal software results, not a benchmark for physical product development. The unit is delivered functionality, not AI activity (or, god forbid, tokenmaxxing).
An attendee asked how to quantify AI. My reaction was that the best AI metric may not be an AI metric at all.
If AI shortens testing, measure testing time. If it accelerates a change decision, measure the time until that decision is made. Look at how quickly stakeholder value arrives, not the local optimization.
Otherwise we risk a new version of the Watermelon Pattern. The AI dashboard is green while the product is still slow.
Free webinar on Anti-Patterns on October 6 with Marc Löffler (German)
The product context matters
Marc was unusually direct:
“Files are bad for AI they’re bad for humans. You basically have to create a context graph.”
Marc Osofsky
The problem is not the existence of files. The problem is that product knowledge remains scattered across them. A file alone does not tell an AI whether it contains the current answer or how it relates to another artifact.
Marc’s answer is a product context layer based on a graph of engineering knowledge. The graph makes relationships explicit while design artifacts remain in their native environments. The deeper change is not the database. It is what happens to engineering knowledge.
“We’re moving to a production system where all that implicit knowledge has to become explicit.”
Marc Osofsky
Much of engineering depends on people knowing what a document really means, which version matters, or whom to ask. AI exposes the cost of that implicit knowledge because automation needs the context that experienced engineers carry in their heads.
What becomes valuable when generation becomes cheap?
This is where Spec-Driven Development becomes interesting.
Marc described a model in which specifications capture intent while downstream artifacts become cheaper to regenerate. Software already shows what this can look like. If code can be recreated quickly, engineers spend less effort preserving every implementation and more effort keeping the specification right.
His formulation was deliberately provocative:
“The specific artifacts become almost disposable in an AI world.”
Marc Osofsky
That statement does not transfer literally to every physical engineering artifact today. A CAD model or validated hardware design is not as disposable as generated software code.
But the direction is relevant. As engineering tools become more programmable, AI can take over more of the generation work. The durable asset then shifts toward the knowledge that tells the system what to generate and how to judge the result.
That makes specifications part of a living product context rather than documents written early and handed over.
The fastest companies were fast before AI
Marc pointed out that some companies now associated with fast development, like Tesla, SpaceX, BYD, had already achieved that speed before generative AI. AI did not create their development system.
My own view is that product architecture and organizational architecture are part of the explanation. I put it this way during the conversation:
“You need to align the architecture of your organization with the architecture of your product.”
Michael Jastram
A company with stable platforms and controlled variation has a different starting point from one where every change crosses organizational boundaries. AI inherits those conditions.
Marc argued that the most established route so far is software-defined development. For physical products, this means moving variation into software where that makes engineering sense and creating interfaces that allow more of the flow to be automated. AI then lands on an architecture already built for speed.
AI should change the process
One audience question asked whether a company should apply AI to existing tasks or redesign the process around the new capability.
Marc used semiconductor development as an example. Rather than making each manual handover a little faster, he described a flow where AI can trigger simulations through CI/CD pipelines and evaluate results without requiring a person to operate each tool step.
That changes the role of the engineer.
“The humans are up front and on the back end and they’re guiding and they’re governing. But they don’t have to be artisans at each step.”
Marc Osofsky
This is a different ambition from adding an AI assistant to the current workflow. It asks which parts of the workflow still need to exist once AI can perform work continuously between human decisions.
Many AI pilots preserve the old process and accelerate one box inside it. That is easy to start with. It is also why the gain often remains local.
The question I took away
Toward the end, Marc returned to measurement. His advice was to focus on the final outcome instead of localized optimization.
That gives me one question for any engineering AI initiative:
If this AI works exactly as promised, which end-to-end delay becomes shorter?
Michael Jastram
If there is no clear answer, the initiative may still be useful. It may save engineering effort or remove tedious work. But that is different from Product Velocity.
The more interesting AI applications change how work moves through the development system. They make product knowledge explicit and remove waiting that used to be built into the process.
We already know that AI can make tasks faster.
Harder question is whether the product moves faster.
