Running Startups at Speed and Scale
What limits speed and scale at a startup? How do you scale without slowing down? What accelerates your production?
Contents
Lightning Talk #
Prior to the age of commonplace LLMs, engineering output itself was somewhat of a “moat” for a business. The goal of a startup was to find product-market fit and deliver the product. “Speed” was measured by traditional output metrics (e.g. lines of code, velocity, pull requests, deployments) and going faster was a relatively simple matter of hiring more engineers, or hiring better engineers. That is no longer the case now that AI helps produce that same traditional engineering output at an unprecedented scale, especially when a non-engineer themselves can quickly produce a usable product.
But it’s important to remember that generating code is not the same as safely delivering a product. Review cycles, verification, refactoring, and more are embedded in the process of delivery. Modern engineering practice has made great strides in automating some of that effort (e.g. CI/CD, testing, detecting vulnerabilities, observability, etc). And with the advent of agentic AI, we see further gains with AI having greater participation in these activities as well. Ultimately, all of these practices serve the same goal: from an engineering perspective, whether the process involves AI or not, we wish to remove as much risk as possible from shipping code. What we are now experiencing is that engineering output is no longer the bottleneck.
But when you eliminate one bottleneck, another always appears.
In fact, that bottleneck doesn’t disappear; it moves. If Engineering can produce a new feature in two hours, but Product takes a week to make a decision, Legal takes three weeks to review it, Sales doesn’t know how to position it, or Customer Support only learns about it when the first user calls, the business is still slow. There is feedback to collect, system health to observe, and more, much of it outside engineering entirely. The rest of the organization itself has to be able to absorb the newfound speed engineering is harnessing.
Real speed comes from removing human hesitation.
So we don’t need AI to simply accelerate engineering delivery; it needs to help shorten the time it takes for the entire company to move from ideation, to decision, to production, to customer experience, and to learning from feedback. The goal isn’t simply to make more software. In an AI-accelerated environment, that becomes an increasingly easy way to generate activity without meaningfully translating it into progress. Making a lot of software is easy now, and can paradoxically be the least helpful thing to do. The question is no longer how much we can produce, but how quickly we can learn what is worth producing.
And if producing software is no longer scarce, neither is access to the tools that produce it. These AI tools are widely available, so having them is no kind of moat either. The real magic is in how they are used to handle more complexity and make more decisions, without a matching increase in people and process. That is what scale really means: not simply serving more users, but handling more customers, more deployments, more complexity, and more decisions without requiring a proportional increase in headcount and process.
Once code becomes cheap, judgement becomes expensive.
AI has made writing code dramatically cheaper. But it has done little to make building a business easier. The hard questions are no longer simply “can we build this?” but “should we build it?”, “does anyone need it?”, “can we earn their trust?”, and “can we operate it reliably?” The startups that will succeed in the modern age aren’t the ones obsessed with making things faster. They will be the ones that master moving the whole organization forward together by learning faster, deploying more confidently, and becoming more efficient without adding proportional complexity via headcount and process.
Lean Coffee: Executive Insights #
The Common Thread #
The five topic sets that followed were, in effect, five field reports on where the bottleneck went: to distribution in the first, to production readiness in the second, to unmeasured quality context in the third, to judgement in the fourth, and to organizational context in the fifth.
One theme cut across nearly all of them: executive alignment. Whoever sets the pace, usually the CEO, is frequently the person least able to tell a prototype from a finished product. So when the wider organization cannot keep up with what engineering can now produce, that does not first show up as a tooling or process gap. It shows up as a disagreement between the executive setting expectations and the team doing the work, and it has to be settled there.
Topic Set 1: Selling, Not Building #
Chosen topics: Effective marketing for a tech startup product · Building is easy, selling is hard · Ideal feedback format and method from the sales team
Anyone can produce a usable product now, so the difficulty moved wholesale to distribution. Marketing has always run on a slower clock than engineering, and buyers are saturated: one participant reported customers telling him plainly to come back in six months because they cannot absorb any more change. And because roughly ten other companies have already built something similar, what it costs a customer to leave you for one of them has collapsed too, so a strong run of new customer signings can mask an organization that is categorically failing.
What seems to work
- Describe the product in the language buyers already use, even where that understates the innovation. EC2 was sold as servers because nobody was searching for virtualized instances in the cloud.
- Find the people who have already said publicly that they have the problem you solve, and go to them.
- Work on keeping customers at the same time as winning them, rather than treating churn as a later problem. You can hit every new-customer target for the quarter and still be shrinking, because the customers you won last year are leaving out the back door.
- Sell against the cost of being wrong rather than the price, which is also the only pricing anchor available when no comparable exists.
Unresolved. How to be found by agents rather than people; everyone agreed it is now the channel that matters and nobody had a method. Whether a startup has any moat at all was a genuine disagreement.
Topic Set 2: Speed vs. Readiness #
Chosen topics: Engineering speed vs. product safety · Speed vs. process in this vibe coding culture · Think small with features
The discussion moved off speed versus process almost immediately. Generating code is not delivering a product; AI made generation cheap and left validation as expensive as ever, so the question the room kept returning to was how you prove something is ready for production. The case that dominated was a CEO comparing a four-hour vibe-coded change against a two-week sprint, and shortening the sprint did not help, because the comparison point is always the afternoon.
What seems to work
- When an executive pushes for speed, shift the conversation to what has to be in place before anything reaches a user, and show him the actual list: data quality testing, regulatory standards, model testing policy, a record of who asked for the work and who signed off where a requirement was not met. Presented as requirements rather than as process, the obligations are far easier for a non-engineer to accept.
- Put a number on the downside and let that number do the arguing. Tie the work to a measurable reduction in organizational risk, then translate it into consequences leadership already pays for: a lower cyber insurance premium, a cleaner audit with fewer findings for customers to read. Not writing the thirty million dollar check is a result an executive recognizes.
- Separate prototyping from implementation explicitly. Prototyping is where AI pays off best, and treating a prototype as a nearly-finished product is what creates the timeline everyone then argues about.
- When the executive is going to proceed regardless, agree to it and build the guardrails behind the scenes rather than trying to block it.
- Where the CEO himself is the obstacle, the board or the investor relationship is the more effective lever. A few pointed questions at the next board meeting about how a specific risk is being managed changes behavior in a way no internal escalation will.
Unresolved. Whether this is a new problem or an old one moving faster. Shadow IT was raised as the precedent, the room did not settle it, and one participant called that the real question; it matters, because one answer calls for culture work and the other for new controls. Related and equally unanswered: when a non-engineer or an outside consultant creates the debt, who pays it down, and when?
Topic Set 3: Debt, Quality, and What Got Cut #
Chosen topic: Tech debt at speed
The trust problem in software is old; the volume is new, and review broke because commits are now massive. More consequentially, organizations are cutting quality capability on asserted rather than validated grounds. One cancelled a mature QA tooling suite for five million a year in claimed savings while its AI spend was too distributed to measure. Another laid off twenty QA engineers; the real loss there is context rather than labor, and context does not appear on a P&L.
What seems to work
- Add the new capability first, then assess each existing control for genuine redundancy. The goal was never zero risk; it is moving risk to a level you are comfortable releasing at.
- Automated testing strategy, not review, is the safety net that makes speed survivable. Where AI does the reviewing, run several agents over the same change and treat what they agree on as the real signal; any one of them alone is not trustworthy.
- When a non-technical leader asks why quality needs its own people when the engineers could do it, the answer is not about skill; they may well have the same skills. Dividing the work into distinct roles is what lets you scale the team up and down as the workload changes. One person can run the fryer and the register at a McDonald’s, and you will not like what comes out when a rush arrives.
Unresolved. Whether measuring health after deployment is a strategy or just a staffing decision. Larger organizations are shifting toward operations reviews and production incident counts as the signal, and nobody in the room believed those organizations had worked out how, the objection being that it measures the loss only once it has already happened. Also open: how to put a number on organizational context before somebody cuts it.
Topic Set 4: Outcome Metrics #
Chosen topics: Outcome metrics were always the most important for the business. Why has AI highlighted the importance of these metrics? · What are the right things to measure? · What should the unit of work be that AI helps create (epic, story, code, test, business plan, marketing plan, website)?
Leadership asks what return AI is producing, and what comes back is cycle time, throughput, and defect counts, useful engineering measures that say nothing about return. Closing that gap is genuinely hard, because nobody can honestly claim that a particular feature produced a particular amount of revenue; the two move together rather than one causing the other. Which means a business case built on a promised revenue number cannot settle an argument with another one built the same way: three teams each promise five million, there is capacity for one, and there is no honest basis for choosing.
Two further observations sat alongside this. Where leadership publishes no goals of its own, teams reverse-engineer them from the earnings call. And nobody wants to kill anything, at a moment when killing things has never been cheaper.
What seems to work
- Name the specific lever the work moves: gross margin, the unit economics of the product, the cost of building software, or a line that genuinely touches revenue. A general claim that something creates value does not survive a finance conversation; a named lever does.
- Require a stated expectation of return from whoever defines the work, before it is built. Building the wrong thing is cheap now and compounds quickly, so writing down what you expect it to move is the only real control left.
- Let each level’s goals be drawn from the level above it, while leaving the decision of what to actually do to the person who owns the goal. A target handed down as an instruction buys compliance and not much effort; a goal someone writes themselves, within their manager’s, buys ownership. It also forces an open conversation about where the bets go, with everyone working from the same information.
Unresolved. The unit-of-work question itself. What do we estimate, review, and measure when AI produces an epic, a test, or an entire business plan at the same cost? The sharpest question asked all session, and untouched.
Topic Set 5: AI Context for Non-Coders #
Chosen topic: How do you manage AI context for non-coders?
An executive connects a tool straight to the data warehouse, produces a wall of charts in an afternoon, and asks why the dashboard team needs two weeks. He is not wrong to ask. But every non-technical user is working in a silo with an agent that has no schema, no templates, and no conventions, producing output that is fast, unrepeatable, and accountable to nobody.
What seems to work
- Treat it as provisioning and identity first: what is installed, what it connects to, own credentials or service accounts, and who may add a connection.
- Centralize the context artifacts (connectors, rules files, templates, verification criteria) and have someone maintain them, so users inherit context instead of rebuilding it badly.
- Make the sanctioned path the better path. A tool that knows your schema beats a consumer chat window, and people choose it without being told to.
Unresolved. Consistency across tools that do not return the same answer twice, and who owns AI infrastructure for non-engineering functions. Nobody claimed the role. Terminology also diverged when someone asked what an agent actually is (a coding tool to one person, a chat with capabilities to another, an orchestrator of sub-agents to a third), and the discussion had to be steered back to the original question.
Threads Across the Conversation #
Executive alignment. The most persistent theme, present in four of the five sets. A prototype and a product are now visually indistinguishable, so leadership calibrates on demo velocity and reads everything after it as obstruction. The work between demo and production was always invisible; it is now invisible and unexplained. Most of what worked was translation rather than engineering: presenting readiness as a list of requirements instead of as process, and expressing debt in money and risk instead of principle. Where translation failed, the only remaining option was structural: going around the CEO to the board or the investors.
Nobody can measure how much change a market will take. The real ceiling on all of this is how much new product a customer base, or an organization itself, can actually take on and use. The room supplied two data points for that ceiling, and both came from outside engineering: customers telling one participant to come back in six months, and a leader describing his own users refusing a velocity he had invested heavily to reach, well before AI. What nobody could offer was a way to know in advance. The ceiling gets discovered by hitting it, which is generally after the thing is already built.
AI made existing problems faster; it did not create them. Shadow IT became shadow AI, proof-of-concept-to-scale failures predate AI, and QA was already being used badly. What AI adds is cover: someone who does not understand an outcome can point at what every other company is doing, feel confirmed by the match, and skip asking where this actually fits. Diagnose the underlying weakness before buying a control for the AI-shaped symptom.
What is left of the moat does not appear on a P&L. The candidates the room settled on were proprietary data, operations only your people perform, and accumulated organizational context. The Spirit auction was the illustration: no buyer for the leases, the gates, or the trained pilots, and ten million dollars for the data. Every asset on that list is unmeasured, which is exactly why the cost decisions described earlier in the same session (cancelling a mature QA suite, laying off twenty QA engineers) cleared approval. You cannot defend what your accounting system cannot see.
Cheap code made bad decisions cheap too. Four examples came up over the course of the session, and they share a shape:
- A strategy generated by AI and adopted without testing it, on the assumption that a plan and a proven plan are the same thing.
- Millions committed to scale a proof of concept from a thousand orders to thirty thousand, on the evidence that it ran well at a thousand.
- AI added to a product so that the product can be sold as an AI product.
- Three teams each promising five million in revenue, with capacity to build one of them.
None of these took much effort to arrive at, all of them survived a room, and not one was tested before it was acted on. That shape is the thing to watch for, rather than any single example of it.
James Nall is Software Engineering Manager in Austin Texas.
