Most public sector pilots fail before they start, because nobody agreed what they were testing

The vocabulary problem is a commercial problem

The guidance opens by acknowledging that piloting has come to mean many different things across government. That sounds like a semantic complaint. It is not. Each mechanism sits at a different point in the project and produces a different output. A policy trial answers whether the idea is worth pursuing. A proof of concept answers whether something can be built and produces a service definition. A scoping phase produces a specification you can procure against. Test and learn finalises requirements so they can be competed. A pilot smooths out the operational and logistical wrinkles before scale.

Get the label wrong and you get the contract wrong. If you call a requirements exercise a pilot, you will have written a contract for implementation when what you actually needed was a route to a second, competitive procurement once the requirements settle. The guidance is explicit that after a test and learn, a two step procurement will usually be appropriate because the requirement may change significantly between the tested model and the final one.

The commercial trap is at the other end too. If a pilot produces material changes to the service specification, and the original tender documents did not provide for them, you may be looking at retendering the rollout. The same applies to substantial cost changes. Naming the test properly forces you to think about that in advance.

What organisations most often get wrong

Three failures recur. The first is running the pilot to justify a decision that has already been made. The guidance is blunt that pilots are not for testing policy appropriateness, exploring alternatives or justifying a business case. A pilot designed to succeed will succeed, and it will teach you nothing you can use.

The second is underestimating time. The guidance notes that a complex test and learn or pilot could take several months or more than a year to test an end to end service and its impact on stakeholder groups, and that time must also be planned in to complete the feedback loop and make changes before scaling. Programmes routinely allow for the test but not for the learning, which is the only part that produces value.

The third is treating supplier involvement casually. Where a third party helps with a trial or scoping phase and may later bid for delivery, the department has to be able to show that involvement gave no real or perceived advantage. That means deciding upfront how information will be used and how the subsequent procurement will run, and it means legal advice, not optimism.

The financial model tells you who really owns the risk

The guidance presents two broad options for paying for a pilot and is honest about both. Costing the pilot as a distinct phase with specific charges is clear and transparent, suppliers perceive it as fair, it limits exposure if the pilot disappoints, it enables a clean break, and it supports the department taking intellectual property. The cost is that most cost risk sits with the department, total project cost may be higher, and scope creep or overruns bite.

Folding pilot costs into total contract pricing transfers cost risk to the provider, drives efficiency and motivates the supplier to make the pilot succeed and move to full implementation. The trade off is that pilot costs stop being transparent, they may be priced into the fully implemented service and raise total cost of ownership, and providers are likely to want payment guaranteed if full implementation is cancelled.

Neither is right in the abstract. But the choice you make reveals what you actually believe about the pilot. If you are confident enough to fold the cost in, you may not need a pilot. If you genuinely expect to learn something that could change the answer, pay for it as a distinct phase and keep the option to stop.

What to do differently on your next programme

Start the testing conversation before the procurement conversation. The guidance says planning should begin at the earliest strategic stages and be incorporated into the delivery model assessment, sourcing strategy and, where relevant, the procurement documents. If your first mention of a pilot appears in the draft specification, it is already late.

Write the evaluation methodology at the same time as the objectives, and write down in advance what you will do if the results are mixed rather than clearly good or bad. The guidance asks teams to consider the potential outcomes of the evaluation, the next steps or alternative approaches if the test is not fully successful, and the impact on overall project timescales. That paragraph is where most of the value sits.

Finally, talk to the market about the test design itself. Suppliers have run these before, in your sector and others, and the guidance repeatedly recommends testing the design with them. A short market engagement on how to structure the pilot is cheap compared with a rollout that stalls because the pilot answered the wrong question.

The takeaways

• Name the mechanism precisely. Policy trial, proof of concept, scoping phase, test and learn and pilot answer different questions.
• A pilot is the final rehearsal before rollout, not a way to test policy, explore options or justify a business case.
• First time outsourcing should include a pilot. If you skip one on a complex project, justify it in the business case.
• Plan time for learning and change, not just for running the test, and decide in advance what a partly successful result means.
• How you pay for the pilot decides who carries the risk, and it should match how genuinely open the outcome is.

Want the full breakdown?

The complete explainer covers the key facts, the requirements in detail and a practical action list, free and printable in the Procurement Library.