Problems with Portas Pilots

There has been a spate of recent coverage on the failure of (some) Portas Pilots to spend money quickly. Defenders of the scheme point to the fact that it is better to spend money wisely and that it is too early to tell whether the pilots are working.

To my mind, this misses the wider point that we will never know whether the Portas Pilots work because of the way that central government set up and run the scheme. To understand whether a pilot has a causal impact on the performance of high streets we need to be able to assess what would have happened in the absence of intervention. To do this, we need a control group that can be compared to the group of high streets that got money. Government had a perfect opportunity to create this control group because it invited bids and then only selected a few of the schemes to get funding. As I explained at the time, there were several ways that this set up could be used to allow effective evaluation of the pilot. Government didn't implement these, so we'll never be able to properly assess whether the scheme worked: if Portas Pilots do better than other high streets, it could be because government chose the places with better potential. If they do worse, it could be because government chose the hardest places to 'turn around'. Learning from pilots involves tackling these issues head-on. As this hasn't happened we will never know.

Interestingly, I've been working on the wider issues for an NAO report. One of the responses we received on that report suggested that spatial policy (like the Portas Pilots) can't be assessed using rigorous impact evaluation. As the Portas Pilots demonstrate, this isn't correct. But as long as parts of government aren't willing to embed evaluation right at the start of policy development it might as well be.