[Posted by Prof Henry G. Overman]
Just before Christmas, the National Audit Office published its long awaited report on Evaluation in Government.
I was part of the LSE team that contributed to that report,
specifically in terms of assessing the quality of evaluations. I thought I’d
highlight both the report (for those not aware of it) and some of our
central findings.
The team – Steve Gibbons, Sandra McNally and
I – looked at 35 UK government evaluations covering active labour
markets, business support, education and local economic growth
(including regeneration). We picked these four because policies in these
areas are targeted differently (e.g. some at firms, some at
individuals) – which then helps illustrate how evaluation deals with
some crucial methodological issues. These are also areas where external
perception of quality varies markedly – and where our team had
considerable expertise.
We were asked to highlight the strengths and weaknesses of
evaluations, to assess the robustness and the usefulness to policy
makers and to suggest improvements. We used the Scientific Maryland
Scale to rank studies – the same tool that will be used in the systematic reviews of the What Works Centre for Local Economic Growth.
What did we find? First, the quality of evaluation varies widely both
within and across these four policy areas. For example, we found
evidence of high quality evaluations in the areas of active labour
markets and education. In contrast, evaluations in the areas of business
support and local economic growth were considerably weaker. Second,
that quality range really matters for policymakers. On the basis of the
reports we saw, we judged that none of the business support or
local economic growth evaluations provided convincing evidence of policy
impacts. In contrast, 6 out of 9 education reports and 7 out of 10
labour market reports were good enough to have some confidence in policy
impacts.
How can policy evaluation get better? We think that using a control
group (or a counterfactual) should be considered a necessary (although
not sufficient) requirement for robust impact assessment and value for
money calculations. Business support and spatial policy evaluations, in
particular, could make better use of use administrative data and
improved evaluation techniques to construct these counterfactuals.
We also make some more technical recommendations about how to handle
policies where people can opt in; improving inference (i.e. how certain
we are about the effects of policy) as well as the interpretation of
impact estimates (do they apply to everyone ‘treated’, or just a
subset?). More care should be taken to distinguish between the analysis
of programme delivery (processes) and the assessment of impact and value
for money (outcomes). Finally, every impact evaluation needs a
technical appendix written for a specialist audience.
Overall, our verdict is mixed. We found a lot of very good
evaluations; and many others that could be easily improved. For other
areas, notably evaluations of business support and spatial policies, the
picture is more worrying. Of course, these are also some of the hardest
kinds of policies to evaluate robustly, but they are also central
elements in many councils and LEPs’ growth strategies. We'll be tackling these issues head on through the work of the What Works Centre for Local Economic Growth.
[A version of this post first appeared on the What Works Centre for Local Economic Growth blog]
CEP Urban and Spatial Programme Blog (formerly SERC)
Friday, 7 February 2014
Monday, 27 January 2014
Cities Outlook 2014: Would UK Cities be better off without London
[Posted by Prof Henry G. Overman]
Great to see Centre for Cities latest 'Cities Outlook' report published today. This year, as well as providing a whole range of up-to-date statistics for Britain's cities, the report focuses on London's relationship with the rest of the UK.
The report considers three broad issues. First, that London sucks talent from the rest of the UK, second, that London's success is a threat to other cities and third, that the rest of the country is a drain on London. I think the report is most convincing on the third of these - London is so reliant on attracting talent from elsewhere in the UK (when people are young) and on commuting from elsewhere in the South East (when people are older) that it make very little sense to think of London as some isolated island that could simply cut itself off from the rest of the UK and still prosper.
The second part of the analysis looks at the way in which firms headquartered in London create jobs elsewhere in Britain. I thought the figures presented here were fascinating, although not totally surprising: London, as you might expect, is far ahead of any other city in this regard. It's not so obvious, however, that this answers the question about the effect of London's success on the rest of Britain - we surely need to recognise that the location of headquarters is, itself, a firm decision that could be affected by the dominance of London.
I'd make a similar point about the first part of the report's analysis - the way in which London sucks talent from the rest of the UK. I don't disagree with the analysis - we know London attracts talent - although it's good to see the basic ideas and statistics so clearly presented. Where I depart from Centre for Cities, is in the way that we talk about this effect and the broader implications for cities across Britain. Personally, I would like to see the debate reframed, by recognising that London offers opportunities to young people that simply aren't available elsewhere. This moves us away from the rather parasitic implication of London 'sucking in talent' and instead raises the question of whether we could generate these opportunities elsewhere. As CfC put it: 'rather than focus only on London’s dominance, the more pertinent question appears to be: why aren’t other large cities offering people enough economic opportunity to stay — and what can be done about it?'
CfC's answer to this question is that we need to give cities greater powers if they are going to create these opportunities. While I am reasonably sympathetic with the plea for greater powers for cities, I am not so convinced that will create comparable opportunities in those cities. Indeed, all the evidence suggests that a big part of London's advantage in creating opportunities comes from its shear size. Diverse cities of nine million plus population simply generate a lot more opportunities. If somewhere else in Britain is going to generate similar opportunities to London, there's an argument to be made that it would also need to be pretty big - perhaps half the size of London. At most, Britain could feasibly support one, may be two cities, of this sort of size (in addition to London).
Of course, recognising the importance of size creates all sorts of problems about which places to focus on outside of London. CfC's report dodges the dilemma by making the case for greater localism for all cities. Most politicians choose to behave likewise, focussing on creating opportunities across Britain. But what if reducing the reliance of Britain on London requires us to significantly increase the size of just one or two other cities? Which cities will they be and how will we help them grow? I don't claim to have the answers, but I at least think we should be asking the question.
Great to see Centre for Cities latest 'Cities Outlook' report published today. This year, as well as providing a whole range of up-to-date statistics for Britain's cities, the report focuses on London's relationship with the rest of the UK.
The report considers three broad issues. First, that London sucks talent from the rest of the UK, second, that London's success is a threat to other cities and third, that the rest of the country is a drain on London. I think the report is most convincing on the third of these - London is so reliant on attracting talent from elsewhere in the UK (when people are young) and on commuting from elsewhere in the South East (when people are older) that it make very little sense to think of London as some isolated island that could simply cut itself off from the rest of the UK and still prosper.
The second part of the analysis looks at the way in which firms headquartered in London create jobs elsewhere in Britain. I thought the figures presented here were fascinating, although not totally surprising: London, as you might expect, is far ahead of any other city in this regard. It's not so obvious, however, that this answers the question about the effect of London's success on the rest of Britain - we surely need to recognise that the location of headquarters is, itself, a firm decision that could be affected by the dominance of London.
I'd make a similar point about the first part of the report's analysis - the way in which London sucks talent from the rest of the UK. I don't disagree with the analysis - we know London attracts talent - although it's good to see the basic ideas and statistics so clearly presented. Where I depart from Centre for Cities, is in the way that we talk about this effect and the broader implications for cities across Britain. Personally, I would like to see the debate reframed, by recognising that London offers opportunities to young people that simply aren't available elsewhere. This moves us away from the rather parasitic implication of London 'sucking in talent' and instead raises the question of whether we could generate these opportunities elsewhere. As CfC put it: 'rather than focus only on London’s dominance, the more pertinent question appears to be: why aren’t other large cities offering people enough economic opportunity to stay — and what can be done about it?'
CfC's answer to this question is that we need to give cities greater powers if they are going to create these opportunities. While I am reasonably sympathetic with the plea for greater powers for cities, I am not so convinced that will create comparable opportunities in those cities. Indeed, all the evidence suggests that a big part of London's advantage in creating opportunities comes from its shear size. Diverse cities of nine million plus population simply generate a lot more opportunities. If somewhere else in Britain is going to generate similar opportunities to London, there's an argument to be made that it would also need to be pretty big - perhaps half the size of London. At most, Britain could feasibly support one, may be two cities, of this sort of size (in addition to London).
Of course, recognising the importance of size creates all sorts of problems about which places to focus on outside of London. CfC's report dodges the dilemma by making the case for greater localism for all cities. Most politicians choose to behave likewise, focussing on creating opportunities across Britain. But what if reducing the reliance of Britain on London requires us to significantly increase the size of just one or two other cities? Which cities will they be and how will we help them grow? I don't claim to have the answers, but I at least think we should be asking the question.
Wednesday, 15 January 2014
Will HS2 end the property price spiral?
[Posted by Prof Henry G. Overman]
I'm puzzled by the latest comments on the impacts of HS2 (as reported in the Times), suggesting that the high speed link will end the property price spiral in London and the South East. How is that possible?
The direct effect, via commuting flows, must be pretty small. Around 400,000 people commute in to Central London by the Underground and around 850,000 on surface rail. According to the Strategic Case for HS2, the combined phases one and two add 19,800 seats at peak hour in to Euston. Even allowing for 100% loading - i.e. full trains - (which seems unlikely) the percentage increase in people who can work outside London and commute in will be tiny, so the effect on property prices will be correspondingly small.
Therefore any indirect effect must operate via the growth and location effects of HS2. But HS2's own figures suggest a large growth boost to London. As I have said before, I have my doubts about the magnitude of these figures but they are surely positive. In the absence of a supply response, this must make housing more expensive not less. Again, according to HS2's own estimates, areas outside London might benefit proportionally more than London and the South East (although, again, this is s.t. a high degree of uncertainty) but all that tells us is that the proportional increases might be larger outside London in other cities well served by the new line(s).
In short it's hard to see how something that boosts growth, and hence demand, in London (both absolutely and relative to other parts of the country not on the line) will do anything other than increase London house prices.
I'm puzzled by the latest comments on the impacts of HS2 (as reported in the Times), suggesting that the high speed link will end the property price spiral in London and the South East. How is that possible?
The direct effect, via commuting flows, must be pretty small. Around 400,000 people commute in to Central London by the Underground and around 850,000 on surface rail. According to the Strategic Case for HS2, the combined phases one and two add 19,800 seats at peak hour in to Euston. Even allowing for 100% loading - i.e. full trains - (which seems unlikely) the percentage increase in people who can work outside London and commute in will be tiny, so the effect on property prices will be correspondingly small.
Therefore any indirect effect must operate via the growth and location effects of HS2. But HS2's own figures suggest a large growth boost to London. As I have said before, I have my doubts about the magnitude of these figures but they are surely positive. In the absence of a supply response, this must make housing more expensive not less. Again, according to HS2's own estimates, areas outside London might benefit proportionally more than London and the South East (although, again, this is s.t. a high degree of uncertainty) but all that tells us is that the proportional increases might be larger outside London in other cities well served by the new line(s).
In short it's hard to see how something that boosts growth, and hence demand, in London (both absolutely and relative to other parts of the country not on the line) will do anything other than increase London house prices.
Monday, 16 December 2013
Land Prices: the dog that’s lost its bark
Posted by Paul Cheshire, SERC
It does not take great economic insight to realise that Britain, especially south east England and above all London, faces a crisis of housing affordability; a crisis that threatens to turn into a wider financial crisis up the line as loan to value ratios increase, with house prices already accelerating ahead of incomes and rising interest rates waiting to jump out of the bushes in maybe less than a year.
The underlying problem - as several SERC researchers have argued over the years - is the restriction on the supply of land our planning system imposes, which has led to a progressively more pathetic rate of house construction, with supply becoming less and less responsive to price changes. For at least 10 years we have known that we need to build about 250,000 homes a year - of the type people want to buy and in locations where people want to live. We have not approached that number since the 1960s and have been averaging about 100,000 a year for the last several years. This rate is the lowest since before WWI.
A reason why our planning system has not responded to the growing crisis of land supply - evident since the 1970s - is that it has not just ignored the impact of land supply on prices and affordability, but until 2010 it was legally obliged to do so. Under the 1947 Town and Country Planning Act and its successors prices were not a ‘material consideration’.
The National Planning Policy Framework (NPPF) of 2012 changed this situation by suggesting that when allocating land, local planners should pay attention to price signals in the market. Prices are an immensely efficient signal of complex information on both demand and supply. Where we observe discontinuities in land prices at the boundary of one permitted use and another, markets are trying to tell us that the higher priced land is in relatively short supply. We should not always obey such signals because there may be social, environmental or amenity benefits in preventing a change of use. Public policy protects National or Urban Parks and Sites of Special Scientific Interest for good social and environmental reasons. But when we observe intensively farmed land fetching £10,000 per ha whereas if it could be used for building houses on its value would jump to £8,000,000 a ha, prices are not just signalling a shortage; they are screaming a shortage. There needs to be a very strong reason indeed in terms of environmental or social benefit to justify policy preventing the change of use.
This simple insight underlay the proposal Steve Sheppard and I made in our 2005 paper arguing for using price signals to help with land allocation decisions. So it was great delight to read the newly issued guidance in relation to the NNPF:
“Land values are determined by the demand for land in particular uses, relative to the supply of land in those uses. The allocation of land supply designated for each different use, independently of price, can result in substantial price discontinuities for adjoining parcels of land (or land with otherwise similar characteristics). Price premiums provide direct information on the shortage of land in any locality for any particular use.”
Splicing the VOA data together with earlier academic studies one could construct a series going back to 1892. This revealed the incredible impact of the Town and Country Planning Act of 1947. In real terms there was no trend increase in land prices at all until the mid-1950s; since then a 15-fold increase. House prices over the same period rose in real terms but only about a quarter as much. It is always possible to substitute land out of house construction and it is land our planning system has been restricting the supply of. So as land prices have risen in real terms over time it has been possible to substitute land out of house production by making houses smaller. We used this data as a part of the evidence drawn on in our 2009 SERC Policy Paper. A version of that Figure is shown below.
Equally the VOA data let us see how the relative price of housing land varied across the whole country from one local authority to another. Again here is a map using the most recent data available:
In an ideal world we would like more than just the revival of the old data series. The Land Registry now provides researchers with access to data for individual house sales. This is an invaluable tool for all kinds of spatial research. It is not obvious why transactions in land should not also be open access data. If it was then there are really interesting things that could be done. We could estimate the value the community placed on transport improvements more accurately. We could even peep into the black box that is the compliance costs of our planning system and the costs of planning obligations. To do that one would need to combine information on sales prices of land parcels with details of planning obligations, its designated use and other data on location, the condition of the land and its accessibility. From that one could potentially learn a great deal to help us plan better and with less economic cost.
----------------------------------------------------------------------
11th December 2013
Dear Chancellor, Dear Secretary of State,
The land market is of vital importance to the economic and social health of the UK. Yet we now have no publicly available and impartial data on UK land market prices. This is a direct result of the decision of the Valuation Office Agency (VOA) to cease to collect and publish its land price indices in 2011.
Understanding land prices is critical to assessing a range of policy questions, from the efficiency of planning reform to the economics of housing development. Terminating the publication of this data makes it much harder for government to understand the consequences of current policy interventions or to assess the case for future reform. As researchers, academics and policy makers we find this situation deeply concerning.
This change also goes directly against developments in nearly all other asset markets. Trillions of dollars of complex and untraded assets are valued every day and there are now strong regulatory obligations for data transparency and disclosure through the new European Market Infrastructure Regulation (EMIR) and the second phase of the Markets in Financial Instruments Directive (MIFID).
While we fully recognise current financial constraints on government, the cost of collecting land price data was small, at approximately £40,000 annually. The benefits in terms of our understanding of this vital market are considerable. For example, as reflected in recently released National Planning Practice Guidance, knowledge of land prices is useful in assessing local housing needs and will help Local Government accurately assess the value of their own land holdings, which is likely to lead to significantly enhanced value for money for the tax payer.
If the government believes that there are better ways to obtain land price data, we would support this. Similarly, simply publishing the data online via the Land Registry would be acceptable. What is important is that reliable and good quality data is in the public domain. The Coalition has pledged an open-data government, and argued that the benefits to such data being available are critical to the modern economy.
We therefore urge the government to set out proposals to publish land values broken down at a regional and sub-regional level from 2014 onwards. This is not an expensive operation to undertake and it would be disproportionately beneficial for understanding this key part of our economy.
Yours sincerely
Kate Barker
Author
Barker Review of Housing Supply
Professor Paul Cheshire
Emeritus Professor of Economic Geography
London School of Economics
Nick Pearce
Director
Institute for Public Policy Research
Tom Papworth
Associate Director - Economic Policy
CentreForum
Jan Crosby
Head of Housing
KPMG
Roger Harding
Director of Communications, Policy and Campaigns
Shelter
Thomas Aubrey
Senior Adviser
Policy Network
Subscribe to:
Posts (Atom)