Evaluation of the New Deal for Lone Parents: early lessons from the Phase One prototype - cost-benefit and econometric analyses
A hard copy of this report summary can be obtained by contacting Paul Noakes [E-Mail: Paul.Noakes@dwp.gsi.gov.uk] or by writing to him at the 'Social Research Division, Department for Work and Pensions, 4th Floor, Adelphi, 1-11 John Adam Street, London WC2N 6HT'.
Research Report No. 110
By Chris Hasluck, Abigail McKnight and Peter Elias
The New Deal for Lone Parents (NDLP) was launched as a Phase One prototype in July and August 1997 in eight Benefits Agency districts across Britain. In addition to the eight prototype areas, six other Benefits Agency districts were selected as a basis for comparison to assist the evaluation of the prototype phase. This report is part of a set of three reports of the evaluation of the prototype phase of NDLP. The Synthesis Report (Hales et al, 2000, Department of Social Security Research Report No. 108) draws together the main results of the evaluation in one volume. The Survey Report (Hales et al 2000b, Department of Social Security Research Report No. 109) is concerned with the findings of surveys conducted for the evaluation.
This report consists of three related papers, all of which provide crucial evidence that informs the conclusions of the evaluation of the NDLP prototype:
Part One by Chris Hasluck contains the Cost-benefit Analysis. This outlines the costs incurred in running the prototype programme and the extent to which these were offset by increases in economic activity, compensating flows of revenue back to Treasury and improved well-being among lone parents who found jobs. A key component of this is an assessment of whether, and how much, the programme stimulated additional movement of lone parents into work, given that many lone parents claiming Income Support would have made this transition in the absence of the programme.
Part Two by Abigail McKnight is about the pattern of Transitions off Income Support. It provides one of the key pieces of information needed to inform the cost-benefit analysis, namely a robust estimate of the additional movement off Income Support, which can be attributed to the programme. In her analysis, the question is approached using a month-by-month database of the total number of current Income Support claims in each of the prototype and comparison areas during the prototype. The analysis uses econometric methods to control statistically for differences in the characteristics of lone parents and in the labour markets of the fourteen areas.
Part Three is by Professor Peter Elias and is also concerned with factors affecting Transitions off Income Support. He reports an econometric analysis performed on the data collected in survey interviews with lone parents. These provided a month-by-month record of labour market and benefit claiming status for each sample member between January 1996 and the date of the interview at the end of 1998. His analysis seeks to test the conclusions reported in Part Two, in particular to assess whether the analysis of transitions off Income Support is enhanced by additional information on the characteristics of lone parents. His analysis also makes use of information on the labour markets of the Benefit Agency districts throughout the prototype (Green 2000 Department of Social Security in-house report No. 63).
Part One - The net economic and exchequer benefits of the New Deal for Lone Parents
This report is part of the output from a programme of evaluation of the New Deal for Lone Parents (NDLP) Phase One prototype. It is concerned with an assessment of the economic and public finance effects of the NDLP and seeks to provide quantitative evidence relating to the net benefits (or costs) of the programme.
The evaluation relates to the cohort of lone parents on Income Support who participated in or were eligible to participate in the NDLP prototype in the period July 1997 to October 1998. Both the costs and benefits extend outside this time frame because of 'set-up' costs before the start of the programme and future benefits (and costs) arising from the employment of lone parents.
Key findings are:
- Total expenditure on NDLP was in the region of £7.86 million, although this is significantly less than the £14 million budget originally allocated for the programme.
- The costs included in NDLP expenditure may not all be appropriate to a cost benefit appraisal of the Prototype programme. For the purpose of the cost-benefit analysis some costs have been excluded from the total and other costs spread over a longer period of time. The effect of this is to reduce the total cost of the NDLP Prototype programme to £4.7 million.
- A critical issue pervading the cost-benefit analysis of the NDLP prototype is whether the programme should be expected to cover its operational costs alone or whether it should cover the total cost of the programme including set-up costs, overheads and central services. In the assessments of net benefits alternative measures are provided
- Estimates of the unit costs of the NDLP Prototype range from £140 per lone parent invited to a NDLP interview (£581 per participant attending an interview) to £1,493 per participant leaving Income Support for employment. These costs take no account of deadweight or other costs and benefits.
- Of the 8,107 lone parents attending for interview, 3,393 entered employment. This figure overstates the impact of the programme because it takes no account of deadweight (lone parents who would
- Additionality (the mirror image of deadweight) was assessed by reference to the subjective opinion of NDLP participants, other survey evidence and evidence of the 'counter-factual' derived from an analysis of administrative data. It was concluded that additionality amounted to around 20 per cent of observed employment amongst NDLP participants. This translates into 645 additional lone parents assisted into work.
- Household circumstances and the nature of employment both affect the outcome of the net benefit calculations. In general, the higher the level of earnings in work (the higher the hourly wage or the longer the weekly hours of work) the greater the economic benefits and the lower the net Exchequer costs. Low gross wages not only generate low economic benefits, they precipitate higher benefit entitlements. The larger the number of dependent children (and the older they are) the greater will be the net Exchequer costs because of the impact of the number of children on benefit entitlements.
- The net economic benefit of NDLP is estimated to have been in the region of £3.6 million (over £5 million if set-up and fixed costs are ignored). This represents a substantial gain for both individual lone parents and for the economy as a whole.
- On the basis of the evidence and assumptions set out in this report, it is concluded that the Phase One prototype programme 'clawed back' the great majority of its costs. We estimate there was small net Exchequer cost of just under £0.5 million over full costs, after accounting for the net public expenditure impacts of the programme.
- The net Exchequer cost of the NDLP prototype would have been eliminated if there were just small differences from the 'base case'. The Exchequer gains from a rate of additionality of 23 per cent (rather than 20 per cent), no substitution effects or a small wage premium for NDLP participants would have each been sufficient to eliminate the net Exchequer cost associated with the prototype programme.
- If the NDLP prototype were required only to cover it's operational costs, the public finance 'balance sheet' is quite favourable. At 20 per cent additionality, the net Exchequer gains from additional employment are estimated to have been in the region of £1.8 million over total operational costs. This Exchequer gain is even greater if additionality were to have exceeded 20 per cent, if substitution effects were negligible or if NDLP participants gained a wage premium on entering work.
- There is a possibility of further Exchequer gain from lone parents who left Income Support but did not enter employment. However, there is too much uncertainty regarding the situations of this group of lone parents (and the Exchequer costs associated with them) to allow these potential gains to be reliably quantified.
- The net benefit calculations are short term in focus and examine immediate outcomes (1-2 years). Any NDLP effect on the 'employability' and lifetime earnings of lone parents has not been taken into account. Such effects, should they exist, would significantly raise the net benefits of the programme. However, these effects cannot be assessed on the basis of the evidence presently available from the evaluation.
Part Two - Transitions off Income Support: estimating the impact of the New Deal for Lone Parents using administrative data
This paper reports on the results from an evaluation of the prototype NDLP (Phase One) using administrative data sources to estimate the effect of NDLP on Income Support claims. Throughout the prototype programme, information was collected on Income Support claims made by lone parents living in eight prototype programme areas and, to aid the evaluation, six comparison areas. The analysis is conducted within an econometric framework, comparing transitions off Income Support for lone parents in the target group living in prototype and comparison areas. Additionally, contrasts are made within prototype areas between lone parents in the target group who experienced NDLP during the early stage of the prototype programme with those experiencing the programme later on. The method employed attempts to isolate the New Deal effect by controlling for other factors that can influence the likelihood of leaving Income Support. These factors influence lone parents' decisions to enter the labour market (particularly the age of youngest dependent child, number of children) and employment opportunities in the local labour market (as proxied by information on local unemployment and vacancies).
The statistical analyses of administrative records of Income Support show that NDLP has been associated with an increase in the probability of lone parents leaving Income Support.
Key findings are:
- At the start of the prototype programme lone parents with existing Income Support claims were invited to participate in NDLP over a period of around ten months through a random process. This allocation process allowed us to identify a random sample of lone parents who were invited to participate in the programme on average seven months earlier than a second group. The difference in transitions off Income Support between these two groups provides us with an estimate of the programme effect. Statistical analysis of Income Support claims revealed that the group invited to enter the programme early had a higher probability of leaving Income Support. A difference which persisted over the 18-month evaluation period even though the size of the difference diminished over time as the later group entered the programme and caught up. In December 1997, six months into the programme, the first group had a 12 per cent higher odds of leaving Income Support, diminishing to seven per cent in June 1998 and five per cent in December 1998.
- Using the within prototype programme area results to compute a monthly depreciation rate in the stock of Income Support claims provides an estimate that after six months NDLP led to a reduction in the stock of Income Support claims by 1.5 per cent, 2.6 per cent after twelve months and 3.3 per cent after eighteen months. This means that, on average, it takes three months less time for half of the original stock of lone parents to leave Income Support in the presence of NDLP. A second technique, using the method of sample enumeration, suggests that the reduction in the stock of Income Support claims after only six months is slightly lower at 1 per cent.
- Comparing transitions off Income Support among existing claims (starting before the prototype programme began) in the prototype and comparison areas fails to identify a significant difference in the odds of lone parents leaving Income Support over the 18 month evaluation period. Concern that this result relies on the ability to control for local labour market conditions led to a number of alternative approaches. Models using different measures of local labour market conditions and separate analyses for areas identified by unemployment types however failed to identify a NDLP effect when comparing the experience of lone parents living in prototype and comparison areas. It remains the case that more detailed indicators of labour market conditions may be more appropriate than the available area level statistics.
- There is some evidence that NDLP has been more effective in helping specific groups of lone parents. Comparisons between the experience of lone parents whose claim for Income Support started in the year before the programme began in the prototype and comparison areas revealed that lone parents in the prototype programme areas had nine per cent higher odds of leaving Income Support.
- Among lone parents with claims for Income Support starting during the prototype programme (the flow sample), NDLP had a positive effect in the high unemployment areas: increasing the odds of leaving Income Support by 12 per cent. No significant effect was identified in the areas with medium or low rates of unemployment.
Part Three: Transitions off Income Support: estimating the impact of the New Deal for Lone Parents using survey data
This report describes quantitative research into the effect of the prototype phase of the New Deal for Lone Parents (NDLP) on the transition off Income Support. The research reported here was designed to complement a more extensive examination of the effects of the prototype programme - an examination which makes use of administrative records for all potential participants in the programme (Part 2).
The objective of the study which forms the basis of this report is to determine whether or not the findings on the scale of the impact of this programme as measured by McKnight can be observed using survey data. Information was collected by survey methods from lone parents in those areas of the country in which the prototype programme was in operation (the prototype areas) and from lone parents in a number of similar areas where it was to be introduced at a later date (the comparison areas).
The analysis is restricted to a particular group of Income Support recipients - those lone parents who were in a spell of Income Support at the time the scheme became operational (July/August 1997). A detailed multivariate statistical examination of factors, which could influence the end of this spell, is conducted. By controlling for differences between the prototype and comparison areas in these factors, the remaining difference between the prototype and comparison areas in the rate at which this spell ends is taken as indicative of the scale of the potential effect of the prototype programme.
The results of this statistical analysis accord with McKnight's findings. McKnight bases her results upon an analysis of administrative records from within the prototype areas, examining the difference in the rate at which the 'July 1997' spell of Income Support ends for those who had an early invitation to the scheme compared with those who were invited later. The analysis presented here, comparing the prototype and comparison areas, shows that, for those respondents who had been in receipt of Income Support on account of lone parenthood for at least eight weeks prior to July 1997, these spells are more likely to end sooner in prototype than in comparison areas. This finding takes into consideration a variety of other factors, not available to McKnight, which could have accounted for this difference between prototype and comparison areas, yet the difference between prototype and comparison areas remains.
The scale of the effect as determined from the multivariate analysis appears to be slightly greater than that noted from a simple contrast between prototype and comparison areas. Without any adjustment for compositional effects and for differences in labour market conditions, the 'stock' of lone parents on Income Support in July 1997 declines by 1.4 percentage points more in the NDLP prototype areas as opposed to the comparison areas over a fifteen month period. After adjusting for a wide variety of factors which could influence this difference between prototype and comparison areas, a residual 3 percentage point difference remains between the decline in the stock of lone parents in these areas after fifteen months. However, it must be stressed that the scale of this effect is at the margin of statistical significance.
Relevant publications
Allison, P. D. (1982). 'Discrete-time methods for the analysis of event histories', in S. Leinhardt, “Sociological Methods 1982,”Jossey-Bass Publishers: San Francisco.
Ashworth, K., Walker, R., Jenkins, S. and Shaw, A. (1997). 'Time Spent on Income Support by Unemployed Claimants', in Gregg, P. (ed) “Jobs, Wages and Poverty: Patterns of Persistence and Mobility in the New Flexible Labour Market, ”London: Centre for Economic Performance.
Bryson A., Ford R., and White, M. (1997). “Making Work Pay: Lone mothers, employment and well-being”, York : Joseph Rowntree Foundation.
Elias, P. (1997). 'Who forgot they were unemployed?' ESRC Research Centre on Microsocial Change. Working Paper No. 97-19.
Green A. E. (2000). “Evaluation of the New Deal for Lone Parents: comparative analysis of the local study areas,” DSS Social research Branch, In-house Research Report Series No.63, London.
Hales J., Shaw A. and Roth, W. (1998). “Evaluation of the New Deal for Lone Parents: A Preliminary Assessment of the 'Counter-factual'”, DSS In-house Research Report Series No.42, London, May.
Hales J., Lessof C., Roth W., Shaw A., Millar J., Barnes M., Elias P., Hasluck C., McKnight A. and Green, A.E. (2000a). “Evaluation of the New Deal for Lone Parents: Early Lessons from the Phase One Prototype, Synthesis Report”, DSS Research Report Series No.108, CDS: Leeds.
Hales J., Lessof C., Roth W., Shaw A., Millar J. and Barnes, M. (2000b). “Evaluation of the New Deal for Lone Parents: Early Lessons from the Phase One Prototype, Findings of Surveys. ” DSS Research Report Series No.109, CDS: Leeds.
Lancaster, T. (1990). “The Econometric Analysis of Transition Data, ”Cambridge: Cambridge University Press.
Layard R. and S. Glaister (eds) (1994). “Cost-benefit analysis” (2nd Edition), Cambridge, Cambridge University Press.
Williams A. (1993). 'Cost-benefit Analysis: Applied Welfare Economics or General Decision Aid?', in Williams, A. and Giardina, E. (eds), “Efficiency in the Public Sector”, Aldershot: Edward Elgar, pp65-79.
Woodfield, K and Finch, H. (1998). “New Deal for Lone Parents: Evaluation of Innovative Schemes.” DSS Research Report No. 89, CDS: Leeds.