Receive at least two EITC in pre and post-period. Previous work has established that health insurance coverage is capable of improving the health outcomes of lower-income families Levy and Meltzer Similar to Baughman , Hoynes et al. The March CPS data allows testing for differences in specific types of insurance. All models control for age, gender, race, marital status as well as the number of people living in the household. Furthermore, state and year fixed effects are controlled for.
Given the assumption that private insurance provides better services than public coverage, this finding provides evidence that health insurance can be viewed as a potential channel underlying the link between increases in income and improved health outcomes. The observed positive effect of expanding EITC on private health insurance coverage is smaller in magnitude than estimates observed by the three previous studies that have examined the effects of EITC on health insurance coverage.
Unlike the findings by Baughman and Duchovny and Hoynes et al. One disadvantage of the analysis is that the CPS data only began providing information on whether respondents purchased their own insurance coverage or whether it is sponsored by their employers starting in , which could strengthens the case that health insurance is a mechanism for the link between income and health.
Nevertheless, previous work has shown that income affects the likelihood with which workers are covered by employer-sponsored insurance. Furthermore, the paper shows that these increased costs were the main reason for why workers did not take up offered insurance plans. The results in this section provide evidence for the role of food expenditures and health insurance coverage in explaining the observed health improvements following increases in income. However, it should be considered that these two factors are by no means the only two potential mechanisms. Other aspects, such as health behaviors and financial stress, are likely to also impact the association and should be examined in future work.
The availability of data regarding the quality of food that individuals consume could furthermore strengthen the evidence suggesting that nutrition explains parts of the improved health outcomes following increases in income. The findings of this study advance the literature on the relationship between income and health by providing evidence for the protective health effects of exogenous sources of income increases to vulnerable parts of the population. The study shows that the expansion of the EITC increased the likelihood of affected heads of household reporting excellent or very good health by 6.
When examining potential explanations for the positive health impacts of additional income, the paper finds that increased spending on food While the magnitude of these effects suggest that food expenditures and health insurance are able to explain how additional income can lead to health improvements, it seems likely that income affects health in several ways. Thus, further examination of other potential channels such as the role of health-related behaviors, health care usage, health expenditures, and stress should be conducted to better understand the link between income and health.
Furthermore, it would be interesting for future work to examine the short- and long-term effects of similar policies on health outcomes of children living in directly affected households. The analysis in this study provides additional evidence for the presence of health benefits related to the EITC. The estimates for the positive health effects for adults are consistent with findings by Hoynes et al. Recent work on the tax credit suggests that further program expansions could help reduce existing health inequalities Fletcher and Wolfe Based on the success of earlier policy changes, other researchers have proposed that the program should be expanded for both families with one child as well as for childless families Hoynes ; Marr et al.
This study shows that the health benefits are largest for people in the plateau phase of the EITC schedule, which has been shown to provide pure income effects Athreya et al. This indicates that, if government policy provides cash transfers that are not conditional on earned income, the relevant effects on health status will correspond to the estimate show in Panel B of Table 7. The findings of this paper furthermore suggest that governmental regulations aimed at assisting lower income families are capable of providing health benefits.
As proposed by Berkman et al. A better understanding of the potentially unintended health benefits of welfare assistance programs could provide additional arguments in favor of certain policy adaptions. Findings in this area of research could help predict the effect of the current development towards mandated health insurance as well as with changes in federal- and state-level minimum wages, which have been discussed intensely by politicians in recent years.
Before the policy changes of OBRA were implemented, seven states had introduced state-level EITC payments and ten additional states adopted it until the end of the period of interest of this study in Today, 25 states have EITC credits at the state level in place, which further highlights the increasing importance of the program.
Details are available upon request. For more information, please see Feenberg and Coutts Dropping individuals with missing self-reported health information in some years of the analysis could bias the results if these respondents were different from the remaining sample, for example in terms of health. Appendix Table A1 shows that there are relatively small differences between the samples with and without missing self-reported health information. The statistics shown in Table A1 are obtained using the sample of people eligible to receive EITC benefits throughout the sample period.
The descriptive statistics are similar for the other two samples used in this study. The PSID provides data for these outcomes starting in The survey questions do not include meals eaten at work or at school. However, the results are unchanged when using the CPS simulations. The graphs looks very similar for the other two samples. They are not shown in the paper due to space restrictions, but are available upon request.
These results are not shown in the paper, but are available upon request. I additionally re-estimate the propensity scores using the two other commonly used estimation techniques for propensity scores, logit and cloglog estimation. The results remain unchanged. Histograms of the propensity scores for the pre- and post-policy period provide evidence that there is a common support for the groups in both periods. The histograms are not shown in the paper, but are available upon request.
In additional models, I test for the effects of the policy on the likelihood of reporting fair or poor health. While finding negative effects, the estimates for the bottom two categories of health status are smaller in magnitude than the estimates for the top two health categories reduction of 4. One reason for the relatively small finding could be that only Thus, while lacking statistical significance, the observed decline of 4. In additional specification, I estimate treatment effects separately for males and females. The results suggest that the positive effects of income on health are larger for women than for men.
In an additional specification, I test for the effect of annual changes in predicted EITC benefits on health status. While the estimates suggests that higher increases in EITC have positive health effects, they are imprecisely estimated. The estimate shown in Panel D of Table 6 uses the narrow sample selection and is therefore comparable with the effect shown in column 1 , Panel C of Table 4. The results remain similar to the main specification for the other two samples. These additional results are not shown, but are available upon request. The second sample is slightly different compared to the analysis on health status.
Given that the PSID only offers two pre-treatment periods with information on food expenditures, the second sample in Table 7 consists of households that received EITC benefits at least twice both before and after the policy change. Due to the magnitude of welfare reforms that were implemented during the late s, all models include controls for the state-specific characteristics shown in the Appendix. These additional findings are not shown in the paper but are available upon request. Skip to main content Skip to sections. Advertisement Hide.
Download PDF. The effects of income on health: new evidence from the Earned Income Tax Credit. Open Access. First Online: 11 August The size of benefits received by eligible families depends on several factors, such as the presence and number of qualifying children in the household.
Table 1 provides an overview of the EITC parameters for families with one and two or more children during the time period of the study. The statistics show that the policy change in the mids substantially altered the credit rates and benefits to eligible families. Table 1 Earned Income Tax Credit parameters — On average, heads of EITC-eligible households with at least two children are more likely to be male and married, while those with one child are slightly older in the least restricted sample only EITC in pre-period.
According to the statistics in Table 2 , it seems that EITC-eligible single mothers in the sample are more likely to have one child. Family incomes are relatively similar for the groups.
The bottom of Table 2 shows summary statistics for health-related outcomes. It is noticeable that heads of households with more than one child are, on average, in better self-reported health than those with one child. Figure 1 shows changes in the share of individuals who report either excellent or very good health across during the period of the study for the sample of individuals that were eligible to receive EITC benefits throughout the pre-expansion period. Open image in new window. Table 3 provides descriptive statistics for the amount of EITC received by households with one and at least two children before and after the EITC expansion.
Statistics for three different sample are presented, that differ in how restrictive the sample is selected. For all three Panels A, B, and C , it is observable that there were very small differences in EITC benefits for eligible families from the two groups prior to the policy expansions. After the implementation of the policy change, however, families with two or more children receive substantially higher payments than those with only once child. The differences in EITC benefits between the two groups following the reform are larger than found by other studies.
Consistent with Table 3 , Fig. The picture shows the amount of EITC which eligible families in the sample receive in dollars for the sample of individuals that receive EITC benefits in all years of the sample. Again, while only small differences in EITC benefits are observable before the expansion for families with one child and those with two or more children, the gap becomes large in the years after the policy change. The structure of the policy changes offers the opportunity for a difference-in-differences DD framework to observe the average treatment effects. In the presence of changes in the composition of the sample, a cross-sectional analysis could provide inaccurate estimates if healthier individuals with two or more children choose to enter the labor force following the incentives of being eligible to higher EITC benefits after the policy change.
Thus, the main specification of this paper uses the longitudinal nature of the PSID to control for individual fixed effects and to purge the estimates of individual time-invariant heterogeneity. I examine treatment effects for three different specifications, which differ in how restrictive the sample was selected: 1 examines all individuals that were eligible to receive EITC benefits in all years before the policy change; 2 examines all individuals that were eligible to receive EITC benefits in at least three years both before and after the policy change; 3 examines individuals who are eligible to receive EITC benefits throughout the sample period.
Since it has been shown that the EITC is often more a short-term safety net for low-income families, the number of observations in the third sample is relatively small. Like any DD model, the estimation of equation 1 makes the key assumption that trends in health outcomes over time are similar across both the treatment and control groups. One way to reduce this potential bias is to explore a difference-in-difference-in-differences DDD framework. The additional comparison groups consist of households with children one and at least two who are, based on the tax simulations, not eligible to receive EITC benefits in any year point during the study period to This section introduces two additional models, which I estimate to test whether the main results are robust to other model specifications.
First, I conduct a falsification test that compares the health outcomes of heads of households from two groups that are equally affected by the policy change. During the period of my study, there were no differences in EITC benefits for families with more than one child. Only following the implantation of the American Recovery and Reinvestment Act ARRA in , benefits for eligible families with three or more children increased significantly. Following the falsification test conducted by Averett and Wang , eligible heads of households with two children form the control group for this specification, while those with at least three children form the treatment group.
Everything else in the falsification test is the same as equation 1. Finding no differences in health outcomes between these two groups can provide evidence that the main analysis is actually capturing health effects due to of the EITC policy change and not due to other time-varying factors that could be correlated with health status Averett and Wang Figure 3 confirms the validity of the falsification test by showing that that EITC credits evolved identically throughout the period of the study for eligible households with two and three or more children.
Second, I estimate a semi-parametric DD model, which was introduced by Abadie and which relaxes the assumption of a linear relationship between income and health. The method captures average treatment effects for the treated group ATT for the case that differences in observed characteristics create non-parallel outcome dynamics between the two observed groups, which violates the main assumption of standard DD models. The main dependent variable is a binary indicator that equals 1 if an individual reports being in either excellent or very good health.
Consistent with the descriptive statistics shown in Tables 2 and 3 , estimates for three different samples are presented. Panel A shows DD results for the sample of individuals that were eligible to receive EITC benefits throughout the pre-treatment period to The baseline estimate in column 1 suggests that being eligible for the increased benefits raises the likelihood of being in the top two health categories by 8.
This effect corresponds to a When additionally accounting for state-specific controls in column 2 , the result remains almost unchanged, which supports the claim that the health effects are not spuriously driven by the other safety net laws passed during the s. Table A2 in the Appendix provides the estimates for these additional state characteristics that can capture the role of welfare reforms on health status.
While no statistically significant effects are noticeable for any of the welfare reform controls, the effects shown in Table 4 could potentially be lower bound effects since previous work has provided evidence for negative effects of welfare reform on health Muennig et al. Table A2 also shows that Medicaid expansions have a negligible effect on health status and that controlling for them does not alter the main estimates. DDD estimates for the impact of the policy change on health are presented in Table 5.
It is noticeable that the results are fairly consistent with the DD effects shown in Table 4. While the results for the sample of households that were eligible to receive EITC benefits throughout the pre-treatment periods are slightly smaller in magnitude Panel A , the observed effects for the other two samples are actually slightly larger than the DD results. While substantially smaller in magnitude, the estimate also show that the policy change increases the likelihood of being in excellent or very good health.
Overall, the findings in Table 5 confirm that the observed positive effects of additional income on health status remain when accounting for potential differential health trends between households forming treatment and control groups and remove concerns that the DD results might be biased. In order to further test for the validity of the main results of the study, estimates for several additional robustness checks are presented in Table 6.
First, I use the amounts of predicted EITC dollars that are obtained from the tax simulator in order to check whether health effects as a result of the expansion were larger for individuals who received higher EITC benefits. This finding provides additional evidence for the positive link between income and health. Table 7 presents fixed effect DD estimates for the effects of additional income following the EITC expansion on food expenditures.
For all three samples, I show estimates for total weekly food expenditures by households as well as for expenditures for food eaten at home and for food eaten out. Additionally, it is noticeable that the effects become larger the more restrictive the sample is selected. The results in Panel B and C show that the majority of this increase in food expenditures is driven by changes in expenditures on food eaten at home, while there are only small changes in expenditures on food eaten out. Given the magnitudes of the findings in Table 7 , the results provide suggestive evidence that food expenditures serve as a channel underlying the positive relationship between income and health.
The DD model shows that treated households are 1. Columns 2 shows that this increase is entirely driven by increases in private insurance coverage, while columns 3 and 4 show that the expansion had small negative effects on public coverage. The DDD findings confirm that the policy change increased the likelihood with which individuals had any coverage and private insurance, even when accounting for potential differential trends between household with one or more children.
The HITC, which was available during two of the four pre-treatment years of this analysis, did not have different eligibility requirement between households with one or at least two children and should therefore not affect the estimates. In an additional model that excludes the years and , I find that the results remain unchanged. This confirms that the observed treatment effects are not driven by the HITC.
Compliance with ethical standards Conflict of interest The authors declare that they have no conflict of interest. Abadie, A. Semiparametric difference-in-differences estimators. Review of Economic Studies , 72 , 1— Google Scholar. Adda, J. Banks, J. The impact of income shocks on health: evidence from cohort data. Journal of the European Association , 7 6 , — Apouey, B. Winning big but feeling no better? The effect of lottery prizes on physical and mental health.
Health Economics , 24 5 , — Athreya, K. Earned Income Tax Credit recipients: income, marginal tax rates, wealth, and credit constraints. Economic Quarterly , 96 3 , — Averett, S. The effects of Earned Income Tax Credit payment expansion on maternal smoking. Health Economics , 22 , — Public Finance Review. Baughman, R. Evaluating the impact of the Earned Income Tax Credit on health insurance coverage.
National Tax Journal , 58 4 , — The Earned Income Tax Credit and fertility. Journal of Population Economics , 22 3 , — State Earned Income Tax Credits and the production of child health: insurance coverage, utilization, and health status. National Tax Journal , 69 1 , — Berkman, L. Social Epidemiology. Biehl, A. Foster care and the Earned Income Tax Credit.
Review of Economics of the Household , 16 3 , — Bitler, M. Welfare reform and health. Journal of Human Resources , 40 2 , — Boyd-Swan, C. The Earned Income Tax Credit, mental health, and happiness. Card, D. Does medicare save lives? Quarterly Journal of Economics , 2 , — Case, A. Economic status and health in childhood: the origins of the gradient. American Economic Review , 92 5 , — The long reach of childhood health and circumstance: evidence from the Whitehall II Study.
The Economic Journal , , F—F Cebi, M. Health insurance tax credits and health insurance coverage of low-earning single mothers. Employment Research: a Journal of Science and its Applications , 16 2 , 1—3. Health insurance tax credits, the Earned Income Tax Credit, and health insurance coverage of single mothers. Health Economics , 23 , — Currie, J. Health insurance eligibility, utilization of medical care, and child health. Saving babies: the efficacy and cost of recent changes in the medicaid eligibility of pregnant women.
Journal of Political Economy , 6 , — Currie, A. Journal of Health Economics , 26 , — Socioeconomic status and child health: why is the relationship stronger for older children? American Economic Review , 93 5 , — Cutler, D. Employee costs and the decline in health insurance coverage. In Cutler, D. Frontiers in Health Policy Research. Dahl, G. The impact of family income on child achievement: evidence from the Earned Income Tax Credit. American Economic Journal , 5 , — Deaton, A. Policy implications of the gradient of health and wealth. Health Affairs , 21 , 13— Aging and inequality in income and health.
American Economic Review , 88 2 , — Dickert-Conlin, S. EITC and Marriage. National Tax Journal , 55 1 , 25— Dowd, T. Income mobility and the Earned Income Tax Credit: short-term safety net or long-term income support. Public Finance Review , 39 5 , — Duchovny, N. Dissertation, University of Maryland at College Park.
Economou, A. Poor and Sick: estimating the R between household income and health. Review of Income and Wealth , 57 3 , — Eissa, N. Journal of Public Economics , 92 , — Labor supply response to the earned income tax credit. Ellwood, D. The impact of the Earned Income Tax Credit and social policy reform on work, marriage, and living arrangements.
National Tax Journal , 53 4 , — Ettner, S. New evidence on the relationship between income and health. Journal of Health Economics , 15 1 , 67— Evans, W. Giving mom a break: the impact of higher EITC payments on maternal health. American Economic Journal: Economic Policy , 6 2 , — Feenberg, D. Journal of Policy Analysis and Management , 12 1 , — Finkelstein, A.
The Oregon health insurance experiment: evidence from the first year. Quarterly Journal of Economics , 3 , — Fletcher, J. Increasing our understanding of the health-income gradient in children. Frijters, P. The causal effect of income on health: evidence from German reunification. Journal of Health Economics , 24 , — Gardner, J. Money and mental wellbeing: a longitudinal study of medium-sized lottery wins. Journal of Health Economics , 26 1 , 49— Grossman, M. On the concept of health capital and the demand for health.
Journal of Political Economy , 80 2 , — Gruber, J. Crowd-out 10 years later: have recent public insurance expansions crowded out private health insurance? Journal of Health Economics , 27 , — Gunter, S. State Earned Income Tax Credits and participation in regular and informal work. National Tax Journal , 66 1 , 33— Hamad, R.
Poverty and child development: a longitudinal study of the impact of the Earned Income Tax Credit. American Journal of Epidemiology , 9 , — Henry, J. The relation of social to biological processes in disease. Social Science and Medicine , 16 , — Hotz, J.
The Earned Income Tax Credit. Moffitt ed. Hoynes, H. Proposal building on the success of the Earned Income Tax Credit. Income, the Earned Income Tax Credit, and infant health. American Economic Journal: Economic Policy , 7 1 , — Effective policy for reducing inequality? The Earned Income Tax Credit and the distribution of income. Idler, E. Self-rated health and mortality: a review of 27 community studies. Journal of Health and Social Behavior , 38 , 21— Internal Revenue Service. Jo, Y. The impact of policy-driven income on childhood obesity. Health Economics.
Kaestner, R. The effect of welfare reform on prenatal care and birth weight. Health Economics , 14 , — Changes in the welfare caseload and the health of low-educated mothers. Journal of Policy Analysis and Management , 25 3 , — Kenkel, D. Is smoking inferior? Evidence from variation in the Earned Income Tax Credit.
Journal of Human Resources , 49 , — Khanam, R. Child health and the income gradient: evidence from Australia. Journal of Health Economics , 28 4 , — Kuehnle, D. The causal effect of family income on child health in the U. Journal of Health Economics , 36 , — Larrimore, J. Does a higher income have positive health effects? Using the Earned Income Tax Credit to explore the income-health gradient. Milbank Quarterly , 89 4 , — Lenhart, O. Do higher minimum wages benefit health? Evidence from the UK. Journal of Policy Analysis and Management , 36 4 , — Levy, H.
The impact of health insurance on health. Annual Review of Public Health , 29 , — Lindahl, M. Estimating the effect of income on health and mortality using lottery prizes as an exogenous source of variation in income. Journal of Human Resources , 40 1 , — Markowitz, S. One intervention group received a labour force attachment LFA intervention, intended to place participants in employment of any kind as rapidly as possible, while the other received a human capital development HCD intervention, aimed at increasing respondents' employability by enhancing their skills.
However, one group Riverside HCD differed systematically from the rest of the sample, since the HCD intervention was only available to respondents who lacked basic skills. All studies collected data on differing age groups of children, with ages ranging from 18 months to 18 years. Figure 2 shows the age groups and subgroups reported by each study at each time point.
In some cases, trials reported child outcomes only by subgroups. Ontario included children aged 2 to 18 years. SSP Recipients reported data on children aged 5. All of the voluntary studies described a process of obtaining informed consent from participants prior to randomisation. The data we report were collected between 18 months and 18 years after randomisation. An independent team of researchers linked data from two studies to mortality data at 15 to 18 years CJF ; FTP At T1 and T2, all data reported were from samples that were still exposed to the intervention.
In CJF and FTP , a proportion of the sample would have reached lifetime limits for welfare receipt and ceased to receive earnings disregards. They would still have been exposed to sanctions, training and case management. At T3, a number of interventions had ended, and sample members were no longer exposed to intervention conditions. There was an expectation that impacts would continue after the interventions had ended because early labour market entry would allow respondents to accrue labour market advantage in terms of job quality and earnings, and that this could contribute to a better environment for children, with lasting health benefits.
Although the overarching aim of all included interventions was to promote employment among lone parents in receipt of welfare benefits, the motivation or ethos underlying this objective differed, as did the approach to achieving it. We describe these differences in detail in Description of the intervention. Briefly, interventions had one of the following motivations. Human capital development HCD aimed to promote skills development in order to secure better quality employment.
Figure 4 provides information about all studies' ethos and approach. Ontario did not fall into any of these categories. However, in practice this typology did not prove as useful as anticipated. Even where study authors stated that the intervention explicitly adopted one of the above approaches, in practice there often seemed to be little variation between interventions of differing types. For instance, a number of LFA interventions offered training, and this did not necessarily differ in level or scope from that offered by HCD interventions.
Study authors often reported that implementation of interventions varied widely within studies. This variation occurred both at the level of intervention ethos and approach, and at the level of individual components, as might be expected in complex interventions with multiple components delivered in different sites and settings. We identified 10 individual components in the interventions see Figure 4.
Except those in UK ERA , control group respondents were also subject to many of these components, such as employment requirements and earnings disregards, to varying degrees. Thus, we describe only those intervention components that represent an incentive, sanction or service over and above what the control group received.
Three studies tested variants of the main intervention with two or more intervention arms. Ontario tested the impact of five different approaches to delivering support to single parents. Two groups within the study received employment training and are included in the review. One of these groups also received child care and support from health visitors. Failure to do so could result in financial sanctions involving partial or total cessation of welfare benefits for a specified period of time.
Supplements were limited to a period of three years. While supplements were being paid, respondents' total income could increase even if their earned income was low. Methods of calculating and levels of generosity varied across studies. Where earned income was disregarded, respondents could claim welfare while earning at much higher levels than previously. While respondents received earnings disregards, total welfare receipt and numbers on welfare were higher. As with supplements, disregards could increase total income even if earned income was low.
Financial contributions toward the cost of child care were made either directly to childcare providers or to parents for a period of one to two years following uptake of employment. Ontario provided a childcare programme to one arm of the intervention only. This differs from requirements to work or to take steps towards work component 1 in that participants were assigned a specific placement in the public, private or voluntary sector , which they had to attend for a set number of hours per week in order to continue receiving benefits, and they were not paid at a normal market rate.
New Hope assigned participants who were unsuccessful in finding work to community service jobs, but these were seen as proper employment and paid at the market rate. The package of welfare reforms passed in the USA in included a federal lifetime limit of 60 months of welfare receipt, with individual states retaining the freedom to apply shorter limits. IFIP did not include time limits over and above those applying to the whole sample under a federal waiver granted in MFIP was able to maintain this under the intervention conditions, but New Hope participants were not held back from lifetime limits after the implementation of Wisconsin Works in The CJF time limit was 21 months.
For recipients who found employment, the period in which they received earnings disregards and other programme benefits counted towards their welfare 'clock'. Thus, there was a transition point where they went from working and receiving many other benefits to relying solely on earned income. Advisors had some discretion in the application of time limits and could grant extensions where they judged recipients to have made a good faith effort or to have been incapacitated through ill health. Sanctions varied in severity across interventions.
Rates of sanctioning also varied within and between interventions. Most of the interventions included some form of education, training or both, whether they were explicitly described as HCD or LFA. FTP developed an extensive set of services around training and development, including assigning specific staff to each participant, funding ongoing training for those who found employment, and developing training work placements in conjunction with local employers. UK ERA also provided information, but in addition paid for training and provided bonuses of up to GBP on completion of training.
CJF provided transitional Medicaid for two years after participants found employment, and IWRE subsidised health insurance while participants' incomes remained below the federal poverty level. MFIP participants were eligible for Minnesota's subsidised health insurance scheme, but this was not an intervention component. In practice, case management differed in terms of levels of contact, flexibility, enforcement and monitoring.
Based on each of these dimensions, we categorised the interventions as having high or low case management. Following the passage of PRWORA, the intervention condition was in fact 'usual care' as the interventions were rolled out statewide while they were being evaluated.
IFIP was terminated after 3. Wisconsin Works was introduced in and affected all respondents in New Hope Under AFDC, conditions varied to some degree from state to state. Receipt of welfare benefits was not subject to time limits. Usual care in Canada varied across states and also changed during the course of the interventions. Initially in both states work requirements were minimal. By contrast, in New Brunswick earnings disregards increased. There was no time limit on benefit receipt. Studies used a range of measures and formats to report primary and secondary outcomes within and between studies and across different time points.
The following provides a summary of which outcomes were reported by each intervention. Appendix 5 includes further details including the time points at which each outcome was reported. Although we searched for parental health outcomes, the vast majority of the sample in all included studies was female. Therefore, we describe adult health outcomes as 'maternal' for the remainder of the review. All 12 studies reported maternal mental health outcomes. These are both well validated and widely used measures of risk of depression in adults.
They were reported both as a continuous measure mean total score , and as a dichotomous measure proportion scoring above a cutpoint defined as 'at risk of depression'. These score each item from 1 to 3 or 1 to 5 depending on the age of the child and calculate the mean of the score for each item in the scale. Investigators collected all of these measures via parent report. Nine studies reported a measure of child physical health.
IFIP reported the percentage of children with fair or poor health, and MFIP reported the percentage with good or excellent health. All of these outcomes were collected via parent report. All employment measures were dichotomous, reporting the percentage of the sample employed or not employed for a given measure. Nine studies reported measures of income.
IWRE reported income for the month prior to the survey annualised to represent the previous year's income. Eleven studies reported a measure of earnings. IWRE reported annualised earnings in the month prior to the survey. NEWWS reported total earnings for years 1 to 5. Many of the interventions included either an earned income disregard or a financial supplement in order to make work pay and ease the transition from welfare to work. Most of these were time limited, with limits ranging from 21 to 36 months although extensions were often available for people with particular difficulties.
However, the periods while working and claiming welfare counted towards the respondent's lifetime limit on welfare receipt. While supplements or disregards were being paid, respondents' total income could increase even if their earned income was low. Obviously when time limits were reached, this effect ceased. In all cases, time limits were reached during the period defined as T2 24 to 48 months.
A number of studies also reported total earnings. We extracted both measures in order to investigate the relationship between earned and total income. IWRE reported the average amount received in the month prior to the survey, annualised, and NEWWS reported the total amount of benefit received between years 1 and 5. UK ERA reported the average amount of benefits received per week. New Hope and Ontario reported the proportion of the sample receiving benefits in the year prior to the survey.
Since lower levels of total welfare paid and of numbers claiming welfare are the desirable outcomes from policy makers' perspectives, we defined these as positive in the analyses. Therefore these studies did not report data on health insurance. Effect sizes were calculated for all reported measures. See Results of the search ; Characteristics of excluded studies.
All studies had at least one item at high risk of bias, with two studies having four domains at high risk NEWWS ; Ontario All but two studies were at low risk of bias for allocation concealment and sequence generation, and it is very likely that these two studies conducted these but did not report it IFIP ; IWRE Blinding of outcome assessment was rare, and only one study reported baseline outcome measurements Ontario All risk of bias judgements are presented in the Characteristics of included studies tables and summarised in Figure 5 and Figure 6.
Since all studies were at high risk in at least one domain, the summary judgement was that all the included studies were at high risk of bias. Risk of bias summary: review authors' judgements about each risk of bias item for each included study. Risk of bias graph: review authors' judgements about each risk of bias item presented as percentages across all included studies. As such, they adopt robust procedures for sequence generation; communication with study authors confirmed this. Where reports explicitly describe allocation concealment, it is clearly conducted correctly, as in the following text:.
One study took place in an academic setting Ontario While authors clearly described adequate methods of sequence generation for this study, they provided no information about allocation concealment, leading to a judgement of unclear risk of bias. Since the trial reports provided no information, we judged the studies to be at unclear risk for both sequence generation and allocation concealment. However, again these are large and very reputable companies, and it is highly likely that they followed correct procedures.
We assessed baseline measures at the level of individual outcomes. We assessed outcomes that were not reported at baseline to be at unclear risk of bias. Where investigators collected and adjusted for baseline measures, or reported them by intervention status with few significant differences, we assessed them to be at low risk. Where studies did not report baseline outcomes by intervention status, or where there were differences between groups at baseline and authors reported no adjustment, we judged them to be at high risk.
Ontario reported all baseline outcome measures, but these differed across intervention groups and authors did not describe any adjustment, so we assessed it as being at high risk of bias. NEWWS reported and adjusted for maternal mental health at baseline but did not collect any other health outcomes at baseline, and we deemed it to be at unclear risk of bias.
We assessed risk of bias in the domain of baseline characteristics at study level. Where studies reported baseline characteristics by intervention group and showed them to have no statistically significant differences, or where they used regression to adjust for baseline differences, we assigned a judgement of low risk of bias.
Blinding of outcome assessment was conducted at the level of individual outcomes. Although it is very unlikely that assessors were blinded, we judged studies to be at unclear risk in the absence of further information. We conducted risk of bias assessment for missing outcome data at study level and at outcome level.
Compared to the larger sample that used administrative data, data from the survey overestimated impacts on earnings, and the authors urge caution in interpreting the findings. Thus, we consider that the risk of bias from missing outcome data is particularly high for this study. We could describe contamination in these studies as either indirect, that is, where the control group were likely to have been influenced by changes in social attitudes towards welfare and by awareness of changing rules affecting the majority of the population, or direct, where there was evidence that the control group were actually subject to the treatment condition at some point during the study.
In Canada, restrictions to welfare benefits for lone parents were also implemented in the late s, and in the UK requirements to seek employment were placed on lone parents of successively younger children. As a result, the control group were directly affected by the new policies in a number of studies. In most cases it is difficult to be sure how much these changes affected controls.
We judged all of these studies to be at high risk of bias from direct contamination. All were successful in this except IFIP , since the intervention was terminated and the control group moved to the new state level policy three and a half years after randomisation.
We judged IFIP to be at high risk of bias from direct contamination and the remainder to be at low risk. Media coverage and publicity, as well as changed attitudes to welfare, accompanied the new policies, and there is evidence that some control group respondents in the CWIE studies believed themselves to be subject to the new rules Moffitt We judged all of the CWIE studies to be at high risk of bias from indirect contamination.
We deemed these to be at high risk of indirect contamination. It is likely that contamination bias would lead to an underestimation of impacts on economic outcomes among the intervention group, as control group members endeavoured to find employment in the mistaken belief that this was now required of them. Underestimation of impacts is not deemed to be as serious as overestimation Higgins a ; however, it is difficult to be sure what effect this type of contamination would have had on health outcomes.
We assessed selective outcome reporting at study level. Protocols were not available for any of the included studies, and studies that reported data for more than one time point or subgroup rarely reported outcomes consistently across groups or times. Government bodies, which arguably had a vested interest in the success of the interventions, funded and participated in all included studies except New Hope Sources of funding are recognised as potential sources of bias. However, as stated, the evaluations involved highly reputable research organisations that have made major contributions to the development of methods for conducting social experiments in their own right.
As such, there is no suggestion that the findings were in any way influenced by the source of funding. See Table 1 ; Table 2. All included studies were at high risk of bias in at least one domain, therefore we downgraded all evidence once for this criterion. As a result, no evidence could attain a quality rating higher than moderate. However, exclusion of this study had only marginal effects on the estimates.
We considered few effects to be at serious risk of inconsistency. Where heterogeneity was high but there was a plausible explanatory hypothesis, we did not downgrade and presented a post hoc sensitivity analysis in Effects of interventions. We discuss these instances in Effects of interventions. Since the populations of all included studies met these criteria, we did not downgrade for indirectness. None of the outcomes included in the review were indirect measures, so we did not downgrade for indirectness in relation to outcomes.
We did downgrade a number of health outcomes for imprecision due to low event rates. Since we had no reason to suspect that other studies have been conducted but remained unpublished, we did not downgrade any outcomes for publication bias. We assessed outcomes for which an effect size could not be calculated as being of unclear quality. Within each domain, there was often a range of quality assessments for different measures. We based an overall assessment for the domain as a whole on the grade assigned to the analysis or analyses with the largest total sample size.
On this basis, of the 12 health domains, we assessed all as moderate quality except T1 maternal mental health low quality , T3 maternal physical health low quality and T3 child mental health unclear quality. We assessed all T1 and T2 economic domains as moderate quality and all T3 ones as low quality.
We report these domain level assessments in the domain summaries in Effects of interventions. See: Table 1 ; Table 2.
The comparison in all cases was with usual care see Description of the intervention. T1 maternal mental health , as this was how studies reported results. We reported these outcomes narratively in the text, and where it was possible to calculate an effect size, we presented it in forest plots. These are designed to summarise the direction and strength of effects, as well as the quality of evidence available, in a way that readers can apprehend visually. Upward and downward pointing arrows indicate positive and negative directions of effect, respectively, defined in terms of the desirability of the outcome e.
A single arrow represents a 'very small' effect, two arrows a 'small' effect, and three a 'modest' effect, as defined in Table A 'o' indicates that there is evidence of no effect. The colour of the arrow denotes the quality: green indicates moderate quality; amber, low quality; and red, very low quality. Where we could not assess quality, we used black. For dichotomous outcomes, we defined the 'event' as reported by study authors, whether it was considered a 'good' or a 'bad' outcome.
For instance, when calculating employment, we defined the good outcome being employed as the event, although traditionally the bad outcome is considered the event Alderson However, we reported the RRs in this way because this is how the original studies reported them. We identify instances where the 'good' outcome is defined as the event as such in the 'Summary of findings' tables Table 1 ; Table 2.
Effect sizes across virtually all outcomes were small i. SMD 0. However, there is debate regarding the utility of these rules for interpreting the effects of population level interventions, since an effect that appears small or even tiny when considered at the level of the individual may be important if replicated across a large population Kunzli , Siontis Cohen has stated that effect sizes observed outside laboratory conditions are likely to be small, and that use of his definitions of effect magnitude warrant caution Cohen Other authors have also argued that in interventions which affect large populations, an SMD of 0.
We present our definitions in Table 32 alongside those recommended by Cohen. The effect magnitude for RRs below 1 is calculated by subtracting 1 from the RR then multiplying by , such that RR 0. These are defined as small and very small effects, respectively. All five studies reporting at T1 reported a measure of maternal mental health. Although the evidence was of moderate quality, the effect was very small SMD 0.
The evidence from Ontario was of very low quality for the same reason and due to high attrition. Forest plot of comparison: 1 Time point 1 Maternal mental health, outcome: 1. Comparison 1 Time point 1 Maternal mental health, Outcome 1 Maternal mental health continuous. Comparison 1 Time point 1 Maternal mental health, Outcome 2 Maternal mental health dichotomous. All of the six included studies that reported at T2 reported maternal mental health.
Forest plot of comparison: 2 Time point 2 Maternal mental health, outcome: 2. Comparison 2 Time point 2 Maternal mental health, Outcome 1 Maternal mental health continuous. Evidence from both studies was of moderate quality, although the result from California GAIN was unlikely to be important as the effect was very small and the CI crossed the line of null effect. Comparison 2 Time point 2 Maternal mental health, Outcome 3 Maternal mental health dichotomous. We calculated effect sizes for the two dichotomous outcomes; there was a very small effect in favour of the intervention for high risk of depression in IFIP RR 0.
Forest plot of comparison: 3 Time point 3 Maternal mental health, outcome: 3. Comparison 3 Time point 3 Maternal mental health, Outcome 1 Maternal mental health continuous. Comparison 3 Time point 3 Maternal mental health, Outcome 2 Maternal mental health dichotomous. At T1 and T3 there were individual studies that reported larger negative effects on maternal mental health, but the evidence was of low or very low quality. One study that reported a very small negative impact at T1 did not report maternal mental health at T3.
At all time points, evidence of moderate quality predominated, therefore the overall quality assessment for maternal mental health at each time point was moderate. One study reported the percentage of the sample in fair or poor health at T1, providing evidence of low quality that the intervention group reported better health than control RR 0. We downgraded this evidence due to imprecision.
Forest plot of comparison: 4 Time point 1 Maternal physical health, outcome: 4. Although the evidence was of moderate quality, the effect is unlikely to be important, as the effect size is very small and the CI crosses the line of null effect. Forest plot of comparison: 5 Time point 2 Maternal physical health, outcome: 5. Event defined as In good or excellent health. This showed a very small effect in favour of control RR 0. However, the evidence was of low quality due to high risk of bias from attrition, and the effect was unlikely to be important as it was very small and the CI crossed the line of null effect.
Forest plot of comparison: 6 Time point 3 Maternal physical health, outcome: 6. Only four studies reported measures of maternal physical health, and all but one reported small to very small positive effects. UK ERA reported a very small negative effect on maternal physical health at T3, but the evidence was of low quality. The evidence on maternal physical health at T1 and T3 was predominantly of low quality; therefore we assessed evidence at both time points to be low quality overall.
At T2, the evidence was of moderate quality. Four studies reported a measure of child behaviour problems at T1. Ontario reported the proportion of the sample with three or fewer behaviour disorders as a categorical variable. We dichotomised the latter variable to create an outcome for the proportion of the sample with two or three behaviour disorders. Evidence from each study was of moderate quality. Forest plot of comparison: 7 Time point 1 Child mental health, outcome: 7. Comparison 7 Time point 1 Child mental health, Outcome 1 Child behaviour problems continuous.
Individual effect sizes for the dichotomous outcomes showed modest negative effects on behaviour problems in the intervention groups in both Ontario RR 1. However, evidence from these outcomes was low quality in CJF Yale and very low quality in Ontario due to wide confidence intervals including no effect and appreciable harm and very high risk of bias in Ontario Comparison 7 Time point 1 Child mental health, Outcome 2 Child behaviour problems dichotomous.
This effect was very small and the CI crossed the line of null effect, so it is unlikely to be important. Forest plot of comparison: 8 Time point 2 Child mental health, outcome: 8. Comparison 8 Time point 2 Child mental health, Outcome 1 Child behaviour problems continuous. Comparison 8 Time point 2 Child mental health, Outcome 2 Adolescent mental health dichotomous. We could identify no plausible hypothesis to explain this heterogeneity. The evidence was of low quality due to this unexplained heterogeneity.
Forest plot of comparison: 9 Time point 3 Child mental health, outcome: 9. Comparison 9 Time point 3 Child mental health, Outcome 1 Child behaviour problems continuous. The intervention had a small positive effect on externalising behaviour, a very small positive effect on internalising behaviour and a very small negative effect on hyperactivity. Behaviour problems were very slightly higher among the IFIP applicant intervention group intervention This difference in effect was possibly related to study characteristics.
Two further studies reported a modest negative effect, but the evidence was of low and very low quality. Since the evidence was primarily of moderate quality at T1 and T2, this was the overall assessment for both time points. Most evidence at T3 was of unclear quality, so this was the overall domain assessment. Only one study reported a measure of child physical health at T1.
As this effect was very small and the CI crossed zero, it is unlikely to be important. Forest plot of comparison: 10 Time point 1 Child physical health, outcome: One study reported the percentage of the sample in good or excellent health MFIP ; this showed a very small effect in favour of control RR 0. As this effect was very small and the CI crossed the line of null effect, it is unlikely to be important. Evidence for all outcomes was of moderate quality. Forest plot of comparison: 11 Time point 2 Child physical health, outcome: Comparison 11 Time point 2 Child physical health, Outcome 1 Child physical health continuous.
Comparison 11 Time point 2 Child physical health, Outcome 2 Child physical health dichotomous. Six studies reported child physical health at T3. No measure of variance was available for SSP Recipients Since standard deviations for four studies reporting the same outcome were available, we imputed a standard deviation for SSP Recipients based on the average for the other four studies. Forest plot of comparison: 12 Time point 3 Child physical health, outcome: Comparison 12 Time point 3 Child physical health, Outcome 1 Child physical health continuous.
Comparison 12 Time point 3 Child physical health, Outcome 2 Child physical health dichotomous. One individual study reported no effect. At each time point, most evidence on child physical health was of moderate quality. Most of the MFIP sample were not subject to employment mandates and could receive earnings disregards for lower levels of employment participation, providing a plausible hypothesis to explain this heterogeneity.
The effects in Analysis Two studies reported the proportion who had ever worked in the fifth year of the study New Hope ; UK ERA , and one study reported the proportion who had ever worked between years 1 and 5 of the study NEWWS Overall, the intervention showed very small to small positive effects on all measures of employment at T1 and T2 ranging from RR 1. All evidence at T1 and T2 was of moderate quality. At T3 the effects on most measures of employment were close to zero, with similar proportions of the control group in employment at 49 to 72 months.
Much of the evidence on employment at T3 was of low quality. At T1 and T2, we assessed most evidence on employment as moderate quality, therefore the domain level quality assessment was also moderate.
In Welfare Reform: Effects of a Decade of Change, Jeffrey Grogger and Lynn Karoly assemble evidence from numerous studies, including nearly three dozen social This report is part of the RAND Corporation commercial book series. Lynn Karoly at RAND Corporation . A number of studies have concluded that welfare reform has led to large declines in caseloads (e.g., Council of Economic.
There were a number of differences between these studies that may have contributed to this, including the lack of any earnings supplement or disregard over and above that received by the control group in the NEWWS intervention. New Hope reported a very small positive effect on intervention group annual earnings SMD 0. However, as this effect was very small and the CI crossed zero, it is unlikely to be important. A possible explanation for this is that earnings disregards had ceased by this point for most CJF respondents.
Both MFIP and SSP Recipients were still providing earnings supplements when T2 data were collected, which may account for their stronger positive effects on income. However, although FTP had also ceased to supplement income, income was higher in the intervention group. We calculated average earnings in year 4 for FTP None of these effects reached statistical significance. Analysis Study authors did not calculate statistical significance. We could not calculate statistical significance.
In the two IFIP groups, there were small differences in favour of control ongoing group and intervention applicant group. Neither reached statistical significance Analysis All five experimental groups in the NEWWS study reported that the intervention groups earned more than control in years 1 to 5 of the study. We could not calculate an effect size for the other study that reported slightly higher earnings among five intervention groups, which were statistically significant in two of the groups.
We could not calculate an effect size for two studies reporting earnings; one study found no statistically significant differences between intervention and control. Another reported very slightly higher earnings for the intervention group. One further study for which we could not calculate an effect size showed a statistically significant effect in favour of control among one subgroup of respondents.
We could not calculate effect sizes for two further studies; one found higher earnings among all five intervention groups, although the difference was statistically significant in just one. The other reported no statistically significant differences and slightly higher earnings in one control subgroup.
Based on the majority of the evidence at T1 and T2, the domain level assessments of income and earnings were of moderate quality. At T3, the evidence was predominantly of low quality, which was reflected in the domain level assessment. There was a very small effect in favour of the intervention group RR 0. This was possibly due to the generous earnings disregards MFIP provided to the intervention group throughout the study, which allowed them to receive welfare benefits while working at higher levels than the control group.
Therefore we conducted a post hoc sensitivity analysis excluding MFIP Although we could not identify any plausible explanation, we did not downgrade the quality of evidence because all effects were in the same direction. Comparison 20 Time point 2 Welfare receipt, Outcome 1 Average annual welfare benefit. Ontario and SSP Recipients reported the proportion of the sample in receipt of welfare at T2.
This indicated that fewer participants in the intervention group were in receipt of welfare RR 0. The evidence was of moderate quality. Comparison 20 Time point 2 Welfare receipt, Outcome 3 Proportion of sample receiving welfare. The measure reported by NEWWS differed from that of the other studies total welfare received in years 1 to 5 rather than in the year prior to data collection , and we therefore analysed it separately.
This evidence was of moderate quality. Comparison 21 Time point 3 Welfare receipt, Outcome 1 Total welfare benefit received. There was a very small effect in favour of the intervention RR 0. Comparison 21 Time point 3 Welfare receipt, Outcome 5 Proportion of sample receiving welfare. At T2, there was evidence of a modest positive effect on total welfare received, which was of moderate quality when we excluded one study that had a modest negative impact on total welfare.
One study reported a modest positive effect moderate quality on welfare received between years 1 and 5. We could not calculate effect sizes for the amount of welfare received in two further studies. One reported that the intervention group received very slightly more welfare than control, while the other reported a large absolute difference in favour of the intervention. The majority of the evidence at T1 and T2 was of moderate quality, therefore these domains were assessed as such. At T3, the evidence was predominantly of low quality. Three studies reported a measure of adult health insurance at T1.
CJF GUP and CJF Yale reported the proportion of the sample with Medicaid at the time of the survey, New Hope reported the proportion of the sample that had ever had Medicaid since randomisation, and NEWWS reported the proportion who ever had health insurance provided by their employer since randomisation. Only one study reported the proportion of focal children ever having health insurance since randomisation NEWWS , finding a very small negative effect for the intervention RR 0. All of the evidence was of moderate quality.
At T2, one study reported the number of adults with Medicaid or other health insurance within 2 to 3 years of randomisation California GAIN , and one study reported the proportion of children having any health insurance continuously in the previous 36 months MFIP The evidence was of moderate quality in both cases, but the result for California GAIN is unlikely to be important, as the effect was very small and the CI crossed the line of null effect.
New Hope reported the proportion of respondents with any type of health insurance and the proportion of respondents whose focal child was insured. However, in all cases the effects are unlikely to be important as they are very small and the CI crosses the line of null effect. At T1 there were very small positive effects on adult health insurance and no effect on child health insurance. At T2, one study found a very small effect in favour of control, while one other found a very small effect in favour of the intervention.
Evidence from T1 and T2 was of moderate quality. Effects on health insurance were very small at T3. The evidence at T1 and T2 was all assessed as moderate quality; therefore both domains were assigned a grade of moderate. At T3, most evidence was of low quality. New Hope reported data at 96 months for a limited set of outcomes. We synthesised the data across three time points 18 to 24 months, 25 to 48 months and 49 to 72 months and eight outcome domains: maternal mental health, maternal physical health, child mental health, child physical health, employment, income, welfare receipt and health insurance.
We were therefore restricted to our planned primary analyses, which included data from all studies. The typology we set out to investigate using subgroup analysis proved less useful than anticipated, as interventions using apparently different approaches were often similar in terms of content and methods. The Canadian provinces and US states in which the evaluations took place were diverse in terms of geography, demographics and local labour markets. All studies were at high risk of bias in at least one domain, although when we incorporated risk of bias and other factors in the GRADE assessment of quality of evidence, most evidence was of moderate quality, implying that further research "is likely to have an important impact on our confidence in the estimate of effect and may change the estimate" GRADEpro GDT Overall, most effects in this review fell below the conventionally accepted threshold for a small effect.
However, as discussed in Effects of interventions , there is some debate regarding the importance of very small effect sizes and suggestions that effect sizes above SMD 0. Nonetheless, the overwhelming majority of effects on health outcomes in this review were below this size, suggesting that there are unlikely to be tangible impacts on health. There is some suggestion that the effects on maternal mental health varied over time, with a tendency toward negative impacts at T1, no effect at T2 and positive impacts at T3. It is possible that intervention group participants experienced higher stress levels at T1, either because they were actively involved in the intervention at that time, due to a period of adjusting to WtW requirements, or because their children were likely to be younger.
However, as the effects are so small, any hypotheses regarding this difference in effects are necessarily speculative. Many economic outcomes at T2 and T3 are likely to have been affected by direct or indirect contamination, which would have led to underestimated impacts. How this might have affected health outcomes is unclear.
Although these analyses included interventions specifically designed to increase income and promote labour market advancement, effects on these outcomes were limited.
In spite of higher employment and earnings, effects on income at T1 and T2 were not always positive. In addition, there is evidence that welfare reform led to an increase in lone parents' expenditure on items such as travel and food consumed away from the home, suggesting that any increase in total income may not have boosted respondents' disposable income Waldfogel In some studies very small effects were due to control groups voluntarily entering employment at a similar rate to intervention groups.
On this basis, we conclude that WtW interventions are unlikely to improve the health of lone parents and their children.
There is some evidence to suggest that there may be small adverse effects on health in some circumstances. Effects on employment and income were perhaps smaller than policy makers might hope or expect. Since economic impacts are hypothesised to mediate health impacts, it is possible that effects on health were very small due to the small economic impacts. These very small effects on maternal and child mental health need to be interpreted against a background of very poor mental health for intervention and control groups at all time points.
The control group risk of depression at any time point ranged from Comparison of effects on income across studies is complicated by variations in tax and transfer systems in different state jurisdictions.
However, overall it is clear that effects on income were unlikely to have important substantive effects. Indeed, although we did not extract data on poverty, most studies noted that poverty remained high for all groups. As noted above, there were insufficient studies possessing similar characteristics to permit statistical subgroup analyses.
Similarly, we could not statistically investigate other intervention characteristics such as whether the intervention was voluntary or mandatory, or whether income was supplemented in any way. However, we used post hoc sensitivity analyses where there was high heterogeneity to generate hypotheses regarding the influence of study or intervention characteristics on effect estimates Haidich These post hoc hypotheses suggested that voluntary interventions that lead to increased income may have positive effects on child mental health, while mandatory interventions that increase employment but do not improve income may lead to negative impacts on maternal and child health.
However, the evidence is limited geographically and temporally, in that most studies took place in North America during a period of economic expansion in the s. We were unable to investigate the role of economic outcomes as mediators of health impacts due to the small number of studies reporting at each time point. The applicability of the findings from the included studies to other contexts is also debatable given that the USA lacks a system of universal health care although most respondents were eligible for Medicaid , and most of the US and Canadian studies were from the s.
Furthermore, while most studies are from only two countries, these are not homogeneous, and economic and political contexts varied across the states and provinces in which studies were conducted. Generalisability may be enhanced by such diversity of contexts Armstrong Various forms of WtW policies and interventions for lone parents have been or are being implemented across the developed world. The age of youngest child at which lone parents are required to be available for work varies internationally but is rarely as young as that tested in these studies often as young as six months.
Many interventions do not provide earnings disregards, extensive case management, training opportunities or childcare subsidies. Welfare reform as implemented in the USA also had many important differences from the interventions reviewed here. These include the universal implementation of lifetime limits on welfare receipt which featured in only three studies reviewed here and the use of diversion policies to prevent eligible lone parents from claiming welfare at all. The included interventions consisted of multiple components in varying combinations.
Individual participants did not receive every intervention component, but few studies reported data on uptake of discrete components, not to mention duration or intensity. Although most of these reports provided a great deal of detail on intervention content, information on some components e. Even if they did provide such data, extraction and analysis would be extremely challenging. In the absence of such information, however, it is not possible to investigate the influence of intervention uptake or individual components.
However, diversity of components and adherence thereof may enhance applicability Armstrong They reported mixed results from different programme designs, but overall found that gains in employment and earnings did not generally persist beyond 5 years. Many studies reported implementation issues that had the potential to affect internal validity. For instance, lack of resources, staff attitudes to welfare reform, cultural differences between sites and caseloads were all mentioned as factors that influenced the nature of the intervention delivered.
In addition, the intervention implemented did not always accord with the explicit ethos or approach. Local economic, social and political contexts also varied. We were unable to statistically investigate the role of implementation issues due to small numbers of studies sharing given characteristics. The role of broader economic and political contexts is discussed below. The intervention was compulsory in 7 of the 12 included studies, and participants were recruited from the existing population of lone parent welfare claimants.
It is very likely that sample populations reflected the target population of the intervention in these cases. In five of the included studies, participation was voluntary. It is likely that this also influenced generalisabilty as only those who were more motivated to gain employment would volunteer to participate. In New Hope , study workers recruited participants in community settings, possibly leading to a less representative population. On the other hand, in both cases this may have led to a more realistic approximation of how the interventions might work outside the trial context.
In a number of studies, some proportion of the sample were married or living with a partner at randomisation. Although some studies reported data on family formation, we did not extract these, as this was not an outcome considered in the review. However, we know that lone parenthood is frequently not a static state, and it is likely that changes in partnership status among the participants again render them more representative of the wider population of lone parents.
This may have encouraged those closest to the labour market to enter employment independently, leaving more disadvantaged welfare claimants on the welfare rolls, although a number of studies made efforts to ensure the control groups were aware of their status.
The nature of the population receiving welfare would also have been influenced by the prevailing economic contexts. All of the included studies were affected by one or more of these factors, but the seven US studies conducted after the implementation of welfare reform in were the most affected. The economy expanded rapidly during this period.
All of these factors are likely to have decreased the potential for positive effects on economic outcomes. During this period there were large decreases in welfare receipt among lone parents in the US; the total caseload declined from 5 million to 2. Analyses of observational evidence suggest that the flourishing economy and the expansion of EITC, rather than welfare reform, were responsible for most of the decline in welfare receipt Grogger b.
The EITC and the economic boom would have affected both intervention and control groups, while the control groups would have been affected by contamination to some extent. Some studies reported that control group respondents left welfare voluntarily in large numbers as a result of the economic conditions, leading to small impacts on employment in the studies.
Given that the contribution of welfare reform to increased employment in the general lone parent population who were exposed to the intervention is considered relatively small, it seems likely that experiences of welfare reform via contamination were responsible for only a small proportion of the control groups' increase in employment. The review includes 12 RCTs, conducted in a variety of settings. Numbers of participants in a given analysis range from to 14, As such, they represent a body of evidence of unusual quality in the field of public health. However, as with any body of evidence, there are some methodological issues that are discussed below.
Using the GRADE approach to assessment, the highest quality attained by any of the evidence was moderate, due to every study being at high risk of bias in at least one domain. Due to the high number of outcome measures within each time point and domain, we developed a domain level GRADE assessment see Risk of bias in included studies. We judged 8 of 12 economic domains to be of moderate quality and the remaining 4 as low quality.
It is normally expected that evidence from public health interventions will be of low or very low quality Burford