External Evaluation
As we grow, independent third party evaluations help assess our program impact for learners, graduates, and employers, so that we can continue to improve and expand our programs.
How does Generation validate impact?
We have begun conducting evaluations in several of our countries with the support of experienced research agencies and academics.
In Generation India, Kenya and France, the evaluations focused on assessing the impact of our program on our learners’ employment trajectory and our employers’ hiring & retention processes.
Specifically in India, the evaluation also compares outcomes for our learners with other skill training programs. In all cases, we have seen strong employment outcomes with a faster time to placement and higher earnings. We have also received great feedback from our employer partners on the ease & quality of hiring from Generation.
The evaluations have also provided great insight into where we can continue to strengthen our programs.
An evaluation of longer term outcomes is currently ongoing in India & Kenya. As we continue to scale up in each of our countries, we are seeking to conduct more independent evaluations that can help to demonstrate our program impact across various programs and geographies.
Mathematica Impact Assessment of Generation Programs in India and Kenya
With support from IKEA Foundation, Mathematica conducted a multi-year impact evaluation of Generation programs in India and Kenya in 2022 and 2023. This evaluation assessed short-term outcomes (3-6 months post-graduation) in Phase 1 and longer-term outcomes (12-18 months post-graduation) in Phase 2 using qualitative and quantitative methods. The study assessed Generation’s impact outcomes for graduates, employers, and delivery partners using a quasi-experimental design in India by comparing outcomes for Generation graduates to those in public training programs for the same or similar professions. In Kenya, Mathematica used a non-experimental design, comparing Generation graduate outcomes to those of applicants of similar profiles who were not selected for our programs.
Key Findings
Generation India:
- 66% of our graduates found employment within 90 days of graduation vs 21% in the comparison group, a 3X multiple
- 85% of Generation graduates were employed in jobs directly related to their Generation training, while 51% in the comparison group were able to do so.
- 14-38% higher wages earned by Generation graduates than the comparison group at three to six months post program, varying by profession.
- 44% of Generation graduates continued to be employed at ~15 months after graduation versus 25% among the comparison group. Mathematica was able to conclude with reasonable confidence that the positive difference in employment of 19 percentage points between the two groups is attributable to Generation’s methodology.
- Generation programs are more cost-effective than those of the comparison group – the cost per percentage point of long-term employment achieved and the cost per dollar of average learner earnings are both 30% lower for Generation programs than for government-funded programs.
Generation Kenya:
- 60% of graduates found paid employment within 90 days of graduation, of whom 83% retained employment for at least three months after starting their first job.
- 75% of graduates found employment directly related to their Generation training.
- While over 80% of learners were unemployed and had no income upon entering Generation Kenya’s programs, they went on to earn monthly wages of ~12,000-26,000 KSh in their first job (Phase I) after graduating.
- 55% of Generation graduates continued to be employed at ~15 months post-graduation compared to 34% of non-selected applicants.
- Long-term job retention is also substantially higher for Generation graduates than for non-selected applicants, with almost 66% of initially employed Generation graduates still employed at ~15 months post-graduationcompared to 50% of non-selected applicants.
Links to Reports
Phase I: Executive Summary
Phase I: Full Report
Phase II: Executive Summary
Phase II: Full Report
Koreis: Impact Study of Generation France
Generation partnered with Koreis to look at impact data from Generation France related to alumni employment status, financial health and professional satisfaction.
Key Findings
- Generation France graduates are over 2x more likely to be employed within 6 months of graduation compared to other job seekers in training (68% vs 30% employment rate).
- 43% of learners are in long term unemployment before entering the Generation France program, compared to 18% in other training programs.
- 72% were earning above 1,200 Euro per month after the program – the minimum wage in France – compared to only 36% who earned this amount prior to joining the program.
Employment Data Lab: Generation UK
We collaborated with the UK Department of Work & Pensions’ Employment Data Lab for an assessment of our learner outcomes in UK, comparing outcomes against matching administrative data.
The evaluation found that 83% of Generation learners found employment within a year. Further, one year after program start, 70% continue to remain employed. Of 5 similar organizations that went through a similar evaluation, Generation has been shown to have the best outcomes to date.
Key Findings
- Generation program is serving learners with higher barriers to employment – 28% qualified for free school meals, 19% for the Special Educational Needs provision, 79% were of black/Asian/other ethnicities, and 61% were enrolled in ‘looking for work’ benefit prior to the program
- 83% of learners found a job within a year of the program, with 73-80% remaining in employment even 2 years after enrolment, compared to 26% before enrolment. Those claiming ‘looking for work’ benefits dropped to 25% 1 year after the program start.
- Learners also had a 3.6x increase in wages from the year before they enrolled in the program to one year after enrolment. Wages increased further by 14% at 18 months after enrolment, indicating sustained employment growth.
- Generation learners were 1.4x more likely to be in employment one year after the program compared to the counterfactual group(70% vs 50%). While a causal impact could not be established due to insufficient data, the largest net positive gap over the counterfactual compared to all organizations evaluated, the rigorous matching process for the current counterfactual(across 154 variables) and the comprehensive nature of our intervention are strong indicators of causality.