Rigorous Impact Evaluation in the RéUsSITE Project

Programme In Côte d'Ivoire, we evaluate a support programme for micro, small and medium-sized enterprises (MSMEs), which is implemented as part of of the Special Initiative “Decent Work for a Just Transition“. The objective of the programme is to strengthen the productivity of the enterprises as well as to create jobs and improve working conditions. The participating firms are each accompanied for six months by an Ivorian management consultancy, which identifies weaknesses in management based on a diagnostic, formulates recommendations, and provides support in implementing the recommendations. The focus is on the areas of financial and human resource management.

Evaluation The evaluation is based on a baseline data collection before the start of the programme activities in April-May 2021 and an endline data collection in early summer 2022, six months after the end of the programme activities. The programme-participating firms were randomly selected after the end of the baseline data collection from among the firms that have applied to participate in the programme, meet the entry requirements, and have participated in the baseline data collection. In addition to the MSME managers, up to ten employees are also interviewed in each case in order to be able to map the effects of the programme on employees.

Research questions The evaluation examines the causal effects of programme participation on firm success and employment in the firms. Of particular interest at the firm level are (i) employment, understood as the number of jobs or hours worked, but also measured in terms of fluctuation and the composition of the workforce, (ii) security and quality of employment, measured e.g. in terms of the proportion of employees with a written employment contract, access to social security or training opportunities, (iii) HR management practices, (iv) firm performance, (v) capital and labor productivity, (vi) access to finance and financial constraints, as well as (vii) financial management practices.

At the employee level, we look at effects of the programme on (i) employment, understood for example as hours worked, job retention/mobility or diversification of income sources, and (ii) indicators of quality of employment, such as income, the existence of a written employment contract, job satisfaction or training opportunities.

Baseline survey and selected initial findings A first look at turnover and the number of employees illustrates the usual size distribution dominated by micro enterprises: more than 60% are micro enterprises with annual turnover of FCFA 30 million or less (approx. EUR 45,700), about a quarter are small enterprises with annual turnover up to FCFA 150 million (EUR 228,700) and less than 15% are medium-sized enterprises with higher annual turnover. About half of the enterprises have less than five employees, 35% have 5-9 employees and 15% have ten or more.

Around 90% of the firms surveyed see access to financing as one of the biggest challenges. Only 11% of the surveyed MSMEs successfully applied for loan financing since the beginning of 2020, while for over 20% the loan applied for was not approved and over 40% of the enterprises did not even apply due to low prospects of success. One major reason for these difficulties in accessing financial services may be weaknesses in financial management: although around 60% of the enterprises state that they prepare annual financial statements, only around 20% are able to provide them when asked.

In terms of quality and security of employment, more than half of the firms do not conclude written employment contracts with their employees and more than a third do not register their employees for social security and health insurance. Even within the firms, the heterogeneity is considerable-in more than a third of the firms, only some of the employees have access to social security and health insurance. In 40% of MSMEs, some or even all employees earn less than the legal minimum wage of FCFA 60,000 (approx. EUR 92) and less than 40% of the firms have offered their employees training in recent years.

Programme The programme to be evaluated offers young Ghanaian women the opportunity for training in the fashion industry and is implemented as part of the Special Initiative “Decent Work for a Just Transition“ in Ghana. The target group are disadvantaged young women from rural and urban areas who complete a training programme in the fashion industry over the course of multiple months and are supported in their subsequent job search. The aim of the programme is to improve the employment status and standard of living of the participants and to provide a skilled workforce for the fashion industry through a labour market-oriented training programme.

Evaluation The evaluation is based on a baseline data collection before the start of the programme and an endline data collection to take place approximately six months after the end of the programme activities. The surveys include questions on current income and living conditions as well as personality traits, among others. Baseline data collection has started in May 2021. Programme activities began in 2021 and 1,529 women could be mobilised until August 2021. Programme activities started from October 2021 onwards and were staggered across regions and training providers. Out of the interested women meeting the entry criteria, over half of the women were randomly selected to participate in the programme, while those not selected form the control group. In the endline survey in 2022, both groups are interviewed again in order to clearly distinguish programme effects from other developments.

Research question The aim of the evaluation is to determine causal effects of programme participation on the labour market situation of the participants. Of particular interest are (i) employment relationships, e.g. in terms of employment status and duration, (ii) indicators of employment quality, e.g. income, the existence of a written employment contract and job satisfaction, (iii) living standards, e.g. property and housing situation, and (iv) mobility, e.g. in terms of migration plans and behaviour.

Baseline survey and next steps The baseline survey was divided into two phases. First, interested women were recruited through so-called pre-registration events, eligibility criteria were checked and their consent to participate was obtained. In the second step, all women meeting the eligibility criteria were invited to a Career Day by SMS. There they receive further information about the programme, got to know the training institutes and took part in the second, more extensive part of the baseline survey. The final training activities will be completed in April 2022, with a view to collecting endline data in late summer 2022.

Programme The programme to be evaluated is part of the Special Initiative “Decent Work for a Just Transition“ and targets skilled artisans in the fields of electrical installation, roofing, metalwork, masonry, glazing, painting, plumbing, steel bending, tiling and air conditioning. The aim is to improve the employment situation of the beneficiaries. A number of job-related offers are made to selected persons over a period of about six months. These include (i) technical training courses (in cooperation with several vocational training institutes), (ii) financial literacy training, (iii) certification and licensing by a national craft register, (iv) digitalisation seminars and access to a mobile application to contact clients, (v) discounted modern tools and related training, and (vi) personal protective equipment and technical advice on occupational health and safety measures.

Evaluation At the time of programme enrolment, we conducted a baseline data collection and then a random selection of beneficiaries in terms of a randomised controlled trial. Approximately six months after the end of the programme period, we will conduct endline interviews with beneficiaries and non-beneficiaries. We will also obtain administrative data on beneficiary programme participation from our partner organisations. The study will then assess the impact on a range of employment and quality of work indicators. A baseline survey and recruitment of skilled artisanal workers has taken place in November and December 2020, with another baseline survey in summer 2021. The programme start date for the already selected beneficiaries was in June 2021, with additional individuals receiving various support services in autumn 2021. The endline data collection is planned for mid-2022.

Research questions Central to this evaluation are possible effects in terms of (i) employment, e.g. in terms of job retention or acquisition and length of employment, (ii) quality of work and life, including e.g. earnings, working conditions, and mobility and external options, (iii) firm-level outcomes, e.g. in terms of performance and personnel.

We are also investigating possible mechanisms that could underlie intervention effects, such as product and production innovations, the use of digital platforms to acquire clients and orders, or changes in participants' professional networks or client base.

Baseline survey and initial findings In November and December 2020, the first phase of baseline data collection took place in Accra, Kumasi and Tamale. Almost all (99%) of the respondents were men, despite efforts to also recruit female artisans. The labour market in the occupational fields served by the programme is clearly male-dominated in Ghana.

There were interested persons in all sectors covered by the programme, although with markedly different frequencies. The training courses offered by the training institutes were adapted accordingly or designed flexibly. In fact, the acquisition of additional craft skills is one of the main reasons for registering, along with the hope of gaining new customers and orders.

Programme The IT Training Academy offers six-month training courses in IT to people with a university degree and IT background. The aim of the programme is to train and develop Ghanaian beneficiaries in the field of information and communication technology and to create jobs. The current trainings focus on Software Development, DevOps, Full Stack Development, Data Science and Salesforce Development. Evaluation Our research includes (i) analysing the impact of IT training courses on labour market outcomes such as employment and wages using a quasi-experimental research design, and (ii) analysing the impact of different soft skills coaching interventions on applicant outcomes in the selection process (test scores, recruiter ratings, intake) and training (performance achieved, completion) using a RIE.

As part of (i), we collect baseline data from all applicants of the training academy. Endline data collection will take place 9 to 12 months after the baseline data collection of each cohort and will be completed by the end of 2023 for the last cohort.

Within the framework of (ii), soft skills coaching sessions are conducted virtually with some of the applicants. Each of these individuals receives 90 minutes of coaching in either organisational skills or communication skills. The coaching sessions are conducted in small groups, with five participants per group. The evaluation is based on the above-mentioned baseline and endline data collections, which are conducted before the start of the soft skills interventions of the respective cohort and one year later, respectively.

Research questions The evaluation of the IT Training Academy uses quasi-experimental methods to examine the effects of participation in the IT training programmes on labour market outcomes such as (the type of) employment and wages. Of particular interest here is the sector in which graduates of the training academy are employed and how dynamic the labour market is for people with IT degrees in Ghana. In the context of the rigorous impact evaluation, we investigate whether low-cost soft skills coaching has a short-term effect on the success of individuals in the job application process and leads to better employment prospects in the medium to long term. In particular, we are interested in whether soft skills coaching can help disadvantaged people into employment and thus compensate for inequalities.