Skip to content

Free University of Bozen-Bolzano

Impact Evaluation

Semester 1 · 27612 · Master in Public Policy and Innovative Governance · 6CP · EN


The course: a) explores how impact evaluation answers the question “what works?” in public policy, introducing Randomized Controlled Trials (RCTs) as the gold standard approach; b) covers additional quantitative techniques for causal analysis, training students to design, implement and analyse evaluations with statistical software; c) develops skills for critically appraising evidence and translating empirical insights into clear, actionable recommendations for governments, public administrations, NGOs and international agencies.

Lecturers: Alexander Moradi

Teaching Hours: 36
Lab Hours: 6
Mandatory Attendance: Attendance is recommended, but not mandatory.

Course Topics
1) The Experimental Ideal: Causal Effects and the Selection Problem 2) Randomized Control Trials, ethical and practical challenges, communication and policy consulting 3) Natural experiments (discovering, analyzing, evaluating) 4) Panel, Difference-in-Differences, Instrumental Variables 5) Regression Discontinuity Designs 6) Synthetic Control

Teaching format
lectures, labs, projects.

Educational objectives
ILO (Intended Learning Outcomes) ILO1 Knowledge and understanding ILO1.1 The student acquires knowledge of organisational models, business factors related to the organisation of public enterprises operating in different sectors (international organisations, central government, local authorities, health care, public enterprises) and the consequences for the community. You will also acquire knowledge necessary for developing, managing and evaluating the impact of public projects, and knowledge useful for analysing organisational innovation processes and interpreting the tools and trends in personnel management in public enterprises. ILO1.2 The student acquires the knowledge of economic theory needed to understand and analyse economic and business phenomena in the public sector in order to support decision-making processes. Knowledge of public policy and the tools necessary for the design of sustainable policies will be consolidated. Knowledge related to the labour market, education and health will also be deepened, functional to the development of public policy analysis and evaluation skills. ILO2 Ability to apply knowledge and understanding ILO2.1 ability to interpret market trends through the application of appropriate economic models and implement economic analysis tools, also using data ILO2.2 ability to apply economic models to describe the behaviour of economic agents and develop sustainable economic policies in various application domains of interest to companies and public bodies ILO2.3 ability to identify and distinguish management concepts and problems in relation to real or hypothetical situations concerning public institutions and companies also in different contexts (e.g. different services/activities/institutions, different local or international institutional levels) ILO2.4 ability to develop and administer projects from the design phase through to the operational implementation phases, including the evaluation of their impact on society, the economy and the environment ILO3Making judgements ILO3.1 ability to apply acquired knowledge to interpret economic and business phenomena in order to make managerial and operational decisions in the context of public administration ILO3.2 ability to select data and use appropriate information to describe a problem concerning the design, implementation and evaluation of public sector projects and policies, aiming at innovation and improvement of processes, products and results ILO3.3 ability to relate models and empirical evidence in the study of public policy phenomena ILO4 Communication skills ILO4.1 ability to communicate effectively in oral and written form the specialised contents of individual disciplines, using different registers according to recipients and communicative and didactic purposes, as well as to evaluate the formative effects of his/her communication ILO5 Learning ability ILO5.1 ability to use information technology autonomously to carry out bibliographical research and investigations and for one's own training and further education.

Assessment
For Attending and Non-Attending Students Project Development: Students choose a topic relevant to the course and develop either (a) an evaluation plan for a public policy, including a complete methodology and proposed data collection and analysis in R, or (b) a replication of an existing policy evaluation, with critical reflection on the study’s methodology, findings and implications. For Attending Students Presentation (30%) Students present their project to the class, summarising purpose, methodology, expected outcomes (evaluation plans) or main findings and critique (replications). (ILO2.3, ILO2.4, ILO3.1, ILO3.3, ILO4.1) Project Report (70%) A 1,500-word report documenting background, methodology, expected or actual results, analysis in R (for replications) and a critical reflection. (ILO1.1, ILO1.2, ILO2.1, ILO2.2, ILO2.4, ILO3.1, ILO3.2, ILO3.3, ILO4.1, ILO5.1) For Non-Attending Students 3. Extended Project Assignment (100%) A 2,500-word report covering the same criteria as attending students, plus a more detailed literature review. Initial contact with the lecturer within the first four weeks is required. (ILO1.1, ILO1.2, ILO2.1, ILO2.2, ILO2.4, ILO3.1, ILO3.2, ILO3.3, ILO4.1, ILO5.1) Project work and presentations are valid for one academic year only.

Evaluation criteria
Presentation: 1. Clarity of Presentation (20%): The student must present content in a manner that is both clear and comprehensible. Complex concepts should be articulated in a way that is accessible to all audience members. 2. Quality of Argumentation (20%): Arguments should be presented in a logical and persuasive manner, with adequate support from empirical data or scholarly literature. 3. Mastery of Technical Terminology (20%): Usage of technical terminology should be precise and contextually appropriate. 4. Interactive Communication Skills (20%): The student’s ability to engage with the audience through responsive Q&A, as well as the effective use of visual aids, will be evaluated. 5. Structure and Organization (20%): The presentation should have a coherent structure with a clear narrative thread throughout. Project Report: 1. Correct Application of Methods (25%): The report should demonstrate that Impact Evaluation methods have been accurately applied and thoroughly described. 2. Depth of Analysis (25%): The report must reflect a comprehensive analysis and profound understanding of the chosen subject matter. 3. Critical Thinking (25%): The report should critically examine the methods employed and the results achieved, showcasing analytical depth. 4. Accuracy and Completeness (25%): The report must be meticulous in considering and presenting all relevant aspects of the project with precision.

Required readings

Cunningham, S. (2025), Causal Inference. The Mixtape.



Supplementary readings

Dunning, T. (2012). Natural Experiments in Social Sciences, Cambridge University Press.

 

Gertler, Paul J.; Martinez, Sebastian; Premand, Patrick; Rawlings, Laura B.; Vermeersch, Christel M. J.. 2016. Impact Evaluation in Practice, Second Edition.  Washington, DC: Inter-American Development Bank and World Bank.

 

Further supplementary reading material will be published regularly on OLE.




Download as pdf

Sustainable Development Goals
This teaching activity contributes to the achievement of the following Sustainable Development Goals.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17

Request info