Career Tracking Pilot Case Study

The development of a questionnaire and methodology guidelines to explore career paths of doctorate holders in a first ESF career tracking pilot project involving five organisations and nearly 500 respondents

 

ABOUT THE PILOT PROJECT

The aim of this pilot was to identify the quality standards and processes involved in the study of a large and diverse population of doctorate holders. 

A large part of the study was to identify and communicate the methodological and statistical steps involved in conducting career tracking over time.

The ESF has championed this area over the years, has facilitated experts coming together and has published their recommendations. The pilot project results from a report called 'Developing Research Careers In and Beyond Europe', produced through the collaborative effort of 23 research organisations. It draws on findings from an international workshop co-hosted by the ESF and the National Research Fund (FNR) of Luxembourg in February 2012. For the first time, this brought career tracking of researchers into the international research policy agenda and provided the backbone for this initiative.

The Challenge

There is great interest in post doctorates, but a lack of data in relation to career tracking. Policy makers, funders and prospective doctoral students all want to know if the long, expensive and difficult route to a PhD is worth the societal and individual investment. 

Organisations have a natural interest in seeing how their investment in grants and programmes has contributed to the career paths of the researchers concerned, and to the benefit of society. They also want to understand the challenges, bottlenecks and opportunities at different career stages, in order to tailor policies and activities to researchers’ needs in the future.

The solution

The European Science Foundation (ESF), together with five research funding and research performing organisations, collaborated in a pilot study of the career paths of early-stage doctorate holders. 

The participating organisations were: 

  • AXA Research Fund, France
  • Fonds National de la Recherche (FNR), Luxembourg
  • Goethe Graduate Academy (GRADE), Goethe University, Frankfurt, Germany
  • Paul Scherrer Institute (PSI), Switzerland
  • The Special Programme for Research and Training in Tropical Diseases (TDR), a co-sponsored programme of UNICEF, UNDP, the World Bank and WHO.

The purposes of the study were to:

  1. Develop a system for evidence-based, knowledge-economy planning across international, institutional and sectoral settings
  2. Produce an outcomes instrument containing standardised and/or validated measures for monitoring and assessing post doctorate career satisfaction levels, mobility and outcome patterns
  3. Provide aggregate and customised institutional analyses of the pilot study’s findings. 

These objectives were fully achieved.

The project had a ‘bottom up,’ partnership-based methodology, which included strong involvement of participating organisations in the design and implementation of the survey. Their involvement was particularly effective in facilitating and encouraging high response rates from their own organisational cohorts of doctorate holders. 

The questionnaire used for the pilot includes items on demographics and virtual, physical, sectoral and occupational mobility. It also includes research outcomes, roles and responsibilities, competence development and skills utilisation. It was designed to allow for a sensitivity analysis of questions of interest, e.g. inter-sectoral mobility between academia and industry, geographic mobility (country of origin/country of doctorate/country where the survey respondent now works), virtual mobility, and inter-disciplinary mobility. 

Each of the five participating organisations identified and contacted a cohort of their own doctorate holders covering a period of up to seven years post-PhD completion. Participating doctorate holders agreed to be included in the pilot and to have their contact details made available to ESF for the survey. 

The final sample comprised 880 doctorate holders, of whom 499 responded (a response rate of 57%). After the survey, five small focus-group meetings were held to explore contextual career and mobility choices. 

The deliverables provided to each participating organisation of the pilot included:

  • Organisational level reports (one per participating organisation) including a statistical analysis of their cohort of doctorates and comparison with the pooled results/data across all participating organisations
  • A focus group report for their relevant cohort
  • The final global report for all participating organisations
  • A process manual detailing the methodology used for the pilot study
  • An anonymised data set for their cohort and the global set of pooled data (in Excel or SPSS).

The results

The primary purpose of the pilot was to design and validate a doctorate-holder career-tracking survey and methodology that could be implemented across different countries and types of research organisation.

The results, while preliminary, are of interest. Some of the headline findings include:

  • Very high levels of employment among doctorate holder respondents (99%), with the majority in full-time employment (89%). However, a high proportion respondents reported insecure forms of employment such as temporary contracts
  • A high level of employment in public sector institutions (82%) followed by non-profit organisations (7%), the private sector (5%) and others including public-private partnerships (5%).
  • Limited inter-sectoral activity in terms of contact with industry or commercial players with some 15% of respondents involved in joint publications with industry partners; 20% collaborating at a distance and 12% working with industry via the web.
  • Strong gender typing in terms of men and women being concentrated in different occupational groupings. A much higher proportion of men than women were employed in management, computer and physical science occupations and a higher proportion of women than men were employed in social science occupations. Similar proportions of men and women work in life sciences, healthcare and education.
  • Significant differences in satisfaction and outcome levels between respondents with employment security and those on temporary contracts. Those on permanent contracts produced higher levels of outputs of societal relevance (patents, policy impacts and public engagement activity) than those on temporary contracts. Those with tenure were also significantly more satisfied with important aspects of their work environment including scientific environment, organisational culture and support available for their career development. 

The report discusses the issue of growing numbers of doctorate holders internationally, and the preference of doctorate holders for a career in academia resulting in bottlenecks and crushing levels of competition for very few posts. The low level of transfer to other employment sectors is noted, as is the need for enhanced guidance, information and policy strategies that encourage alternative career choices. 

The mobility patterns that emerged indicate that doctorate holders are highly mobile with some 90% having worked or studied in another country for a period of at least three months. 

The findings are in keeping with the pattern reported in the OECD Careers of Doctorate Holders (2013) survey.  

Geographic mobility tends to be largely intra-European. North America is also a draw for this group. Within Europe, the mobility pattern seems to be uni-directional, i.e. from Southern or peripheral European countries into Northern European countries. Those who move country to undertake their doctoral study seem to base themselves in those countries subsequently. Concerns are raised in the report about this uneven distribution. It was acknowledged that a scaled-up study would be required to allow more in-depth exploration of the apparent geographic trends from the pilot. Such movement, unchecked, could further weaken already fragile economies.

The existence of an explicit funding policy to encourage the return of doctorate holders (or their equivalent) appears to be one way to tackle this issue - as evidenced by the experience of TDR, a participating organisation whose alumni do return to their country of origin after qualifying or post-doctoral experience abroad.

The report, which was published in May 2015, culminated in a set of recommendations aimed at policy makers and research organisations.

Participating Organisations

AXA Research Fund - AXA RF

Protecting its clients and the community from risks is at the core of AXA's purpose. Convinced that researching today will help better protect tomorrow, the AXA Group created the AXA Research Fund in 2007. Its mission is to boost scientific progress and discoveries that contribute to understanding and better preparation against environmental, life and socio-economic risks.

The Goethe Graduate Academy - GRADE

The Goethe Graduate Academy - GRADE supports junior researchers from all faculties at the Goethe University Frankfurt. Its purpose is to educate outstanding leaders - inside and outside academia. GRADE thus offers a diverse and high-quality programme for doctoral candidates and postdocs.

Fonds National de la Recherche - FNR

Since its creation in 1999, the Fonds National de la Recherche (FNR - National Research Fund) has become a key actor in building a high-quality research system in Luxembourg. The FNR provides funding for all branches of science and the humanities with an emphasis on strategically aligned research domains.

Paul Scherrer Institute - PSI

The Paul Scherrer Institute, PSI, is the largest research centre for natural and engineering sciences within Switzerland. It performs world-class research in three main subject areas: Matter and Material; Energy and the Environment; and Human Health. By conducting fundamental and applied research, PSI works on long-term solutions for major challenges facing society, industry and science.

UNICEF/UNDP/World Bank/WHO Special Programme for Research and Training in Tropical Diseases - WHO/TDR

TDR, the Special Programme for Research and Training in Tropical Diseases, is a global programme of scientific collaboration that helps facilitate, support and influence efforts to combat diseases of poverty. It is hosted at the World Health Organization (WHO), and is sponsored by the United Nations Children’s Fund (UNICEF), the United Nations Development Programme (UNDP), the World Bank and WHO.

www.who.int/tdr

 

Testimonial

“We were very pleased to see confirmation in this study of TDR’s success in helping researchers from low- and middle-income countries return home after study abroad. Building and retaining research capacity is a critical goal of ours, and this pilot study helps us in our continuing evaluation of what has the strongest impact.”

John Reeder, TDR - the Special Programme for Research and Training in Tropical Diseases