Outcome Measures and Evaluation

Outcome Measures

An outcome measure is a tool designed to measure the effect (such as changes or improvements) on a student's mental health as the result of intervention. UMHAN members have been debating and discussing the use of outcome measures since UMHAN was created! There are no particular outcome measures designed specifically for use with students in an educational context, or that are consistently used across the sector.

Part of this discussion has focused on this lack of consistency, and the need for standardised terminology to make sure that any measurements are analysing the same thing. Please visit our page on Language and Terminology for more information about this.

University counselling services often use CORE (and the associated software CORE-NET) and so some mental health teams embedded within counselling services also use this measure. However, a study by Broglia et al. in 2018 found that half of all counselling services did not use a validated clinical measure (1).

In our most recent Member Survey Report 2023, we asked members about monitoring and evaluation of interventions and activities with students. The majority of members collect data, monitor and evaluate the work they do with students and many use service or institutionally designed surveys. The outcome measures most cited by members include:

GAD-7 - Generalised Anxiety Disorder Assessment is an online, score-as-you go tool which can be used during telephone and online sessions with students.

PHQ 9 - Patient Health Questionnaire 9 is another online measure for use in screening depression, often used in conjunction with GAD.

CORE-10 - Clinical Outcomes in Routine Evaluation - 10 item scale. This asks how a person has felt over the past week.

CIAO - Counselling Impact on Academic Outcomes - this is a new measure available to CORE Net FE/HE subscribers.

Other outcome measures mentioned but used less frequently included:

CORE-OM - Clinical Outcomes in Routine Evaluation – Outcome Measure. It has 34 items.

WEMWBS - Warwick-Edinburgh Mental Wellbeing Scale. It has 14 items. SWEMWBS - the short WEMWBS - 7 items.

See our report for more information.

SCORE Consortium

The Student Counselling Outcomes Research and Evaluation (SCORE) consortium comes out of a long-running wish in the university and college sector to create a shared routine outcomes database to provide evidence for the sector and to improve service delivery. It is a researcher-practitioner consortium developing a data set of student counselling outcomes. The Project Manager is Dr. Emma Broglia at the University of Sheffield.

Progress to date has included working with four measures – the Clinical Outcomes in Routine Evaluation Outcome Measure (CORE-OM), the Counseling Center Assessment of Psychological Symptoms (CCAPS), the Patient Health Questionnaire (PHQ-9), and the Generalized Anxiety Disorder (GAD-7).

The Consortium publish frequently on their work in this area.

For example, in 2017, Broglia et al. explored the Counseling Centre Assessment of Psychological Symptoms (CCAPS-62) compared with the CORE-10 with almost 300 students at UK universities. They concluded that CCAPS-62 was a reliable assessment tool to use with UK students in university counselling services (3).

A recent article about the adoption of routine outcome monitoring (ROM) by Barkham and Broglia on behalf of SCORE, looked at the purpose of ROM, burden of measurement, impact of measurement on clients and the concerns of practitioners (4).

In 2024, Broglia and Barkham's 30 page article was published which presents evidence-based recommendations aimed at enhancing data capture and usage (5). In this paper they explore the pros and cons of everyone using the same measure. They also put forward 10 recommendations for supporting universities and colleges to move towards adopting the principles of learning health systems to support students’ mental health.

SCORE Consortium

Impactosaurus

This is a free tool which contains some established measures, such as CORE-10 and GAD-7 as well as the ability to create your own.

Impactosaurus

Child Outcomes Research Consortium

The Child Outcomes Research Consortium collects and uses evidence to improve children and young people’s mental health and wellbeing. Founded in 2002 by a group of mental health professionals determined to understand the impact of their work, their members include mental health service providers, schools, professional bodies and research institutions from across Europe and beyond.

They have a variety of resources on their website, as well as discussions about why outcome measures are used, and how to use them successfully.

CORC

Evaluation

TASO

Transforming Access and Student Outcomes (TASO) is an affiliate What Works Centre set up in 2019 and funded by the Office for Students. Their vision is to eliminate equality gaps in higher education using evidence-informed practice in Higher Education. In 2023 they launched their Student Mental Health Evidence Hub. This includes collated examples of practice, the best available evidence, and evaluation guidance to improve student mental health support in HE.

TASO Student Mental Health Evidence Hub

Their evaluation guidance also includes information on outcome measures for non-clinical contexts (Byrom et al., 2023) (2).

AMOSSHE

AMOSSHE the Student Services Organisation produced an online toolkit aimed at helping departments undertake performance monitoring, assess what is most appropriate to their organisation and students’ needs, and support professionalisation in the sector.

The toolkit is intended to support evaluations of the value and impact of services that support students. It includes:

Guidance on the issues to consider when thinking about undertaking an evaluation
Practical steps for undertaking evaluation
Methods and instruments to use during the evaluation process
Examples of practice taken from the literature, developed by the universities involved in the pilot phase of the project, and elsewhere

It was published in 2011 but is a useful starting point for members new to impact evaluation.

AMOSSHE Value and Impact Toolkit

SMaRteN

The Student Mental Health Research Network has also undertaken some research into appropriate wellbeing and mental health measures for longitudinal research and evaluation with students. Heron et al. (2023) (6) held consensus groups exploring the set up and running of student wellbeing cohorts for universities interested in undertaking longitudinal studies. The groups were made up of student representatives, academics, clinicians, and clinical academics with experience. They reached consensus that a core set of areas should be measured and recommended the following four measures to be used a “core battery”:

GP-CORE (general psychological symptoms)
WEMWBS (general wellbeing symptoms)
PHQ-9 (anxiety symptoms)
GAD-7 (depressive symptoms).

Whole School and College Approach (WCSA) Measurement Toolkit

Launched in 2024, this is a free, online tool to help schools and colleges measure and self-assess the impact of their whole school and college approach to emotional wellbeing and mental health. It was produced by partners across Sussex and Kent to support school and college staff and Mental Health Support Teams (MHSTs) to better track and monitor their approach to promoting emotional wellbeing.

A report detailing how the toolkit was developed and piloted by Robinson et al. can be found here. The authors also consider its strengths, limitations and applicability.

WSCA Measurement Toolkit