Scaling of Marks

Scaling is where an assessment or module marks are adjusted (up or down) for an entire cohort or group of similarly affected students to ensure that the academic standards achieved by students are equivalent to previous years.

• Scaling is not used to adjust individual student marks.
• Scaling is only be used to adjust the marks of an assessment or module.
• Scaling is not used to change marks to achieve an ‘ideal’ distribution of results.
• Scaling is not be used to adjust the final mark at the programme level when considering overall degree classifications.

The decision to adjust marks using scaling is made by an Examining Board.

Examining Boards review the marks initially assigned by the examiners to ensure that the final marks for individual modules are an accurate reflection of the academic standards achieved by the students.

Examining Boards will consider statistical data for each module, including assessments within the module (for the current academic year and up to 4 previous academic years). If an Examining Board considers that the marks are not an accurate reflection of the expected academic standards achieved by the students, the Board may consider taking remedial action by scaling marks.

The use of scaling should move the marks into the range that would accurately reflect the academic standards achieved by students. Where a decision is taken to scale the marks of students to address a significant and identifiable issue, the scaling will be applied to all affected students.

Scaling will be used to either increase a spread of students across the mark range or to re-align a high or low average attainment. The exact approach adopted to achieve this will depend on the issue to be addressed. Examples of simple scaling methods are:

• Addition of marks – an agreed percentage is added to each mark;
• Multiplication by a factor – each mark is multiplied by an agreed factor.

Other methods of scaling may be considered and applied.

Full details of the University's Scaling  Policy is provided below.

The decision to scale

The decision to adjust marks using scaling is made by an Examining Board exercising its collective academic judgement after considering specific statistical data. The Policy is very clear that it should not be used to change the marks to achieve an ‘ideal’ distribution of results.

The University anticipates that scaling will not normally be needed because of variations already made to module design and assessments. Clear marking criteria will illustrate what students need to demonstrate within each performance threshold, and examiners are able to assign marks within the spectrum associated with the level of performance they judge to have been achieved by each student.

Scaling might still be considered in exceptional circumstances where, for example, it appears to the Board that material covered in lectures was inconsistent with the assessment questions set. Again, the University anticipates that this will not normally be needed because, in such cases, the module mark scheme can be “developed” to address the inconsistency and ensure that results accurately reflect the standards achieved.

Each Examining Board should be made aware of all the adjustments made to assessments and programmes when considering marks.

If, when considering those marks, an Examining Board identifies that there is an unexpected change in performance and/or that the marks achieved do not reflect the standard achieved, they can also consider scaling. When considering scaling, Examining Boards must look at the design and assessment of each module and ensure they meet the following principles:

• assessments are appropriate to the level of study;
• assessments enable students to demonstrate achievement of the learning outcomes;
• assessments cover the subject content appropriately;
• assessments are accessible and fair so that all students have equal opportunity to demonstrate achievement of the learning outcomes;
• there are clear marking criteria indicating what students must achieve in order to be awarded marks within a particular banding.

If there are questions relating to the application of these principles, then there may be adequate grounds for scaling the marks of an assessment or a module.

Application of scaling

• Examining Boards are only able to use scaling where the measured performance of a cohort differs significantly from previous cohorts and when there is no evidence that the module has changed.
• Scaling is not used to adjust individual marks.
• The Board should only use scaling at the assessment or module level.
• Scaling will not be used at the programme level when considering overall degree classifications.
• There are many methods of scaling that may be considered and applied but each Examining Board must follow the University's Scaling Policy and ensure that consideration of the principles is detailed in the minutes of its meeting.
• An Examining Board can scale marks down as well as up if that is deemed necessary to ensure marks are an accurate reflection of the academic standards achieved.
• If the marks count towards a classification, the External Examiner must be consulted and their view considered by the Examining Board.

The decision to use scaling should be recorded in the minutes of the meeting of the Examining Board and should include:

• a rationale; and
• the data used to support the decision; and
• any External Examiner’s comments.

The information in the minutes should be used to communicate any scaling decisions to any students affected.

The Examining Board should also report any instances of scaling to the relevant Board of Studies/School Board, with a view to ensuring that module marks which required scaling in one year will not need to be scaled in the next. Boards of Studies are expected to monitor instances of scaling.

Scaling Policy guidance for Examining Boards

Consideration of Module Marks

The Examining Board should review and analyse the marks and statistical data for each module including component assessments within the module (for the current academic year and up to 4 previous academic years for skewed or unrepresentative features. These data should include:

• A comparative review of the sets of module/assessment marks awarded for up to 4 previous academic years. This review would normally include:

• the mean mark before and after the proposed group scaling with comparative data from previous years;
• the standard deviation before and after the proposed group scaling with comparative data from previous years.

• A review of module/assessment marks including averages and standard deviation awarded across the programme in the same year for that cohort of students;
• Consideration of any unusual or structural mitigating circumstances that might have contributed to a significant change to the mark distribution (e.g. a change in the staff delivering a module or marking assessments, change to marking criteria, particular acknowledged problems with a particular question or questions on an examination paper);
• Information with respect to how the Examining Board has dealt with similar cases in previous years.

If there are identified issues and the Examining Board considers that the marks are not an accurate reflection of the expected academic standards achieved by the students, the Board may consider taking remedial action by scaling. Scaling should be undertaken only after full consideration of a prescribed set of statistical data that could support the decision.

The distribution of marks can be affected by such factors as a small cohort size for a module (e.g., <10).  Analysis of the over-arching statistical monitoring across the module taken within a year, or across modules taken over several years, remains an important aspect of the Examining Board discussion and decision-making process.

Examples of when it may be appropriate to scale

Typical mark ranges vary across assessments, so it is not practicable to define precise institutional guidelines. However, generally, a module’s mean will be expected to fall within a certain range. These should be roughly comparable across modules on which the same students are registered. There are however explainable differences between modules that might result in significantly different module/assessment means. Examples include, where students have engaged well with a module and therefore have performed exceptionally well in an assessment or where some of the students registered on the module are unrepresentative of the cohort of a particular programme, or where poor results are potentially explained by other factors.

On rare occasions, it might be necessary to consider scaling marks down. For instance, a member of staff working with qualitative evaluation may have misjudged an academic level of study. Such changes are unlikely to be necessary with quantitative/template marking since such issues should have been resolved through standard shadowing/mentoring approaches.

Scaling may be considered when:

there is a significant, known and clearly identifiable issue with an assessment such as an error or ambiguity; or

the range of marks significantly fails to match student performance, for instance, failing to fit onto the marking criteria/descriptor which might be evidenced by one or more of the following:

• an atypical mean, distribution (i.e. unusual patterns of high or low marks) or overall mark spread;
• the range of marks is not in line with what would be expected from past performance on this module;
• the range of marks is not in line with what has been achieved by the same students registered on other modules at that level;
• the number of fails is not in line with what has been achieved by the same students registered on other modules at that level;
• the mark profile is not what would be expected from students’ past performance on this module.

The above criteria are not defined to require an Examining Board to scale. Rather, they are guidelines as to when it might be appropriate to consider scaling. Any final decision as to whether to scale should always focus on whether there is a significant misalignment between the student outcomes and the marking criteria/descriptors.

Examples of when not to scale

Scaling is difficult to do accurately when a cohort is small, i.e. less than 15 students. This is because statistical comparisons are unlikely to be valid. In such cases, all assessed work should ideally be re-moderated/remarked, but it is recognised for certain methods of assessment (such as multiple-choice questions) scaling may be the only alternative to change the distribution of marks. In this case scaling may be used but only in if the issue is deemed to be significant and re-moderating/remarking would not resolve the issue.

Key areas for consideration

Module or assessment marks should only be scaled if there is evidence that the marks initially recorded do not accurately reflect the academic standards achieved by the students, and any scaling that is required should move the marks to the range that would accurately reflect the standards achieved. This does not mean, for example, that a higher average should simply be scaled to the point that lies just inside the range deemed to be typical.

Accurately reflecting the academic standards achieved by students DOES NOT depend on a norm-referenced assumption that a set percentage of students should receive first-class marks (either for a single module or overall for a degree), but on assessing performance against the criteria recognised as indicating such achievement.

Monitoring the relationship between marks and perceived academic standards of achievement in individual modules will inevitably raise the question as to whether typical ranges of performance will be encountered in all cases, and what the causes of any differences might be. Since the aim is not to increase or reduce all ranges of performance to match a typical range, reasoned argument must then be applied to establishing whether a set of results for a module should be scaled (and how far), or whether the difference now being encountered is produced by a group that is more or less able than might usually be observed; sometimes influenced by the small size of a group.

Care should be taken when considering the scaling of optional modules. It might be expected that the outcomes of required or core modules would tend to show similar ranges of performance over time whereas some of the optional modules could show, at least occasionally, much higher or much lower results. This might be a consequence of very small numbers of students in some modules, or it may reflect the profile of students taking the optional modules. It would be a mistake to scale marks of optional modules to represent the same range of marks as would be encountered across the required or core modules. The different results would be justified by showing that the students involved in each were representative of different levels of performance.

Scaling should not be applied to modules which only assign marks of 0 and 100 (fail/pass).

A critical part of setting and maintaining academic and professional body standards is ensuring minimum standards for a pass are set in advance e.g. Examining Boards might only scale between 40% and 90% if 40% still reflects the expected standard of a pass.

Timing of scaling

Scaling should be applied prior to the Examining Board meeting as regulations do not permit marks to be changed once marks are confirmed by the Board. It is also necessary to undertake scaling before the calculation of students’ recommended degree classification.

When considering scaling it is important to remember that, in accordance with the University Principles of Assessment and Feedback, marks are an important form of feedback to students on their progress.

Ideally scaling should be applied before assessment marks are returned to students, but only once the appropriate quality procedures have been completed. Creating a long delay on returning feedback to students on their course is undesirable so sometimes it will be necessary to release provisional marks to students before scaling has taken place and provide students with an explanation of the reason for the difference between the marks.

Any marks returned to students should always include the statement that marks are provisional until approved by the Examining Board. However, any changes made to marks through the application of scaling must at some point be communicated to students.

Application of Scaling

Where a decision is taken to scale the marks of students to address a significant and identifiable issue, the scaling must be applied to all affected students. Different decisions may be appropriate for a subset of the whole module cohort if those students were not affected by the abnormal assessment circumstances.

Where a decision is taken to scale marks which contribute to the degree classification, the external examiner should be informed of the full circumstances of the case, provided with any relevant paperwork, and invited to comment on the proposed method for scaling.

Scaling will be used to either increase a spread of students across the mark range or to re-align a high or low average attainment. The exact approach adopted to achieve this will depend on the issue to be addressed. Examples of simple scaling methods are:

• Addition of marks – an agreed notional percentage is added to each mark
• Multiplication by a factor – each mark is multiplied by an agreed factor.

Other methods of scaling may be considered and applied if the Examining Board believes they are more appropriate.

If scaling is applied, the Examining Board should record in the minutes the reason for scaling the marks and the method of scaling employed.

As noted above the need for scaling is a clear indication of an issue with an assessment, so where such cases occur it is anticipated that some form of investigation will be carried out to mitigate for the issue in future years.

Contact Student Advice

Advice@cardiff.ac.uk
+44 (0)2920 781410