Assessing prevocational trainees: a brief guide for Term Supervisors

New Prevocational Progress Review forms were introduced in January 2009 for the assessment of prevocational trainees in PGY 1 and 2 (interns and resident medical officers).

The new forms provide more information for trainees, Term Supervisors and hospital Directors of Prevocational Education and Training (DPETs). The forms are designed to measure training outcomes in terms of the Australian Curriculum Framework for Junior Doctors (ACF), ensuring that prevocational trainees are meeting the standard required for registration with the NSW Medical Board.

The following information is a guide for Term Supervisors on how to use the forms to aid effective assessment of junior doctors.

There are separate forms for mid term and end of term

• Mid Term Formative Appraisal (Orange form)

This form is completed around week 5. The aim is to review the trainee’s performance, identify any areas in which they have been having difficulty or require further education and training, and plan the future direction of the term.

• End of Term Summative Assessment (Red form)

This form is completed in the final week of the term. For year 1 trainees this assessment is required by the NSW Medical Board for registration. For year 2 trainees the assessment forms part of their record of achievement and will be useful in evaluating their progress to vocational training.

A large part of each form is completed by the trainee before meeting with the Supervisor.

Procedures for structured assessment

Australian Curriculum Framework for Junior Doctors

The Australian Curriculum Framework for Junior Doctors has been developed by the Confederation of Postgraduate Medical Education Councils. It provides a detailed description of the knowledge and skills that junior doctors need to learn in order to become safe and effective in the Australian health system. By breaking down the requirements into specific performance elements, the framework makes it easier to plan and assess training in the workplace.

The new Prevocational Progress Review forms are designed to measure outcomes related to the Curriculum Framework. Term Supervisors can use the Curriculum Framework as a guide to appropriate training goals for the terms they are supervising.

Three meetings

For assessment purposes, Term Supervisors should arrange three meetings with each prevocational trainee in each term:

1 Term orientation:

The Term Supervisor reviews the term description and discusses:

• the major focus and goals of the clinical unit and the expectations of the JMO’s role
• term learning objectives and skills training goals
• supervision needs and the process of performance assessment.

The Term Supervisor may also want to review the trainee’s current level of knowledge and experience.

Service information about work practices, protocols and guidelines may also be provided by the Supervisor at this time, or this may be tasked to another senior clinician.

2 Mid term appraisal:

Takes place at about week 5 of the term.

The trainee completes page 1 and the self-assessment in sections 1 and 2 of the form before meeting with the Term Supervisor.

Self-assessment provides a basis for discussing progress and planning the future direction of training during the term. The trainee is asked to outline personal strengths and achievements and identify weaknesses or needs for further development. The Supervisor then offers feedback in the same manner, filling in sections 2 and 3 of the form.

Should the trainee receive a grade of “Borderline/requires assistance” or “Clearly below expected level”, the Director of Prevocational Education and Training (DPET) should be informed and a progress plan should be identified.

3 End of term assessment:

Takes place in the last week of term.

The trainee completes page 1 and the self-assessment in sections 1 and 2 of the form before meeting with the Term Supervisor.

Self-assessment provides a basis for discussing the outcomes of the term. The Supervisor completes the assessment by filling in sections 2 and 3.

The assessment should indicate the trainee’s standard of performance at the end of the term. Gaps or weaknesses in performance are only relevant if there are reasonable grounds for believing that the trainee is still performing at that level. If the trainee is assessed as not demonstrating overall competency in the term, the DPET should be informed.

Standard of assessment

Prevocational trainees are rated against the expected performance of average trainees at an equivalent stage of training. This expectation will be higher in the fifth term than the first. Supervisors should consider all trainees as having commenced internship with an Australian medical school graduation or equivalent, such as AMC assessment.

The Progress Review forms use four ratings:

1 Clearly below expected level: The trainee is substantially below the standard expected for someone at this stage of training and needs planned assistance to meet the standard. If any rating is “Clearly below expected level”, the Term Supervisor should consult the Director of Prevocational Education and Training.

2 Borderline/requires assistance: The trainee requires further development relative to the average trainee performing this term at this stage of training and needs planned assistance to meet the standard. If any ratings are “Borderline/requires assistance” the Term Supervisor should consider consulting the Director of Prevocational Education and Training.

3 At expected level: The trainee is performing at the level expected of someone at this stage of training.

4 Clearly above expected level: The trainee is performing at a level that should be recognised and highly commended.

Why self-assessment?

Skill in self-assessment is now an integral part of professional development. Self-assessment followed by discussion with the Term Supervisor gives prevocational trainees the opportunity to reflect on how they perceive they are performing, and insight into how they are actually performing. This may provide encouragement or impetus for change, and further enhance their skill in self assessment, which will be required in most vocational programs.

Effective mid-term appraisal and action plans

Key features of helpful, effective appraisal include accurate evidence, constructive discussion, and cooperation to develop a plan to specifically address any problem areas.

When weaknesses and problems in the trainee's performance are identified at the mid-term appraisal, the Term Supervisor should work with the trainee to develop an action plan. Action plans might call for participation in structured training sessions, individual learning programs, supplementary tutorials, or observation of or participation in particular procedures.

As a simple example, prescribing difficulties may require some focused study in pharmacology with additional practical assistance from the team pharmacist. Close supervision of all prescribing by the registrar or CMO may be required until a further review to ensure that the appropriate skills and knowledge have been acquired.

For some trainees, an action plan may give attention to language or communication skills or other aspects of personal development. Assistance from the DPET may be required when designing an action plan.

Responsibility for assessment, and consultation with others

It is the Term Supervisor's responsibility to complete the mid-term appraisal and end-of-term assessment, but it is appropriate to consult other members of the clinical team to form a picture of the trainee's performance during the term. The progress review forms ask Supervisors to note which other staff members have been consulted. In some circumstances (such as relief terms in which the trainee has moved frequently from team to team) the Term Supervisor may not have sufficient knowledge of the trainee to form a reasonable assessment even after consulting other staff members. In this case, the DPET should be consulted, and the DPET may agree to assume the responsibility for completing the assessment.

Administration

Prevocational trainees should keep a copy of their completed assessment forms. Completed forms are collected by the JMO Management Unit, which makes a copy and sends the originals to NSW IMET.

The new forms are part of a better information system. The new system will make it easier for DPETs to identify the strengths and weaknesses of the training program and to assist Term Supervisors with thorough feedback.

For further information

The Medical Journal of Australia has published succinct and valuable articles on supervising and assessment in the "Teaching on the run" series. See:

Lake FR, Ryan G. Teaching on the run tips 8: assessment and appraisal. Med J Aust 2005; 182: 580-581. http://www.mja.com.au/public/issues/182_11_060605/lak10146_fm.pdf

Lake FR. Teaching on the run tips 9: in-training assessment. Med J Aust 2005; 183: 33-34. http://www.mja.com.au/public/issues/183_01_040705/lak10147_fm.pdf

Vickery AW, Lake FR. Teaching on the run tips 10: giving feedback. Med J Aust 2005; 183: 267-268. http://www.mja.com.au/public/issues/183_05_050905/vic10464_fm.pdf

Lake FR, Ryan G. Teaching on the run tips 13: being a good supervisor — preventing problems. Med J Aust 2005; 184: 414-415. http://www.mja.com.au/public/issues/184_08_170406/lak10081_fm.pdf

The DPET at your hospital can advise you on specific issues relating to assessment procedures for prevocational trainees at your hospital.

For a pdf copy of this guide, please click here