1 / ARCP advice for Panel Members
Drs. Ramesh Mehay, Paul Johnson & Mike Tomson

Intro & Help

ARCP stands for Annual Review of Competency Progression (an annual review to see whether trainees are progressing well towards competence). This formal review has to happen EVERY year AND before they move up an ST year. For most full-time trainees who have not taken any extra time out of training than is allowed – their ARCP can be orchestrated to happen just once a year to cover both things – an annual review and prior to an ST change. It gets more complicated when a trainee become part-time or has taken extended additional leave (like sick leave).
This document gives advice on what to look for in the review. Reviewing a trainee’s e-portfolio (for ARCP or ES) involves making qualitative judgments more than quantitative ones. We are not asking you to look for excellence. All we’re asking you is to determine whether there is ‘good enough’ evidence of satisfactory progression. Don’t lose sight of this.
These judgments need to be holistic too – so, if evidence for a particular competency is lacking in one part of the e-portfolio, check if it is lacking elsewhere; or perhaps it is demonstrated elsewhere! The WPBA is designed so that each professional competency is tested in a number of different places.
The ultimate question for the final ST3 is whether they are ready for independent practice. As the trainee progresses through training, the evidence of competence demonstrated and degree of readiness should gradually accumulate. The picture becomes clearer as more evidence is gathered – look at that big picture, determine whether the evidence is there and remember that ‘good enough is good enough’.

Preparation at Home

  • There will be preparation for each ARCP assessor before the meeting. You will basically be given 4-5 trainees to assess at home. This means logging into their ePortfolios and determining whether they are making progress.
  • This document provides detailed guidance for a form called‘ARCP checklist in 5 steps’ - which tells you exactly what to look for in the ARCP review.
  • Your ARCP review of each eP should only take 30minutes – 1 hour. Probably 1 hour if you’re a newbie, but you will speed up as you become more familiar. Remember to slow down if you start sensing areas for concern.
  • Please note down any areas which are unsatisfactory or for which there is missing evidence. Use the ARCP Checklist in 5 Steps form to record this information. The panel chair will ask you for specifics on the day and this information will be fed back to the trainee and their GP educator(s).

ARCP panel membertraining

If this is the first time you are an ARCP panel member, there are some prerequisites..

  • You should have been on an ARCP training workshop – contact your scheme administrator if you have not. These are usually Deanery-led and only happen a few times a year (at Autumn/Spring schools). All panel members (lay people & doctors) should do ARCP training every 3 years – unless you have ‘inherited rights’ through your wealth of experience in ARCP panelling.
  • You should also been on some Equal Opportunities training in the last 3 years – if not, do some online training:
  • Please read this document in its entirety (and look at the ARCP checklist in 10 Stepsassessment form).
  • Consider taking this MCQ to boost your confidence:

Making notes for the ARCP

  • Please use the ARCP Checklist in 10 Stepsand follow it through. It can be found at ES/ARCP tab in top navigation menu).
  • Fill this form in electronically and bring a copy on the day. Please send a copy to the scheme’s administrator so that they can start collating feedback for the trainee.
  • If you make notes, remember to think about your wording. Your wording is important because
  • It helps the panel to create SMART objectives against which progress will be measured and
  • The final ARCP feedbackis the thing that will be examined for clarity and reasonableness at appeal or in legal challenge.

2 main questions your review needs to answer

The role of the panel member is not to do ‘another’ ES type review – that would simply be too time consuming, unnecessary and it is not good duplicating work already done. Yes, you are quality assuring the ES report to some extent but the 2 main questions that need answering are:

1. Has this trainee met all the stipulated requirements AND

2. Are they doing okay? (i.e. no major concerns)

And that’s really it.

Most of the info for your ARCP review will be in the ES report.

Most of the information for the ARCP review can be gleaned from the ES report. And as it’s all in one place, the review really shouldn’t take too long. You may have to dip into other areas of the ePortfolio to get a more informed opinion about the trainee, but again, you’re only dipping in to get a good enough flavour rather than doing a detailed analysis and review. Even if there are concerns your review shouldn’t take too long – because the bottom line is answering the two questions above. If the trainee has not met the stipulated requirements or is not progressing well – don’t give them the green light and instead refer them to the panel to discuss.

Using the ARCP Checklist in 5 Steps form

Using the ARCP Checklist in 5 Steps form

Using the ARCP Checklist in 5 Steps form should help you do the review in 30 minutes to 1 hour (unlike the ES reviews which take 3-4 hours).

  • The ES report is the main thing you will need to look at. Remember, you are reviewing an ST year (and not just the last 6 months) – so there may be more than one ESR to read.
  • You shouldn’t have to spend very long on each section of the ARCP Checklist in 5 Steps form. It all depends on the quality of the ES report.
  • If it is well-written, you have faith in its reliability and the trainee is doing well, we recommend a light touch approach  make the judgement and swiftly move onto the next trainee. Simply whizz through most things, perhaps dip into the ePortfolio here and there and get it done within 20-30 mins. However, slow down if you come across areas concerns.
  • But if the report is shabby, then you will have to dig deeper into the eP in order to get the information to help make your recommendation. The review will take longer (and hence why feedback to the ES is important) – but remember, you only have to dig to the level where you have enough reliable information to make your decision.
  • To determine whether an ESR is reliable –
  • Look at the comments the ES has made…
  1. Are they specific, measurable, achievable, relevant and timely or SMART?
  2. Are they backed up with evidence? (objective more than subjective)
  • Two areas of the ES report which in particular will help you decide are
  1. the ‘Actions’section for the Competency Ratings on the 1st page and
  2. the ‘Agreed Learning Plan’ sectionon the last page

An invaluable technical tip

The guidance below relies heavily on the ES report. However, sometimes you will need to dip into other areas of the ePortfolio. It can be quite ‘clunky’, cumbersome and frustrating flitting between the ES report and other areas of the ePortfolio. In the guidance below, we suggest the following. If you need to come out of the ES report and look at another part of the ePortfolio, instead of coming out, stay there. Instead, RIGHT CLICK on the other place you want to go into (usually one of the links in the ePortfolio’s left hand navigation menu) and select the option‘Open link in a new tab’. This will open that link in a new internet page which you can then view by clicking on its tab at the top. When you’ve read it, you can then click on the tab for the main ES report (which you will not have clicked out of). This is a real time-saver. This ‘right click’ method is a really good tip for Ed Supervisors and Trainers too – for instance, when reading the learning log entries – open multiple log entries all in one go  read each one and validate them rather than going in and out of a log entry and the learning log entry list.

Before anything – workout what stage the trainee is at

Log into the ePortfolio and click on Summary and figure out where the trainee is in their training programme. For example,are they coming to the end of their final year (ST3) and heading for their Certificate of Completion of Training (CCT) or are they transitioning from their first year (ST1) to their second year (ST2)? This is important because it determines your ARCP approach and how rigid to be with ‘the training rules’. Someone who is moving up an ST year (i.e. transitioning) needs to have all the right numbers of assessments and has to meet all the national requirements. Someone who is having a progress review but not transitioning (usually part-timers, those out-of-sync or returning from maternity), only needs to do the right number of things for where they are at (i.e. pro rata) – in which case you’re simply making sure that they’re making good progress and are on track for training. And, when considering recommendations for CCT, you need to consider whether any concerns are so serious as to need say 6 months extra in training at a minimum cost of £40,000.

Guidance Notes: ‘ARCP Checklist in 5 Steps’

1. Click Progress to Certification – read last ARCP report

□This will give you a good idea of what the trainee was like during their last review.

□Were there previous areas of concern? Have they been rectified or are similar problems still going on? Issues may be in relation to WPBA, their post, other parts of their training and even their home/social life. Or it might include more serious stuff like complaints, significant untoward incidents and other serious investigations.

□You don’t need to spend too long here if everything looks okay

2. Click Review Preparation – read last 2 ESRs

□Most of the information for your ARCP review should be available from the ES report.

□For each ST year, there will be a minimum of two ES reports to read. Spend time reading the latest ESR first because it presents cumulative data from previous reports on things like Curriculum Coverage, DOPS, Linked evidence to the Professional Competencies etc.. The last ESR will guide you how quickly to go through the others. If there are no concerns, swiftly read the others (otherwise, slow down).

Look at the following sections of the ESR. Don't spend too long on each section (unless concerns). Also remember, good enough is good enough.

RATING SCALES - both trainee's and the ES's

□ESR competency ratings: if an Educational Supervisor has rated the trainee BELOW EXPECTATIONS in 3 or more competencies, the ESR should say ‘unsatisfactory progress’ or ‘panel opinion requested’ and the local panel needs to decide whether the trainee is making progress or not. If not, refer to a central deanery panel.

□Trainee's self-rating: read the 'Evidence' section - is what they have written objective rather than subjective? If not, has the ES written a more objective report?.

□ES's rating: read 'Actions’ section–are the suggestions SMART (i.e Specific, Measurable, Achievable, Realistic and Timely)?

CURRICULUM COVERAGE

□Look at 'Linked learning Logs TOTAL' –this will tell you how many total log entries there have been for each curriculum area so far. In terms of where the trainee is at, is there an appropriate level of coverage? Are there deficient areas the trainee needs to work on?

□Look at 'Linked Learning Log in Current ST Year' –this tells you how well the trainee has been adding things into their eP in their current ST year. In other words, how well engaged they have been with the eP during this ST year. Compare to previous – better, worse, the same? Word of warning though: if numbers low, don’t automatically assume the trainee is not engaging or experiencing difficulty - it may be that the trainee's log entries have been linked to the old curriculum in which case click 'Expand Curriculum Statement Headings 2010'.

□Overall, what does ES say about coverage? Reasonable?

□By the end of ST3

  • There cannot be zero entries for any one curriculum area.
  • But there is no set number of log entries for each curriculum heading.
  • However, for those areas which are frequently encountered in GP - we would expect double figures. Smaller numbers for areas less frequently encountered.
  • Common problems: Management – is not Clinical Management but about Organisational stuff, Teaching – have to do, not simply attend, and cannot link every patient encounter to the Consultation – must write about cons. skills.

DOPS

□Good progression? (i.e. reasonable for where the trainee is at?) Remember, they have 3 years to get them done. An ST1 might have only got one done – which might be okay. An ST2 should have got 2-3 done. The super keen ones will have got all of the mandatory ones done by end of ST2.

□What does ES say?

WPBA

□CBDs, COTs, CEXs Numbers:

  • This bit isn't covered well in the ESR because it is only listed as the number of assessments done in the period the ESR reviewed and not the whole ST year (which is what the ARCP review period is interested in).
  • Therefore, right click on 'Evidence' in the left hand menu and select'Open in a new tab'. Select the ST year under 'Review Type' and then select 'All Reviews' to see how many have been done in the current ST year so far from its beginning.
  • Now look at the table on the summary page: make sure all minimum numbers are satisfied.
  • Remember, for trainees who are NOT transitioning (i.e. not moving up an ST year and often part-time) – in terms of numbers, have they done what would be expected on a pro-rata basis? For example, at the end of the ST year for a full-time ST2, one would expect 6 CBDs. For a person who has just come back from maternity and has done 4 months at 50% part-time (which is equal to 2 months at full-time equivalent), one might expect 1 CBD (2/12 months x 6 CBDs).

□Click MSF

  • You only really need to read 'Comments from supervisor' who should have summarised 'highlights in performance' and 'areas for development'.
  • What does all of this tell you about the trainee? Doing okay or concerns?

□Click PSQ

  • Briefly look at the numbers. (But) Remember that numbers DO NOT TELL YOU THE STORY behind them. For instance, a doctor who gives the patient everything they want may be rated more highly in the PSQ than a doctor with higher skills who appropriately challenges them.
  • Read the ES comment at the end. What does all of this tell you about the trainee? Doing okay, or are there concerns?

□Click CSR

  • Skim through the report to ensure no big issues. What does this report tell you about the trainee? Doing okay, or are there concerns?

COMPETENCY AREAS

□Look at columns 'Linked Forms TOTAL'– this will tell you how many assessments have been done which have looked at each competency. For where the trainee is at, does this look reasonable? Any big deficiencies?

□'Linked Learning Logs TOTAL' - this will tell you how many learning logs have been validated against each competency. For where the trainee is at, does this look reasonable? Any big deficiencies?

□If you see low numbers of validated entries…

Validation of learning log entries to competencies can only be done by the Educational or Clinical Supervisor – not the trainee themselves. So, if you see low numbers, quickly check to make sure it is not because there are loads of log entries but the educator has not read and validated them. Right click ‘Learning Log’ in left hand navigation menu and ‘Open in a new tab’ and look at the column which says ‘Read’ and see whether most entries have a tick in them. If they are, then pick a few entries to look at the top to see if the educator has linked them to the professional competencies (i.e. validation).

PDP

□What does the ES say (esp about quality)?

□Take a peek at the PDP entries themselves - right click PDP in left hand navigation menu and select 'Open in new tab'. Is there reasonable engagement (i.e. at least 4 PDP entries per post)? It is unacceptable to have zero PDP entries for a post.

□The ES report itself should generate at least 3 action points of which can be converted into PDP items. The trainee should generate at least one extra item based on their current post.

QUALITY OF EVIDENCE

□What does the ES say about the quality of (usually) the log entries?

□Dip in and have a look at a few to get a better idea of the trainee. Right click 'Learning Log' and select 'Open link in a new tab'. Don't spend too long - only sample enough to give you a reliable flavour of what the trainee is like as a learner. You’re mainly looking for the level of reflection within the entries (superficial or deep & meaningful?).