ACMD 512

Milestones Project Summary

Simi Rahman, MD

January 11, 2011

Concept Summary:

-   The ACGME & ABP have partnered to start the Pediatrics Milestones project.

-   ACGME Outcome Project: institutions struggle to define the optimal methods for assessment of competencies.

-   The Milestones Project will document KSA ‘milestones’ with increasing mastery of each competency.

-   In 2009 the goals were: reframe and further define the 6 competencies in Pediatrics; identify markers of achievement in Pediatrics; identify tools that could be used by the Pediatric community.

-   Guiding Principles: Central Goal - UME-->GME-->MOC

-   Highlight of Conceptual Consideration:

-   Create milestones that speak directly to the learner making explicit essential objectives and expectations of residency.

-   Narrative anchors of behavior --> learning roadmap of next steps in developmental progression.

-   Establish normative ranges, expectation of variation across learners and contexts.

-   Redefine transitions from artificial developmental progression to more meaningful ones.

-   Critical benchmarks for progression.

-   Scholarly approach, Glassick’s second standard for assessing scholarship: adequate preparation. Searched the ontogeny of behaviors that represent the ‘does’ level of Miller’s pyramid. Iterative process using Harris’ “succession of lenses” was used to construct the milestones.

Drawbacks:

-   Case specificity: the inability to translate learner performance from one context to another. The milestones are written in generic language with the addition of real-world examples. Therefore assessment of a milestone will give outcomes that may not be transferable to another context.

-   Development may take place at different rates in various anchors, like with milestone development in children.

-   Halo effect is possible if student fulfills one part of a particular milestone to the exclusion of the other, and the assessor compensates or assumes completion of the milestone.

-   Achievement of a milestone does not reflect consistency or habit. Milestones measured over time, in various contexts will need to be compiled to reflect internalization of behaviors.

Advantages:

-   Milestones are intended to measure individual learner outcome, and to be used as a guide by learners.

-   Acknowledge the complexity of assessing learners, and therefore provide a continuum instead of specific, discrete benchmarks; this leaves room for overlap between various competencies, variation between learners.

-   Critical Transition Points: stopping points, level that would need to be reached before progression to the next level.

Beyond Competencies & Milestones: Adding Meaning Through Context:

Barriers:

-   confusion about how to integrate ACGME competencies into training in a meaningful way.

-   lack of understanding of how the KSA of each competency develops over time.

-   perception of lack of applicability to real world practice.

How to bridge from the world of competencies to the ‘real-world’ practice?

-   EPAs: Entrustable Professional Activities: the routine professional-life activities of physicians based on their specialty and subspecialty.

-   To perform the professional activity the practitioner, eg pediatric hospitalist, must:

-   have K of Signs and Symptoms

-   Perform a physical exam

-   search for outcomes assc. with interventions

-   communicate with patient/ family about management plan

-   relate to family in a way they understand

-   act as a liaison to PCP

-   “Entrustable”: overcomes the challenge of assessing complex behaviors that span various competencies. When resident is able to perform ‘without direct supervision’, supervisor will be able to assess it from observation. Specific milestones must be reached for entrustment to occur.

-   EPA’s can be written for medical students as well.

How the EPAs fulfill the vision of the Carnegie Foundation Report on Medical Education:

-   Standardization of learning outcomes and individualization of learning process

-   integration of formal knowledge and clinical experience

-   development of habits of inquiry and innovation

-   focus on professional-identity formation

How the articles pertain to my practice.

As a pediatrician and hospitalist I found the articles extremely valid and timely. I can relate to my experience in trying to write objectives for a pediatric hospitalist elective for medical students and struggling to reduce the competencies to meaningful behaviors. I became aware that these behaviors develop at different rates for different learners, and reflect their life experience, prior exposure to pediatrics/ medicine, their maturity level and prior experiences in other rotations. Much of what I was teaching and assessing as an educator related to behaviors that vaguely fell under professionalism or communication skills. And every student presented a unique palette of challenges. How then to reduce them to a scale that assumed they all come to the hospitalist experience with similar experiences, mind-set or skills, basically a similar substrate?

The concept of a continuum that extends from UGE through to clinical practice is timely. I have also experienced the other end of the spectrum through my own MOC experience. There is controversy in the pediatric community towards the MOC experience in that it tries to reduce clinical practice to very concrete standards that are then applied uniformly to general pediatricians and subspecialists alike without differentiation between the two. A milestone approach to MOC would allow different physicians to be at different levels without needing to equate the two. It would allow the mastery approach to medical learning to be applied to clinical practice, with each subspecialty creating their Critical Transition Points that a physician could achieve at their own pace, reflecting their true level of expertise instead of an artificial construct.