Connect with us
  • Elysium


Escaping rehab’s ‘black box’ problem



Deep-rooted in the rehabilitation field is the scourge of the “black box”.

It prevents research progress, hinders patient outcomes and makes it harder to justify spending on rehab services.

Its origins as a popular phrase among healthcare professionals can be traced back to the early 90s.

Its relevance has endured to present day, however; much to the dissatisfaction of a band of clinicians and researchers in the US.

As one paper they produced earlier this year puts it, the black box of rehabilitation is: “Our inability to characterise treatments in a systematic fashion across diagnoses, settings, and disciplines, so as to identify and disseminate the active ingredients of those treatments.”

Another paper in the same series notes: “Most often, (rehabilitation) treatments are defined by either discipline (“X hours of occupational therapy”) or the problem being treated (“gait training”), neither of which describes what the clinician actually does to affect functioning.

Research reports that include detailed protocols often lack information about how a treatment was administered; for example, instead of reporting what quantities of active treatment ingredients were provided, treatment dose descriptions simply state the duration or number of sessions.

“Even published treatment manuals frequently lack sufficient details to enable other researchers to replicate findings or build on previous results or for clinicians to confidently implement published treatments in everyday care.”

Dr John Whyte (pictured) is among those who have been proactively rallying against the black box problem for over a decade.

Whyte is founding director of the Moss Rehabilitation Research Institute in Philadelphia and a specialist in traumatic brain injury and disorders of consciousness.

He has also spent over 10 years on a journey to solve the limitations of classifications in rehabilitation.

“The vast majority of our treatments are not drugs, but experiences that patients have in collaboration with a clinician such as a physical or occupational therapist,” he says.

“We’ve never had a good way of defining and labelling them. One of the most common ways of talking about the treatments people get in rehabilitation is by discipline – ‘he got six weeks of physical therapy’ – as though either everything a physical therapist does is effective, or nothing they do is effective, and all you need to know is how many times they met.

“Alternatively, we name treatments by the problem that they’re treating, like ‘gait training’, ‘memory remediation’, ‘social skills training’, but again the classification gives us no idea what actually transpired in those treatment sessions.”

The American rehab community, with international support, has been leading the way in unravelling the black box puzzle. In 2005, an American Congress of Rehabilitation Medicine taskforce became a hub for discussion of the black box problem.

The (then) National Institute on Disability and Rehabilitation Research organised a funding announcement that invited development of a new approach to treatment classification and measurement in rehabilitation.

A consortium of researchers and clinicians was then awarded a five-year grant and embarked on what has ultimately been a journey spanning over a decade.

Whyte has been a key participant in developing the concepts in that time and is now chair of the ACRM task force that continues to promote their implementation.

Thanks to a research contract from the Patient Centered Outcomes Research Institute, these procedures have this year been distilled into the Manual for Rehabilitation Treatment Specification (which can be downloaded here http://mrri.org/innovations/manual-for-rehabilitation-treatment-specification/).

Whyte says: “Essentially, we need to get to the point where the label of the treatment allows clinicians and researchers to say ‘this works’ or ‘that doesn’t work’, because everything with that label works or doesn’t. And, everything with that label can be studied and compared to something with a different label.

“The Rehabilitation Treatment Specification System (RTSS) gives a set of rules and procedures by which we would argue every rehabilitation treatment can be specified with respect to its known or hypothesised active ingredients.

“From here we can begin to use it as a taxonomy for grouping treatments. We can use it to aggregate treatments and show how one active ingredient is effective versus another active ingredient.”

The RTSS could be pivotal to improving rehabilitation research as well as patient outcomes.

Its development comes amidheightened pressure on the rehabilitation field to establish an evidence base for its interventions.

“Increasingly, this means the need for comparative effectiveness research on treatments with established efficacy, as well as sophisticated methods for synthesising evidence across studies,” says a paper published by those behind the RTSS initiative.

The same report also highlights the complications of proving the efficacy of rehabilitation treatments. There is the field’s complex, multidisciplinary team structure and its emphasis on behavioural treatments, “which depend heavily on the training of new skills and on the interpersonal dynamics between patient and provider”.

Also, “rehabilitation applies to a great diversity of populations, settings, and clinical problems, from impairments through activities to participation”.

The paper also notes that rehabilitation is patient-centred in a way that fosters “a nearly infinite variety” of treatment goals.

RTSS remains a work in progress for Whyte and his fellow black box crusaders.

There is an ongoing forum for discussion with rehabilitation professionals with diverse backgrounds who are interested in issues related to treatment specification (see more here).

Various projects are also underway to inform further refinement and implementation of the RTSS.

Educational efforts to increase awareness of the RTSS and enable its use by rehabilitation professionals are also planned.

Click on the links below to read more about the black box problem and the RTSS that could help to solve it.

A Theory-Driven System for the Specification of Rehabilitation Treatments (2019, Jan.)

The Rehabilitation Treatment Specification System: Implications for Improvements in Research Design, Reporting, Replication, and Synthesis (2019, Jan.)

Advancing Rehabilitation Practice Through Improved Specification of Interventions (2019, Jan.)

The Importance of Voluntary Behavior in Rehabilitation Treatment and Outcomes (2019, Jan.)

Editorial: An End to the Black Box of Rehabilitation? (2019, Jan.)

Click to comment

Leave a Reply

Your email address will not be published. Required fields are marked *