Samba Safety Badge

The Only Platform to Combine CSA & MVR Data

Resource

The State of the CSA Score: IRT and Facing the Future – Fully Informed

April 25, 2019

Steve Bryan

In the dozen years that I have been involved in the Trucking Industry, I have met many amazing people and seen advances in safety technology, data, and analytics that are mind-boggling.  I have met hundreds of business owners and executives who operate their businesses with a vision of literally delivering America, but a mission built on Safety.  If I have heard it once, I’ve heard it 100 times – competition stops at safety’s door.

In 2007, I founded Vigillo and now proudly serve as an executive with SambaSafety. While SambaSafety is a supplier to the industry and exists as a for-profit business, we live and breathe the mission of safety.

Change has been constant in my industry tenure and perhaps none as momentous as the pending changes to the foundation of the CSA scoring methodology – used by industry and the FMCSA to gauge a carrier’s commitment to safety.

The specific topic is the work SambaSafety, and previously Vigillo, has done to work through the transition from CSA to the new IRT Model as defined by the National Academies of Sciences, Engineering and Medicine (NAS) in their report of June 2017:

https://www.fmcsa.dot.gov/sites/fmcsa.dot.gov/files/docs/mission/policy/82226/nas-report-final-june-2017.pdf

Change as big as the shift from CSA Classic to an IRT model has led to a robust debate among industry participants. Below are excerpts from some concerns we’ve heard and our response.

Question: While it is possible to guess what the new IRT model will look like, how can any early representation produce results that will align with the final model?

We didn’t guess at the design of the model. Just the opposite. NAS provided a 130-page roadmap that describes in detail the IRT model.  With counsel from IRT experts, we followed that roadmap to produce our preview.

Moreover, we have a strong track record helping the industry to anticipate and interpret the CSA scoring methodology. Vigillo released the industry’s first CSA management platform in 2008, two full years before the official launch by FMCSA.

We have helped support changes to the CSA program, culminating in the FAST ACT and the IRT recommendation. In its early years, I provided data to illustrate CSA deficiencies to Congress as CSA criticism grew after its launch. I collaborated with the American Trucking Associations and others in the development of the language that was inserted into the FAST Act which ultimately launched CSA Reform.  At the request of the ATA, I gave the industry comments at the first meeting of the NAS and accompanied the ATA Chairman in the delivery of those comments. Over the next 18 months, I attended every public meeting held by NAS. Before the draft report was published publicly, I was one of the primary reviewers. Ultimately, I led the efforts at SambaSafety to retain the services of IRT experts including one of the NAS Panelists.

Back in 2008, Vigillo gave thousands of motor carriers a preview of what CSA was, how it worked, and how it would impact their safety programs. Advance knowledge of these kinds of significant changes at FMCSA is essential. That’s why we’re doing it again with IRT.

Question: Why worry about IRT now when it’s not near implementation?

The NAS report provided for a two-year timeline following the initial 18-month study.  That would call for implementation in June of 2019. FMCSA has stated publicly that its target is September 2019, not significantly behind.

Our best information says that we’ll see careful rollout and solicitation of industry feedback over the following six months.  Full implementation is expected in Q2 2020. The reforms are moving along pretty much as outlined in the FAST Act.

IRT produces very different results for many carriers.  IRT uses a methodology that is easy to interpret but nearly impossible for a non-statistician to calculate. Many of our customers want a preview of their IRT score to be included in the discussion, right from the start. It’s imperative for them to know how their safety culture will be assessed and it allows them to provide informed feedback to the FMCSA during the solicitation period.

At SambaSafety, we live the mission of safety, and we innovate. We invest in R&D and are continually building new products and services to serve that mission.  We think our job is to get there first. We have a proven track record of doing that time and time again. Our customers benefit from our ability to anticipate change, understand formerly hidden risks, participate in the discussion of the new model and prepare for a two-year look back.

It’s a choice to be proactive and get involved early to anticipate significant industry changes like IRT.  Hundreds of carriers are making that choice.  We think that’s a good thing for safety.

Question: Does IRT really improve upon legacy CSA?

It does, no question about it.

Let’s look at three of the defects of CSA as it exists today:

  1. Safety Event Groups – Spiking Scores
  2. Disparate State Enforcement – Geography sways Scores
  3. A relatively small number of scored carriers – Less than 20% of carriers have sufficient data for any CSA Score

First, Safety Event Groups, as constructed today, group carriers in an attempt to compare similar carriers.  It has never worked well and has some problematic side effects.  Five of the seven Groups are based on inspection count.  A carrier can move from one group to another by virtue of just one inspection, even a clean inspection, and scores spike dramatically as a carrier moves up the ladder in Safety Event Groups. Related to this, a Carrier’s CSA Measure produces very different CSA Percentile Scores depending on what group they are in.  As a result, Carrier A with a BASIC Score of say 50% is not comparable to Carrier B with 50% in a different Group.

IRT does away with Safety Event Groups in favor of a Risk Exposure Index.  IRT utilizes this Exposure Index which is created by blending Power Unit Count, Driver Count, VMT and Inspection Count to normalize carriers for comparison purposes. No groups to leap between and all carriers scores are comparable to each other.

Second, Law Enforcement Disparity has always plagued CSA.  CMV Enforcement on the New Jersey Turnpike is a very different animal than enforcement at the Texas border.  Our country is vast, diverse, and immensely challenging from an enforcement perspective. Enforcement knows their unique challenges and focuses on the specifics that they believe enhances safety in their own back yards. CSA does not formally recognize these challenges and punishes carriers who operate in targeted enforcement zones vs. carriers who do business elsewhere.

IRT is not about the frequency of violations.  IRT looks at patterns of violations; it’s not just counting them.  As a result, the wild swings we see in CSA Scores due to disparate enforcement are largely smoothed out.

Third, CSA today cannot score a carrier with insufficient inspection data.  As a result, only about 100,000 carriers receive any CSA Score at all. IRT has a lower threshold for what constitutes sufficient data for it to evaluate and provides scores for almost 200,000 carriers.

There are other improvements that IRT brings in terms of a more scientific foundation, ability to adapt to changing patterns in the data and it is set up well for additional data to be added in the future.  We can hardly imagine the magnitude of varying types and amounts of data that trucking operations will generate in the future. IRT is uniquely capable of incorporating new types of data to make continuous improvements to our understanding of what constitutes an excellent Safety Culture in the Trucking Industry.

Steve Bryan