������ý

[Stay on top of transportation news: .]

The is readying changes to the after a long, fierce debate over its ability to accurately identify unsafe carriers. Along the way, even the Government Accountability Office and Congress got involved.

The changes include simplification of the severity weights assigned to violations, reorganization of measurement categories, increased focus on frequent violations and adjustments to the program’s “intervention thresholds.” Critics of the system hope these changes improve CSA or — in their view — fix what is wrong with it: namely, its capacity to identify carriers likely to have crashes and, moreover, absolve those that are not.

That might be “Mission Impossible.”

According to GAO, many fleets that CSA has historically labeled as unsafe haven’t gone on to have crashes. GAO said most regulations used to calculate CSA SMS scores aren’t violated often enough to strongly associate them with crash risk. This is likely because most of the data in the system consists of vehicle violations captured during roadside inspections. These violations don’t have a strong relationship to crash risk, especially compared with driver violations. Also, crashes are rare, so it’s difficult to tie crashes to causal factors. Causal relationships aren’t necessarily nonexistent — just difficult to prove.

Image

�������dzٳ���

So, Congress asked the National Academies of Sciences to determine the degree to which CSA scores “predict or are correlated with future crash risk.” The use of “correlated” was deliberate; FMCSA defends its scoring system by pointing out one such correlation: The group of carriers with poor average scores has a higher average crash rate than the group of carriers with better scores. But FMCSA doesn’t contend that it can predict which carriers will be involved in crashes. Poor scores are defined as those that exceed the enforcement intervention threshold in at least one category.

Congress also directed NAS “to determine if CSA scores provide comparable precision and confidence for the relative crash risk of individual carriers” (emphasis added) and to review “the tie between crash risk and specific regulatory violations, with respect to accurately identifying and predicting future crash risk for motor carriers.”

FMCSA’s analysis reminds us of the difference between correlation and causation. Correlations can have predictive value but differ from causation. For example, the group of drivers who refuse to wear seat belts may have a higher average crash rate than the group of drivers who wear them, but seat belt use doesn’t prevent crashes. Instead, the correlation likely points to the drivers’ propensity to take risks and their attitudes toward compliance. But not every driver in the first group drives unsafely or is likely to have a crash.

Left with nothing else, FMCSA had to rely on its group-level analysis. Predicting crashes using existing data is almost impossible. And many fleets with poor scores won’t have crashes — either because of luck or because the violations they committed don’t have strong causal relationships to crash risk. FMCSA even acknowledges that its approach focuses on crash prevention, not prediction.

FMCSA’s system is similar to the one used to set insurance premiums. Groups of drivers (e.g., 18-year-olds) pay higher rates because they’re more prone to being involved in crashes. Yet, not every 18-year-old will be in a crash, and some are safer than others. But generally, 18-year-olds have more crashes, and the safer ones will feel the characterization is unfair.

Given the correlation between high CSA scores and higher crash rates at the group level, FMCSA’s use of the scores to prioritize enforcement resources is reasonable and a much better approach than the previous methods of relying on a carrier’s safety rating or the agency’s now-retired Selective Compliance Enforcement system.

Using CSA and SMS, FMCSA can flag fleets with high scores for further investigation. But use of the scores alone to predict an individual fleet’s propensity to be involved in a crash will continue to yield unreliable results.

Fleetworthy’s Rob Abbott previously served as American Trucking Associations’ VP of safety policy during the CSA rollout.

Want more news? Listen to today's daily briefing below��or go here for more info: