Safety Assurance in CCAM Systems: The Gap Between Innovation and Validation

The Validation Challenge

Carlos Lujan, Head of Homologation CAV at Applus+ IDIADA, a designated Technical Service for vehicle type approval, identifies a fundamental challenge facing the industry: "SAE Levels 2 and beyond require a different validation and assessment approach, and the creation of a SAF covering those levels would be beneficial."

This isn't just an incremental problem; it's a paradigm shift. The methodologies that worked for conventional vehicles and even early ADAS features are proving inadequate for today's AI-driven automated systems.

The AI Validation Gap

According to Lujan: "There is not yet a harmonized methodology adopted by the regulatory bodies (specifically, UN ECE)."

This creates a paradox: The automotive industry is deploying increasingly sophisticated AI-powered systems, yet the frameworks to validate their safety remain undefined at the regulatory level.

Three Critical Challenges Facing Technical Services Today

1. Low Level of Methodological Detail

The current NATM (New Assessment and Testing Methodology for Automated Driving) framework provides general guidance, but lacks the specificity needed for consistent application:

"The main limitation we are confronting is the low level of specific detail of the existing methodologies, together with the general lack of experience in the regulatory bodies with actual cases, which leads to difficulties with the interpretation of particular cases."

Impact: Each validation case requires individual interpretation, leading to variability in decision-making and reduced repeatability across different systems.

2. Time-Intensive Preparatory Phases

Technical services currently spend significant resources defining validation criteria case-by-case:

"This phase is critical nowadays, as this is where the technical service shall define the criteria, which may be particular to each different case (depending on the system, the ODD and other factors)."

Impact: Extended timelines, increased costs, and potential inconsistencies across evaluations.

3. Multi-Pillar Integration Complexity

While NATM defines a multi-pillar approach integrating simulation, track testing, and real-world validation, the practical implementation remains subjective:

"Only general guidance is provided, and the final decision is always subject to the interpretation of the technical service."

What the Industry Needs: A Vision for Comprehensive Safety Assurance

When asked about expectations for an ideal Safety Assurance Framework (SAF), the priorities are clear:

Clear Workflows with Integrated Tools

"I expect a clear workflow with integrated tools in order to avoid variability in the decision-making process."

Repeatability and Resource Efficiency

"Repeatability and comparability of the results obtained among systems under evaluation, with a drastic reduction on time and resources needed."

Detailed Guidance Over General Principles

"A complete guideline would be much more beneficial, considering the wide portfolio of activities carried out by IDIADA."

The need isn't for incremental improvement, it's for comprehensive frameworks that eliminate ambiguity and enable consistent, efficient validation across diverse CCAM technologies.

The Standardization Imperative

For technical services operating in a global automotive market, standardization carries exceptional weight. In Lujan's words: "This is a key element for a Technical Service, if not the most important."

With CCAM technologies deploying at different timelines across regions, harmonized validation frameworks become essential for enabling global type approval processes and ensuring consistent safety standards worldwide.

Emerging Trends Shaping CCAM Validation

The industry is witnessing several converging trends:

"Confirmation of the trend already started some time ago with the spread of the multi-pillar approach and increasing relevance of virtual tools. Other current trends are the need for a life-cycle assessment and the identification of the challenges of the integration of AI in the CCAM systems."

These four key trends define the evolution of CCAM validation:

1. Expansion of multi-pillar approaches beyond current NATM guidelines
2. Increasing relevance of virtual tools alongside physical testing
3. Life-cycle assessment requirements covering development through decommissioning
4. Recognition of AI integration challenges as a critical validation frontier

The Path Forward

The stakeholder perspective reveals an urgent need for action:

• Develop harmonized validation methodologies at the regulatory level (UN ECE)
• Create detailed, prescriptive safety assurance frameworks that reduce interpretative variability
• Integrate validation tools into cohesive workflows that span simulation, track testing, and real-world validation
• Establish global standardization that accommodates varied deployment timelines while ensuring consistent safety outcomes
• Build regulatory experience through actual case studies and shared learning across technical services

Final Thoughts…

As the automotive industry races toward higher levels of automation, the validation infrastructure must evolve just as rapidly. The insights from technical service stakeholders make clear that incremental adjustments to existing frameworks won't suffice.

The question isn't whether we can validate AI-driven CCAM systems’ safety, it's whether we'll develop the comprehensive frameworks needed to do so efficiently, consistently, and at scale.

For technical services, OEMs, regulators, and technology providers alike, the message is clear: The time for comprehensive safety assurance frameworks is CERTAINly now.

About this article…

This analysis draws from stakeholder interviews conducted as part of the CERTAIN project | CCAM (Resilient and Continuous Safety Assurance Methodology for CCAM and its HMI Components) , a European Union initiative focused on developing harmonized safety assurance frameworks for Connected, Cooperative, and Automated Mobility (CCAM) systems.