The vehicle of the future will drive autonomously - all manufacturers are currently working towards this goal. One of the biggest challenges is to test an autonomous vehicle. The new driving functions constantly increase the complexity of vehicle software. And the more responsibility the vehicle takes away from the driver, the more it must be ensured that no errors occur during operation.

According to current studies, more than 100 million test kilometers have to be driven before an autonomous vehicle can be allowed on the market. This corresponds to a testing period of about 2.3 years - with classical methods this is no longer economically feasible.

In addition, there are the changes in the ecosystem of the automotive industry: testing is increasingly shifting towards system suppliers. Long before their systems can be integrated into the overall vehicle, they must ensure that the systems function flawlessly in the end product. Instead of testing on the finished vehicle, as much functionality as possible must be tested in earlier development phases. Virtualization methods are used to test as many different scenarios as possible and achieve a high level of test coverage.

One of the currently established methods is software-in-the-loop, in which the fully implemented software is tested on the test bench. The earliest stage of validation at present is "model-in-the-loop": functional models of the software are created and then tested.

Testing in the concept phase

These processes often focus on realistic models and include the validation of models, code, single or multiple ECUs or entire virtual vehicles. But in the earliest phase of development - the concept phase - there is hardly any validation. This is precisely where one of the keys to increasing efficiency in development lies: testing a pure concept or an operating strategy can improve product quality and accelerate development.

Current tests using model-in-the-loop or software-in-the-loop only take place after the specifications have been developed and subsequently implemented as a complex functional model or ECU software. Up to this stage, considerable development effort has already gone into the software or the software model. This development effort is based on the assumption that the underlying concept is correct.

Errors in the concept are identified at the earliest in the first safety loop Model-in-the-Loop - i.e. after the implementation of the model. However, if these tests reveal that an assumption contained in the concept leads to poor or even erroneous behavior in a particular scenario, the concept, specification, model, or software must be adapted and re-tested.

Especially in the field of autonomous driving, however, concepts become more and more complex and contain hundreds of assumptions and dependencies that are difficult for the specifier to oversee. A validation loop in the concept phase can create clarity and reduce effort in later development phases.

Finding the first bugs even before implementation

So why not start one step earlier with the concept itself? Concept-in-the-Loop - a high-level test procedure that is positioned before Model-in-the-Loop - is a solution to reduce this effort. The goal: to detect concept errors as early as possible and with minimal effort before further development effort is put into the subsequent development steps.

To achieve this, the complexity of the function is reduced to such an extent that only the parts relevant to the concept, such as behavior strategies, are simulated. Code standards, manually created scenarios or specific ECU functions are deliberately not taken into account in order to focus all attention on the concept.

This approach takes the term "frontloading" one step further: since Concept-in-the-Loop (CiL) can be controlled and evaluated in the same way as the existing Model-in-the-Loop (MiL), Software-in-the-Loop (SiL) and Hardware-in-the-Loop (HiL) processes (for example by using common test automation tools, big data platforms and data analytics processes), integration into the existing backup tool chain is ensured. Thus, the basic models can be gradually extended with finer models (e.g. for sensor or environment simulation) as the process progresses.

Find the right concept faster with Concept-in-the-Loop

Especially with specifying autonomous driving functions, very early validation can be helpful: Due to the complexity of the functions, the "right" concept is often not visible at a glance. One example might be a seemingly simple task, such as pulling into a motorway entrance:

If the autonomous vehicle is located directly next to a vehicle driving on the motorway, it has two options: Accelerate and pull in front of the other vehicle - possibly exceeding an existing speed limit or crossing the end of the entry lane. Or braking and getting into line behind the other road user - and possibly losing too much speed in the process or endangering or slowing down the following traffic.

So which operating strategy is the right one? The answer is not necessarily obvious. When making a decision, a human driver unconsciously considers a large number of other parameters such as his or her own speed, traffic density and flow, the length of the entry lane or the behaviour of other road users, to name a few. In order to determine a functional concept, the specification engineer must consider a multitude of possible scenarios.

Whether this specification leads to the desired behaviour in reality can only be determined during actual testing. Model-in-the-loop testing is currently available as the first validation loop. The function is specified initially, then implemented as a function model and only then tested in a simulation environment using various manually created scenarios. A considerable development effort, without which no conclusions about the effectiveness of the operational strategy are possible.

Fine tuning in the concept phase by means of simulation

The Concept-in-the-Loop approach provides much earlier and easier clarity about the effectiveness of the concept: In the example of the motorway entry, the concept can be modelled within a very short time. Instead of modeling the complete function (taking into account a large number of parameters such as bus signals or ECU-internal variables), only a few parameters are required to represent a simple state machine.

Implementation in the test environment is also possible with considerably less effort for concept tests: Model-in-the-loop tests typically use complex 3D simulation environments that allow realistic simulation of the entire vehicle. The entire bus environment and thousands of parameters relating to driving physics - from the damping level to rolling resistance - are taken into account. These are important test parameters in later development stages. However, in order to test the concept of threading mentioned in the example, these data are unimportant and do not have to be parameterized first.

A faster alternative is offered by simple traffic flow simulators such as SUMO or comparable software. These require significantly less input when designing test scenarios and thus allow faster testing of the concept with much less detail. Thousands of random scenarios can be generated and tested with relatively little effort. The results of these tests can then be evaluated using big data applications, for example.

With the help of this validation solution, the specifier quickly arrives at an assessment of how the vehicle should behave depending on which parameters. Even in the concept phase, fine-tuning of the functions is possible, which would currently only be possible in much later development stages. Only when the concept has been fundamentally tested and found to be effective the complex model will be implemented and re-tested in the next safety level Model-in-the-Loop.

More efficiency

The potential increases in efficiency with this approach are enormous: For a concept-in-the-loop test, only the implementation of a rough concept as a simple state machine is necessary. For model-in-the-loop tests, on the other hand, an elaborated functional model of the ECU software must already be developed. In our estimation, the amount of work involved in some examples is about a factor of one hundred.

This early validation enables specification errors to be found very early, and the first validation takes place during the specification phase. The subsequent development effort is reduced because the effectiveness of the concept can already be proven during the specification phase and fewer changes to the software are necessary in later phases.

Possible applications of such a virtual concept confirmation include the validation of operating strategies for autonomous and electrified vehicles, concepts in Car2Car communication or cloud-based concepts (e.g. traffic warning, congestion avoidance).

Integration into the tool chains

For a concept-in-the-loop approach to work, the test procedure should be seamlessly integrated into the tool chains of the other validation steps: From test automation through automated test evaluation to data analytics procedures, many different procedures are already in use today to control, monitor and evaluate the validation process. Due to the increasing number of tests and test procedures, these processes are becoming more and more important and allow a structured, development-accompanying validation.

Concept-in-the-Loop can be integrated into existing tool chains and thus controlled and evaluated in the same way as later test procedures. The advantage is that the models used can be extended step by step, thus allowing more and more accurate validation. This reduces double efforts and enables prompt validation as soon as new functions or models become available.