Realistic models of human behavior for simulation of autonomous driving at scale

Simulation is at the core of development at all self-driving companies.

Simulation allows to develop and validate new features in thousands of scenarios, far beyond what is possible on the real road. This massively accelerates development of the autonomous driving software, saves costs, and increases safety. Realistic models of the driving dynamics of the car, sensors, and static world are well developed and broadly available on the market. A crucial element hard to replicate is human behavior.

cogniBIT develops algorithms simulating the behavior of human road users (drivers, cyclists, pedestrians, etc.) with all their characteristics (e.g., limited visual field, lack of attention, diverse emotions).

We enable leading autonomous driving companies to shift the development and validation from the real world to the simulation environment, thereby reducing costs and time to market by a factor of 1000, while increasing safety.

cogniBIT’s simulation models are applied in a variety of use cases, such as, for example, reinforcement learning of motion planning algorithms. Use of highly realistic traffic agents in this use case is crucial to prevent overfitting of machine-learning-based algorithms.

Realism and validity of algorithms describing human behavior.

Limitations of conventional ML solutions become evident by shifting the focus from highly logical problems (such as playing chess) or problems restricted in dimensionality (such as recognizing objects/text) to less clearly defined puzzles.

In our opinion, conventional AI approaches alone cannot solve high-dimensional challenges such as modeling unpredictable human behavior in traffic. Interdisciplinary solutions combining engineering with neuroscience, cybernetics, and data science, are the only way to sufficiently capture human factors in traffic.

Our unique technology combines machine learning with a novel neurocognitive-based approach, which reproduces the human sensorimotor processing chain from perception to cognition and action. Elements of the chain is represented by a scientifically-proven closed-loop model. This approach allows simulation of human characteristics, such as a limited field of view, lack of attention, or distractions. The elements of the chain (sub-models) and the resulting closed-loop model of the human behavior are validated against real-world data, such as traffic data or physiological data.

This approach ensures the most realistic behavior of traffic agents in every scenario including edge cases. The advantages of cogniBIT models become even more clear in complex scenes such as construction sites on the highway or intersections in the city with multiple cars and pedestrians.

One of the fascinating features of our models is the ability to anticipate how the scene will evolve, e.g., our agents are capable to do human-like prediction of emergency breaking of the lead vehicle reacting to a pedestrian crossing the road resulting in breaking in advance. This shows a high level of realism of our models.

Our models ensure very high realism, improved computational performance, and are most suited to reliably cover edge cases. Our road user models are platform-independent and can be integrated into any simulation platform, supporting common industry standards.

Scalability of cogniBIT’s approach

cogniBIT’s models are closed-loop models, meaning, that unlike predefined datasets they are able to react to changing environment (e.g., changing trajectory of the autonomous vehicle under test). Another advantage of our models is the transparency with regard to the cause effect chain, which enable the user to understand the reasons for agent behaviours.

Our models are also fully deterministic in the sense of producing reproducible results. At the same time, they contain stochastic elements that automatically generate variations of a scene depending on the random seed (demo video). Another source of variation is given by the different agent types, allowing easy tests of different traffic styles. Using the variation of these parameters thousands of scenarios, enabling automated simulation at scale, can be created fast and efficiently.


We also plan to use our technology for predicting the behavior of road users (human intent prediction) as a major component of the software for autonomous vehicles control. Our vision is to enable robust AI-augmented human-machine interaction.

by Denis Azarov