APTIV - SMARTER SAFETY THROUGH INTERIOR AND EXTERIOR SENSOR FUSION
In the realm of active safety, every radar, every camera, every lidar — every sensor in a vehicle — doesn’t just add to the safety capabilities; it multiplies them. With more inputs and data points to work with, the right software can combine the data to make decisions in ways that go far beyond what is traditionally thought of in sensor fusion.
Usually, sensor fusion refers to the combination of data inputs from two or more external sensors to get a better read on what obstacles might be around a vehicle. We might combine the inputs from two radars, for example, to classify an object that appears at the edge of their respective fields of view. Or we might combine the input from a camera and a radar to take advantage of the strengths of both modalities and more accurately identify what’s near the vehicle.
Now we’re seeing the emergence of interior sensors. These often take the form of a camera inside the cabin, and they can make many safety-related determinations, such as whether drivers have their eyes on the road or whether passengers are wearing their seatbelts.
Things get even more interesting when we combine the information from our sensors about what is going on outside the vehicle with the information we have about the driver’s state inside the vehicle, using interior and exterior sensor fusion. If we know drivers are drowsy or distracted, we could proactively warn them about a dangerous situation well before an intervention is required.
Let’s say a car aggressively cuts in front of your vehicle. As long as you’re actively observing traffic and you see the other car merging into your lane early enough, your vehicle probably doesn’t have to intervene to avoid a collision.
But if you’re distracted — changing the radio station, perhaps — you might not see what’s happening until it’s too late.
A more complete picture
With interior sensing, we can see that your eyes are looking down at the radio instead of out at the road, while the exterior radars see the car cutting into your lane. We can immediately trigger a warning that something is going on outside the vehicle that requires your attention, and even direct you with visual and audio cues to where it is taking place.
By fusing interior and exterior sensors, much more proactive safety solutions are now possible. Consider a scenario where you are stopped at an intersection waiting to make a right-hand turn. You’re looking to your left, watching for a gap in the traffic so you can quickly make the right turn. Meanwhile, a pedestrian starts to cross in front of you from the right.
The interior sensor sees where you are looking, but one of the exterior radars sees the pedestrian. The system knows you haven’t looked over to see the pedestrian, so it warns you that the person is there — before it becomes an emergency.
This isn’t just speculation. Aptiv demonstrated these technologies at the recent CES 2020 show in Las Vegas, as part of our broader demonstrations on the benefits of interior sensing. And we will continue to use our whole-car perspective to create even more safety innovations that leverage all of the data generated by the car.
Author - A.J Lasley, Global Director of AS&UX Advanced Engineering
