Driving at night, in fog, in shadow or glaring light can be dangerous because drivers can’t always ‘see’ what is on the road. Thermal cameras, also called ‘night vision cameras’, can detect anything that emits heat. They may prove to be the perfect driving companion technology for Advanced Driver Assistance Systems (ADAS) and autonomous vehicles.
Thermal cameras are also useful in smart cities, automotive repair and Covid-19 screening. Auto Futures talks with Chris Posch, Director of Engineering for Automotive Products, FLIR, a leader in the design and manufacture of thermal imaging infrared cameras.
What Can Thermal Cameras Detect?
The use of automotive thermal cameras has been growing, says Posch, The first thermal camera applications were in vehicles for animal detection in which drivers are shown an image with lines around the animal and then given a warning. Collisions with animals on the road are dangerous and frequent. In fact, the Insurance Institute for Highway Safety estimates that there are 1.3 million collisions with deer every year in the U.S.
Posch reports that thermal cameras are especially adept at detecting living objects especially pedestrians. Pedestrian detection is now included in many vehicle ratings. However, the testing conditions are not always real-world.
“FLIR is working with OEMs and regulators to make sure that the automatic emergency braking testing with pedestrians simulates real-world situations,” says Posch.
He reports that NHSTA (National Highway Safety Traffic Administration) found that 75% of all pedestrian deaths occurred after sunset. In fact, Governors Highway Safety Association (GHSA) estimates 6,590 pedestrians were killed on U.S. roads in 2019, the highest number in more than 30 years.
Recent real-world tests by AAA of pedestrian detection found that the cars travelling in night conditions at 25 mph(40 km/h) hit the soft pedestrian target 100% of the time. In daylight when driving 20 mph (32 km/h) the vehicles hit the stationary test dummy 60% of the time.
Thermal camera data fused with radar data provides much better pedestrian detection with automatic emergency braking results in testing.
FLIR contracted VSL Labs to test thermal camera data fused with radar at the American Center for Mobility (ACM) near Detroit to check the viability of thermal cameras to detect pedestrians. The radar detected the objects then the thermal camera along with the object database identified the pedestrian dummy in winter weather.
Test vehicles were programmed to automatically stop if a pedestrian was in the path of the vehicle. The test dummies were outfitted with special heat emitting technology. Vehicles were tested in three use cases.
1. The pedestrian standing in the middle of the lane.
2. The pedestrian crossing the lane.
3. The pedestrian crossing in front of the vehicle from an obscured place.
In all cases, the pedestrian was detected and the car stopped before reaching the dummy. FLIR plans on testing the system in summer weather.
Why Thermal Cameras Are Good for Autonomous Driving Systems
After the Uber autonomous vehicle hit and killed Elaine Herzberg while crossing with a bicycle on a roadway in Arizona in March 2018 at night, autonomous vehicle companies realized that they needed to deploy more sensors.
The Uber vehicle didn’t detect Herzberg as a human because of the bicycle and she was not in a crosswalk. After the incident, AV companies began contacting FLIR about thermal cameras.
Object detection and identification is very important to AVs, says Posch. FLIR has compiled a database of objects all over the world to help identify objects.
What Can Thermal Cameras See in Total Darkness?
While compiling the object data, Posch was in one of the vehicles that showed the images from the visual camera and the thermal camera. He drove around the corner and there were police officers that were dressed in black in shadow. The visual camera did not detect the officers. The thermal camera was able to detect them.
“Most people wear black and often at night,” says Posch, who notes that FLIR cameras detect in the 8-14 micron range that will still detect pedestrians wearing black at night as well as different costumes. The cameras can detect temperature differences within half of a degree and see four-times further than headlights.
FLIR, through Veoneer, will offer the first automotive system to include multiple thermal sensing cameras for narrow and wide fields-of-view for self-driving vehicles.
Thermal cameras can also help car mechanics diagnose engine and electrical problems. FLIR EST (Elevated Skin Temperature) Thermal Screening Solutions are being used to detect elevated body and skin temperatures for Covid-19 symptom screening.
Thermal cameras have become so useful that they have been installed in rugged mobile phones so that building contractors and maintenance workers can evaluate different temperatures in structures to discover electrical flaws and more.
Thermal cameras can also be used for smart cities to detect pedestrians in the road where cars are blinded, detect accidents and vehicles for smart traffic control. FLIR smart city cameras detect chemical, biological, explosive, and radiation hazards as well as emergency situations and crimes.
Previously, night vision was used in the military and was expensive, says Posch. Now the cost of thermal cameras has come down significantly, and the more that are made, the cost will be reduced even further.