Figure 1

Image Sensors with high dynamic range, together with powerful processors and sophisticated algorithms, are enabling a new generation of vision-based systems that can detect and distinguish objects in a wide range of lighting conditions.

“The introduction of in-car video cameras is opening up new application areas for driver assistance systems,” said Bernd-Josef Schäfer, vice-president of the driver assistance systems business unit at Robert Bosch GmbH. “As each new functional enhancement appears, the car is gradually learning to ‘see,’” Schäfer said.

Vision systems are helping automakers differentiate their vehicles on the basis of safety. The rear vision camera system on the Buick Enclave, for example, has a caution symbol that changes in size and color to draw the driver's eye to the closest detected object (Figure 1).

Systems are looking inward as well as outward. Saab, for example, is developing a driver attention warning system that uses infrared cameras on the driver's door and the center console to record and analyze eye movement. The system sounds audible and visual alerts if the driver's eyes close for longer than a normal blink. If the driver's condition persists, audible warnings become more urgent and the driver's seat vibrates.

Volvo in January launched a collision avoidance package that includes lane-departure warning, adaptive cruise control, collision warning with auto brake, distance alert and driver alert. In the driver- alert application, a camera continuously measures the distance between the car and road lane markings. Sensors register the car's movement while a control unit stores the information and calculates whether or not the driver risks losing control of the vehicle. If the risk is high, the driver is alerted via an audible signal and a text message. The lane-departure warning system employs a camera to monitor the car's position between the road markings.

Figure 2

Using image sensors from Sensata Technologies B.V., Bosch developed the night-vision system deployed on Mercedes S and CL class vehicles (Figure 2). Infrared high-beam headlights illuminate an area of more than 150 meters in front of the vehicle. The infrared image is picked up by a video camera, where electronics convert the signals into an image on the central display that enables drivers to identify dangerous situations more quickly and gives them more time to react. The system, which includes a 400 MHz PowerPC and a field-programmable gate array (FPGA), can also recognize and display pre-defined road signs.

Bosch is readying Night Vision Plus for 2008 production. The Plus version adds color, and is said to be capable of distinguishing between pedestrians who are standing or moving. Bosch is also developing video-based driver-assistance systems that use sensor data fusion to process spatial information. Its technology can merge signals from a video camera and a 77 GHz radar sensor, or those from two video cameras. The system can recognize and analyze critical situations, predefined or otherwise.

Bosch is using sensor data fusion technology to develop a Predictive Emergency Brake application that will automatically apply the brakes and minimize the severity of the consequence if it recognizes that the driver is failing to react to a pending collision.

Delphi is using sensor fusion in a collision mitigation system that combines cameras and long-range radar for applications including lane- departure warning, collision warning with auto brake, driver alert control, and adaptive cruise control.

Omron Automotive Electronics has fitted a vehicle with a sensor fusion demonstration that combines Omron's lidar sensor and high-dynamic range camera to facilitate full-speed range adaptive cruise control, lane- departure warning, and heading control systems. The heading control system, integrated with electric power steering, provides a gentle tug on the wheel in the proper direction when the vehicle drifts over a lane marker.

Hella KGaA Hueck & Company markets a rear-view CMOS color camera with a 130-degree aperture angle, and plans to launch an ultrasonic park-assist application in Europe this year, and is readying other camera-based systems for launch in 2009.

Hella's active driver-assistance systems are based on camera and ultrasonic technology as well as lidar and 24 GHz radar. The Chrysler 300 offers Hella's lidar-based adaptive cruise control technology while Audi's Q7 SUV contains Hella's radar-based lane-change assistant system, also known as the Audi Side Assist. The lane-change assistant relies on two 24 GHz radar sensors integrated into the vehicle's bumper.

Siemens VDO, now part of Continental, has developed an Intelligent Passive and Active Safety (IPAS) system that networks driver- assistance systems with active and passive safety systems. The IPAS platform includes a “LiCam” sensor, which combines a CMOS camera and a lidar sensor to provide environmental and traffic information. The lidar sensor delivers data on vehicles in front, while the CMOS camera monitors lane markings in varying light conditions and provides information for traffic sign recognition and high beam assist. Fusing sensor data from lidar and CMOS censors results in optimum lane attribution and object recognition, for full-speed range adaptive cruise control functionality, according to Dean McConnell, director, Occupant Safety & Driver Assistance Systems, at Continental, N.A.

Significant vision technology development is occurring at the chip level. Texas Instruments has numerous design wins for vision applications, according to automotive and marketing business manager Brooke Williams. Digital-signal- processor-based devices employing TI's DaVinci technology include the TMS320DM642 series and the newer TMS320DM643x. DM642 devices are deployed in rear-view and birds-eye view applications that use multiple cameras, as well as in night-vision applications with data feeds from infra-red sensors.