Key applications for image sensing

As early as 1948, George Orwell foretold of a world where ‘Big Brother' would be watching everyone, in his classic novel ‘1984’. At the time, this was largely dismissed as science fiction, although many would argue it is now coming true - with some justification. In the fast-moving world of image sensing, statistics are rarely correct for long.

However, current reports suggest that there are upwards of 25 million CCTV cameras deployed globally, with more added daily. They are particularly prevalent in the UK, with some four million cameras in operation, and 500,000 of these located in London. In fact, it is estimated that the average Londoner may be captured on camera up to 300 times each day.

Security is just one application sector for modern image sensor technology though. Within the industrial arena, human inspectors are being replaced by all-seeing camera systems - that don't make mistakes, fall asleep, or need a break. As our cars get smarter, moving to the point that they will eventually drive themselves, up to a dozen cameras may be deployed to check for hazards on the road ahead, help us with parking and even monitor if we are in a fit state to drive.

Security applications

CCTV cameras are deployed both indoors and outdoors. They can be found everywhere from large commercial buildings (such as sports stadiums, airports, and hospitals) through to small-scale installations or residential homes.

Indoor installations tend to be less demanding from a technology perspective, as the light levels can be controlled (thereby allowing the cameras to detect the scene easily). However, outdoors light levels and operational conditions can be more of a challenge, not least as the undesirable behaviour these cameras are looking to detect won’t always take place on bright, sunny days.

One of the key considerations in selecting a sensor for security applications is the field of view (FoV), which is the angle over which the sensor can detect incoming light. This is related to the lens used for the camera, which, in turn, is influenced by the sensor format and size. Of the three formats commonly used in CCTV cameras, a 2/3” sensor is generally intended for long-range capture in very low light situations, while a 1/3” sensor offers faster frame rates. Sitting in the middle of these is the 1/2” sensor format, which provides a good compromise and is popular in many applications.

Above: factories are increasingly moving towards a fully automated approach, often known as automated optical inspection

There are two types of fundamental technology for image sensors - charge-coupled device (CCD) and complementary metal oxide semiconductor (CMOS). In general, CMOS devices offer high resolution - something that is of importance for identifying individuals through their facial features. They also offer reasonably strong dynamic range and fast frame rates, as well as supporting low power operation (which is useful in remotely situated battery powered installations). Limitations of CMOS, like relatively high noise and lower sensitivity, can be addressed through signal processing and, as CMOS devices now offer global shutter to avoid the blurring of moving subjects, they are rapidly becoming the preferred choice for security applications (supplanting CCDs).

One CMOS image sensor that is specifically aimed at security applications is the AR0521 5.1MPixel 1/2.5” format device from ON Semiconductor. With an active pixel array of 2,592 (H) x 1,944 (V), it captures images in either linear or high dynamic range mode. It is designed for superior low light performance, giving clear and sharp images. The ability to capture both continuous video and single frames makes it highly effective for surveillance tasks.

Industrial inspection

For many years, humans have performed end-of-line quality checks in factories. While this system has generally worked well, humans are occasionally prone to errors and can be distracted, as well as representing an ongoing (and generally increasing) cost over time. As a result, factories are increasingly moving towards a fully automated approach, often known as automated optical inspection (AOI) – which overcomes these drawbacks.

Camera systems can examine products, or even sub-assemblies, after (or during different phases of) the production process. Often this is achieved by the use of reference images of a known ‘good’ item, against which all others are compared by sophisticated software. AOI systems can be programmed to detect many issues - including incorrect colouring, poor assembly, and the presence of erroneous elements.

Above: The KAF-09001 9.1MP CCD from ON Semiconductor

As the camera and AOI station will be placed indoors, and lighting may be controlled to illuminate the product properly, the low light issues that exist in security applications are not a concern here. However, to inspect a product thoroughly, good colour rendition is a key requirement to ensure that there is no ‘drift’ in an upstream process that has affected (for example) the colour of paint sprayed onto the product.

When inspecting intricate or detailed products, such as identifying the component values on modern electronic printed circuit boards, the resolving power of the camera is of prime importance. Small passive components can have dimensions in the sub-mm range, so reading text accurately requires a high-resolution sensor with low noise performance. This would typically be what is termed a ‘full frame’ sensor, which has the same dimensions as 35mm film (approximately 36mmx24mm), as the larger pixel sizes in this format provide better detail.

ON Semiconductor’s KAF-09001 sensor provides elevated imaging performance for demanding applications - including intricate factory inspection work, medical diagnostics, and scientific research. The large sensor size (36.3mmx36.3mm) houses a 9.1MPixel imaging array that comprises 12µm x 12µm pixels. This device delivers high degrees of image quality and uniformity necessary for advanced AOI applications.

Automotive image sensing

Image sensing is not restricted to fixed installations such as security and factory inspection. Quite the opposite is true - with automotive image sensing being one of the largest growth sectors at the moment. As vehicles continue to move towards semi-autonomous and eventually fully-autonomous operation, their ability to sense and understand the environment around them will be critical - replacing the eyes of the human driver.

One of the earliest uses of image sensing technology in an automotive context was the rear-view cameras that allow drivers to see the blind spot behind them. This has been made mandatory for new vehicles in some jurisdictions, following unfortunate accidents where pets and children were crushed while a driver reversed. Implementations are increasing though, with forward-looking cameras providing a variety of functions within advanced driver assistance systems (ADAS) - including the ability to detect objects within the path of the vehicle and to determine if they are static or moving, inanimate or living. Forward-looking cameras are also able to read speed limit signs, and either advise the driver or even restrict the vehicle speed directly.

Given the importance of the images being detected by the vehicle’s cameras, of which there may now be ten or more in some vehicles, the ability to process the image quickly and obtain meaningful data in every possible weather/lighting condition is paramount. Ultra-high resolution is not always necessary, as often just detecting that an object is within the vehicle’s trajectory is sufficient. In many cases, high resolution may slow the system down (which in itself can be dangerous) or require a hugely powerful (and thus expensive) processor.

Most image sensors used in automotive applications will be required to have AEC-Q100 qualification, to show that they are suited to the harsh environment, and can cope with extreme temperatures, heavy vibration, and EMI exposure. Additionally, those that affect the braking and/or steering functions will need to be capable of working within systems that are designed to meet the corresponding automotive safety integration levels (ASIL).

One of the newer areas within automotive image sensing is to provide cameras that are pointed directly at the drivers themselves. Through monitoring the eyes of the driver, the camera can decide if they are looking at the road ahead, as well as detecting the early stages of fatigue.

Under such circumstances, the ADAS can subsequently suggest a rest stop, or even apply the brakes. Designed explicitly for in-cabin sensing of this kind, ON Semiconductor’s AR0135AT incorporates a global shutter to address the flickering light that can come from the LED-based headlamps of other vehicles or street lighting. The 1/3” format, 1.2MPixel sensor incorporates a new pixel design that allows for dramatically improved low light performance - thereby producing the clear, low noise images needed for eye-tracking functionality.