Empowering Intelligent IoT with 'True Black Light': Aixin Zhimou® AI-ISP Breaks Through Night Imaging Limitations

Release time：2023-07-24

From the initial million-pixel mobile phones to today's tens of millions or even hundreds of millions of pixels in DSLR cameras and smartphones, the iterations of various imaging devices are driven by people's endless pursuit of perfect image quality. After all, in the face of a super camera with a pixel count of up to 576 million in the human eye, everything still falls short of being truly lifelike.

The development of cameras has always been intertwined with the support of ISP technology. For traditional ISP technology, night scenes and high dynamic range have always posed significant challenges. Especially in dynamic nighttime scenarios, issues like noise, motion blur, blurriness, highlights overexposure, and low-light information loss have not only failed to meet the increasingly high demands for image recording but also plagued industries such as video surveillance and traffic management.

However, continuous technological breakthroughs have the power to overturn our expectations. Just as people continue to find fault with the flaws of traditional ISPs, the rapid advancement of AI offers us more possibilities. The emergence of AI-ISP technology makes high-definition full-color imaging in extremely low-light environments possible. Cameras are transitioning from merely 'seeing' to gradually 'seeing clearly' and even 'understanding'.

The era of 'True Black Light' is officially upon us.

True Black Light: Empowering Intelligent IoT

True Black Light" is a term used to describe a camera's performance under low-light conditions. Different names, such as "dim light," "starlight," "super starlight," and "black light," are used to differentiate the night vision capabilities of cameras based on their low-light performance. While there isn't a unified standard for these terms, the focus remains on whether a camera can maintain full-color imaging under specific low-light brightness conditions. Generally, the term "black light" implies that a camera can operate in conditions with an illumination level of 0.01 lux or lower.

Traditionally, achieving black light effects typically involved using large sensor formats, low-resolution sensors (often with larger individual pixel sizes), widening aperture modules, or reducing video frame rates to increase the amount of incoming light. These methods aimed to capture more light and color information in low-light scenarios. However, with these methods, achieving 100% color saturation and preventing motion artifacts or ghosting effects in extremely dark scenes remained challenging. Moreover, increasing sensor formats or widening apertures could lead to higher costs, which didn't fully meet industry demands.

In other words, in extremely dark environments, it's often difficult to achieve the best of both worlds.

Nevertheless, there are solutions. The emergence of AXERA's self-developed Aixin Zhimou® AI-ISP technology has brought true black light full-color imaging into reality. Testing has shown that cameras enhanced with Aixin Zhimou® AI-ISP can achieve real-time color restoration at 25 to 30 frames per second in lighting conditions below 0.01 lux. This is achieved without the need for white light or infrared fill light. Additionally, these cameras can maintain color saturation of 100% or higher, and ensure that moving objects exhibit no significant motion blur, perspective distortion, or ghosting effects. AXERA also believes that only when six major elements are met, a camera can truly be referred to as having "true black light," providing users with improved video quality in extremely dark scenarios.

It is precisely due to the breakthrough of 'True Black Light' that smart cities and intelligent IoT have gained richer application scenarios. In extremely dark environments such as intersections, mountaintops, ponds, and farmlands, video brightness can be enhanced without sacrificing image quality. This allows cameras to capture well-structured images with accurate color reproduction in full color videos. Furthermore, this technology enables comprehensive scene coverage. In locations like ponds and farmlands where supplemental lighting is impractical, the same technology achieves more precise intelligent analysis.

Key Modules AI-Enabled: Achieving Optimal AI-ISP Performance with Limited Edge Computing Power

How Does Aixin Zhimou® AI-ISP Achieve 'True Black Light'? On July 19th, at the '2023 Yangtze River Delta Integrated Circuit Industry Innovation and Development Forum' held during the 5th World Semiconductor Conference, AXERA's ISP system expert, Zhang Xing, introduced the architectural innovations of Aixin Zhimou® AI-ISP.

ISP stands for Image Signal Processor, which is responsible for converting the signals captured by an image sensor into recognizable and distinguishable information for humans and machines. It plays a crucial role in image quality. Metaphorically, an ISP can be compared to the brain of the human visual system. It processes, enhances, and refines the images collected by the lens sensor, ultimately delivering an image quality that is as realistic or even better.

Traditional ISPs process the raw data output from CIS sensors by converting it into YUV data in a specific pipeline sequence to make it visible to the human eye. This processing is based on mathematical modeling and human understanding of images. On the other hand, AI-ISP, a relatively new technological concept in recent years, builds upon the traditional ISP pipeline by addressing problems that cannot be effectively solved through traditional mathematical modeling. It employs AI neural networks to simulate the human brain using richer parameters, establishing more complex models, and thereby surpassing the limitations of traditional ISPs to achieve superior image results.

Currently, Aixin Zhimou® AI-ISP enhances the four key modules in the ISP pipeline—HDR, 3DNR, RLTM, and Demosaic—through AI empowerment. By continuously improving the performance of these four modules through AI training, it breaks through the constraints of traditional ISP imaging.

The reason for choosing to AI-enable key modules is because edge computing power is often limited. Considering practical considerations such as power consumption and cost in real-world applications, applying AI processing to the entire pipeline is not feasible. It can be said that Aixin Zhimou® AI-ISP is more geared towards true productization. By AI-enabling crucial modules, it concentrates the limited computing power on the most critical and perceptible functions, achieving the optimal performance of the entire AI-ISP.

Therefore, in practical applications, Aixin Zhimou® AI-ISP can enhance performance through iterative visual models, enabling rapid upgrades in chip product image quality. This implies that AXERA can cover various scenarios and markets with a single chip, not only reducing the overall cost of AI applications but also significantly enhancing productivity.

Currently, AXERA is not only optimizing and iterating AI algorithms related to HDR, 3DNR, RLTM, and Demosaic, but also exploring AI-enabling other modules in the traditional ISP pipeline. Furthermore, to meet the high computational requirements of Aixin Zhimou® AI-ISP, AXERA has also developed the AiXin Tongyuan® hybrid precision NPU, accelerating the practicality of AI development.

Regarding the relationship between Aixin Zhimou® AI-ISP and 'True Black Light,' Zhang Xing mentioned that in the era of 0.01lux 'True Black Light,' improvements in lenses and sensors alone cannot address the fundamental issue. Traditional ISPs cannot meet the technological demands of true black light, which has led to confusion and the adoption of methods such as frame reduction or additional lighting. AiXin believes that a qualified black light full-color camera should function in 0.01lux scenarios without supplemental lighting, achieving real-time (25-30)fps, accurately identifying moving objects, and correctly reproducing colors. This, indeed, is the unique advantage of Aixin Zhimou® AI-ISP.

In fact, Aixin Zhimou® AI-ISP's entry into the market in 2020 played a pivotal role in driving the rapid development of the 'True Black Light Full-Color Camera' as a new product category.

This year, the popularity of large models has propelled AI into a new stage. AXERA believes that due to the high demands for real-time and reliable image processing, as well as privacy protection, AI will transition from cloud to edge computing, placing higher requirements on edge computing power. Currently, AXERA's third and fourth-generation chips equipped with AiXin Tongyuan® hybrid precision NPU support the deployment of large Transformer models. With the advancement of large models, AXERA is actively exploring how these models can enhance AI-ISP performance.

It can be observed that as imaging demands increase in extreme scenarios such as low light, high dynamic range, and light pollution, AI-ISP is becoming increasingly vital in the field of artificial intelligence visual imaging. By solving core pain points in the industry such as low light and high dynamic range through technological innovation, new product categories like the 'True Black Light Full-Color Camera' created by Aixin Zhimou® AI-ISP are unlocking more visual possibilities. This might lead to the dawn of a new era in imaging.