Share this on social media:

Greg Blackman talked with Isak du Preez, his new method of 3D deflection measurement aroused the interest of BMW and other car manufacturers

One of the early 3D deflection scanners used a security camera, but still produced high-quality 3D surface contours. Image credit: Isak du Preez/Axiscan

After talking with a friend who calculated hail dents on cars in the summer, Isak du Preez first thought of the idea of ​​installing a 3D scanner for the reflective surface in South Africa. Domestic insurance companies pay for the dents to repair the hail damage; they want to know the number, size and location of the dents. Du Preez believes that it is possible to scan the surface of a car in 3D to find dents.

In 2016, he built a miniature prototype of the scanner. A basic plotter moves a patterned light arch on the toy car. He used the phone camera to capture the reflections of the shiny car body and developed software to calculate the 3D surface profile.

Since then, six large scanners have been built, mainly for scanning cars, and du Preez is manufacturing a seventh for BMW. Calculating hail dents is no longer the goal; it is now a car’s offline inspection, which is usually a manual task or the use of a robotic arm to scan the surface from different angles. Scanning car windshields is another potential application, or large-area curved architectural glass. Aerospace companies are also interested in Du Preez checking some of their components. Any reflective surface-usually a large smooth surface-can benefit from this technology.

This technique is a deflection measurement method, although when Du Prez first started, he had never heard of the term. The advantage of the du Preez method is that the accuracy of the 3D deflection measurement method can be applied to large surfaces in a practical and economical way-it could not be solved by the deflection measurement method before. It is now possible to scan objects several meters wide with sufficient accuracy to detect local shape distortions on the order of micrometers.

Isak du Preez's first prototype scanner, made from hardware store materials in 2016. Image source: Isak du Preez

Traditionally, cameras and flat-panel displays are used for deflection measurement. Each pixel of the camera is mapped to a scene point on the display, usually a display pixel. When a reflective surface is introduced, what each pixel of the camera sees can be traced back to points in the surrounding scene through the specular reflection of the surface. The mapping is based entirely on geometric information; it only depends on the shape of the surface and the location of the camera and display.

The technology relies on specular reflections, so the surface must be shiny or at least shiny, but any reflections are very sensitive to shape-therefore, a hail dent that is only a micrometer deep can be detected with a normal camera. It is more sensitive to depth than structured light scanning, which is based on triangulation between the camera and projector, and treats specular reflections as noise.

The disadvantage of the deflection measurement method is that the visible reflections in objects (especially convex objects) are usually very small. Therefore, using standard methods, only a small area of ​​the surface can be mapped at a time, and the process is repeated and all the pieces are stitched together to capture the entire 3D shape. For example, when inspecting the paint on the car body, the deflection sensor installed on the robot will take hundreds of different images for each small block.

"The contribution of this technology is to reinvent the scene encoder," he said. 'Electronic displays are very convenient hardware for generating flat scenes, but if you need to fully enclose an object like a car, then the practicality is much worse. It is also over-engineered because we don't need to display arbitrary images to encode the scene.

Isak du Preez integrates Axiscan's software with the first full-scale machine. Image credit: Isak du Preez/Axiscan

Du Preez's method is not to display, but to rely on static patterns printed on the inside of the arch. Moving the arch to the car will encode a uniquely decodable scene. This mode is composed of a set of latitude codewords. When translated with longitudinal motion, the latitude and longitude of each scene point are encoded. In order to distinguish it from various other deflection measurement methods, du Preez calls this method the latitude code scanning deflection measurement method.

The design of this mode is optimized using a technology similar to channel coding in telecommunications. The codewords are packed in one code space as efficiently as possible. "But in the case where the discrete communication code is optimized by sphere packaging, what we have done can be more accurately described as spaghetti packaging because we are encoding continuous variables," Duprez explained.

At some stage, du Preez said that the easiest way to design patterns is to use physical analogy. He fixed a pyramid-shaped fixture to represent the code space, and tried different ways to install a section of flexible pipe into the volume. "Efficient coding ultimately allows for shorter codewords, reducing the depth and quality of the arches and the entire machine required," he explained.

Du Preez is a software engineer, and software is where the complexity lies; the machine itself is relatively simple. Through his company Axiscan, du Preez provides the model and the software to decode it, while the large scanner is built or commissioned by his customer according to the requirements of the application. Finally, Axiscan's software accepts camera video as input and generates a 3D mesh of the surface as output.

A 3D grid contains a vertex per camera pixel; its lateral density is only limited by the camera resolution. In addition, the system is self-calibrating, and the software automatically calculates the position of each camera from a direct view of the encoded scene.

3D data from Mini Scan. Image credit: Isak du Preez/Axiscan

Early scanners used to assess car hail damage used security cameras, but because they compress video, they are not ideal. "This is the last thing you want," Du Prez said. 'For this, you want the pixels to be as independent as possible.

"But it still works," he added, adding that the dent reconstructions from different cameras are consistent with each other, as small as about 1 micron. "Considering low camera specifications, video compression, and generally uncontrolled environmental conditions, this level of software repeatability is encouraging."

He continued: "The hardware specifications for this project are very loose. I don't think we have reached the upper limit of achievable accuracy.

Most metal plates used to make cars scatter light far beyond specular reflection. Usually it only becomes reflective after the car is painted. But when imaging in long-wave infrared, the rougher surface type can still behave like a mirror. If car manufacturers can find defects before the expensive car painting step, that will add a lot of value.

Past attempts at infrared deflection measurement usually involve moving a pattern of complex motion with many degrees of freedom composed of heating elements, which still only encodes a flat scene. This technology is difficult to design due to the lack of ready-made infrared displays and thermal elements that only slowly change the temperature.

A 3D scan showing the curvature of a car's windshield. Image credit: Isak du Preez/Axiscan

Du Preez believes that Latitude Code Scanning Deflectometry may be the ideal choice for infrared work, because it already relies on a mobile scene generator to encode large surfaces. Just replace the camera with an infrared camera, and then switch the pattern to the heat emission pattern painted on the heated arch. Testing this potential is one of the goals of cooperation with BMW Research Laboratories.

Currently, du Preez's software takes about 7 minutes to produce 3D results, but it runs on a single-threaded CPU. He hopes to reduce the time to 15 seconds or less through parallel processing, which will suit the natural cycle time of arching motion.

Du Preez hopes that Latitude Code Scanning Deflectometry will bring the accuracy of deflection measurement to many new applications, and the interest of BMW and other companies will only help improve the technology. He said that the latitude code scanning deflection measurement method is not as accurate as the interferometric method, nor is it as versatile as structured light scanning, but the combination of accuracy and versatility, while also cost-effective, shows that the method has a promising future. "There are many things to explore," he said.

Photo: Dennis Simonov/shutterstock.com

Use laser triangulation for electronic quality checks. Credit: sick

Figure 1: The inspected object with multiple illumination angles in front of the camera. Credit: Euresys

Photo: Dennis Simonov/shutterstock.com

Photo: Constantine Zibert/shutterstock.com

Greg Blackman discovers how seed producers reap the rewards of imaging

Photo: Dennis Simonov/shutterstock.com

Tahir Rabbani of Viztronics talked about a 3D solution for imaging shiny orthopedic implants at the Vision exhibition. Greg Blackman reports

Greg Blackman and James Falconer of Pleora talk about the latest version of GigE Vision

Photo: Sonpichit Salangsing/shutterstock.com

As solar energy becomes an increasingly important energy source, Abigail Williams looks at how infrared imaging can be used to evaluate solar panels

Anne Wendel, Director of Machine Vision at VDMA, said that people are increasingly worried about chip supply bottlenecks and material shortages.

Vision experts say that neural networks are making great progress, but the implementation of machine vision solutions still faces major challenges

Picture: Sony Semiconductor and Prophesee

Prophesee won the Vision Award for its event-based imaging method. Greg Blackman talked with company CEO Luca Verre about the future of the technology