3D Vision Put to the Test

12 Jul 2022

With the wide range of demands placed on them – the FRAMOS D400 camera series is amazingly versatile.

3D vision is becoming commonplace in industrial applications; however, it must meet different demands from those in the consumer sector. A couple of examples are the need to capture fast-moving objects, and the need to place the camera far from its controlling computer. Along with this, stable and latency-free data transmission over these extended distances are as vital as its need for simple integrations.

Until recently, 3D vision technology was considered to be expensive, time-consuming to configure, and complex. A great deal of expertise was needed to implement them, especially for customized solutions. Now, new and existing applications are able to take advantage of this technology that would not have been viable a few years ago. The increase in demand in the consumer market has made 3D technology more affordable for industrial applications. Off-the-shelf products are making this technology easier to access and speeding up the development of 3D vision-based applications in new industries. Real-time depth data provides the basis for fully automated processes on production lines, and in logistics and industrial applications where intelligent devices make their own decisions. Intel’s® RealSense™ technology is innovative and relatively inexpensive which has been a major factor in why 3D Vision is becoming more adopted in these designs.

Intel® RealSense’s™ Versatile 3D Technology Portfolio

FRAMOS has been distributing Intel® RealSense’s™ products worldwide since 2017. They contain space-saving depth modules with a dedicated vision processor that allows for stereo depth sensing to be achieved directly on the camera. Their working principle is based on two cameras – similar to how humans see. Viewing objects with two eyes produces true, quantified depth perception. Equipment and machines can use the cameras to capture, learn about, and interact with their surroundings. This type of approach is classified as “Perceptual Computing”. With 3D vision, for example, robots can reach for objects just like people do. At the same time, they can ensure that they do not collide with their surroundings during movement.

The Intel® RealSense™ D400 stereo camera series currently consists of four cameras – D415, D435, D435i (with IMU), and the D455 – which all differ in terms of the baseline (this affects the distance of the stereo setup), field of view and shutter type. They can be used both indoors and outdoors, and in all sorts of different lighting conditions thanks, in part, to the additional infrared laser projector. Besides this, they can also be used in multi-camera configurations without causing interference and the need for any calibration. All the models have a USB-C connection, the preferred choice for the consumer sector.

FRAMOS D400e Camera Series with Gigabit Ethernet

Industrial applications require cameras that meet tougher demands with respect to their connectivity and suitability in harsher environments. As a result, FRAMOS has developed its own series of depth cameras, based on Intel’s® RealSense™ technology, that address these more demanding requirements of. USB connections have been replaced with industrial-grade gigabit Ethernet connectors and protocols, and the plastic enclosures have been upgraded with an IP66 rated dust and waterproof, sturdy aluminium housings. These modifications allow the cameras to operate reliably in ambient temperatures of 0°C to 55°C, making them ideal for use in industrial environments.

The locking Gigabit Ethernet (M12), and power supply and GPIO (M8, for 12 or 24 V) connections can withstand movement and vibration, better than USB cables. With Power over Ethernet (PoE), the cameras can be operated without a separate power supply, making wiring simpler. Long transmission distances are not a problem with Gigabit Ethernet. Image data is pre-processed in the camera and can be sent over cables up to 100 m long.

The required working distances and precision requirements are important criteria in choosing the right camera. Not only that, moving objects, specifically how quickly they move, must also be considered. FRAMOS draws on the entire Intel® RealSense™ D400 series so the ideal solution can be achieved for any application.

• D415e has a high level of precision in its 65°x40° FOV for applications in the medium capturing range from approx. 30 cm with normal working speeds and not particularly fast-moving objects (rolling shutter, depth sensors, and RGB sensor)
• D435e has a wider field of view (FOV 87”x57°) which is best for close-up range from approx. 20 cm and fast-moving objects (global shutter; depth sensors, and IR sensor)
• D455e uses the largest baseline in the series giving it the highest level of precision (measurement errors less than 2% at 4m) which is suitable from approx. 50 cm and is good for tough requirements and/or very fast-moving objects (global shutter; depth sensors, and RGB sensor)

This powerful technology is well protected inside these light, compact cameras. Their ease of operation, uncomplicated plug-and-play 3D vision solution, and affordable price make the robust FRAMOS D400e cameras suitable for numerous applications, as more and more industries are demanding innovative 3D vision solutions.

The D400e cameras differ mainly by the Intel® RealSense™ depth module used and the features that this module offers. In addition, the new D455e offers other details that make it stand out from the D415e and D435e models. Because of its broad baseline, it is wider with its cable connections on the back of the camera, while sturdier mounting points are used (M4 instead of M3 used on the D415e and D435e). The camera enhances the depth data from the stereo setup (1280 x 720 pixels, max. 90 fps) with image data from a global shutter RGB sensor (1280 x 800 pixels, max. 30 fps). Since the depth sensors and RGB sensor have the same field of view, the resulting data are a perfect match of depth information and colour images. The global shutter sensors make it possible to capture very fast objects without distortion.

Board-Level Modules That Can Be Flexibly Integrated

As one of the world’s leading suppliers of image processing technologies, FRAMOS helps its customers identify their needs in the design of 3D sensing projects, select the best Intel® RealSense^TM product including all the necessary components, and implement the 3D system into their application. On top of this, they also offer Intel® RealSense^TM board-level modules that can be easily integrated into customer-specific camera solutions where space is tight.

The D400e board-level modules come without an enclosure and offer gigabit Ethernet in a compact PCB format. They provide the same performance as their enclosed counterparts in the FRAMOS D400e camera family but can be more easily integrated into new or existing designs and configurations. All camera modules are supplied with a depth camera module, the FRAMOS D4 Vision Processing Board and the necessary interposer cables.

Typical Applications

These plug-and-play solutions allow real-time positioning, orientation and tracking of robots, automatic guided vehicles (AGVs) along with many other smart machines, such as those used in medical technology and in logistics, the motor industry and smart agriculture.

A “seeing” robot, for example, can recognise different parts in pick-and-place applications and set them down accurately on a processing station, a conveyor, or a driverless transport system. 3D vision gives these driverless systems fully autonomous mobility, enabling them to move around in complete safety and avoid people or other obstacles without additional installations such as rails or markers.

Robotics Solution for Premium Car Producer

One example of successful use is an application by the BMW Group, where a robot (“SortBot”) sorts and palletises small load carriers (SLC) using the D435e. The camera supplies the depth data needed so that the SortBot recognises the position of the SLCs, and the vacuum gripper moves into the best position using AI technology. This means that it can efficiently grip, align and palletise the SLCs.

The FRAMOS D435e depth camera is ideal for this application. It takes full advantage of the Intel®RealSense® camera module to provide the necessary data. Using the integrated Intel®RealSense ™Vision Processor D4, depth information is generated directly in the camera and made available via the gigabit Ethernet interface. As a result, no additional processing power is required in the host system to create the depth image. Furthermore, color information and depth data can be overlaid to ensure accurate identification and orientation of the SLCs.

Smart Farming

Cameras and depth sensors are also essential for automation in smart farming. Overall, stereo cameras have proven to be the sturdiest depth sensing systems, both indoors, and outdoors. They offer good performance in any lighting condition, are immune to interference in multi-camera setups or from external light sources and are inexpensive. The integrated RGB module, which supplements the 3D data with color information, allows the farmer, for example, to better assess the health of a plant or the ripeness of a fruit.

• Monitoring: Depth data can be used to monitor both a plant’s growth and its health through the combination of both the 2D-RGB and/or NIR imaging. This information, in turn, can help to reduce the use of pesticides on healthy plants.
• Navigating: Both the visible spectrum and the 3D data can be used for navigating machines, allowing them to drive round plants and other obstacles. This can improve safety in agricultural operations and increase efficiency by lowering staffing costs.
• Harvesting: The ripeness of a fruit can be assessed by automatic picking machines using the depth data that is combined with 2D-RGB and/or NIR imaging. Autonomous self-driving harvesting machines are able to locate fruit on a plant, assess their ripeness and pick them, all without damaging the plant.

In all of  these tasks, the industrial-grade D400e is the heart of the automation system. The user’s imagination is practically the only limit on the innovative possibilities that 3D vision opens up.

FRAMOS Offers Unlimited Possibilities with the D400e Camera Family

3D cameras for industrial applications have to operate in harsh production environments while reliably detecting fast-moving objects at speed. FRAMOS has developed a number of industry grade cameras based on the Intel® RealSense™ technology for a wide range of applications. All D400e FRAMOS cameras offer gigabit Ethernet connectivity and an IP66-rated dust-proof, water-proof housing providing an easy way to integrate 3D vision into even the harshest environments. This plug-and-play solution enables real-time positioning, orientation and tracking for robots, navigation for automated guided vehicles (AGVs), and many other “smart” machines, e.g. in medical engineering.

With the advent of more products that offer various options for capturing depth information, more applications and use cases are opening up to take advantage of the data that they provide. Their lower costs and ease of implementation are making them much more appealing. However, many are not designed to withstand the harsh environments seen in many industrialized implementations. As such, these designers are looking for more ruggedized products that can ensure 24/7 runtime when under stress of constant movement or ranging environmental conditions. FRAMOS’s D400e series of 3D Depth cameras have been designed with this in mind and have shown that they can be used reliably in many challenging applications.

Want to know more?  Click here.