SeaSplat sees true colors of the underwater world

03 Jun 2025

MIT optical processing reveals accurate details of undersea geometry for better 3D modelling.

A project from MIT and the Woods Hole Oceanographic Institution (WHOI) has developed an image analysis tool intended to render truer and more accurate representations of underwater scenes.

Capturing the true colors of objects in the ocean is challenging, thanks to the range of scattering and obstruction effects introduced as light passes through the water.

In particular underwater images tend to lack red colors alongside an excess of blue and green, commented MIT, while backscatter causes underwater images to exhibit a veiling or hazy effect.

As well as preventing a true picture of any scene, this interference also hinders the recreation of accurate 3D images of underwater objects and structures from the optical data, leaving their geometry unclear.

The project's solution is SeaSplat, a method to enable real-time rendering of underwater scenes leveraging recent advances in 3D radiance fields, unveiled at the 2025 IEEE International Conference on Robotics and Automation (ICRA) (preprint pdf).

SeaSplat, named for both its underwater application and the 3D Gaussian splatting (3DGS) numerical method it employs, takes images of a scene and stitches them together to generate a complete, three-dimensional representation that can be viewed in detail, from any perspective.

"With SeaSplat we can model explicitly what the water is doing, and as a result can in some ways remove the water and produce better 3D models of an underwater scene," said MIT graduate student Daniel Yang.

Assess health of coral reef and marine communities

The Gaussian splatting approach involves representing the individual points of a visual field with 3D Gaussian data and then applying a computational model designed to produce photorealistic rendering, although until now that method has generally assumed that an object's color is consistent with distance - true in air, but not necessarily so in water.

"Efforts to adapt 3D reconstruction to underwater imagery have been hampered, mainly by two optical underwater effects: backscatter and attenuation," commented MIT.

"Backscatter occurs when light reflects off of tiny particles in the ocean, creating a veil-like haze. Attenuation is the phenomenon by which light of certain wavelengths attenuates, or fades with distance. In the ocean, for instance, red objects appear to fade more than blue objects when viewed from farther away."

SeaSplat tackles this via a color-correcting algorithm that determines the degree to which every pixel in an image must have been distorted by both the backscatter and attenuation, and then takes away those aquatic effects to compute what the pixel's true color must be.

The operation combines its 3DGS method with an underwater image formation model, so that the imaging operation can accurately estimate a scene's geometry while also correcting its colors.

In trials carried out in the Red Sea, off the coast of Curaçao, and in the Pacific Ocean near Panama, SeaSplat successfully generated a true-color 3D world within which objects kept their authentic coloration and retained it when viewed from different perspectives, rather than fading as they would if viewed through the actual ocean.

Although at present the technique requires a tether to a desktop computer, and is therefore too bulky for small underwater robots, it could still be useful for tethered operations onboard a vehicle connected to a larger ship.

"This is the first approach that can very quickly build high-quality 3D models with accurate colors, underwater, and it can create them and render them fast," said Yogesh Girdhar from WHOI. "That will help to quantify biodiversity, and assess the health of coral reef and other marine communities."