Does Your Simulation Capture the World Your Mission Requires?

This blog is part of a series exploring the technologies and principles shaping the future of simulation.

A helicopter crew flies a night route in a simulator. The copilot/gunner switches to thermal to identify a building near the landing zone. The building is in the right place, but does it look as it would through the aircraft’s actual sensor?

The gunner uses the display to practice recognizing objects and making engagement decisions. If the simulated sensor shows features the real equipment would not reveal, the gunner can learn to rely on cues that will be missing on the mission.

Each sensor view needs to draw from the same building data, including its location, shape and surface materials. That shared description is digital ground truth.

One World for Every Sensor

Digital ground truth is a single 3D description of the terrain and everything on it. Every visual and sensor view is built from it, so the helicopter crew and an unmanned aircraft flying overhead see the building in the same place. This alignment across views is called correlation.

Along with the building’s geometry, such as its shape, location and height, the description records its type and what its surfaces are made of. The roof has a color for the optical camera. It also has properties that tell the thermal model how the roof emits infrared energy and tell the radar model how it reflects radar energy.

A shared synthetic environment of an airfield and the surrounding town, defined once for every visual and sensor view.

Materials and Conditions Shape Each Sensor View

What differentiates high fidelity sensor modeling? In a few words, material attribution. Thermal sights, for example, respond to surface materials. Concrete, grass and glass heat and cool differently, and their surfaces emit and reflect infrared energy differently.

A thermal sight detects that energy within its operating wavelength spectrum. The image combines the energy the building emits, energy reflected from its surroundings, and the effect of the ambient air along the viewing path. Recent heating and cooling also affect the picture, so two surfaces at the same temperature can look different through the sight. These factors decide how clearly the building stands out from the surrounding terrain.

An infrared view of the same airfield. Pavement, trees, buildings and the parked helicopters each show different contrasts based on their material properties and the conditions in the scene.

The models for the optical camera, night vision goggles, and radar make the same kind of calculation for their own part of the wavelength spectrum. Each model has to respond as conditions change. A building that stands out at one time of day may blend into its surroundings at another, or fade in poor weather conditions.

The same airfield at dusk. The terrain and buildings are unchanged, and the lighting changes what the crew can see.

Consistently rendering every view under dynamic environmental conditions is a key image generator function. Aechelon’s Nucleus, an integrated image generation foundation for high fidelity simulation systems, generates visual and sensor views from the same worldwide database, so each view reflects the same lighting and weather.

Taking it further, some environmental conditions come from the aircraft’s interaction with the terrain. On approach, rotor wash lifts dust and soil particles from the landing zone. Aechelon’s Aero-Turbulence and Brown-Out Server (ATBOS) uses real time computational fluid dynamics, a method for calculating how air moves, to model how the rotor’s airflow meets the terrain surface and builds the dust cloud.

The cockpit view and each sensor display show that cloud as their sensor would detect it. When ATBOS is integrated with the host simulator, the same airflow also affects how the simulated aircraft handles.

An ATBOS visualization of modeled airflow around a helicopter. As rotor airflow interacts with the terrain, it can lift dust and reduce the crew’s visibility during landing.

Wrong Cues Can Teach the Wrong Response

When a view departs from reality, training can teach the wrong thing. Suppose the simulated thermal view shows a feature the gunner’s actual sight would not show under the same conditions. The gunner may learn to depend on it, and repeated practice reinforces the habit. On the mission, the cue is missing.

This is negative training. The simulator teaches a response that can impair performance in the real world.

Several sensor views can agree with one another and still carry the same error. Correlation alone does not establish that the views match what the real equipment would show. The shared data and the resulting sensor views both need validation.

A night vision view of the same airfield. If simulated goggles showed more detail than real goggles could at this light level, crews would learn to expect cues they won’t get on the mission.

Validate the Data and the Sensor Views

The data used to build the exercise also gives instructors a reference for after-action review. They can compare the building’s defined location and features with what appeared through the gunner’s sight and how the gunner responded.

That comparison helps explain a missed identification. If the gunner overlooked a visible feature, the crew needs more practice. If the simulated sensor failed to show it correctly, the training system needs a fix, and engineers can correct the shared data once for every view.

Database developers track where source data came from, when it was collected, how accurately it represents the area and whether it is still current. They also check the resulting sensor views against the conditions being trained.

Those checks continue as the mission changes. A new image of a landing zone may show a building’s shape and location without the material information needed to reproduce its thermal appearance. The update is ready for training once engineers have checked both the data and the resulting sensor views against the conditions in the exercise.

Training programs live with change. Missions move to new regions and sensors are upgraded, and each change asks the simulator to show something it has not shown before. A training environment built on one validated description of the world can take on those changes and keep every view tied to what crews will see on the mission. Over time, that is how simulated sensor views earn the trust crews place in them.

Trust also depends on knowing how much detail a training task needs. In the next post, we explore how to choose the right fidelity for the training objective, including where added detail improves training and where it adds little value.

About the Author

Randall Toth is a Market Innovation and Product Ecosystem Strategist at Shield AI. He brings over 38 years of experience in geospatial development for military modeling, simulation and mission systems. His work spans terrain data production, training system development, systems engineering and program leadership.