For Navigation Mesh, custom costs on entering navigation areas, and some further control, can be achieved by extending FRecastQueryFilter. This requires NavigationSystem and probably NavMesh in your Build.cs file, otherwise a Detour related .h file will not include correctly.

You can override getVirtualCost on the filter subclass. It includes some others also, see the parent’s parent dtQueryFilter.

The query filter works using recast/detour’s settings, which have a number of challenges:

  • Detour uses X and Z for X and Y coordinates, and Y coordinate for Unreal’s Z coordinate. A conversion can be done using Unreal2RecastPoint and Recast2UnrealPoint.
  • It seems the coordinates don’t exactly match UE’s coordinate system in general, but this may be affected by the location of the navmesh actor.
  • The coordinates are for positions of the navmesh polygons, not the actual path your actor would take. A separate process called string pulling is done to postprocess the path to what the actor will then use. Think of it like literally pulling a piece of string - it removes the “slack” from the path.

To use your custom Recast Query Filter class, you should also create a subclass of UNavigationQueryFilter. Override InitializeFilter, and set the following code into it:

Super::InitializeFilter(NavData, Querier, Filter);