AP_NavEKF3: fix computation of rho #40

Job	Run time
build (stm32f7, chibios, 10)	15m 32s
build (stm32h7, chibios, 10)	15m 3s
build (fmuv2-plane, chibios, 10)	7m 55s
build (periph-build, chibios, 10)	14m 48s
build (iofirmware, chibios, 10)	4m 35s
build (CubeOrange-bootloader, chibios, 10)	3m 22s
build (fmuv3-bootloader, chibios, 10)	3m 32s
build (revo-bootloader, chibios, 10)	3m 36s
build (stm32h7-debug, chibios, 10)	9m 9s
build (fmuv3, chibios, 10)	15m 58s
build (revo-mini, chibios, 10)	12m 49s
build (MatekF405-Wing, chibios, 10)	13m 8s
build (CubeOrange-ODID, chibios, 10)	10m 26s
build (CubeRedPrimary-bootloader, chibios, 10)	3m 18s
build (configure-all, chibios, 10)	12m 50s
build (build-options-defaults-test, chibios, 10)	26m 57s
build (signing, chibios, 10)	11m 27s
build (CubeOrange-PPP, chibios, 10)	9m 8s
build (CubeRed-EKF2, chibios, 10)	8m 47s
build (SOHW, chibios, 10)	12m 59s
build (Pixhawk6X-PPPGW, chibios, 10)	5m 25s
build (new-check, chibios, 10)	15m 24s
	3h 56m 8s

Provide feedback