How to use RTKLIB and Eagle Eye for Localization

[joewong00]

Hi,

After reading through the documentations, I'm still a little vague about the whole idea of using eagle eye for localization, so here are my questions:

1. Is the input from RTKLIB necessary? I realised that the rosbag contains the topic "/rtklib_nav", which is from RTKLIB ros bridge. It seems like this input is required to run eagle eye rt based on the config file.

2. In RTKLIB configurations, I noticed that a base station input is required. However, I do not have that so does that mean that I cannot use RTKLIB to output the rtklib_nav topics?

3. Currently, I only have the IMU topic as well as a fix type GPS data. I assume that I can use the eagle eye rt with the canless mode, but if I have the wheel odometry data, I can use the CAN mode by just using the odometry twist data right?

These are my questions and I hope that you can assist me here. I am hoping to perform localization of my robot with the use of GPS data along with the IMU for a more precise pose in the map.

Thank you.

[rsasaki0109]

1
Input from RTKLIB is not required. GNSS-related input can be selected from the following options.
https://github.com/MapIV/eagleye/blob/main-ros1/eagleye_rt/config/eagleye_config.yaml#L11-L15

2
You can use RTKLIB to output rtklib_nav topics without needing base station input, but in that case this package is lane-level accurate.
Base station input is required if you want to get more accurate poses with cm accuracy on the map.

3
If wheel odometry data is available, CAN mode can be used. Note, however, that the kinematic model assumes a car and does not support actuated two-wheeled or omnidirectional robots.

[joewong00]

Hi, thanks for the reply.

Regarding the first question, if I do not have any input from RTKLIB, what should I put in the GNSS velocity source? I tried using the odometry twist data, but when I tried running, the "Eagleye status" shows that the latitude and longitude under rtklib (input) shows a zero value.

About your second reply, may I know how can I obtain the rtklib_nav topics without the input from base station? I'm a little confused with the configurations in RTKLIB.

Thank you.

[rsasaki0109]

1
The GNSS input options for latitude/longitude and velocity information include the following

gnss:
  velocity_source_type: 0 # rtklib_msgs/RtklibNav: 0, nmea_msgs/Sentence: 1, ublox_msgs/NavPVT: 2, geometry_msgs/TwistWithCovarianceStamped: 3
  velocity_source_topic:  /rtklib_nav
  llh_source_type: 0 # rtklib_msgs/RtklibNav: 0, nmea_msgs/Sentence: 1, sensor_msgs/NavSatFix: 2
  llh_source_topic:  /rtklib_nav

2

For more information, please see the RTKLIB documentation and how to use it, but the following is a good reference for setting up without a base station.
The key point is to set pos1-posmode to single.

https://github.com/MapIV/RTKLIB/blob/rtklib_ros_bridge_b34/app/consapp/rtkrcv/conf/rtklib_ros_bridge_single.conf

[joewong00]

Yes, for the gnss input this is what I used:
velocity_source_type: 3, velocity_source_topic: /odometry_twist
llh_source_type: 2, llh_source_topic: /gps/fix

I checked that these topics are being subsrcibed. However running eagleye rt, the status still show that I do not have any gnss input. Did I do anything wrong?

[rsasaki0109]

gnss.velocity_source_topic should be put in GNSS Doppler velocity information.
/odometry_twist should be put in twist.twist_topic.

[rsasaki0109]

I will also check on it because there may be a possibility of bugs with that combination of GNSS

[joewong00]

I see, thank you.

My issue is that I'm still trying to explore more on RTKLIB so that it can be used in this case. In the meantime, I suppose I can just set "use_gnss_mode" to NMEA, twist topic to /odometry_twist, gnss velocity source to /nmea_sentence and llh source to either /nmea_sentence or /fix topic?

[joewong00]

Unfortunately with the above set up, it is not working.

The status is always showing this.

[rsasaki0109]

Just set "use_gss_mode" to NMEA, "twist topic" to /odometry_twist, "gnss velocity source" to /nmea_sentence, and "llh source" to either /nmea_sentence.
Does /nmea_sentence contain RMC (velocity information) as well as GGA?

[joewong00]

Hi, yes I tried using the above config you mentioned, and my nmea_sentence topic does have GNRMC and GNGGA, but the result is the same. This is my rosbag.

rosbag

My topics are:

1. imu: /wit/imu
2. /odometry/twist
3. /nmea_sentence
4. /fix

I realised that the gnss_converter node has died halfway when I was running. Because the status was too quick I didn't notice it at first. What might be the reason that is causing the process to die?

[rsasaki0109]

@joewong00
Thanks for providing rosbag. I will check it out. By the way, is the branch you are using main-ros1?

[joewong00]

Yes I'm using main-ros1 branch. Thanks for helping me checking it out.

[rsasaki0109]

@joewong00
I have checked your rosbag. The frequency of the IMU is too low at 5Hz.
Probably 30 Hz is required. Recommendation is 50Hz or higher.

Also, please set the imu_rate appropriately in your yaml.
https://github.com/MapIV/eagleye/blob/main-ros1/eagleye_rt/config/eagleye_config.yaml#L32

Furthermore, the default "moving_judgment_threshold" is set for cars, so you need to make it a bit smaller for smaller vehicles. Let's set it to 0.834m/s (which is about 3km/h).

https://github.com/MapIV/eagleye/blob/main-ros1/eagleye_rt/config/eagleye_config.yaml#L36

In addition, to make Eagleye function properly, initialization is necessary. If you do not perform initialization, the output of the twist will remain raw and you cannot utilize pose data.
The first step is static initialization, where you need to keep Eagleye stationary for about 4 (yawrate_offset_stop.estated_interval) seconds after startup to estimate the yaw rate offset.
The next step is dynamic initialization. This involves driving Eagleye straight for about 30 seconds. In this process, the wheel speed and azimuth scale factor are estimated. Once dynamic initialization is complete, Eagleye will be able to provide corrected twist and pose data.
If you don't want dynamic initialization, you need two gnss receivers.

[joewong00]

Ohh I see, so my issue is at the imu rate and moving judgement threshold, which are also responsible for the process died in gnss_converter node? Is there any problem with my nmea sentence or gnss receiver?

So this initialization part explains why in the rosbag, the results will be output after around 100 seconds, because it takes time to do both static and dynamic initialization and the same goes to real vehicle.

That being said, I wont be able to get any output without performing both initialization. I am however, quite interested in the use of two gnss receivers with no dynamic initialization. Is there any resources I can look into about this?

Thank you.

[rsasaki0109]

There is probably no problem with your NMEA sentence or GNSS receiver. For your reference, here is the YAML configuration file when I tested your rosbag:

# Estimate mode
-use_gnss_mode: RTKLIB
+use_gnss_mode: NMEA
 use_canless_mode: false

# Topic
twist:
-  twist_type: 0 # TwistStamped : 0, TwistWithCovarianceStamped: 1
-  twist_topic: /can_twist
+  twist_type: 1 # TwistStamped : 0, TwistWithCovarianceStamped: 1
+  twist_topic: /odometry/twist
imu_topic: /wit/imu
gnss:
-  velocity_source_type: 0 # rtklib_msgs/RtklibNav: 0, nmea_msgs/Sentence: 1, ublox_msgs/NavPVT: 2, geometry_msgs/TwistWithCovarianceStamped: 3
-  velocity_source_topic:  /rtklib_nav
-  llh_source_type: 0 # rtklib_msgs/RtklibNav: 0, nmea_msgs/Sentence: 1, sensor_msgs/NavSatFix: 2
-  llh_source_topic:  /rtklib_nav
+  velocity_source_type: 1 # rtklib_msgs/RtklibNav: 0, nmea_msgs/Sentence: 1, ublox_msgs/NavPVT: 2, geometry_msgs/TwistWithCovarianceStamped: 3
+  velocity_source_topic: /nmea_sentence
+  llh_source_type: 1 # rtklib_msgs/RtklibNav: 0, nmea_msgs/Sentence: 1, sensor_msgs/NavSatFix: 2
+  llh_source_topic:  /nmea_sentence

# TF
tf_gnss_frame:
@@ -29,11 +29,11 @@ ecef_base_pos:
# Eagleye Navigation Parameters
# Basic Navigation Functions
common:
-  imu_rate: 50
-  gnss_rate: 5
+  imu_rate: 5
+  gnss_rate: 1
   stop_judgment_threshold: 0.01
   slow_judgment_threshold: 0.278
-  moving_judgment_threshold: 2.78
+  moving_judgment_threshold: 0.834

velocity_scale_factor:
   estimated_minimum_interval: 20

The multi-antenna feature is still available in the current version, but the configuration is a bit difficult. It will become easier to use in the next release of Eagleye (expected at the end of May), so please wait a little while.
https://github.com/MapIV/eagleye/tree/develop-ros1#dual-antenna-mode

[joewong00]

Thanks. I have changed my imu publishing rate to be 50 hz and followed the same configurations like the one you set above. However I realised there might be some issue in my nmea sentence topic that is causing the node to die. This is the error message output when I launched it separately.

[rsasaki0109]

Have you made any changes to the gnss_converter package?
If not, it may be a ros melodic specific problem.
We only support ros noetic in ros1 since ros melodic is going EOL at the end of this month.

[rsasaki0109]

If you create another workspace and build eagleye, the problem may go away.
I've dealt with this kind of mysterious error before.

[joewong00]

I see, I will try that as well.
However I realised it might be the issue of the nmea navsat driver that I'm using, it occasionally received an invalid checksum and which is a 2 digits sentence as shown here, this is causing the error in GNSS converter node.

I will try to fix this issue before continuing.

[rsasaki0109]

That may be the problem. Please confirm, thank you.

[joewong00]

Yes, I have verifed that the issue is with my nmea sentence, which is not supposed to be just 2 characters otherwise it would cause the error. I'm still trying to find a solutions for that.

Other than that, I would like to know that if I only have the navsatfix topic without any velocity source from the GNSS receiver (no nmea sentence / twist), then what should be the input of velocity source under GNSS?

[rsasaki0109]

The key to eagleye is GNSS Doppler velocity, and it will not move without that information.

[joewong00]

Alright, understood.

I tried running on a real test, and below is the status I got:

It seems like my position status is not enabled. I assume that this "position" is supposed to be having the same frequency as my IMU. However the latitude and longitude update is not 50Hz. What did I do wrong?

[rsasaki0109]

There are two stages in dynamic initialization. The first stage is where the azimuth is estimated and the pose is published (which means the pose is the value of GNSS). The second stage is where the trajectory combined with GNSS/IMU is established, and at this point, the position status becomes true.

[joewong00]

In the gga input the "rtk status" shows that there is No Fix. Is this an issue or is this normal?

[rsasaki0109]

The rtk fix is more likely to occur in open skies, and no fix is normal in urban areas or environments with buildings in the vicinity.

[joewong00]

Thanks for the clarification. Here is my updated rosbag, can you help me verify that everything is okay? After the static initialization I should be getting the estimated twist that can be used to integrate into autoware with NDT pose. The only issue I'm having now is not being able to complete dynamic initialization and get the eagleye pose.

One important note is that I had to modify the gnss_converter to handle the exception of invalid sentence in my /nmea_sentence that output a weird string occasionally.

rosbag_updated

[rsasaki0109]

I will check on it when I have time.

[rsasaki0109]

I have confirmed that the pose is being published with the following parameters.

 # Estimate mode
-use_gnss_mode: RTKLIB
+use_gnss_mode: NMEA
 use_canless_mode: false  
 
 # Topic
 twist:
-  twist_type: 0 # TwistStamped : 0, TwistWithCovarianceStamped: 1
-  twist_topic: /can_twist
-imu_topic: /imu/data_raw
+  twist_type: 1 # TwistStamped : 0, TwistWithCovarianceStamped: 1
+  twist_topic: /odometry/twist
+imu_topic: /wit/imu
+imu_topic: /wit/imu
 gnss:
-  velocity_source_type: 0 # rtklib_msgs/RtklibNav: 0, nmea_msgs/Sentence: 1, ublox_msgs/NavPVT: 2, geometry_msgs/TwistWithCovarianceStamped: 3
-  velocity_source_topic:  /rtklib_nav
-  llh_source_type: 0 # rtklib_msgs/RtklibNav: 0, nmea_msgs/Sentence: 1, sensor_msgs/NavSatFix: 2
-  llh_source_topic:  /rtklib_nav
+  velocity_source_type: 1 # rtklib_msgs/RtklibNav: 0, nmea_msgs/Sentence: 1, ublox_msgs/NavPVT: 2, geometry_msgs/TwistWithCovarianceStamped: 3
+  velocity_source_topic:  /nmea_sentence
+  llh_source_type: 1 # rtklib_msgs/RtklibNav: 0, nmea_msgs/Sentence: 1, sensor_msgs/NavSatFix: 2
+  llh_source_topic:  /nmea_sentence
 
 # TF
 tf_gnss_frame:
@@ -30,10 +30,10 @@ ecef_base_pos:
 # Basic Navigation Functions
 common:
   imu_rate: 50
-  gnss_rate: 5
+  gnss_rate: 1
   stop_judgment_threshold: 0.01
   slow_judgment_threshold: 0.278
-  moving_judgment_threshold: 2.78
+  moving_judgment_threshold: 0.556
   
 velocity_scale_factor:                                
   estimated_minimum_interval: 20
@@ -45,7 +45,7 @@ velocity_scale_factor:
   th_velocity_scale_factor_percent: 10.0          
 
 yawrate_offset_stop:                                  
-  estimated_interval: 4
+  estimated_interval: 3
   outlier_threshold: 0.002
 
 yawrate_offset:               

[joewong00]

Thank you. Yes I'm able to get the pose as well, but I realised that the x y pose is updating at the rate of 1Hz (same as GNSS), but I think it is supposed to be 50Hz.
I also realised that the x y pose I'm getting is the MGRS coordinates, and this pose is relative to the map because it publishes the TF from map to base link. Does that mean my map's origin needs to be at the origin of the MGRS grid?

[rsasaki0109]

To get to 50hz, you need to run position status until it is true or rtk and then
use_rtk_deadreckoning must be set to true
https://github.com/MapIV/eagleye/blob/main-ros1/eagleye_rt/launch/eagleye_rt.launch#L9
I will check the code in a bit.

Map must be MGRS grid

[joewong00]

About the GNSS velocity source, I only have the GPS yaw and GPS ground velocity. Can these be used?

[rsasaki0109]

Thank you. Yes I'm able to get the pose as well, but I realised that the x y pose is updating at the rate of 1Hz (same as GNSS), but I think it is supposed to be 50Hz.

In my environment it was 50hz.

---  yawrate offset --------------------------
 yawrate offset  -0.000000 [rad/s]
 status enable  False

---  slip angle ------------------------------
 coefficient  0.000000
 slip angle  0.000000 [rad]
 status enable  False

---  heading ---------------------------------
 heading  2.713759 [rad/s]
 status enable  True

---  pitching --------------------------------
 pitching  0.000000 [rad]
 status enable  False

---  height ----------------------------------
 height  0.0000 [m]
 status enable  False

---  position --------------------------------
 latitude  3.12940017 [deg]
 longitude  101.72385950 [deg]
 altitude  56.4000 [m]
 status enable  False

~/eagleye_main_ws$ rostopic hz /eagleye/pose
subscribed to [/eagleye/pose]
average rate: 51.103
	min: 0.010s max: 0.022s std dev: 0.00272s window: 15
average rate: 50.225
	min: 0.009s max: 0.031s std dev: 0.00374s window: 64

About the GNSS velocity source, I only have the GPS yaw and GPS ground velocity. Can these be used?

When use_gnss_mode is NMEA, GPS ground velocity is used.

[joewong00]

Thanks for the reply. I have tried running using rtklib, it gave me a warning of estimated velocity scale factor is too large or too small. What does it mean and how can I resolve this warning? Thanks.

[rsasaki0109]

The warning you received means that the estimated value for the velocity scale factor is either too large or too small. In response to this, the estimation value is being rejected. May I ask what value you have estimated? If it seems to be a reasonable value, you can relax the threshold for anomaly detection using the following param.

param

th_velocity_scale_factor_percent: 10.0

https://github.com/MapIV/eagleye/blob/develop-ros1/eagleye_rt/config/eagleye_config.yaml#L48

code
https://github.com/MapIV/eagleye/blob/develop-ros1/eagleye_rt/src/velocity_scale_factor_node.cpp#L113-L120

[joewong00]

I'm using the same default parameters. The velocity scale factors I'm getting is always remaining at 1. I also notice there is no changes in the value of yawrate offset and yawrate offset stop. For this rosbag, I'm using RTKLIB mode. I know that it is not long enough to finish the initialization for getting the position. Otherwise, I wanted to make sure that everything is working.

I would appreciate it if you could take a look at my new rosbag. Thank you.

rosbag

[joewong00]

Unfortunately I tried running this on a real robot using the same setup as the above rosbag and it was not able to provide me the position. I made sure that both static and dynamic intialization was done. Everything else was fine except that I was not able to obtain the position, thus no pose was published. What could have gone wrong?

[joewong00]

Hi, is there any updates on this?

[rsasaki0109]

Sorry, I haven't been able to address that yet because I'm busy.