update speed calculation based on STL comparison #482
Conversation
There was a problem hiding this comment.
when
lane = 1
and speed_smoothed < 3
then 3
when
lane = 2
and speed_smoothed < 3
then 3
when
lane = 3
and speed_smoothed < 3
then 3
when
lane = 4
and speed_smoothed < 3
then 3
when
lane = 5
and speed_smoothed < 3
then 3
when
lane = 6
and speed_smoothed < 3
then 3
The above code can be replaced with a single line code
when lane in (1, 2, 3, 4, 5, 6) and speed_smoothed < 3 then 3
There was a problem hiding this comment.
Thank you! This is great. I will revise it.
There was a problem hiding this comment.
If there are lane 7 and 8, I believe that your model will not be applicable, include the condition for lane 7 and 8 also, so that those lanes speed also can be imputed
There was a problem hiding this comment.
Great question! Do you have any idea which route has lane 7 and 8? I will double check and include them.
There was a problem hiding this comment.
Please consider add ymal or update ymal if already exists for this particular model
There was a problem hiding this comment.
Sure! I will update the yaml file accordingly
|
@thehanggit , how did you handle extremly high speed greater than 120 mph?? I can see thousands of speed are extremely high ranges from 120 to 236,00 see the this snowflake work sheet: https://app.snowflake.com/vsb79059/dse_caltrans_pems/w14xEbjZWdd3#query |
Hi, @mmmiah. These extreme values come from occupancy that has extreme value as denominators. For example, 0.001 could easily cause anomalies, which I believe happens a lot in the existing and modernized PeMS system. The g-factor speed model can not handle it intrinsically because it only has the function of smoothing instead of capping. It is very simple to drop them, you just need to set up the lower and upper bounds, like what I did in the final query. The "imputed speed" considers this bound based on existing PeMS speed data. Taking lane 1 speed for SR91 on 10/04/2024, you can see the speed is bounded by [3, 86.5] |
|
@thehanggit , still I can see 589 imputed speed is more than 100 mph. Could you please check again. I will say if any speed exceed more than 100 mph, just replace that with free flow speed. When vehicle occupancy is super small, vehicle will move at free flow speed or more, therefore we can easily fill that with free flow speed. Let me know when your PR is ready for final review! Thank you! |
Again, @mmmiah. The logic is already there. I'm not sure what's happening there but will double check once dbt is running normally. |
Maybe it's due to lane 7 and lane 8 as segments have them. Will incorporate this part into the current logic and see what happens. |
yes, include lane 7 and 8 that will solve the issue as you can see this coming from lane greater than 6 |
Thanks a lot for investigation! I will fix the code once dbt is working and invite you for the review! |
Update the speed calculation based on speed comparison documentation. The current g-factor is detector based with 12 steps smoothing