🍏 🛜 iOS BLE Integration

[the-bay-kay]

Purpose

The purpose of this PR is to incorporate BLE Scanning into the OpenPATH Finite-State Machine, such that trips are only tracked when a registered BLE Beacon is present at the beginning of the trip.

Timeline

• Set up the skeleton for the new FSM Changes (Will include diagram in a later post, once the FSM Changes are finalized)
• Alter the FSM, such that trips begin (e.g., a transition to the "ongoingTrip" state) only occurs when:
  1. The user has exited the established Geofence, and then
  2. There is a beacon within range, shortly after the geofence exit
• Plumb through the BLE Data that will be recorded, such that beacon data is recorded to the database
• Run A-B Tests to determine that only beacon trips are being checked
  • Perform a trip with the beacon in range
  • Perform the same trip without the beacon in range
• Brainstorm additional travel edge cases, which may potentially include:
  • The presence of multiple beacons within range during the trip start
  • Lack of an accurate geofence.
    • E.x.: If a user commutes to work with a fleet car, walks to lunch, and drives elsewhere within another fleet car -- as written, the geofence wouldn't match beacon location)
  • False Positive when testing for beacon within range
    • E.x.: Personal car is parked next to fleet vehicle, what happens if beacon is picked up during this trip?
• Adjust FSM / BLE Scan to address the issues described above.

Further Reading

For information on this PR, please consult the following:

• Determine what bluetooth signals vehicles give off and how they can be read while in the foreground and background e-mission-docs#1046
• Plumb through new "matched bluetooth vehicle" from the phone to the server and to the trip e-mission-docs#1062
• 🤖🛜 Android BLE Integration #219

[the-bay-kay]

Just reached the first design hurdle...

As discussed here, CLBeaconRegion is deprecated (docs). Naturally, we want to instead use the more contemporary CLBeaconIdentity (docs). Unfortunately, this is only availabile in iOS 17.0+.

Because (i) iOS 17.0 was released relatively recently, (ii) We currently support iOS 13.0+ (iii) We cannot reliably expect users to update their phones for OpenPATH, I'm very hesitant to use this moving forward. As I see it, we've got a few choices:

1. Go ahead with CLBeaconIdentity, add some sort of prompt for users to update their OS if need be
2. Continue with CLBeaconRegion, either:
   a. Adding a "Future Work" Tag to eventually update this to CLBeaconIdentity, or
   b. Time permitting, case on the iOS Version to use the appropriate function as needed

My current plan is to begin tests with the older CLBeaconRegion, per my concerns above. @shankari , let me know if you'd like me to proceed another way! I'll be using edu.berkeley.eecs.emission as the identifier for my tests.

[the-bay-kay]

Looking to separate the Bluetooth functionality into a separate function, similar to GeofenceActions. I believe we'll be able to make this confirm to the CLLocationManagerDelegate protocol. Currently looking at this swift tutorial as reference, and digging through the cordova-plugin-ibeacon to see how they handle the wrapper.

[the-bay-kay]

This article from 2017 seems to suggest the following about CLLocationManager...

If you start inside of a region, you will not receive an event until you leave it (and trigger an exit event).

If this is true, this:

1. Contradicts the behavior observed in cordova-plugin-ibeacon (our test results can be found in this thread)
2. Is worrisome, as this would result in edge cases where trips could not be tracked (e.g., a user is presumably already in range of a beacon when they drive away in a fleet vehicle and exit the geofence)

I haven't been able to find any articles / threads corroborating this, will investigate further...

[the-bay-kay]

Looking to separate the Bluetooth functionality into a separate function, similar to GeofenceActions. I believe we'll be able to make this confirm to the CLLocationManagerDelegate protocol.

Probably barking up the wrong tree here, don't think we can do this. If I understand the documentation correctly, we can only have one delegate at a time: and, in the case of TripDiaryStateMachine, we already use TripDiaryDelegate...

Considering the bluetooth scanning is technically an action performed in the TDSM, I'll try adding the scanner to the preexisting delegate instead.

[the-bay-kay]

Currently working out how we're going to check if BLE Scanning has "timed out" (i.e., if we exit a geofence and there is no beacon available, what should we do?)

CLLocationMonitor supposedly has a ~20 second window where it will monitor for an iBeacon Region, but times out shortly therafter. Since there is no notification/callback for a region monitor "timing out" (it seems monitoringDidFailForRegion is not called), I believe we need to manually check the status of the region with didDetermineState. My solution to this is to start monitoring before the transition to kScanningForBLEState , and then check the status of the region monitoring while in that state.

EDIT: maybe I misunderstand CLLocationMonitor's interraction with iBeacon Regions... Does it actually timeout? I remember reading this in a thread, but cannot find it again; this thread also seems to suggest that the monitoring is a more sustainable way to detect iBeacons... I haven't found any info in the documentation... will update further once I have a better understanding of the iBeacon region monitoring behavior.

[shankari]

Considering the bluetooth scanning is technically an action performed in the TDSM, I'll try adding the scanner to the preexisting delegate instead.

Correct. Just to clarify, each instance of location manager can only have one delegate. And since we are going to reuse the location manager between both the location and the bluetooth regions, we must use the same delegate. I was going to clarify this tonight; I'm glad you figured it out yourself 😄

[the-bay-kay]

More of a general design question, for my own understanding -- what is the purpose of including the notifications within TripDiaryActions.m? E.g., this function has the notification line...

+ (void) startTracking:(NSString*) transition withLocationMgr:(CLLocationManager*)locMgr {
    [self startTrackingLocation:locMgr];
    [[NSNotificationCenter defaultCenter] postNotificationName:CFCTransitionNotificationName // <- The line in question...
                                                        object:CFCTransitionTripStarted];
}

But, when called in TripDiaryStateMachine.m, we often immediately send the same notification, after calling this method...

- (void) handleWaitingForTripStart:(NSString*) transition  withUserInfo:(NSDictionary*) userInfo {
    // More code...
    else if ([transition isEqualToString:CFCTransitionVisitEnded]) { 
        if ([ConfigManager instance].ios_use_visit_notifications_for_detection) {
            // We first start tracking and then delete the geofence to make sure that we are always tracking something
            [TripDiaryActions startTracking:transition withLocationMgr:self.locMgr];
            [TripDiaryActions deleteGeofence:self.locMgr];
            [[NSNotificationCenter defaultCenter] postNotificationName:CFCTransitionNotificationName // <- Same notification
                                                                object:CFCTransitionTripStarted];
        }
        // ....
}

I have to assume this was done intentionally -- why not just have the notification fire off once? What is the reason to having it occur twice? I'll follow this design pattern for the BLE integration, but I'm curious as to justification for doing so!

[the-bay-kay]

My current plan is to begin tests with the older CLBeaconRegion, per my concerns above... Let me know if you'd like me to proceed another way! I'll be using edu.berkeley.eecs.emission as the identifier for my tests.

Well, there may be some issues with using "edu.berkeley.eecs.emission" as our identifier. When working with the BK011, the Beacon Name / Identifier has a limit of 17 ASCII characters... For testing sake, I'll shorten this to "nrel.emission". We may be able to set a larger identifier with the iBKS 105, but this is unlikely.

[the-bay-kay]

Testing Setup

Spent ~3 hours running errands today with the following OpenPATH configurations:

• OpenPATH "Release" version
• OpenPATH "Staging" Version
• OpenPATH "Fleet" Version (with iBeacon checks enabled)

Tests Performed

Here are the two tests I performed:

• Driving with the iBeacon, stopping at several locations. For these tests, the Beacon was already on and broadcasting within my car prior to exiting my apartment and starting the trip.
• Driving without the iBeacon, stopping at several locations. For these tests, the Beacon was offline in the car prior to entering exiting my apartment and starting the trip.

Results:

iBeacon Broadcasting:

• Preliminary results suggest that the iBeacon Deployment is successfully tracking within the presence of the beacon. My first set of trips recorded correctly, though there were some issues with one leg of the test, which I will document later.

No Beacon Present:

• I just got back from my test trips 15 minutes ago, and am waiting for the data on the control phones (Staging, Release) to be processed in the server pipeline. Once the control tests are back, I'll compare to the iBeacon phone and report my results.

EDIT: It seems that the Fleet version of the app does not start tracking, even when an iBeacon is present. My initial hope was caused by a false positive, where two phones (Staging & Fleet) were logged into the same opcode. Having addressed this issue, it appears that the tracking is not starting. Unfortunate, but data is data. Calling it a night for now will check the logs and reevaluate tomorrow morning.

[the-bay-kay]

Ran a second round of tests, with the following config:

• iOS Fleet Release
• Android Fleet Release
• iOS Production Release

For these tests, I took two trips as described above. Will report back on the results!

[the-bay-kay]

Second Round of Tests

The data has been processed through the pipeline, with some unexpected results. I'll describe them in the table below. In the interest of not disclosing my address, I won't be including screenshots on the public repo -- feel free to shoot me a message if you'd like more details!

As with past tests, the setup of the "With Beacon" tests and "Without Beacon" tests is as follows:

• With Beacon: The Beacon has been turned on in the car, without the phones in range. I then get the phones from my apartment, and go about my trip as I would normally.
• WIthout Beacon: The beacon is powered off within the car, which I then confirm via the Config App on my Control phone, and then I fetch Fleet phones and travel as I would normally.

Test Results

Deployment Type	To Store, With Beacon	From Store, with Beacon	Neighborhood Loop, No Beacon
iOS Control	✓	✓	✓
iOS Fleet	✓	✗	✗
Android Fleet	✓	✓	✓

✓: Trip was recorded
✗: Trip was not recorded

Analysis

As we can see, the behavior we're recording isn't exactly what we'd expect. From these two (limited) tests, it seems we can draw two conclusions:

1. The iOS Fleet version is successfully recording when a beacon is present, but the recording is not reliable
2. The Android Fleet version appears to be over fitting, and recording all trips.

• Link to the Android Fleet development branch can be found in PR 219.

My plan is to spend some time sifting through the logs to determine exactly what happened during these tests!

[the-bay-kay]

Note: Looking into the logs, it seems I was wrong about the "20-second timeout period" when running CLLocationMonitor's monitoringForRegion. Sifting through the logs, the majority are callbacks from didRangeBeaconsInRegion -- often occurring for far longer than ~20 seconds at a time. We'll have to either:

1. Add a timeout feature, that ends scanning shortly after exiting the geofence
2. Rething how scanning fits into the FSM -- perhaps the consistent detection of a beacon region will be the trip start? Not as enthusiastic about this, but just adding it for the brainstorm.

[the-bay-kay]

Region Monitoring Observations

Well, it may not be the culprit, but I do think I've found an issue with my code...

Current Code

Right now, we're checking the region entrance in TripDiaryDelegate.m as follows:

// After scanning is set up (as above), this will be called when a given beacon is in range
- (void)locationManager:(CLLocationManager *)manager didEnterRegion:(CLRegion *)region {
    // We only monitor the entrance of BLE Regions...
    if([region.identifier compare:OpenPATHBeaconIdentifier] != NSOrderedSame) {
        [[NSNotificationCenter defaultCenter] postNotificationName:CFCTransitionNotificationName
                                              object:CFCTransitionBLEFound];
       return;
    }

Compare this to how the logs display the region:

Beacon Detected

11636,1712334526.1338,2024-04-05T09:28:46.133800-07:00,[BLE] didRangeBeaconsInRegion
11637,1712334526.14286,2024-04-05T09:28:46.142860-07:00,"{
  ""region"": {
    ""typeName"": ""BeaconRegion"",
    ""uuid"": ""426C7565-4368-6172-6D42-6561636F6E73"",
    ""identifier"": ""edu.berkeley.eecs.emission""
  },
  ""eventType"": ""didRangeBeaconsInRegion"",
  ""beacons"": [
    {
      ""minor"": 1972,
      ""rssi"": -64,
      ""major"": 57280,
      ""proximity"": ""ProximityImmediate"",
      ""accuracy"": 0.18,
      ""uuid"": ""426C7565-4368-6172-6D42-6561636F6E73""
    }
  ]
}"

No Beacon Detected

11668,1712334604.71172,2024-04-05T09:30:04.711720-07:00,[BLE] didRangeBeaconsInRegion
11669,1712334604.71852,2024-04-05T09:30:04.718520-07:00,"{
  ""region"": {
    ""typeName"": ""BeaconRegion"",
    ""uuid"": ""426C7565-4368-6172-6D42-6561636F6E73"",
    ""identifier"": ""edu.berkeley.eecs.emission""
  },
  ""eventType"": ""didRangeBeaconsInRegion"",
  ""beacons"": []
}"

Observation

Regardless of whether or not the region was within range, it appears that the region.identifier will be the same. Presumably, didEnterRegion is only called when the corresponding beacon appears within the beacons field. I've got half a mind to modify didRangeBeacons link, and attach the transition to it -- but considering (i) it's deprecated status, and (ii) the fact didEnterRegion should be sufficient, I think I'm barking up the wrong tree.

Next Steps

If I am unable to find an obvious culprit for the tracking issues, I think my next plan is to add some logs to the didEnterRegion function, and attempt to debug from there.

[the-bay-kay]

Standardizing OpenPATH UUID

Just wanted to add a quick note about what UUID's we'll be using! As Jack discussed in this thread, we'll be using the iBeacon signature as follows:

• UUID: Signifies the beacon is part of an OpenPATH Deployment
• MajorID: Signifies which deployment the beacon belongs to
  • For our current project, this is the hex value dfc0
• MinorID: A Unique identifier, corresponding to the specific vehicle

Since we need a unique UUID, Shankari and I decided to generate one with Python as follows:

>>> import uuid
>>> uuid.uuid3(uuid.NAMESPACE_DNS, 'gov.nrel.cims.openpath')
UUID('bf3df3b1-6e46-35fa-86e5-927c95dd096c')
>>> 

Going forward, I will be configuring my test beacons to use the following:

• UUID: bf3df3b1-6e46-35fa-86e5-927c95dd096c
• MajorID: dfc0
• MinorID: [UniqueID]

[the-bay-kay]

iOS Change Brainstorming

Just some notes on the next changes / rounds of tests I'm going to make:

• Adjust BLE Monitoring to begin with the FSA. Originally, I designed it so that the scan would start with the geofence exit, but that appears to cause too much noise. Instead, the workflow will be:
• If BLE detected OR Geofence exit, check for the other: then, if the second change is detected, begin tracking.
  • Rather than having a separate BLE state, I'm thinking we have a "Waiting for trip to start", then "Preliminary Check" state.
  • It'd be nice if they were a single state, but that doesn't feel great -- that is, transition function out of kPreliminaryCheck shouldn't rely on the transition taken from qWaitingForTripToStart -- there should probably be a separate state for (i) 2x Checking Geofence Exit, and (ii) 2x Checking BLE Beacon
• Add logs to the "EnterRegion", so that we have a better understanding of when it occurs relative to the geofence exit

FSA Graphs

This was sketched up mostly for my understanding (so, excuse the chicken-scratch handwriting! 😁 ), but here's the rough plan for the FSA changes, if folks are curious:

[shankari]

@the-bay-kay I think this looks fine at a high level.

Some potential cleanups:

• I don't think that the create geofence and check BLE in range need to happen in serial
• why is there not a create BLE monitoring, parallel to create geofence
• you also need to recreate the geofence and go back to the monitoring state when exiting out of ongoing_trip

Please update this with how it worked when you implemented it!

[the-bay-kay]

So far, not so good -- It appears none of my trips got tracked yesterday... Going to 2xcheck that (i) Our app recognizes the beacon, then (ii) Sift through the logs to figure out why it never started tracking

EDIT:
Good news: The beacon is being detected by the Dev Scanner, and the UUID matches the one I'm scanning for in the Native Code.
Bad News: There's something wrong with my scanner logic... time to dive into the logs!

[the-bay-kay]

Trip Tracking Debug:

Below are the trips I'm using as a control for debugging. The trips themselves are blurred, due to PII: all that matters in this instance are the start / stop timestamps.

Looking at the logs, we can see the following outputs roughly at the same timestamp (3:22 PM, 15:22):

770,1712785886.41506,2024-04-10T14:51:26.415060-07:00,new AppState: background
771,1712787704.05533,2024-04-10T15:21:44.055330-07:00,"In TripDiaryStateMachine, received transition T_BLE_ENTER_REGION in state STATE_WAITING_FOR_TRIP_START"
772,1712787704.07481,2024-04-10T15:21:44.074810-07:00,"In TripDiaryStateMachine, received transition T_BLE_ENTER_REGION in state STATE_WAITING_FOR_TRIP_START"
773,1712787704.07975,2024-04-10T15:21:44.079750-07:00,"In TripDiaryStateMachine, received transition T_BLE_ENTER_REGION in state STATE_WAITING_FOR_TRIP_START"

# The logs continue until...

1060,1712787704.9993,2024-04-10T15:21:44.999300-07:00,"In TripDiaryStateMachine, received transition T_BLE_ENTER_REGION in state STATE_WAITING_FOR_TRIP_START"
1061,1712787847.16503,2024-04-10T15:24:07.165030-07:00,BEMJWTAuth:pluginInitialize singleton -> initialize completion handler
1062,1712787847.17186,2024-04-10T15:24:07.171860-07:00,"intro_done result = {
    ""intro_done"" = ""2024-04-10T14:49:55.536-07:00"";
}"
1063,1712787847.17318,2024-04-10T15:24:07.173180-07:00,"Auth found in local storage, now it should be stable"
1064,1712787847.17429,2024-04-10T15:24:07.174290-07:00,finished init of iOS native code
1065,1712787847.17543,2024-04-10T15:24:07.175430-07:00,truncating obsolete entries before 1710195847.175374
1066,1712787847.18251,2024-04-10T15:24:07.182510-07:00,"reqConsent = 2016-07-14, currConfig.approval_date = 2016-07-14"
1067,1712787847.18352,2024-04-10T15:24:07.183520-07:00,isConsented = YES
1068,1712787847.18575,2024-04-10T15:24:07.185750-07:00,"in getPriorConsent, currConfig = <ConsentConfig: 0x283194380>"
1069,1712787847.18733,2024-04-10T15:24:07.187330-07:00,registered for visit notifications
1070,1712787847.18928,2024-04-10T15:24:07.189280-07:00,initializing TripDiaryStateMachine with state = STATE_WAITING_FOR_TRIP_START
1071,1712787847.1959,2024-04-10T15:24:07.195900-07:00,Returning combination of always = 1 and precise 1
1072,1712787847.20334,2024-04-10T15:24:07.203340-07:00,"All settings valid, nothing to prompt"
1073,1712787847.20455,2024-04-10T15:24:07.204550-07:00,"In valid state STATE_WAITING_FOR_TRIP_START, nothing to do..."

As we can see, the transition is being detected, and the transition notification is being set, but We are not transitioning along that edge! When I exit the geofence later, it appears we properly traverse the edge...

1080,1712787847.26691,2024-04-10T15:24:07.266910-07:00,"In TripDiaryStateMachine, received transition T_EXITED_GEOFENCE in state STATE_WAITING_FOR_TRIP_START"
1081,1712787847.2717,2024-04-10T15:24:07.271700-07:00,"In TripDiaryStateMachine, received transition T_CHECK_BLE_STATUS in state STATE_WAITING_FOR_TRIP_START"
1082,1712787847.27583,2024-04-10T15:24:07.275830-07:00,Moved from STATE_WAITING_FOR_TRIP_START to STATE_CHECKING_BLE
1083,1712787847.27755,2024-04-10T15:24:07.277550-07:00,Returning combination of always = 1 and precise 1
1084,1712787847.28893,2024-04-10T15:24:07.288930-07:00,"All settings valid, nothing to prompt"
1085,1712787847.28998,2024-04-10T15:24:07.289980-07:00,"In valid state STATE_CHECKING_BLE, nothing to do..."
1086,1712787847.29113,2024-04-10T15:24:07.291130-07:00,Current state of region STATIONARY_GEOFENCE_LOCATION is 2 (outside)
1087,1712787847.29218,2024-04-10T15:24:07.292180-07:00,Received OUTSIDE geofence state when currState = STATE_CHECKING_BLE
1088,1712787847.29332,2024-04-10T15:24:07.293320-07:00,"In TripDiaryStateMachine, received transition (null) in state STATE_CHECKING_BLE"
1089,1712787847.29772,2024-04-10T15:24:07.297720-07:00,"In state STATE_CHECKING_BLE, Recieved location <+34.00163387,-118.43585250> +/- 24.00m (speed 0.00 mps / course -1.00) @ 4/10/24, 1:33:59 PM Pacific Daylight Time"
1090,1712787847.30017,2024-04-10T15:24:07.300170-07:00,"Considering region with id = nrel.emission, location 0.000000, 0.000000"
1091,1712787847.3013,2024-04-10T15:24:07.301300-07:00,"Considering region with id = STATIONARY_GEOFENCE_LOCATION, location -118.436062, 34.001434"
1092,1712787847.30258,2024-04-10T15:24:07.302580-07:00,Deleting it!!
1093,1712787847.30393,2024-04-10T15:24:07.303930-07:00,started monitoring for region nrel.emission
1094,1712787847.81224,2024-04-10T15:24:07.812240-07:00,Current state of region STATIONARY_GEOFENCE_LOCATION is 2 (outside)
1095,1712787847.82161,2024-04-10T15:24:07.821610-07:00,Received OUTSIDE geofence state when currState = STATE_CHECKING_BLE
1096,1712787847.82607,2024-04-10T15:24:07.826070-07:00,"In TripDiaryStateMachine, received transition (null) in state STATE_CHECKING_BLE"
1097,1712787847.83647,2024-04-10T15:24:07.836470-07:00,Current state of region nrel.emission is 1 (inside)
1098,1712787847.83903,2024-04-10T15:24:07.839030-07:00,Received INSIDE geofence state when currState = STATE_CHECKING_BLE
1099,1712787847.84153,2024-04-10T15:24:07.841530-07:00,"In TripDiaryStateMachine, received transition (null) in state STATE_CHECKING_BLE"
1100,1712787847.84913,2024-04-10T15:24:07.849130-07:00,Current state of region nrel.emission is 1 (inside)

Shortly after this, at 15:33, we receive the following transitions: curiously, it seems we lap over STATE_ONGOING_TRIP before almost immediately going back to STATE_WAITING_FOR_TRIP_START...

1296,1712788426.64859,2024-04-10T15:33:46.648590-07:00,Received OUTSIDE geofence state when currState = STATE_CHECKING_BLE
1297,1712788426.65375,2024-04-10T15:33:46.653750-07:00,"In TripDiaryStateMachine, received transition (null) in state STATE_CHECKING_BLE"
1298,1712788427.26313,2024-04-10T15:33:47.263130-07:00,"In TripDiaryStateMachine, received transition T_BLE_ENTER_REGION in state STATE_CHECKING_BLE"
1299,1712788427.28084,2024-04-10T15:33:47.280840-07:00,"In TripDiaryStateMachine, received transition T_TRIP_STARTED in state STATE_CHECKING_BLE"
1300,1712788427.29179,2024-04-10T15:33:47.291790-07:00,Moved from STATE_CHECKING_BLE to STATE_ONGOING_TRIP
1301,1712788427.29615,2024-04-10T15:33:47.296150-07:00,Returning combination of always = 1 and precise 1
1302,1712788427.30357,2024-04-10T15:33:47.303570-07:00,"All settings valid, nothing to prompt"
1303,1712788427.30602,2024-04-10T15:33:47.306020-07:00,"In valid state STATE_ONGOING_TRIP, nothing to do..."
1304,1712788427.30818,2024-04-10T15:33:47.308180-07:00,Current state of region nrel.emission is 1 (inside)
1305,1712788427.31028,2024-04-10T15:33:47.310280-07:00,Received INSIDE geofence state when currState = STATE_ONGOING_TRIP
1306,1712788427.31232,2024-04-10T15:33:47.312320-07:00,"In TripDiaryStateMachine, received transition T_END_TRIP_TRACKING in state STATE_ONGOING_TRIP"
1307,1712788427.31659,2024-04-10T15:33:47.316590-07:00,stopped fine location tracking
1308,1712788427.31821,2024-04-10T15:33:47.318210-07:00,"In TripDiaryStateMachine, received transition T_TRIP_ENDED in state STATE_ONGOING_TRIP"
1309,1712788427.32298,2024-04-10T15:33:47.322980-07:00,Moved from STATE_ONGOING_TRIP to STATE_WAITING_FOR_TRIP_START

Next Steps

I think a clear first step is to fix the transition between STATE_WAITING_FOR_TRIP_START and STATE_CHECKING_GEOFENCE, along the T_BLE_ENTER_REGION transition. Will report my results once this is finished and tested!

[the-bay-kay]

Well, that was an embarrassing bug 🤦‍♀️ Turns out, having two checks for the same region would cause some issues....

    } else if ([transition isEqualToString:CFCTransitionEnterBLERegion]) { // <- The Problem!!
        // FLEET: If we detected a BLE Region before 
        // Now, we'll have to check for the GeoFENCE Exit
        [[NSNotificationCenter defaultCenter] postNotificationName:CFCTransitionNotificationName
                                                                object:CFCTransitionEnterBLERegion];

    } else if ([transition isEqualToString:CFCTransitionVisitEnded]) { 
        if ([ConfigManager instance].ios_use_visit_notifications_for_detection) {
            // We first start tracking and then delete the geofence to make sure that we are always tracking something
            [TripDiaryActions startTracking:transition withLocationMgr:self.locMgr];
            [TripDiaryActions deleteGeofence:self.locMgr];
            [[NSNotificationCenter defaultCenter] postNotificationName:CFCTransitionNotificationName
                                                                object:CFCTransitionTripStarted];
        }
    } else if ([transition isEqualToString:CFCTransitionTripStarted]) {
        [self setState:kOngoingTripState withChecks:TRUE];
    } else if ([transition isEqualToString:CFCTransitionEnterBLERegion]) {
        [self setState:kCheckingGeofenceState withChecks:TRUE];
    }

At least it's a quick fix. Recompiling, then I'll go on a few walks (Car's low on gas, don't wanna waste fuel driving circles!) and report my findings.

[the-bay-kay]

Back from a walk with the beacon, waiting for the data to process. In the mean time, to address your questions @shankari :

---

I don't think that the create geofence and check BLE in range need to happen in serial

EDIT I misread that -- I believe you're correct, the two initializations can happen in parallel!

Is that the case? Perhaps I misunderstood -- I thought we broke the iOS tracking down into two cases (discussion here):

1. Keep the existing geofence exit but when we get the EXITED_GEOFENCE event, check fleet deployment or not. If fleet, scan to see if a known bluetooth is nearby and only start tracking in that case. TBD: record the bluetooth scan that we saw so we can map the trip to the vehicle during post-processing
2. Register for iBeacon enter/exit and if we get an enter with a known bluetooth, then:
   a. start tracking location
   b. check to see that there has been sufficient duration or distance or something from previous trip end and then start tracking location

With Method 1, we need this to occur in series -- First EXIT_GEOFENCE, then config check, then finally confirm a BLE beacon is within range.

With Method 2, I think it would also be wise to check for the geofence exit after detecting an iBeacon within range. My justification for this is, I think doing so prevents false-positives when tracking trips.

As an example: Say a cooworker is leaving in a fleet vehicle, and as she drives away, parks near the user to wave goodbye. The beacon may have been ranged, but we don't want track that user's bus ride home! Thus, I figured we should check for the geofence exit while within the BLE region.

Perhaps this sort of circumstance is too niche. Furthermore, Since we plan to continuously range for the beacon during the trip, this will most likely be a moot point. If you think it'll perform better without the second check, I'll go ahead and remove that state!

---

why is there not a create BLE monitoring, parallel to create geofence

That step is missing from the diagram, but already in the code! I'll update my diagram accordingly -- once this is fully working, I'll draw up a new rendition of the FSM.

---

you also need to recreate the geofence and go back to the monitoring state when exiting out of ongoing_trip

Correct! I believe this will occur as it already does -- I shouldn't be touching any of the outgoing edges from ongoing_trip. If we end up removing the geofence check after BLE ranging, I'll be sure to add this step!

[the-bay-kay]

Another aside. When approaching this method of tracking:

Keep the existing geofence exit but when we get the EXITED_GEOFENCE event, check fleet deployment or not. If fleet, scan to see if a known bluetooth is nearby and only start tracking in that case. TBD: record the bluetooth scan that we saw so we can map the trip to the vehicle during post-processing

Within this scenario, what is the condition for returning back to the waitingForTripToStart? Visually...

What should this question-mark edge be? The BLE Monitoring does not have a "timeout" function. Should this state have a timeout attatched, where it waits N Seconds before tranistioning back? Does the geofence get destroyed and recreated?

I'm 90% sure my current solution is wrong. As written, I'm checking for if the user re-enters the goefence. The justification being -- the GeoFence was presumably created near the fleet vehicle being used. If a user instead walks out of the geofence to stretch their legs at lunch, they'll eventually end up back at the office, and re-enter the old geofence.

While I think this makes sense in theory, I don't know how reliable the geofence status is. As currently written, the FSM creates a new geofence under a few different circumstances:

• Right after oneTimeInit(), before the waiting phase
• After a trip end has been detected.
  • Within this section, we also check if the user has remained in the goefence, to tell whether or not it is a "true trip ending"

Are there any situations I'm missing? From what I can tell, there is no point where the geofence would be destroyed without entering the onging_trip state. I think my logic feels sound, but my gut feeling tells me there are some issues with this. Specifically, I'm concerned by these logs from yesterday's tests...

1296,1712788426.64859,2024-04-10T15:33:46.648590-07:00,Received OUTSIDE geofence state when currState = STATE_CHECKING_BLE
1297,1712788426.65375,2024-04-10T15:33:46.653750-07:00,"In TripDiaryStateMachine, received transition (null) in state STATE_CHECKING_BLE"
1298,1712788427.26313,2024-04-10T15:33:47.263130-07:00,"In TripDiaryStateMachine, received transition T_BLE_ENTER_REGION in state STATE_CHECKING_BLE"
1299,1712788427.28084,2024-04-10T15:33:47.280840-07:00,"In TripDiaryStateMachine, received transition T_TRIP_STARTED in state STATE_CHECKING_BLE"
1300,1712788427.29179,2024-04-10T15:33:47.291790-07:00,Moved from STATE_CHECKING_BLE to STATE_ONGOING_TRIP
1301,1712788427.29615,2024-04-10T15:33:47.296150-07:00,Returning combination of always = 1 and precise 1
1302,1712788427.30357,2024-04-10T15:33:47.303570-07:00,"All settings valid, nothing to prompt"
1303,1712788427.30602,2024-04-10T15:33:47.306020-07:00,"In valid state STATE_ONGOING_TRIP, nothing to do..."
1304,1712788427.30818,2024-04-10T15:33:47.308180-07:00,Current state of region nrel.emission is 1 (inside)
1305,1712788427.31028,2024-04-10T15:33:47.310280-07:00,Received INSIDE geofence state when currState = STATE_ONGOING_TRIP
1306,1712788427.31232,2024-04-10T15:33:47.312320-07:00,"In TripDiaryStateMachine, received transition T_END_TRIP_TRACKING in state STATE_ONGOING_TRIP"
1307,1712788427.31659,2024-04-10T15:33:47.316590-07:00,stopped fine location tracking
1308,1712788427.31821,2024-04-10T15:33:47.318210-07:00,"In TripDiaryStateMachine, received transition T_TRIP_ENDED in state STATE_ONGOING_TRIP"
1309,1712788427.32298,2024-04-10T15:33:47.322980-07:00,Moved from STATE_ONGOING_TRIP to STATE_WAITING_FOR_TRIP_START

I need to spend more time piecing apart the logic of this section to better understand what occurred.

[the-bay-kay]

More issues with the current version, my walk did not record. Looking at the logs, it appears I successfully entered the region and transitioned to STATE_CHECKING_GEOFENCE, but did not properly detect the geofence exit:

1414,1712863583.01609,2024-04-11T12:26:23.016090-07:00,"Received geofence exit in state STATE_CHECKING_GEOFENCE, ignoring"
1415,1712863583.01946,2024-04-11T12:26:23.019460-07:00,Current state of region STATIONARY_GEOFENCE_LOCATION is 2 (outside)

When at the STATE_CHECKING_GEOFENCE, I'm handling the check as follows:

/*
 * handleCheckGeofenceStatus is an function for the Fleet Deployment, that checks
 * for a Geofence exit after a BLE Beacon has been detected.
 */
- (void) handleCheckGeofenceStatus:(NSString*) transition withUserInfo:(NSDictionary*) userInfo {
    if ([transition isEqualToString:CFCTransitionExitedGeofence]) {
        [TripDiaryActions deleteGeofence:self.locMgr];
        [[NSNotificationCenter defaultCenter] postNotificationName:CFCTransitionNotificationName
                                                            object:CFCTransitionTripStarted];
    } else if ([transition isEqualToString:CFCTransitionExitBLERegion]) {
        // Left BLE Region without geofence exit, don't start tracking
        [[NSNotificationCenter defaultCenter] postNotificationName:CFCTransitionNotificationName
                                                            object:CFCTransitionFailedToExitGeofence];
    } else if ([transition isEqualToString:CFCTransitionFailedToExitGeofence]) {
        [self setState:kWaitingForTripStartState withChecks:TRUE];
    } else if ([transition isEqualToString:CFCTransitionTripStarted]) {
    [self setState:kOngoingTripState withChecks:TRUE];
    } else if ([transition isEqualToString:CFCTransitionEnterBLERegion]) {
        // If we just entered this state, NOOP to stay here
        [[NSNotificationCenter defaultCenter] postNotificationName:CFCTransitionNotificationName
                                                            object:CFCTransitionNOP];
    } else  {
        NSLog(@"Got unexpected transition %@ in state %@, ignoring",
              transition,
              [TripDiaryStateMachine getStateName:self.currState]);
    }
}

So, my question is: how is it that we exited the geofence, without CFCTransitionExitedGeofence being sent to the NotificatioNcenter??? Need to do more digging.

EDIT

Turns out the answer was pretty obvious. When the geofence exit is checked, we also check the current state, before sending out the notification (link). Will add the checking_Geofence state to this check, and do another lap around the neighborhood. Gonna be pretty fit after all of this testing 😁

[shankari]

@the-bay-kay I think you should draw out the FSM carefully (maybe in a powerpoint) and make sure that you have thought through it carefully. If the FSM is right, the implementation will be right.

[the-bay-kay]

Updated FSM OUTDATED

Below is the latest iteration for the iOS Fleet FSM. This, to the best of my understanding, should give a comprehensive coverage for most edge cases, and should track correctly. Any feedback is greatly appreciated!

[the-bay-kay]

FSM Changes

Shankari and I met this morning to discuss the structure of the FSM. Considering Alpha testing begins in 3 days, and we noticed that the BLE Region status was the deciding factor along all of the steps in the FSM, we opted to remove the Geofence for the time being. After alpha testing is complete, we'll most likely re-incorporate the geofence.

Below is my updated FSM, utilizing only the the BLE Region Status / Ranging Status. Following that are some additional notes on my design:

Some notes on this design:

• I opted to add Double Check if In BLE Region state because I noticed some delay in between ranging and a region enterance. See the following logs as an example:

1390,1712863393.14896,2024-04-11T12:23:13.148960-07:00,[BLE] didDetermineStateForRegion
1391,1712863393.15425,2024-04-11T12:23:13.154250-07:00,"{
  ""region"": {
    ""typeName"": ""BeaconRegion"",
    ""uuid"": ""BF3DF3B1-6E46-35FA-86E5-927C95DD096C"",
    ""identifier"": ""nrel.emission""
  },
  ""eventType"": ""didDetermineStateForRegion"",
  ""state"": ""CLRegionStateInside""
}"
1392,1712863393.16959,2024-04-11T12:23:13.169590-07:00,[BLE] didRangeBeaconsInRegion
1393,1712863393.17201,2024-04-11T12:23:13.172010-07:00,"{
  ""region"": {
    ""typeName"": ""BeaconRegion"",
    ""uuid"": ""BF3DF3B1-6E46-35FA-86E5-927C95DD096C"",
    ""identifier"": ""edu.berkeley.eecs.emission""
  },
  ""eventType"": ""didRangeBeaconsInRegion"",
  ""beacons"": []
}"
1394,1712863394.24039,2024-04-11T12:23:14.240390-07:00,[BLE] didRangeBeaconsInRegion
1395,1712863394.24869,2024-04-11T12:23:14.248690-07:00,"{
  ""region"": {
    ""typeName"": ""BeaconRegion"",
    ""uuid"": ""BF3DF3B1-6E46-35FA-86E5-927C95DD096C"",
    ""identifier"": ""edu.berkeley.eecs.emission""
  },
  ""eventType"": ""didRangeBeaconsInRegion"",
  ""beacons"": [
    {
      ""minor"": 1972,
      ""rssi"": -67,
      ""major"": 57280,
      ""proximity"": ""ProximityImmediate"",
      ""accuracy"": 0.22,
      ""uuid"": ""BF3DF3B1-6E46-35FA-86E5-927C95DD096C""
    }
  ]
}"

• While I believe the trip end detected is still important, I'm still unsure as to whether or not we want to include this. Looking at all of the potential choices that could be made at this psuedo-state, it appears that we will effectively make the same choices that we would have during the Double Check if in BLE Region state.
  

While these two feel incredibly similar, I do believe the detectTripEnd is still relevant, so I'll include it in my design going forward. @shankari , pinging you again to confirm this looks good for alpha testing!

[shankari]

@the-bay-kay for this initial implementation, I think this is way too complicated. You should use only beacon ranging as the trigger out of ongoing_trip as well. If we get a CLOutside callback, we should stop tracking. Using the location tracking as a backup is a future fix.

Can you clarify why you need Double Check if In BLE Region? I would just generate the BLE_Found or similar transition after I have received at least a few range messages. See e-mission/e-mission-phone#1144 (comment)

[the-bay-kay]

@shankari I'm somewhat confused -- In this version, I am using the beacon ranging as the trigger out of ongoing_trip. I'll go ahead and cut the Double Check If In BLE Region, and cut the CheckIfTripEnded.

I added the Double Check, because I noticed some noise within the signal during my trip logs (where the beacon would not be detected as 'in range' for the first few occurrences, between the end of monitoring and the start of ranging. I don't know how I would transition "after a few range messages", do you have a suggestion for how to tally them? For the time being, this is my latest iteration of the FSM. LMK if I understood your edits correctly

[the-bay-kay]

iOS Updates, Future Work

I was able to install the latest version of the BLE build -- there are still several issues to work out, but we are now successfully only tracking within the presence of a BLE Beacon [Image 1]

Because:

• We have a tight deadline of alpha testing beginning tomorrow
• While we are not tracking all trips within BLE range, we are only tracking trips within range

I believe the best course of action is to implement the following remaining features, before debugging the FSM Changes further:

• Add Bluetooth permissions to the initial permissions pop-up
• Merge & Integrate the data storage from PR 221

Once these two pieces have been integrated, I plan to:

• Go through the logs to diagnose why the trip below tracked, but others have not
• Implement changes to amend errors found

Proof of trip tracking:

[shankari]

@the-bay-kay I have pulled these changes and simplified it. Concretely, you have a bunch of code which had commented out the non-fleet version. We cannot push a version to production with those changes (it will break everybody else), and we cannot really have the GSA folks all use staging builds.

The non-fleet version needs to stay production quality - only the fleet version can be alpha quality. I don't have a beacon, so let us please allocate some time today for testing.

[the-bay-kay]

@shankari Understood -- I forgot to add it to my initial post, but adding the config check was on my list of changes to make this evening. Almost done incorporating the BLE permission check in the popup -- I'll link to the PR once I've confirmed it's working.

Once that is done, I'll pivot to the config check. I'll allocate some time to drive around this evening for tests as well, once the pop-up is done. Let me know if there are any other components I'm missing.

[shankari]

@the-bay-kay let me clarify - I am done with the config checks. I am adding code to save the entries, and after testing, I am going to push that to staging for people with actual BLE beacons to test. Will then pull the last UI changes and hopefully send it off to production tonight/early tomorrow.

[the-bay-kay]

done with the config checks. I am adding code to save the entries, and after testing, I am going to push that to staging for people with actual BLE beacon

Ah, I misunderstood -- When those changes are ready, LMK which branch to pull from, so my permissions changes are up to date with that.

After I am done with this, what would you like me to prioritize as to not double up on work -- debugging and re-testing the beacon tracking?

[shankari]

yes, testing. I don't have a beacon, and we are going to push this to production with very little time on staging. I would really like thorough testing of both the fleet and non-fleet versions. You could start with android for now, where my changes are done; I can test with simulating transitions, but I can't easily invoke the BluetoothService in the way that the actual beacon/library combination does.

EDIT: Note that you don't actually have to go on a trip for the first round of testing. You should be able to start and stop trips from the developer zone. You can start a trip from there and then move the beacon near and far and see what happens in the device logs (adb logcat or xcode logs)

[shankari]

@the-bay-kay as you can see, I am able to push changes to your branch directly. Once I am done, I will merge this to integrate_ble and you can push your authentication changes there as well

[shankari]

Simulated processed trips	Simulated draft trips	Error with the current showIf on draft trips

DFC fermata with fleet mode on init	DFC fermata with fleet mode on non-init	DFC Fermata (checked in version) with no fleet mode on init

[shankari]

@the-bay-kay I don't have a beacon, so I can't test this. What is the type of object that we receive in didExitRegion and didEnterRegion? We want to copy values out so that we can save them to the database? It looks like it used to be CLBeaconRegion but that is now deprecated.

EDIT: It looks like it is in fact CLBeaconRegion (by looking at the code in the delegate). Not sure that has been tested, though...

[the-bay-kay]

It looks like it is in fact CLBeaconRegion (by looking at the code in the delegate). Not sure that has been tested, though...

Correct -- we are using CLBeaconRegion, despite the fact it is deprecated, because the contemporary CLBeaconIdentityCondition is only available for iOS 17.0+ (docs). (I wrote more about this at the beginning of the PR here)

I'm somewhat confused though -- when you ask "has it been tested", what do you mean? This API is the same as the one used in cordova-ibeacon, which we have tested and confirmed works.

[shankari]

I'm somewhat confused though -- when you ask "has it been tested", what do you mean? This API is the same as the one used in cordova-ibeacon, which we have tested and confirmed works.

I meant, have we added a breakpoint here and confirmed that the value passed in is a CLBeaconRegion? I was looking at the implementation for "exit" and before my change, we weren't casting the value, or trying to use it in any way. So I wasn't sure what would happen if I cast it to (CLBeaconRegion*) and tried to access the fields from it

[shankari]

@the-bay-kay I am almost done with my changes, but have not been able to test the native callbacks (e.g. the code in TripDiaryDelegate since I don't have a real BLE beacon. Could you please get on Teams or call me so we can coordinate that testing before I push this to staging?

[shankari]

Merging this to clear the decks to start building for staging

[shankari]

future fix: should see what happens if the phone bluetooth is turned off. We may need to add a check for that, and remind the user to turn it on, at least in the fleet case

[shankari]

Future fix: note TODO here

[shankari]

don't you need to check for the UUID as well here?

[the-bay-kay]

I believe we do not -- as I understand CLBeacon, the (CLBeaconIdentityConstraint *) within the function signature will be the constraint set when we begin ranging for beacons. That is, when we call startRangingBeaconSatisfyingConstraint on line 341, we will only receive notifications for beacons that match that constraint.