Middle "screen" tier between permit and block?

[roycewilliams]

When using broad regular expressions (such as when blocking entire area codes, etc.), I'm not aware of a good way today for a more-specific number to be neither permitted nor blocked, but simply screened. It's almost like there needs to be a "middle tier" for screening.

I'd be willing to tackle an implementation, and first seek consensus on the approach. Worth pursuing? Considerations / ideas? Obviously, precedence (order of enforcement) and associated UX would need to be worked out.

[thess]

It sounds like raw Permit and Block lists along with RegEx Permit and Block patterns are not sufficient. As it stands now, callers that do not fall into one these categories (or denied by a block service lookup) are screened by default. Are you feeling the need to cover exceptions like "Block all numbers by pattern" except for those on another list or RegEx? Same goes for permitted RegEx, but screen exceptions in any event. After all mechanisms are exercised, and the call is neither permitted nor blocked - it is screened. What else would happen. I'd like to also keep the self-permitting logic intact.

It seems if we were to go down this path, perhaps we should look at some other rules-based screening which includes RegEx handling and leave the explicit Permit and Block lists intact or incorporate them in the rules logic somehow. I suspect we could make this arbitrarily complex with not too much work. What would the UI look like?

More thought and prototyping required.

[SteveInTx]

I would agree with the need to better define "Screened". From my perspective I would like to pass all calls that are not found in the 3 current criteria: Permitted, Blocked, or RegEx. My system today screens calls and if they are not found in the coded criteria, the system marks them as screened and hangs up on them. My RegEx eliminates by pattern most of the robocalls etc. I would like a config line directing calls that fail to meet any coded criteria. Option A: Pass through to the phone. Option B: Hang up. That should satisfy every option.

[roycewilliams]

Exactly.

One specific use case I'm thinking of is "a regex blocks or allows all calls from this area code, but I want to explicitly screen calls from this specific number."

[whitedavidp]

Royce, the more I think about this scenario, the more confused I become. I am sure you have good reasons. But can you explain why you would want to keep a specific number in a perpetually screened state? Perhaps it might help me to understand if I see the following line from your config:

SCREENED_ACTIONS = ("answer", "greeting", "voice_mail")

Best...

[b-morgan]

Regular expressions can get pretty complex so the example of block an area code except for this number should be possible.

As long as the regex lists are tested in a pre-determined order a third pair of regex lists could be added for screened names and screened numbers which might make things easier to write.

[whitedavidp]

I also think this is true. However, I think that such regex's would be beyond the skill of many/most users (including myself) to achieve this.

Does there need to specifically be an "always screened list" as there is currently a "always permit list" or "always block list"? I admit that I am now not quite sure of the precedence of the application of the permit, block, and regex lists.

Ok, I checked the code <<<

In app.py's run loop here is what happens in sequence (checking stops with the 1st true condition encountered):

1. Permit next is checked.
2. If whitelist is in the screening mode, permitted lists are checked. In callscreener.py here is what happens in sequence:
   2a. The permitted numbers in the database are checked
   2b. The permitted names patterns are checked
   2c. The permitted numbers patterns are checked
3. If blacklist is in the screening mode, blocked lists are checked. in callscreener.py here is what happens in sequence:
   3a. The blocked numbers in the database are checked
   3b. The blocked names patterns are checked
   3c. The blocked numbers patterns are checked
4. If none of the above, the call is screened

Ok, I checked the code <<<

I also wonder how frequent this seemingly rare use case might be?

[roycewilliams]

Fair question. The overall idea is that some kinds of calls can be reviewed later without having to get my attention to decide whether or not I want to take the call right that minute. Some scenarios, some of which are permanent, but others would just be "set until no longer needed"):

• Known automatic calls from doctor's offices reminding you about an upcoming appointment
• An acquaintance or customer who will talk to you for an hour if they get you "live", but will leave a much more compact message :D
• A persistent recurring caller that's been blocked, that you want to test whether or not they are "legit enough" to leave a message
• [Edit]: Unless a call is explicitly permitted, screen all calls from a specific area code

[whitedavidp]

For all of these (which I also face), I would think that one would want those numbers to be permitted. However, there are certainly times I do NOT want to pick up or speak to permitted callers and let them simply ring into your voice mail for later attention. And it can be hard to ascertain who the caller actually is without looking at a caller-id enabled handset (of which I have one and am never close to it - also the caller id info from the utility can be very unhelpful - as in the case of most incoming cell phone calls where they are simply "WIRELESS CALLER" all the time) or at CA's desktop. So here is what I do:

1. I enable MQTT and have mosquito running on the same box as CA. It turns out this is not hard to do. In addition to adding mosquito, you do have to add a package so CA can do MQTT.
2. I then enable mechanisms on all of my computers and cell phones to monitor the topic "callattendant/CallerID" which was recently added (in 2.0.4, I think). These mechanisms ignore all but Action = 'Permitted' messages. And in those cases, they flash/pop up the caller name/number from the MQTT payload on the screen (which contain name info from CA, if it exists, rather than whatever the utility provides). In this manner, for example, when the verbose customer is calling, I can easily see that is the case and decide to simply NOT pick up - letting it ring into voice mail.
3. On Android, I use Tasker (along with a MQTT plug in) for the mechanism. On Windows, I use EventGhost (also with an MQTT plug in). I am not an Apple fan-boy but I would suppose that there are similar mechanisms to be had on those platforms. I also have not pursued this on Linux but I am sure there would be a way to do it there as well.
4. All of this happens in very near real time. So the flash/pop up happens in sync with the first ring.

I would be happy to share my work that was enabled by Ted. And mine is restricted to use on my LAN (although I am sure that it could be otherwise) as that is all I need.

Or perhaps I am befuddled and missing the point...

Best

[roycewilliams]

Thinking out loud ... a key to both resolving this, and also enabling much more sophisticated flexibility, might be to think about how making the enforcement hierarchy more abstract, and then cherry-pick how best to get there if it's worth it.

For example, we could expand each "layer" of check into a separate configurable layer, and then allow the user to configure both the order of enforcement (with the default config identical to today's behavior) and the action taken.

Today, these layers are:

• permitted
• blocked
• regex blocked by name
• regex blocked by number
• regex permitted by name
• regex permitted by number

There is an enforcement order that is partially explicit (SCREENING_MODE), and partially implicit (depending on order of how things are checked, order of regular expressions in the list, etc.)

There is also a mapping of behaviors to these queues (BLOCKED_ACTIONS, SCREENED_ACTIONS, PERMITTED_ACTIONS).

I'm not suggesting that we have to over-abstract / YAGNI this. But I think it may be useful to document the full order of enforcement/action to inform how best to address this class of problem. A little bit of abstraction could yield some design tweaks that both "make the simple things easy, and the difficult things possible".

[b-morgan]

After more thought, I think the only list that might need to be added would be a screened number list added between permitted and blocked. Maybe add a "greylist" to the screening mode to control its use. Screened regex could be added for completeness but I think they would be mostly unused.

I am also in favor of documentation!

[whitedavidp]

I agree that having a screened list (and possibly regex) would "solve" this particular conundrum (and possibly others)...
But to put any specific number on any list certainly seems to imply that the number is a "known" number to the user of CA.

But why one would want to perpetually keep a number being screened is still not quite clear to me. If the number is indeed someone who you know and MIGHT - at some point - want to speak with, why not add that number to the permitted list and, when a call comes in from that number, simply not pick up the call - letting it ring into voice mail (either in CA or the utility's system)? In this case, the key is knowing who the caller is in at least semi-real-time and where ever you are that is connected, and then using that info to decide to actually pick up the phone (or not). If you are not around the phone or not actually connected to CA (directly or via MQTT), the permitted number should ring into voice mail eventually anyhow - which seems to be what Royce wants to happen in the cases he mentions.

But maybe I just mis-understand.

[roycewilliams]

I think you're tracking. :D One may not want to perpetually screen a specific individual number, but one might want to perpetually screen all calls matching certain criteria, in a way not possible with today's order of precedence.

[whitedavidp]

Yeah, I am starting to get it. Best

[whitedavidp]

Perhaps related to this is the question of having screened calls categorized as permitted... From all I can tell, calls that are screened eventually get thrown in with calls that are permitted. Yes, they are tagged as screened. I just find that confusing and I think it ends up obfuscating just how many calls are screened - which, in our case - is huge. I think they are much more logically put in the blocked category, if not in a category of their own. Best

[whitedavidp]

To be clear here, I am speaking about the following code found in webapp.py:

sql = """SELECT COUNT(Number), Number,
       CASE
           WHEN b.PhoneNo is not null then b.Name
           WHEN c.PhoneNo is not null then c.Name
           ELSE a.Name
       END Name
       FROM CallLog AS a
       LEFT JOIN Whitelist AS b ON a.Number = b.PhoneNo
       LEFT JOIN Blacklist AS c ON a.Number = c.PhoneNo
       WHERE Action IN ('Permitted', 'Screened')
       GROUP BY Number
       ORDER BY COUNT(Number) DESC LIMIT 10"""
   g.cur.execute(sql)
   result_set = g.cur.fetchall()
   top_permitted = []
   for row in result_set:
       top_permitted.append(dict(
           count=row[0],
           phone_no=format_phone_no(row[1], current_app.config["MASTER_CONFIG"]),
           name=row[2]))