Protect community from confusable homoglyphs

[MartinDelille]

This PR aims to fix #1469 by protecting the creation of confusable name for community.

I learned how to use the library during the implementation but there is something I don't know very well how to do.

The documentation says that we have to update the data file by executing the two following commands:

pip install confusable_homoglyphs[cli]
confusable_homoglyphs update

I suppose the call the confusable_homoglyphs update has to be added to the Makefile during the make env step, am I right?

[mattbk]

How often do they need to be updated? Could this be done in cron, or in the payday script?

I'm not certain make env gets run very often in production unless needed by code changes.

[MartinDelille]

I think updating it every time the site is updated would be fair enough. The script update if from this page: http://ftp.unicode.org/Public/security/latest/confusables.txt

Last update was november 2018.

[Changaco]

This is a very inefficient implementation, the more communities there are the slower it becomes. Also you're repeating an identical function call (cls._unconfusable(name)) in every loop iteration.

[MartinDelille]

Fixed here: 9b496bc

[Changaco]

You only fixed the repeated call, not the wider issue: this implementation is still inefficient.

[MartinDelille]

Which part is the most inefficient:

• the database query?
• the unconfusable lookup?
• the unconfusable_string function?

[Changaco]

The fetching of all the existing community names from the database and the subsequent loop. That's linear complexity, O(n), with n being the number of existing communities.

[Changaco]

We should probably do something like this instead:

                possible_collisions = get_confusables(name)
                possible_collisions.append(name)
                collides_with = cursor.one("""
                    SELECT name
                      FROM communities
                     WHERE name IN %s
                     LIMIT 1
                """, (tuple(possible_collisions),))
                if collides_with:
                    raise CommunityAlreadyExists(collides_with)

[Changaco]

I'm not certain make env gets run very often in production unless needed by code changes.

make is not used at all in production.

[Changaco]

We should probably have a copy of the confusables.txt file in the repo so that we can fall back to it if fetching the current file from the ftp.unicode.org server fails.

[Changaco]

Strike that, the confusable_homoglyphs package already ships with a copy of the data transformed into JSON, so we don't need to deal with fetching the confusables.txt file at all.

[MartinDelille]

Ok I removed the update in the make env step here: 64033bd

[Changaco]

You only fixed the repeated call, not the wider issue: this implementation is still inefficient.

[Changaco]

It would be nice to have a docstring explaining what this function does.

[MartinDelille]

Done here: a73af92

Tell me if there is a specific format for that.

[Changaco]

The fetching of all the existing community names from the database and the subsequent loop. That's linear complexity, O(n), with n being the number of existing communities.

[Changaco]

That's a comment, not a docstring.