Localized emoji names

[12people]

Right now, the current emoji search only searches by emoji name, which seems to be hard-coded to the English variant only.

It'd be great if emoji names could be localized.

This would be useful not just for search, but also for accessibility, letting people show tooltips over emoji in the emoji picker to clarify what the emoji is called.

[Fintasys]

Currently there is no direct support for multiple languages, but you can still achieve it by creating your own custom emoji sets (default_emoji_set.dart) and set it in your config depending on the language you need (Config(emojiSet: ...))

[12people]

Is there a reason why there's no direct support for languages other than English?
Having it in the plug-in itself would be useful to a number of multilingual apps, I'm sure, without each of those apps having to manually translate the emoji names.

[Fintasys]

Open source unfortunately doesn't work as easy as this. Of course it makes sense, but I personally don't have the need and currently the time to work on that feature. If you or anyone is willing to spend some work on supporting this feature I'm happy to review.
Another aspect is that everyone has different requirements and some people prefer a small size library because they don't need language support.

And as described above, currently it's possible to support it if you really need it. I appreciate your request, so I can learn what needs the users have and sometimes I find time and motivation to address those, otherwise I'm happy to see contributions 😬

[12people]

I think the bulk of this feature is primarily the translation work, right? (If I understand you correctly, the code support for a basic implementation is there through the custom emoji sets?)

As for how to do the translation, I would suggest setting up a free Weblate project and letting volunteers contribute their translations. To qualify for their free "libre tier", the project must meet these conditions:

Libre project with open-source code

• Translated content has to be released under a license approved by OSI or recognized as libre by FSF.
• All the source code has to be libre-licensed and publicly available in a supported version control system.
• Translations have to be in the same repository as the source code, or under the same project/organization as the source code repository.
• Non-code projects are welcome as well.

Project approval

• Using Weblate has been approved by the upstream project.
• Mention you use Weblate for translations in the README. It’ll attract new translators and also help Weblate to be free for more projects.

Established projects

• The project receives a reasonable number of contributions.
• The project has had active development for at least three months.

Your trial

• You will have 14 days to set up your project.
• Please request project approval once you finish the setup.
• Unapproved projects will be removed.
• We reserve our right to reject inappropriate content (erotic, gambling, etc.).

If I have official permission from the project, I can set this up under my own account, but I'm guessing you might prefer to set this up under your account to have better control over the project?

[Fintasys]

Hey @12people
I checked it out and created an account. But seems the file needs to be brought into a readable format before we could use the service for translations. For best performance I left it in the current format, I'm worried if we read the emoji's from a json or arb file it would affect the loading speed. Any suggestions?

[12people]

Hey @Fintasys
If you use ARB files along with Flutter's built-in code generation, then I don't think it should affect performance. Flutter generates Dart files from ARB files, so all Strings are compiled along with the rest of the code, no files are read while running the code.

[Fintasys]

@12people I created a project, but I can't find time working on some arb format that then can be used with the library and also making sure that it doesn't affect performance. If you are able to come up with a PR that helps to setup weblate that would be great, unfortunately the project I created is determined to deleted in 5 days due to inactivity 😅 Maybe it's possible to create a new one after that, but not sure. I'm sorry that I couldn't make any progress so far 😮‍💨

[12people]

No problem. I'll take a look at it when I have a bit more time. I think you can reapply after the project is closed (and if not, there are also other translation services that have free tiers for open-source projects).

[emavgl]

Hi, thank you for your work on this package. I am also working on an app which supports multiple languages, and searching emoji by localized name seems to me an important feature to have. I don't think however that a crowd-sourcing solution to fill all the emoji names in different languages would be easy, fast to achieve.

I saw on Github many projects which had this issue already. Check this comment tom-james-watson/Emote#65 (comment)

Check here for example: https://github.com/Armaldio/localize-emoji-db/tree/master/dist/emojis

So in my opinion it would make sense to provide a new parameter Locale and choose the proper set of emoji from there.

[Fintasys]

I think the easiest solution might be to just create translated files for the emoji-lists like I drafted in this PR (used GeminiPro to translate the values)
Then users can set the config file for their language in the Config settings. This will impact the size of the package (around 64kb per language) but won't affect the performance.
The project above includes multiple descriptions per emoji which we are currently not supporting in this package and I'm not sure if there is even an official standard for those descriptions.
I wonder if we use arb files and combine them into categories during runtime, it might affected the performance.
Would like to hear your opinions 🙏

[emavgl]

Translating the emoji-lists using LLM seems also good, but I don't believe it is a long term solution for the project:

• You would need to translate the same file in a lot of locales (you can also stop at the most common ~ 10)
• There could be errors
• The list of emoji may get updated, and translations needs to be updated too

Personally, I believe that fetching emoji and translations from some official source is the only way to keep the emoji list and translations up-to-date.

There is an official list of emoji and annotations https://github.com/unicode-org/cldr/tree/main/common/annotations

[Fintasys]

That's true, but this solution requires more effort. Nevertheless, I agree that it would be nice to have some automation and official data behind all of this.

Steps to achieve above goal would be

1. Allow multiple descriptions (actually like the way the official source is doing it ... | ... | ... . We could use this and don't need to make changes to the current structure.

2. Create automation to iterate through each emoji and find it's description in the official source. Then create emoji-set based on the data.

3. Migration to arb files I think is optional at this point. emoji-sets should be just fine to generate.

Let me see if can create some automation easily with help of AI 💪

[emavgl]

1. Yes, that seems strange indeed. Also using some upper arrows. I believe it is some sort of convention to avoid repetitions between translations file, for example, if a translation is the same between Austria de_AT and Germany for example, it indicates to take the translations from de instead. But I don't know the detail of it.

The one above seems a good plan to me.

Unfortunately, I can't contribute directly to the project, as I don't have much time available. Also, the project is yours and you can decide of course the direction it has to take :)

I am nevertheless grateful for your work 🙏

[Fintasys]

I had some time right now and came up with a script that seems to work.
One bummer although is that flags for some reason are not part of the language files, any idea why? I couldn't find any explanation or alternative source.

[emavgl]

I did not find anything official, but ChatGPT says that for flags no annotations are provided and it is the platform / application itself to assign some annotation to them. I would suggest to keep the English annotations only for those flags, and maybe, adding the locale status code. It should be possible in my opinion to have the flag Austria when typing AT or Austria.

Most likely the reason is to avoid some political controversial issue.

[Fintasys]

I see, make sense.
Also used existing descriptions as fallback when there is no translation. If you are curious, you can take a look at my script and the generated locale files.
#223

[Fintasys]

Released it as dev version for now, feel free to test 🙏
https://pub.dev/packages/emoji_picker_flutter/versions/4.0.0-dev.2

[emavgl]

Thanks @Fintasys, I'll test it over the weekend! Have a nice day

[emavgl]

I tried and it works fine :)

[Fintasys]

Thanks for testing, appreciate your support 🙏