Normalization of microtext

[salonipriyani]

Is your feature request related to a problem? Please describe.
The text from the user could be erroneous (due to typos or spelling mistakes) or have abbreviated words (BLR for Bangalore Airport).

Describe the solution you'd like
First, using a lexical approach, the abbreviations and acronyms will be handled and then using a phonetic algorithm, Soundex, the spelling mistakes will be corrected.

Describe alternatives you've considered

Additional context

[vishakha-lall]

@salonipriyani What is the update on this?

[Rukmini-Meda]

@vishakha-lall I would like to work on this issue. Can you please assign this to me as part of GSSoC'20?

[vishakha-lall]

@Rukmini-Meda Please share your approach for this issue, what kind of normalisation would you be doing, where will you get the data to train the normalisation on?

[shreyanshi2228]

Is it like gathering data for some common abbreviations?

[shreyanshi2228]

https://github.com/npuliyang/Microtext_Normalization
Will it work?

[vishakha-lall]

@shreyanshi2228 Please elaborate on how you plan to use the shared resource with respect to the requirements of this project. Abbreviations to common cities and locations would be more relevant to this project.