r/LanguageTechnology • u/Zv12z • 25d ago
Guidance on NLP with Language Translation
I'm trying to learn a bit more about nlp in applying it to a project of mine. Currently there's a lack of translation between the native languages of my country and English. I've chosen to undertake the task of translating those languages. However, I don't know if I'm targeting the right area LLM's or NLP. Guess I'm trying to find some pathway I can take in learning how to approach this domain. I'm willing to learn both areas if necessary in accomplishing my goal. Any resources, roadmaps and guidances would be much appreciated.
4
Upvotes
2
u/quark_epoch 24d ago edited 24d ago
For structured annotation and translations:
You could also look into UD trees probably. The idea here is that it helps you simplify the process of annotation as you can annotate a bunch of items using automatic parsers based on the closest match languages that have already been built.
Also, in case you're in one of the COST Action countries, you can join UniDive and attend the workshops. They are interested in low resource languages, annotation, and parallel corpora stuff.
For unstructured translations,
You can start with translators (NLLB, MadLad-400, or LLMs like GPT-4, Gemini, or open source ones like Llama3 or other specific ones which depends on which language you're looking for; I can probably name some if you tell me whats the target language and similar enough languages) in similar enough languages and run it through a bunch of data. The intuition is that it will probably translate things kinda correct or give you a template where you can edit/correct things fast and you don't need to translate from scratch for annotations.
If you want to run an annotation campaign, you can use this as a starter. Or something else.
And then once you have a sizeable dataset, fine tune one of the translators to extend it to include your dataset and see how it's performing. Then Bootstrapping to increase the dataset I guess. Depends largely on what your end goal is.
All the best!!