Translation is common among the applications of Natural Language Processing and machine learning. Though due to advancements in technology mainly in the pre-trained models, and transformers, it becomes easier to create a suitable language translation tool.

Here in this article, we will create the language translation application from the pre-trained model available in Hugging Face Transformers.

Understanding Language Translation Models

Pre-trained translation models are advanced machine learning models that have been trained on extensive multilingual datasets to perform translation tasks. These models leverage large amounts of text data from multiple languages to learn the nuances of translating text from one language to another. The primary advantage of using pre-trained models is their ability to perform translation tasks out-of-the-box without requiring extensive training from scratch on specific datasets.

Popular Pre-trained Models

1. MarianMT

MarianMT is a collection of pre-trained translation models that support a wide range of language pairs. Developed by the Microsoft Translator team, MarianMT models are based on the Marian neural machine translation architecture, which is efficient and designed for high-quality translation across numerous languages.

2. BERT (Bidirectional Encoder Representations from Transformers):

While BERT itself is not a translation model, it has been used as a component in translation systems to improve contextual understanding. BERT’s bidirectional training helps models grasp the context of words in a sentence, which is crucial for accurate translation. It is often used in conjunction with other models to enhance performance.

3. GPT-3 (Generative Pre-trained Transformer 3):

GPT-3 is a language model developed by OpenAI that can generate coherent and contextually relevant text. Although its primary use is not translation, GPT-3’s large-scale pre-training on diverse text data allows it to perform translation tasks effectively, especially when fine-tuned for specific languages.

How Pre-Trained Models Work?

Pre-trained translation models generally use the Transformer architecture, which has become the standard for many state-of-the-art natural language processing tasks.

The Transformer architecture consists of the following key components:

1. Encoder-Decoder Structure:

• Encoder: The encoder processes the input text (source language) and converts it into a set of continuous representations, capturing the semantic meaning of the input.
• Decoder: The decoder takes the encoder’s output and generates the translated text (target language) by producing a sequence of tokens.

2. Self-Attention Mechanism

Self-attention allows the model to weigh the importance of different words in a sentence when encoding or decoding. This mechanism helps the model focus on relevant parts of the input text while generating the translation.

3. Positional Encoding

Since Transformers do not have a built-in sense of order like recurrent neural networks, positional encoding is used to provide information about the position of each word in the sequence, ensuring that the model understands the order of words.

Process of Translation

1. Tokenization: The input text is divided into smaller units called tokens. Tokenization helps the model handle variable-length text and convert it into a format suitable for processing.
2. Encoding: The tokenized input is processed by the encoder, which converts the tokens into embeddings—a form of numerical representation capturing the meaning of each token. The encoder’s self-attention mechanism processes these embeddings to create contextual representations.
3. Decoding: The decoder uses the contextual representations generated by the encoder to produce the translated text. It generates tokens one by one, using previously generated tokens and the encoder’s output to create coherent and contextually accurate translations.

English to Hindi Translator using Pre-trained Translation Model

Step 1: Import Necessary Libraries

The first step involves importing the libraries required for translation and building the interactive interface. In this case, we need the transformers library for the translation model and gradio for creating a web interface.

You can install the gradio library using the following code:

!pip install gradio 

from transformers import MarianMTModel, MarianTokenizer
import gradio as gr

Step 2: Define the Model Name

Specify the pre-trained translation model you want to use. Here, we are using the Helsinki-NLP/opus-mt-en-hi model, which translates from English to Hindi.

# Define the model name
model_name = 'Helsinki-NLP/opus-mt-en-hi'

Step 3: Load the Tokenizer and Model

Load the tokenizer and model from the pre-trained model specified in the previous step. The tokenizer converts text into a format suitable for the model, while the model performs the actual translation.

# Load the tokenizer and model
tokenizer = MarianTokenizer.from_pretrained(model_name)
model = MarianMTModel.from_pretrained(model_name)

Step 4: Define the Translation Function

Create a function to handle the translation process. This function will tokenize the input text, generate the translation using the model, and decode the translated text.

def translate(text):
    # Tokenize the input text
    tokenized_text = tokenizer.prepare_seq2seq_batch(, return_tensors='pt')
    
    # Perform the translation
    translation = model.generate(**tokenized_text)
    
    # Decode the translated text
    translated_text = tokenizer.decode(translation[0], skip_special_tokens=True)
    
    return translated_text

Step 5: Create a Gradio Interface

Set up an interactive web interface using Gradio. This allows users to input English text and receive Hindi translations. The gr.Interface function creates the user interface with text input and output fields.

# Create a Gradio interface
iface = gr.Interface(
    fn=translate, 
    inputs=gr.Textbox(label="Enter text to translate"), 
    outputs=gr.Textbox(label="Translated text"),
    title="English to Hindi Translator",
    description="Enter English text and get the Hindi translation."
)

Step 6: Launch the Gradio Interface

Finally, launch the Gradio app. This will start a local web server where users can interact with the translation tool.

# Launch the Gradio interface
iface.launch()

Complete Code:

# Import necessary libraries
from transformers import MarianMTModel, MarianTokenizer
import gradio as gr

# Define the model name
model_name = 'Helsinki-NLP/opus-mt-en-hi'

# Load the tokenizer and model
tokenizer = MarianTokenizer.from_pretrained(model_name)
model = MarianMTModel.from_pretrained(model_name)

def translate(text):
    # Tokenize the input text
    tokenized_text = tokenizer.prepare_seq2seq_batch([text], return_tensors='pt')
    
    # Perform the translation
    translation = model.generate(**tokenized_text)
    
    # Decode the translated text
    translated_text = tokenizer.decode(translation[0], skip_special_tokens=True)
    
    return translated_text

# Create a Gradio interface
iface = gr.Interface(
    fn=translate, 
    inputs=gr.Textbox(label="Enter text to translate"), 
    outputs=gr.Textbox(label="Translated text"),
    title="English to Hindi Translator",
    description="Enter English text and get the Hindi translation."
)

# Launch the Gradio interface
iface.launch()

Output:

Gradio Interface for English to Hindi Translator

English to French Translator using Pre-trained Translation Model

We can create French translator using the Helsinki-NLP/opus-mt-en-fr model.

# Import necessary libraries
from transformers import MarianMTModel, MarianTokenizer
import gradio as gr

# Define the model name
model_name = 'Helsinki-NLP/opus-mt-en-fr'

# Load the tokenizer and model
tokenizer = MarianTokenizer.from_pretrained(model_name)
model = MarianMTModel.from_pretrained(model_name)

def translate(text):
    # Tokenize the input text
    tokenized_text = tokenizer.prepare_seq2seq_batch([text], return_tensors='pt')
    
    # Perform the translation
    translation = model.generate(**tokenized_text)
    
    # Decode the translated text
    translated_text = tokenizer.decode(translation[0], skip_special_tokens=True)
    
    return translated_text

# Create a Gradio interface
iface = gr.Interface(
    fn=translate, 
    inputs=gr.Textbox(label="Enter text to translate"), 
    outputs=gr.Textbox(label="Translated text"),
    title="English to French Translator",
    description="Enter English text and get the French translation."
)

# Launch the Gradio interface
iface.launch()

Output:

Gradio Interface for English to French Translator

English to German Translator using Pre-trained Translation Model

We can create German translator using the Helsinki-NLP/opus-mt-en-de model.

# Import necessary libraries
from transformers import MarianMTModel, MarianTokenizer
import gradio as gr

# Define the model name
model_name = 'Helsinki-NLP/opus-mt-en-de'

# Load the tokenizer and model
tokenizer = MarianTokenizer.from_pretrained(model_name)
model = MarianMTModel.from_pretrained(model_name)

def translate(text):
    # Tokenize the input text
    tokenized_text = tokenizer.prepare_seq2seq_batch([text], return_tensors='pt')

    # Perform the translation
    translation = model.generate(**tokenized_text)

    # Decode the translated text
    translated_text = tokenizer.decode(translation[0], skip_special_tokens=True)

    return translated_text

# Create a Gradio interface
iface = gr.Interface(
    fn=translate,
    inputs=gr.Textbox(label="Enter text to translate"),
    outputs=gr.Textbox(label="Translated text"),
    title="English to German Translator",
    description="Enter English text and get the German translation."
)

# Launch the Gradio interface
iface.launch()

Output:

Gradio Interface for English to German Translator

Conclusion

In this article, we successfully built a simple language translation tool using the MarianMT model for English-to-Hindi translation. By leveraging pre-trained models from Hugging Face Transformers and creating an interactive interface with Gradio, we demonstrated how accessible and effective modern translation tools can be. This approach not only simplifies the translation process but also provides a foundation for developing similar tools for other languages. As technology evolves, these methods will continue to enhance language translation applications, making them more versatile and user-friendly.