Chapter 10: Machine Translation
10.5 Practical Exercises of Chapter 10: Machine Translation
Exercise 1: Implementing a Simple Seq2Seq Model
Implement a simple sequence-to-sequence model using LSTM layers for both the encoder and decoder. Use this model to translate short English sentences into French. You can use a small dataset such as the English-French sentence pairs dataset available on manylanguagepairs.com for this exercise.
Here is a skeleton code to help you get started:
import tensorflow as tf
from tensorflow.keras.models import Model
from tensorflow.keras.layers import Input, LSTM, Dense
# Hyperparameters
batch_size = 64
epochs = 100
latent_dim = 256
num_samples = 10000
# Prepare the data...
# Define an input sequence and process it.
encoder_inputs = Input(shape=(None, num_encoder_tokens))
encoder = LSTM(latent_dim, return_state=True)
encoder_outputs, state_h, state_c = encoder(encoder_inputs)
encoder_states = [state_h, state_c]
# Set up the decoder, using `encoder_states` as initial state.
decoder_inputs = Input(shape=(None, num_decoder_tokens))
decoder_lstm = LSTM(latent_dim, return_sequences=True, return_state=True)
decoder_outputs, _, _ = decoder_lstm(decoder_inputs, initial_state=encoder_states)
decoder_dense = Dense(num_decoder_tokens, activation='softmax')
decoder_outputs = decoder_dense(decoder_outputs)
# Define the model
model = Model([encoder_inputs, decoder_inputs], decoder_outputs)
# Compile & train the model
model.compile(optimizer='rmsprop', loss='categorical_crossentropy')
model.fit([encoder_input_data, decoder_input_data], decoder_target_data,
batch_size=batch_size,
epochs=epochs,
validation_split=0.2)Exercise 2: Attention Mechanism
Try to implement an attention mechanism and incorporate it into the seq2seq model you created in the first exercise.
Exercise 3: Experimenting with Transformer Models
Use the Hugging Face's Transformers library to load a pre-trained Transformer model and use it for translation tasks. Compare the performance with the seq2seq model you created in the first exercise.
from transformers import pipeline
# Use the 'translation_en_to_fr' pipeline
translator = pipeline('translation_en_to_fr', model='Helsinki-NLP/opus-mt-en-fr')
# Translate a sentence
translation = translator("Hello, how are you?", max_length=40)
print(translation[0]['translation_text'])Exercise 4: Evaluation Metrics
Write a function to calculate the BLEU score for the translations produced by your models. Use this function to compare the performance of different models or different versions of a model.
from nltk.translate.bleu_score import sentence_bleu
def calculate_bleu_score(reference, candidate):
reference = [reference.split()]
candidate = candidate.split()
score = sentence_bleu(reference, candidate)
return score
reference = "The cat is on the mat."
candidate = "There is a cat on the mat."
print("BLEU Score:", calculate_bleu_score(reference, candidate))Remember to adjust the models and parameters according to the resources you have, as training these models can be very resource-intensive. Also, these exercises are just starting points, feel free to expand upon them and experiment with different settings and architectures.
10.5 Practical Exercises of Chapter 10: Machine Translation
Exercise 1: Implementing a Simple Seq2Seq Model
Implement a simple sequence-to-sequence model using LSTM layers for both the encoder and decoder. Use this model to translate short English sentences into French. You can use a small dataset such as the English-French sentence pairs dataset available on manylanguagepairs.com for this exercise.
Here is a skeleton code to help you get started:
import tensorflow as tf
from tensorflow.keras.models import Model
from tensorflow.keras.layers import Input, LSTM, Dense
# Hyperparameters
batch_size = 64
epochs = 100
latent_dim = 256
num_samples = 10000
# Prepare the data...
# Define an input sequence and process it.
encoder_inputs = Input(shape=(None, num_encoder_tokens))
encoder = LSTM(latent_dim, return_state=True)
encoder_outputs, state_h, state_c = encoder(encoder_inputs)
encoder_states = [state_h, state_c]
# Set up the decoder, using `encoder_states` as initial state.
decoder_inputs = Input(shape=(None, num_decoder_tokens))
decoder_lstm = LSTM(latent_dim, return_sequences=True, return_state=True)
decoder_outputs, _, _ = decoder_lstm(decoder_inputs, initial_state=encoder_states)
decoder_dense = Dense(num_decoder_tokens, activation='softmax')
decoder_outputs = decoder_dense(decoder_outputs)
# Define the model
model = Model([encoder_inputs, decoder_inputs], decoder_outputs)
# Compile & train the model
model.compile(optimizer='rmsprop', loss='categorical_crossentropy')
model.fit([encoder_input_data, decoder_input_data], decoder_target_data,
batch_size=batch_size,
epochs=epochs,
validation_split=0.2)Exercise 2: Attention Mechanism
Try to implement an attention mechanism and incorporate it into the seq2seq model you created in the first exercise.
Exercise 3: Experimenting with Transformer Models
Use the Hugging Face's Transformers library to load a pre-trained Transformer model and use it for translation tasks. Compare the performance with the seq2seq model you created in the first exercise.
from transformers import pipeline
# Use the 'translation_en_to_fr' pipeline
translator = pipeline('translation_en_to_fr', model='Helsinki-NLP/opus-mt-en-fr')
# Translate a sentence
translation = translator("Hello, how are you?", max_length=40)
print(translation[0]['translation_text'])Exercise 4: Evaluation Metrics
Write a function to calculate the BLEU score for the translations produced by your models. Use this function to compare the performance of different models or different versions of a model.
from nltk.translate.bleu_score import sentence_bleu
def calculate_bleu_score(reference, candidate):
reference = [reference.split()]
candidate = candidate.split()
score = sentence_bleu(reference, candidate)
return score
reference = "The cat is on the mat."
candidate = "There is a cat on the mat."
print("BLEU Score:", calculate_bleu_score(reference, candidate))Remember to adjust the models and parameters according to the resources you have, as training these models can be very resource-intensive. Also, these exercises are just starting points, feel free to expand upon them and experiment with different settings and architectures.
10.5 Practical Exercises of Chapter 10: Machine Translation
Exercise 1: Implementing a Simple Seq2Seq Model
Implement a simple sequence-to-sequence model using LSTM layers for both the encoder and decoder. Use this model to translate short English sentences into French. You can use a small dataset such as the English-French sentence pairs dataset available on manylanguagepairs.com for this exercise.
Here is a skeleton code to help you get started:
import tensorflow as tf
from tensorflow.keras.models import Model
from tensorflow.keras.layers import Input, LSTM, Dense
# Hyperparameters
batch_size = 64
epochs = 100
latent_dim = 256
num_samples = 10000
# Prepare the data...
# Define an input sequence and process it.
encoder_inputs = Input(shape=(None, num_encoder_tokens))
encoder = LSTM(latent_dim, return_state=True)
encoder_outputs, state_h, state_c = encoder(encoder_inputs)
encoder_states = [state_h, state_c]
# Set up the decoder, using `encoder_states` as initial state.
decoder_inputs = Input(shape=(None, num_decoder_tokens))
decoder_lstm = LSTM(latent_dim, return_sequences=True, return_state=True)
decoder_outputs, _, _ = decoder_lstm(decoder_inputs, initial_state=encoder_states)
decoder_dense = Dense(num_decoder_tokens, activation='softmax')
decoder_outputs = decoder_dense(decoder_outputs)
# Define the model
model = Model([encoder_inputs, decoder_inputs], decoder_outputs)
# Compile & train the model
model.compile(optimizer='rmsprop', loss='categorical_crossentropy')
model.fit([encoder_input_data, decoder_input_data], decoder_target_data,
batch_size=batch_size,
epochs=epochs,
validation_split=0.2)Exercise 2: Attention Mechanism
Try to implement an attention mechanism and incorporate it into the seq2seq model you created in the first exercise.
Exercise 3: Experimenting with Transformer Models
Use the Hugging Face's Transformers library to load a pre-trained Transformer model and use it for translation tasks. Compare the performance with the seq2seq model you created in the first exercise.
from transformers import pipeline
# Use the 'translation_en_to_fr' pipeline
translator = pipeline('translation_en_to_fr', model='Helsinki-NLP/opus-mt-en-fr')
# Translate a sentence
translation = translator("Hello, how are you?", max_length=40)
print(translation[0]['translation_text'])Exercise 4: Evaluation Metrics
Write a function to calculate the BLEU score for the translations produced by your models. Use this function to compare the performance of different models or different versions of a model.
from nltk.translate.bleu_score import sentence_bleu
def calculate_bleu_score(reference, candidate):
reference = [reference.split()]
candidate = candidate.split()
score = sentence_bleu(reference, candidate)
return score
reference = "The cat is on the mat."
candidate = "There is a cat on the mat."
print("BLEU Score:", calculate_bleu_score(reference, candidate))Remember to adjust the models and parameters according to the resources you have, as training these models can be very resource-intensive. Also, these exercises are just starting points, feel free to expand upon them and experiment with different settings and architectures.
10.5 Practical Exercises of Chapter 10: Machine Translation
Exercise 1: Implementing a Simple Seq2Seq Model
Implement a simple sequence-to-sequence model using LSTM layers for both the encoder and decoder. Use this model to translate short English sentences into French. You can use a small dataset such as the English-French sentence pairs dataset available on manylanguagepairs.com for this exercise.
Here is a skeleton code to help you get started:
import tensorflow as tf
from tensorflow.keras.models import Model
from tensorflow.keras.layers import Input, LSTM, Dense
# Hyperparameters
batch_size = 64
epochs = 100
latent_dim = 256
num_samples = 10000
# Prepare the data...
# Define an input sequence and process it.
encoder_inputs = Input(shape=(None, num_encoder_tokens))
encoder = LSTM(latent_dim, return_state=True)
encoder_outputs, state_h, state_c = encoder(encoder_inputs)
encoder_states = [state_h, state_c]
# Set up the decoder, using `encoder_states` as initial state.
decoder_inputs = Input(shape=(None, num_decoder_tokens))
decoder_lstm = LSTM(latent_dim, return_sequences=True, return_state=True)
decoder_outputs, _, _ = decoder_lstm(decoder_inputs, initial_state=encoder_states)
decoder_dense = Dense(num_decoder_tokens, activation='softmax')
decoder_outputs = decoder_dense(decoder_outputs)
# Define the model
model = Model([encoder_inputs, decoder_inputs], decoder_outputs)
# Compile & train the model
model.compile(optimizer='rmsprop', loss='categorical_crossentropy')
model.fit([encoder_input_data, decoder_input_data], decoder_target_data,
batch_size=batch_size,
epochs=epochs,
validation_split=0.2)Exercise 2: Attention Mechanism
Try to implement an attention mechanism and incorporate it into the seq2seq model you created in the first exercise.
Exercise 3: Experimenting with Transformer Models
Use the Hugging Face's Transformers library to load a pre-trained Transformer model and use it for translation tasks. Compare the performance with the seq2seq model you created in the first exercise.
from transformers import pipeline
# Use the 'translation_en_to_fr' pipeline
translator = pipeline('translation_en_to_fr', model='Helsinki-NLP/opus-mt-en-fr')
# Translate a sentence
translation = translator("Hello, how are you?", max_length=40)
print(translation[0]['translation_text'])Exercise 4: Evaluation Metrics
Write a function to calculate the BLEU score for the translations produced by your models. Use this function to compare the performance of different models or different versions of a model.
from nltk.translate.bleu_score import sentence_bleu
def calculate_bleu_score(reference, candidate):
reference = [reference.split()]
candidate = candidate.split()
score = sentence_bleu(reference, candidate)
return score
reference = "The cat is on the mat."
candidate = "There is a cat on the mat."
print("BLEU Score:", calculate_bleu_score(reference, candidate))Remember to adjust the models and parameters according to the resources you have, as training these models can be very resource-intensive. Also, these exercises are just starting points, feel free to expand upon them and experiment with different settings and architectures.