Project 3: Sentiment Analysis API with Fine-Tuned Transformer
Step 3: Fine-Tune the Model
Fine-tune a pretrained transformer model for sentiment analysis. This crucial step involves taking a pre-trained model that already understands language patterns and further training it on our specific sentiment analysis task. Fine-tuning allows the model to learn the nuances of sentiment expression in movie reviews while retaining its foundational language understanding.
The process involves adjusting the model's parameters through supervised learning on labeled examples, where each review is paired with its corresponding sentiment label. This approach is more efficient than training a model from scratch, as it leverages the model's existing knowledge while adapting it to our specific use case.
from transformers import AutoModelForSequenceClassification, TrainingArguments, Trainer
from sklearn.metrics import accuracy_score, precision_recall_fscore_support
# Load the model
model = AutoModelForSequenceClassification.from_pretrained(model_name, num_labels=2)
# Define training arguments
training_args = TrainingArguments(
output_dir="./results",
evaluation_strategy="epoch",
learning_rate=2e-5,
per_device_train_batch_size=16,
num_train_epochs=3,
weight_decay=0.01,
save_steps=500,
logging_dir="./logs",
logging_steps=100,
)
# Define a compute_metrics function
def compute_metrics(eval_pred):
logits, labels = eval_pred
predictions = logits.argmax(axis=-1)
precision, recall, f1, _ = precision_recall_fscore_support(labels, predictions, average="binary")
acc = accuracy_score(labels, predictions)
return {"accuracy": acc, "f1": f1, "precision": precision, "recall": recall}
# Initialize the trainer
trainer = Trainer(
model=model,
args=training_args,
train_dataset=tokenized_datasets["train"],
eval_dataset=tokenized_datasets["test"],
tokenizer=tokenizer,
compute_metrics=compute_metrics,
)
# Train the model
trainer.train()This code implements the model fine-tuning process for sentiment analysis. Let's break it down into key components:
1. Model Setup
- The code loads a pre-trained model (AutoModelForSequenceClassification) with 2 output labels for binary classification (positive/negative)
2. Training Configuration
The TrainingArguments sets up the training parameters:
- Learning rate of 2e-5
- Batch size of 16 samples per device
- 3 training epochs
- Weight decay of 0.01 for regularization
- Evaluation performed after each epoch
3. Metrics Calculation
The compute_metrics function calculates important performance metrics:
- Accuracy: Overall correct predictions
- Precision: Accuracy of positive predictions
- Recall: Ability to find all positive cases
- F1 score: Balanced measure of precision and recall
4. Training Setup
The Trainer class combines all components:
- The model itself
- Training arguments
- Training and test datasets
- Tokenizer for processing text
- Metrics computation
This approach is particularly efficient as it leverages the model's existing language understanding while adapting it specifically for sentiment analysis
Step 3: Fine-Tune the Model
Fine-tune a pretrained transformer model for sentiment analysis. This crucial step involves taking a pre-trained model that already understands language patterns and further training it on our specific sentiment analysis task. Fine-tuning allows the model to learn the nuances of sentiment expression in movie reviews while retaining its foundational language understanding.
The process involves adjusting the model's parameters through supervised learning on labeled examples, where each review is paired with its corresponding sentiment label. This approach is more efficient than training a model from scratch, as it leverages the model's existing knowledge while adapting it to our specific use case.
from transformers import AutoModelForSequenceClassification, TrainingArguments, Trainer
from sklearn.metrics import accuracy_score, precision_recall_fscore_support
# Load the model
model = AutoModelForSequenceClassification.from_pretrained(model_name, num_labels=2)
# Define training arguments
training_args = TrainingArguments(
output_dir="./results",
evaluation_strategy="epoch",
learning_rate=2e-5,
per_device_train_batch_size=16,
num_train_epochs=3,
weight_decay=0.01,
save_steps=500,
logging_dir="./logs",
logging_steps=100,
)
# Define a compute_metrics function
def compute_metrics(eval_pred):
logits, labels = eval_pred
predictions = logits.argmax(axis=-1)
precision, recall, f1, _ = precision_recall_fscore_support(labels, predictions, average="binary")
acc = accuracy_score(labels, predictions)
return {"accuracy": acc, "f1": f1, "precision": precision, "recall": recall}
# Initialize the trainer
trainer = Trainer(
model=model,
args=training_args,
train_dataset=tokenized_datasets["train"],
eval_dataset=tokenized_datasets["test"],
tokenizer=tokenizer,
compute_metrics=compute_metrics,
)
# Train the model
trainer.train()This code implements the model fine-tuning process for sentiment analysis. Let's break it down into key components:
1. Model Setup
- The code loads a pre-trained model (AutoModelForSequenceClassification) with 2 output labels for binary classification (positive/negative)
2. Training Configuration
The TrainingArguments sets up the training parameters:
- Learning rate of 2e-5
- Batch size of 16 samples per device
- 3 training epochs
- Weight decay of 0.01 for regularization
- Evaluation performed after each epoch
3. Metrics Calculation
The compute_metrics function calculates important performance metrics:
- Accuracy: Overall correct predictions
- Precision: Accuracy of positive predictions
- Recall: Ability to find all positive cases
- F1 score: Balanced measure of precision and recall
4. Training Setup
The Trainer class combines all components:
- The model itself
- Training arguments
- Training and test datasets
- Tokenizer for processing text
- Metrics computation
This approach is particularly efficient as it leverages the model's existing language understanding while adapting it specifically for sentiment analysis
Step 3: Fine-Tune the Model
Fine-tune a pretrained transformer model for sentiment analysis. This crucial step involves taking a pre-trained model that already understands language patterns and further training it on our specific sentiment analysis task. Fine-tuning allows the model to learn the nuances of sentiment expression in movie reviews while retaining its foundational language understanding.
The process involves adjusting the model's parameters through supervised learning on labeled examples, where each review is paired with its corresponding sentiment label. This approach is more efficient than training a model from scratch, as it leverages the model's existing knowledge while adapting it to our specific use case.
from transformers import AutoModelForSequenceClassification, TrainingArguments, Trainer
from sklearn.metrics import accuracy_score, precision_recall_fscore_support
# Load the model
model = AutoModelForSequenceClassification.from_pretrained(model_name, num_labels=2)
# Define training arguments
training_args = TrainingArguments(
output_dir="./results",
evaluation_strategy="epoch",
learning_rate=2e-5,
per_device_train_batch_size=16,
num_train_epochs=3,
weight_decay=0.01,
save_steps=500,
logging_dir="./logs",
logging_steps=100,
)
# Define a compute_metrics function
def compute_metrics(eval_pred):
logits, labels = eval_pred
predictions = logits.argmax(axis=-1)
precision, recall, f1, _ = precision_recall_fscore_support(labels, predictions, average="binary")
acc = accuracy_score(labels, predictions)
return {"accuracy": acc, "f1": f1, "precision": precision, "recall": recall}
# Initialize the trainer
trainer = Trainer(
model=model,
args=training_args,
train_dataset=tokenized_datasets["train"],
eval_dataset=tokenized_datasets["test"],
tokenizer=tokenizer,
compute_metrics=compute_metrics,
)
# Train the model
trainer.train()This code implements the model fine-tuning process for sentiment analysis. Let's break it down into key components:
1. Model Setup
- The code loads a pre-trained model (AutoModelForSequenceClassification) with 2 output labels for binary classification (positive/negative)
2. Training Configuration
The TrainingArguments sets up the training parameters:
- Learning rate of 2e-5
- Batch size of 16 samples per device
- 3 training epochs
- Weight decay of 0.01 for regularization
- Evaluation performed after each epoch
3. Metrics Calculation
The compute_metrics function calculates important performance metrics:
- Accuracy: Overall correct predictions
- Precision: Accuracy of positive predictions
- Recall: Ability to find all positive cases
- F1 score: Balanced measure of precision and recall
4. Training Setup
The Trainer class combines all components:
- The model itself
- Training arguments
- Training and test datasets
- Tokenizer for processing text
- Metrics computation
This approach is particularly efficient as it leverages the model's existing language understanding while adapting it specifically for sentiment analysis
Step 3: Fine-Tune the Model
Fine-tune a pretrained transformer model for sentiment analysis. This crucial step involves taking a pre-trained model that already understands language patterns and further training it on our specific sentiment analysis task. Fine-tuning allows the model to learn the nuances of sentiment expression in movie reviews while retaining its foundational language understanding.
The process involves adjusting the model's parameters through supervised learning on labeled examples, where each review is paired with its corresponding sentiment label. This approach is more efficient than training a model from scratch, as it leverages the model's existing knowledge while adapting it to our specific use case.
from transformers import AutoModelForSequenceClassification, TrainingArguments, Trainer
from sklearn.metrics import accuracy_score, precision_recall_fscore_support
# Load the model
model = AutoModelForSequenceClassification.from_pretrained(model_name, num_labels=2)
# Define training arguments
training_args = TrainingArguments(
output_dir="./results",
evaluation_strategy="epoch",
learning_rate=2e-5,
per_device_train_batch_size=16,
num_train_epochs=3,
weight_decay=0.01,
save_steps=500,
logging_dir="./logs",
logging_steps=100,
)
# Define a compute_metrics function
def compute_metrics(eval_pred):
logits, labels = eval_pred
predictions = logits.argmax(axis=-1)
precision, recall, f1, _ = precision_recall_fscore_support(labels, predictions, average="binary")
acc = accuracy_score(labels, predictions)
return {"accuracy": acc, "f1": f1, "precision": precision, "recall": recall}
# Initialize the trainer
trainer = Trainer(
model=model,
args=training_args,
train_dataset=tokenized_datasets["train"],
eval_dataset=tokenized_datasets["test"],
tokenizer=tokenizer,
compute_metrics=compute_metrics,
)
# Train the model
trainer.train()This code implements the model fine-tuning process for sentiment analysis. Let's break it down into key components:
1. Model Setup
- The code loads a pre-trained model (AutoModelForSequenceClassification) with 2 output labels for binary classification (positive/negative)
2. Training Configuration
The TrainingArguments sets up the training parameters:
- Learning rate of 2e-5
- Batch size of 16 samples per device
- 3 training epochs
- Weight decay of 0.01 for regularization
- Evaluation performed after each epoch
3. Metrics Calculation
The compute_metrics function calculates important performance metrics:
- Accuracy: Overall correct predictions
- Precision: Accuracy of positive predictions
- Recall: Ability to find all positive cases
- F1 score: Balanced measure of precision and recall
4. Training Setup
The Trainer class combines all components:
- The model itself
- Training arguments
- Training and test datasets
- Tokenizer for processing text
- Metrics computation
This approach is particularly efficient as it leverages the model's existing language understanding while adapting it specifically for sentiment analysis