Click here to view the next lesson.

Capítulo 5: Modelos de Transformadores Clave e Innovaciones

Ejercicios Prácticos para el Capítulo 5

Los siguientes ejercicios están diseñados para consolidar tu comprensión de modelos Transformer especializados como BioBERT y LegalBERT. Cada ejercicio incluye una explicación detallada, requisitos de la tarea y soluciones con ejemplos de código.

Ejercicio 1: Uso de BioBERT para el Reconocimiento de Entidades Nombradas (REN)

Tarea:

Extraer entidades nombradas de texto biomédico usando BioBERT. Identificar entidades relacionadas con enfermedades, productos químicos y genes.

Solución:

from transformers import AutoTokenizer, AutoModelForTokenClassification, pipeline

# Load pre-trained BioBERT model and tokenizer
tokenizer = AutoTokenizer.from_pretrained("dmis-lab/biobert-base-cased-v1.1")
model = AutoModelForTokenClassification.from_pretrained("dmis-lab/biobert-base-cased-v1.1")

# Biomedical text
text = "BRCA1 is a gene associated with breast cancer. Aspirin is a chemical used for pain relief."

# Initialize pipeline for NER
ner_pipeline = pipeline("ner", model=model, tokenizer=tokenizer)

# Perform named entity recognition
results = ner_pipeline(text)

# Display entities
print("Extracted Entities:")
for entity in results:
    print(f"Entity: {entity['word']}, Label: {entity['entity']}")

Salida Esperada:

Extracted Entities:
Entity: BRCA1, Label: B-Gene
Entity: breast cancer, Label: B-Disease
Entity: Aspirin, Label: B-Chemical

Ejercicio 2: Ajuste fino de BioBERT para Extracción de Relaciones

Tarea:

Realizar un ajuste fino de BioBERT en un conjunto de datos de extracción de relaciones para predecir la relación entre un gen y una enfermedad.

Solución:

import torch
from transformers import AutoTokenizer, AutoModelForSequenceClassification, Trainer, TrainingArguments

# Load BioBERT tokenizer and model
tokenizer = AutoTokenizer.from_pretrained("dmis-lab/biobert-base-cased-v1.1")
model = AutoModelForSequenceClassification.from_pretrained("dmis-lab/biobert-base-cased-v1.1", num_labels=2)

# Example dataset
texts = ["BRCA1 is associated with breast cancer.", "EGFR is unrelated to heart disease."]
labels = [1, 0]  # 1: Related, 0: Unrelated

# Tokenize dataset
inputs = tokenizer(texts, padding=True, truncation=True, return_tensors="pt")
dataset = torch.utils.data.TensorDataset(inputs["input_ids"], inputs["attention_mask"], torch.tensor(labels))

# Define training arguments
training_args = TrainingArguments(
    output_dir="./results",
    evaluation_strategy="epoch",
    learning_rate=2e-5,
    per_device_train_batch_size=8,
    num_train_epochs=3,
    weight_decay=0.01,
)

# Initialize Trainer
trainer = Trainer(
    model=model,
    args=training_args,
    train_dataset=dataset,
)

# Train the model
trainer.train()

Ejercicio 3: Clasificación de Cláusulas Contractuales Usando LegalBERT

Tarea:

Clasificar cláusulas contractuales en categorías como "Cláusula de Pago" o "Cláusula de Terminación" usando LegalBERT.

Solución:

from transformers import AutoTokenizer, AutoModelForSequenceClassification, pipeline

# Load LegalBERT tokenizer and model
tokenizer = AutoTokenizer.from_pretrained("nlpaueb/legal-bert-base-uncased")
model = AutoModelForSequenceClassification.from_pretrained("nlpaueb/legal-bert-base-uncased", num_labels=2)

# Legal text example
text = "The tenant shall pay rent on the first day of each month without demand."

# Define clause types
labels = {0: "Payment Clause", 1: "Termination Clause"}

# Use pipeline for classification
classification_pipeline = pipeline("text-classification", model=model, tokenizer=tokenizer)
result = classification_pipeline(text)

# Map prediction to label
predicted_label = labels[int(result[0]['label'].split('_')[-1])]
print(f"Predicted Clause Type: {predicted_label}")

Salida Esperada:

Predicted Clause Type: Payment Clause

Ejercicio 4: Clasificación Zero-Shot Usando CLIP

Tarea:

Clasificar una imagen basada en descripciones textuales usando CLIP. Hacer coincidir una imagen de un perro con etiquetas de texto apropiadas.

Solución:

from transformers import CLIPProcessor, CLIPModel
from PIL import Image

# Load pre-trained CLIP model and processor
model = CLIPModel.from_pretrained("openai/clip-vit-base-patch32")
processor = CLIPProcessor.from_pretrained("openai/clip-vit-base-patch32")

# Input image and text descriptions
image = Image.open("dog.jpg")  # Replace with your image file
texts = ["a photo of a dog", "a photo of a cat", "a photo of a car"]

# Process inputs
inputs = processor(text=texts, images=image, return_tensors="pt", padding=True)
outputs = model(**inputs)

# Compute similarity
logits_per_image = outputs.logits_per_image  # Image-to-text similarity scores
probs = logits_per_image.softmax(dim=1)  # Probabilities
print("Similarity Scores:", probs)

Salida Esperada:

Similarity Scores: [[0.80, 0.15, 0.05]]

Ejercicio 5: Generación de Resúmenes Legales con LegalBERT

Tarea:

Utilizar LegalBERT para resumir un documento legal, extrayendo los puntos principales.

Solución:

from transformers import pipeline

# Load summarization pipeline
summarizer = pipeline("summarization", model="nlpaueb/legal-bert-base-uncased")

# Input legal text
legal_text = """
The tenant agrees to pay rent on the first day of each month. Failure to do so will result in penalties as outlined in Section 5.
Additionally, the landlord may terminate this agreement if the tenant violates any clauses.
"""

# Summarize text
summary = summarizer(legal_text, max_length=50, min_length=20, do_sample=False)
print("Legal Summary:", summary[0]['summary_text'])

Salida Esperada:

Legal Summary: The tenant agrees to pay rent monthly and faces penalties or termination for non-compliance.

Estos ejercicios proporcionan experiencia práctica con BioBERT, LegalBERT y CLIP, demostrando su utilidad práctica en dominios especializados. Al completar estas tareas, obtienes una comprensión más profunda de cómo los modelos basados en Transformer pueden adaptarse para tareas biomédicas y legales, así como para el aprendizaje multimodal.

Ejercicios Prácticos para el Capítulo 5

Ejercicio 1: Uso de BioBERT para el Reconocimiento de Entidades Nombradas (REN)

Tarea:

Extraer entidades nombradas de texto biomédico usando BioBERT. Identificar entidades relacionadas con enfermedades, productos químicos y genes.

Solución:

from transformers import AutoTokenizer, AutoModelForTokenClassification, pipeline

# Load pre-trained BioBERT model and tokenizer
tokenizer = AutoTokenizer.from_pretrained("dmis-lab/biobert-base-cased-v1.1")
model = AutoModelForTokenClassification.from_pretrained("dmis-lab/biobert-base-cased-v1.1")

# Biomedical text
text = "BRCA1 is a gene associated with breast cancer. Aspirin is a chemical used for pain relief."

# Initialize pipeline for NER
ner_pipeline = pipeline("ner", model=model, tokenizer=tokenizer)

# Perform named entity recognition
results = ner_pipeline(text)

# Display entities
print("Extracted Entities:")
for entity in results:
    print(f"Entity: {entity['word']}, Label: {entity['entity']}")

Salida Esperada:

Extracted Entities:
Entity: BRCA1, Label: B-Gene
Entity: breast cancer, Label: B-Disease
Entity: Aspirin, Label: B-Chemical

Ejercicio 2: Ajuste fino de BioBERT para Extracción de Relaciones

Tarea:

Realizar un ajuste fino de BioBERT en un conjunto de datos de extracción de relaciones para predecir la relación entre un gen y una enfermedad.

Solución:

import torch
from transformers import AutoTokenizer, AutoModelForSequenceClassification, Trainer, TrainingArguments

# Load BioBERT tokenizer and model
tokenizer = AutoTokenizer.from_pretrained("dmis-lab/biobert-base-cased-v1.1")
model = AutoModelForSequenceClassification.from_pretrained("dmis-lab/biobert-base-cased-v1.1", num_labels=2)

# Example dataset
texts = ["BRCA1 is associated with breast cancer.", "EGFR is unrelated to heart disease."]
labels = [1, 0]  # 1: Related, 0: Unrelated

# Tokenize dataset
inputs = tokenizer(texts, padding=True, truncation=True, return_tensors="pt")
dataset = torch.utils.data.TensorDataset(inputs["input_ids"], inputs["attention_mask"], torch.tensor(labels))

# Define training arguments
training_args = TrainingArguments(
    output_dir="./results",
    evaluation_strategy="epoch",
    learning_rate=2e-5,
    per_device_train_batch_size=8,
    num_train_epochs=3,
    weight_decay=0.01,
)

# Initialize Trainer
trainer = Trainer(
    model=model,
    args=training_args,
    train_dataset=dataset,
)

# Train the model
trainer.train()

Ejercicio 3: Clasificación de Cláusulas Contractuales Usando LegalBERT

Tarea:

Clasificar cláusulas contractuales en categorías como "Cláusula de Pago" o "Cláusula de Terminación" usando LegalBERT.

Solución:

from transformers import AutoTokenizer, AutoModelForSequenceClassification, pipeline

# Load LegalBERT tokenizer and model
tokenizer = AutoTokenizer.from_pretrained("nlpaueb/legal-bert-base-uncased")
model = AutoModelForSequenceClassification.from_pretrained("nlpaueb/legal-bert-base-uncased", num_labels=2)

# Legal text example
text = "The tenant shall pay rent on the first day of each month without demand."

# Define clause types
labels = {0: "Payment Clause", 1: "Termination Clause"}

# Use pipeline for classification
classification_pipeline = pipeline("text-classification", model=model, tokenizer=tokenizer)
result = classification_pipeline(text)

# Map prediction to label
predicted_label = labels[int(result[0]['label'].split('_')[-1])]
print(f"Predicted Clause Type: {predicted_label}")

Salida Esperada:

Predicted Clause Type: Payment Clause

Ejercicio 4: Clasificación Zero-Shot Usando CLIP

Tarea:

Clasificar una imagen basada en descripciones textuales usando CLIP. Hacer coincidir una imagen de un perro con etiquetas de texto apropiadas.

Solución:

from transformers import CLIPProcessor, CLIPModel
from PIL import Image

# Load pre-trained CLIP model and processor
model = CLIPModel.from_pretrained("openai/clip-vit-base-patch32")
processor = CLIPProcessor.from_pretrained("openai/clip-vit-base-patch32")

# Input image and text descriptions
image = Image.open("dog.jpg")  # Replace with your image file
texts = ["a photo of a dog", "a photo of a cat", "a photo of a car"]

# Process inputs
inputs = processor(text=texts, images=image, return_tensors="pt", padding=True)
outputs = model(**inputs)

# Compute similarity
logits_per_image = outputs.logits_per_image  # Image-to-text similarity scores
probs = logits_per_image.softmax(dim=1)  # Probabilities
print("Similarity Scores:", probs)

Salida Esperada:

Similarity Scores: [[0.80, 0.15, 0.05]]

Ejercicio 5: Generación de Resúmenes Legales con LegalBERT

Tarea:

Utilizar LegalBERT para resumir un documento legal, extrayendo los puntos principales.

Solución:

from transformers import pipeline

# Load summarization pipeline
summarizer = pipeline("summarization", model="nlpaueb/legal-bert-base-uncased")

# Input legal text
legal_text = """
The tenant agrees to pay rent on the first day of each month. Failure to do so will result in penalties as outlined in Section 5.
Additionally, the landlord may terminate this agreement if the tenant violates any clauses.
"""

# Summarize text
summary = summarizer(legal_text, max_length=50, min_length=20, do_sample=False)
print("Legal Summary:", summary[0]['summary_text'])

Salida Esperada:

Legal Summary: The tenant agrees to pay rent monthly and faces penalties or termination for non-compliance.

Ejercicios Prácticos para el Capítulo 5

Ejercicio 1: Uso de BioBERT para el Reconocimiento de Entidades Nombradas (REN)

Tarea:

Extraer entidades nombradas de texto biomédico usando BioBERT. Identificar entidades relacionadas con enfermedades, productos químicos y genes.

Solución:

from transformers import AutoTokenizer, AutoModelForTokenClassification, pipeline

# Load pre-trained BioBERT model and tokenizer
tokenizer = AutoTokenizer.from_pretrained("dmis-lab/biobert-base-cased-v1.1")
model = AutoModelForTokenClassification.from_pretrained("dmis-lab/biobert-base-cased-v1.1")

# Biomedical text
text = "BRCA1 is a gene associated with breast cancer. Aspirin is a chemical used for pain relief."

# Initialize pipeline for NER
ner_pipeline = pipeline("ner", model=model, tokenizer=tokenizer)

# Perform named entity recognition
results = ner_pipeline(text)

# Display entities
print("Extracted Entities:")
for entity in results:
    print(f"Entity: {entity['word']}, Label: {entity['entity']}")

Salida Esperada:

Extracted Entities:
Entity: BRCA1, Label: B-Gene
Entity: breast cancer, Label: B-Disease
Entity: Aspirin, Label: B-Chemical

Ejercicio 2: Ajuste fino de BioBERT para Extracción de Relaciones

Tarea:

Realizar un ajuste fino de BioBERT en un conjunto de datos de extracción de relaciones para predecir la relación entre un gen y una enfermedad.

Solución:

import torch
from transformers import AutoTokenizer, AutoModelForSequenceClassification, Trainer, TrainingArguments

# Load BioBERT tokenizer and model
tokenizer = AutoTokenizer.from_pretrained("dmis-lab/biobert-base-cased-v1.1")
model = AutoModelForSequenceClassification.from_pretrained("dmis-lab/biobert-base-cased-v1.1", num_labels=2)

# Example dataset
texts = ["BRCA1 is associated with breast cancer.", "EGFR is unrelated to heart disease."]
labels = [1, 0]  # 1: Related, 0: Unrelated

# Tokenize dataset
inputs = tokenizer(texts, padding=True, truncation=True, return_tensors="pt")
dataset = torch.utils.data.TensorDataset(inputs["input_ids"], inputs["attention_mask"], torch.tensor(labels))

# Define training arguments
training_args = TrainingArguments(
    output_dir="./results",
    evaluation_strategy="epoch",
    learning_rate=2e-5,
    per_device_train_batch_size=8,
    num_train_epochs=3,
    weight_decay=0.01,
)

# Initialize Trainer
trainer = Trainer(
    model=model,
    args=training_args,
    train_dataset=dataset,
)

# Train the model
trainer.train()

Ejercicio 3: Clasificación de Cláusulas Contractuales Usando LegalBERT

Tarea:

Clasificar cláusulas contractuales en categorías como "Cláusula de Pago" o "Cláusula de Terminación" usando LegalBERT.

Solución:

from transformers import AutoTokenizer, AutoModelForSequenceClassification, pipeline

# Load LegalBERT tokenizer and model
tokenizer = AutoTokenizer.from_pretrained("nlpaueb/legal-bert-base-uncased")
model = AutoModelForSequenceClassification.from_pretrained("nlpaueb/legal-bert-base-uncased", num_labels=2)

# Legal text example
text = "The tenant shall pay rent on the first day of each month without demand."

# Define clause types
labels = {0: "Payment Clause", 1: "Termination Clause"}

# Use pipeline for classification
classification_pipeline = pipeline("text-classification", model=model, tokenizer=tokenizer)
result = classification_pipeline(text)

# Map prediction to label
predicted_label = labels[int(result[0]['label'].split('_')[-1])]
print(f"Predicted Clause Type: {predicted_label}")

Salida Esperada:

Predicted Clause Type: Payment Clause

Ejercicio 4: Clasificación Zero-Shot Usando CLIP

Tarea:

Clasificar una imagen basada en descripciones textuales usando CLIP. Hacer coincidir una imagen de un perro con etiquetas de texto apropiadas.

Solución:

from transformers import CLIPProcessor, CLIPModel
from PIL import Image

# Load pre-trained CLIP model and processor
model = CLIPModel.from_pretrained("openai/clip-vit-base-patch32")
processor = CLIPProcessor.from_pretrained("openai/clip-vit-base-patch32")

# Input image and text descriptions
image = Image.open("dog.jpg")  # Replace with your image file
texts = ["a photo of a dog", "a photo of a cat", "a photo of a car"]

# Process inputs
inputs = processor(text=texts, images=image, return_tensors="pt", padding=True)
outputs = model(**inputs)

# Compute similarity
logits_per_image = outputs.logits_per_image  # Image-to-text similarity scores
probs = logits_per_image.softmax(dim=1)  # Probabilities
print("Similarity Scores:", probs)

Salida Esperada:

Similarity Scores: [[0.80, 0.15, 0.05]]

Ejercicio 5: Generación de Resúmenes Legales con LegalBERT

Tarea:

Utilizar LegalBERT para resumir un documento legal, extrayendo los puntos principales.

Solución:

from transformers import pipeline

# Load summarization pipeline
summarizer = pipeline("summarization", model="nlpaueb/legal-bert-base-uncased")

# Input legal text
legal_text = """
The tenant agrees to pay rent on the first day of each month. Failure to do so will result in penalties as outlined in Section 5.
Additionally, the landlord may terminate this agreement if the tenant violates any clauses.
"""

# Summarize text
summary = summarizer(legal_text, max_length=50, min_length=20, do_sample=False)
print("Legal Summary:", summary[0]['summary_text'])

Salida Esperada:

Legal Summary: The tenant agrees to pay rent monthly and faces penalties or termination for non-compliance.

Ejercicios Prácticos para el Capítulo 5

Ejercicio 1: Uso de BioBERT para el Reconocimiento de Entidades Nombradas (REN)

Tarea:

Extraer entidades nombradas de texto biomédico usando BioBERT. Identificar entidades relacionadas con enfermedades, productos químicos y genes.

Solución:

from transformers import AutoTokenizer, AutoModelForTokenClassification, pipeline

# Load pre-trained BioBERT model and tokenizer
tokenizer = AutoTokenizer.from_pretrained("dmis-lab/biobert-base-cased-v1.1")
model = AutoModelForTokenClassification.from_pretrained("dmis-lab/biobert-base-cased-v1.1")

# Biomedical text
text = "BRCA1 is a gene associated with breast cancer. Aspirin is a chemical used for pain relief."

# Initialize pipeline for NER
ner_pipeline = pipeline("ner", model=model, tokenizer=tokenizer)

# Perform named entity recognition
results = ner_pipeline(text)

# Display entities
print("Extracted Entities:")
for entity in results:
    print(f"Entity: {entity['word']}, Label: {entity['entity']}")

Salida Esperada:

Extracted Entities:
Entity: BRCA1, Label: B-Gene
Entity: breast cancer, Label: B-Disease
Entity: Aspirin, Label: B-Chemical

Ejercicio 2: Ajuste fino de BioBERT para Extracción de Relaciones

Tarea:

Realizar un ajuste fino de BioBERT en un conjunto de datos de extracción de relaciones para predecir la relación entre un gen y una enfermedad.

Solución:

import torch
from transformers import AutoTokenizer, AutoModelForSequenceClassification, Trainer, TrainingArguments

# Load BioBERT tokenizer and model
tokenizer = AutoTokenizer.from_pretrained("dmis-lab/biobert-base-cased-v1.1")
model = AutoModelForSequenceClassification.from_pretrained("dmis-lab/biobert-base-cased-v1.1", num_labels=2)

# Example dataset
texts = ["BRCA1 is associated with breast cancer.", "EGFR is unrelated to heart disease."]
labels = [1, 0]  # 1: Related, 0: Unrelated

# Tokenize dataset
inputs = tokenizer(texts, padding=True, truncation=True, return_tensors="pt")
dataset = torch.utils.data.TensorDataset(inputs["input_ids"], inputs["attention_mask"], torch.tensor(labels))

# Define training arguments
training_args = TrainingArguments(
    output_dir="./results",
    evaluation_strategy="epoch",
    learning_rate=2e-5,
    per_device_train_batch_size=8,
    num_train_epochs=3,
    weight_decay=0.01,
)

# Initialize Trainer
trainer = Trainer(
    model=model,
    args=training_args,
    train_dataset=dataset,
)

# Train the model
trainer.train()

Ejercicio 3: Clasificación de Cláusulas Contractuales Usando LegalBERT

Tarea:

Clasificar cláusulas contractuales en categorías como "Cláusula de Pago" o "Cláusula de Terminación" usando LegalBERT.

Solución:

from transformers import AutoTokenizer, AutoModelForSequenceClassification, pipeline

# Load LegalBERT tokenizer and model
tokenizer = AutoTokenizer.from_pretrained("nlpaueb/legal-bert-base-uncased")
model = AutoModelForSequenceClassification.from_pretrained("nlpaueb/legal-bert-base-uncased", num_labels=2)

# Legal text example
text = "The tenant shall pay rent on the first day of each month without demand."

# Define clause types
labels = {0: "Payment Clause", 1: "Termination Clause"}

# Use pipeline for classification
classification_pipeline = pipeline("text-classification", model=model, tokenizer=tokenizer)
result = classification_pipeline(text)

# Map prediction to label
predicted_label = labels[int(result[0]['label'].split('_')[-1])]
print(f"Predicted Clause Type: {predicted_label}")

Salida Esperada:

Predicted Clause Type: Payment Clause

Ejercicio 4: Clasificación Zero-Shot Usando CLIP

Tarea:

Clasificar una imagen basada en descripciones textuales usando CLIP. Hacer coincidir una imagen de un perro con etiquetas de texto apropiadas.

Solución:

from transformers import CLIPProcessor, CLIPModel
from PIL import Image

# Load pre-trained CLIP model and processor
model = CLIPModel.from_pretrained("openai/clip-vit-base-patch32")
processor = CLIPProcessor.from_pretrained("openai/clip-vit-base-patch32")

# Input image and text descriptions
image = Image.open("dog.jpg")  # Replace with your image file
texts = ["a photo of a dog", "a photo of a cat", "a photo of a car"]

# Process inputs
inputs = processor(text=texts, images=image, return_tensors="pt", padding=True)
outputs = model(**inputs)

# Compute similarity
logits_per_image = outputs.logits_per_image  # Image-to-text similarity scores
probs = logits_per_image.softmax(dim=1)  # Probabilities
print("Similarity Scores:", probs)

Salida Esperada:

Similarity Scores: [[0.80, 0.15, 0.05]]

Ejercicio 5: Generación de Resúmenes Legales con LegalBERT

Tarea:

Utilizar LegalBERT para resumir un documento legal, extrayendo los puntos principales.

Solución:

from transformers import pipeline

# Load summarization pipeline
summarizer = pipeline("summarization", model="nlpaueb/legal-bert-base-uncased")

# Input legal text
legal_text = """
The tenant agrees to pay rent on the first day of each month. Failure to do so will result in penalties as outlined in Section 5.
Additionally, the landlord may terminate this agreement if the tenant violates any clauses.
"""

# Summarize text
summary = summarizer(legal_text, max_length=50, min_length=20, do_sample=False)
print("Legal Summary:", summary[0]['summary_text'])

Salida Esperada:

Legal Summary: The tenant agrees to pay rent monthly and faces penalties or termination for non-compliance.

Compra este libro