Chapter 12: Project: News Aggregator
12.4 Building the User Interface
In this section, we will focus on building the user interface (UI) for the news aggregator chatbot. The UI is crucial as it allows users to interact with the chatbot and access the news articles. We will create a web-based interface using Flask, a lightweight web framework for Python. This interface will enable users to request news updates, view summaries, and explore categorized news articles.
12.4.1 Setting Up Flask
First, we need to install Flask if we haven't already:
pip install FlaskNext, let's set up the basic structure of our Flask application.
from flask import Flask, request, jsonify, render_template
from nlp_engine import preprocess_text, categorize_article
from summarizer import summarize_text, abstractive_summarize_text
from news_fetcher import fetch_news
app = Flask(__name__)
@app.route('/')
def home():
return render_template('index.html')
@app.route('/get_news', methods=['POST'])
def get_news():
category = request.form.get('category')
articles = fetch_news(category)
return jsonify(articles)
@app.route('/summarize', methods=['POST'])
def summarize():
article = request.form.get('article')
summary_type = request.form.get('summary_type')
if summary_type == 'extractive':
summary = summarize_text(article)
else:
summary = abstractive_summarize_text(article)
return jsonify({'summary': summary})
if __name__ == '__main__':
app.run(debug=True)In this script, we define the basic routes for our Flask application:
- The
homeroute renders the home page. - The
get_newsroute fetches news articles based on the selected category. - The
summarizeroute summarizes a given article using either extractive or abstractive summarization.
12.4.2 Creating HTML Templates
Next, we will create HTML templates to render the web pages. We will use Bootstrap for styling to create a responsive and visually appealing interface.
templates/index.html:
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>News Aggregator Chatbot</title>
<link rel="stylesheet" href="<https://stackpath.bootstrapcdn.com/bootstrap/4.5.2/css/bootstrap.min.css>">
</head>
<body>
<div class="container">
<h1 class="mt-5">News Aggregator Chatbot</h1>
<form id="news-form" class="mt-4">
<div class="form-group">
<label for="category">Select News Category</label>
<select class="form-control" id="category" name="category">
<option value="general">General</option>
<option value="business">Business</option>
<option value="technology">Technology</option>
<option value="sports">Sports</option>
<option value="entertainment">Entertainment</option>
</select>
</div>
<button type="submit" class="btn btn-primary">Get News</button>
</form>
<div id="news-articles" class="mt-4"></div>
</div>
<script src="<https://code.jquery.com/jquery-3.5.1.min.js>"></script>
<script>
$(document).ready(function() {
$('#news-form').on('submit', function(event) {
event.preventDefault();
const category = $('#category').val();
$.post('/get_news', { category: category }, function(data) {
let articlesHtml = '<div class="list-group">';
data.forEach(article => {
articlesHtml += `
<a href="${article.url}" target="_blank" class="list-group-item list-group-item-action">
<h5 class="mb-1">${article.title}</h5>
<p class="mb-1">${article.description}</p>
<small>${article.source} - ${article.publishedAt}</small>
</a>
`;
});
articlesHtml += '</div>';
$('#news-articles').html(articlesHtml);
});
});
});
</script>
</body>
</html>In this HTML template, we create a simple form to select the news category and a div to display the fetched news articles. We use jQuery to handle the form submission and dynamically update the page with the fetched news articles.
12.4.3 Integrating Summarization and Categorization
To enable users to view summaries and categorize articles, we need to add additional functionality to the frontend and backend.
app.py (continued):
@app.route('/categorize', methods=['POST'])
def categorize():
article = request.form.get('article')
category = categorize_article(article)
return jsonify({'category': category})
@app.route('/summarize_article', methods=['POST'])
def summarize_article():
article = request.form.get('article')
summary_type = request.form.get('summary_type')
if summary_type == 'extractive':
summary = summarize_text(article)
else:
summary = abstractive_summarize_text(article)
return jsonify({'summary': summary})
# Updated HTML template to include summarization and categorization
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>News Aggregator Chatbot</title>
<link rel="stylesheet" href="<https://stackpath.bootstrapcdn.com/bootstrap/4.5.2/css/bootstrap.min.css>">
</head>
<body>
<div class="container">
<h1 class="mt-5">News Aggregator Chatbot</h1>
<form id="news-form" class="mt-4">
<div class="form-group">
<label for="category">Select News Category</label>
<select class="form-control" id="category" name="category">
<option value="general">General</option>
<option value="business">Business</option>
<option value="technology">Technology</option>
<option value="sports">Sports</option>
<option value="entertainment">Entertainment</option>
</select>
</div>
<button type="submit" class="btn btn-primary">Get News</button>
</form>
<div id="news-articles" class="mt-4"></div>
</div>
<script src="<https://code.jquery.com/jquery-3.5.1.min.js>"></script>
<script>
$(document).ready(function() {
$('#news-form').on('submit', function(event) {
event.preventDefault();
const category = $('#category').val();
$.post('/get_news', { category: category }, function(data) {
let articlesHtml = '<div class="list-group">';
data.forEach(article => {
articlesHtml += `
<div class="list-group-item">
<h5 class="mb-1">${article.title}</h5>
<p class="mb-1">${article.description}</p>
<small>${article.source} - ${article.publishedAt}</small>
<button class="btn btn-secondary btn-sm mt-2" onclick="summarizeArticle('${article.content}')">Summarize</button>
<div class="summary mt-2" id="summary-${article.title}"></div>
</div>
`;
});
articlesHtml += '</div>';
$('#news-articles').html(articlesHtml);
});
});
});
function summarizeArticle(article) {
const summaryType = 'extractive'; // or 'abstractive'
$.post('/summarize_article', { article: article, summary_type: summaryType }, function(data) {
$('#summary-' + article.title).html('<p>' + data.summary + '</p>');
});
}
</script>
</body>
</html>In this updated template, we add buttons for summarizing each article. The summarizeArticle function sends a request to the /summarize_article endpoint and displays the summary within the page.
12.4.4 Categorizing Articles
We will integrate the categorization functionality to automatically categorize news articles into different topics.
nlp_engine.py (continued):
# Define function to categorize article
def categorize_article(article):
bow = dictionary.doc2bow(nltk.word_tokenize(article.lower()))
topics = lda_model.get_document_topics(bow)
sorted_topics = sorted(topics, key=lambda x: x[1], reverse=True)
topic_index = sorted_topics[0][0]
topic_terms = lda_model.show_topic(topic_index, topn=4)
topic_keywords = [term[0] for term in topic_terms]
return topic_keywordsThis function uses the trained LDA model to categorize the article into topics based on the most relevant keywords.
Updated Flask Application:
@app.route('/categorize_article', methods=['POST'])
def categorize_article_route():
article = request.form.get('article')
category = categorize_article(article)
return jsonify({'category': category})
# HTML template for categorizing articles
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width,
initial-scale=1.0">
<title>News Aggregator Chatbot</title>
<link rel="stylesheet" href="<https://stackpath.bootstrapcdn.com/bootstrap/4.5.2/css/bootstrap.min.css>">
</head>
<body>
<div class="container">
<h1 class="mt-5">News Aggregator Chatbot</h1>
<form id="news-form" class="mt-4">
<div class="form-group">
<label for="category">Select News Category</label>
<select class="form-control" id="category" name="category">
<option value="general">General</option>
<option value="business">Business</option>
<option value="technology">Technology</option>
<option value="sports">Sports</option>
<option value="entertainment">Entertainment</option>
</select>
</div>
<button type="submit" class="btn btn-primary">Get News</button>
</form>
<div id="news-articles" class="mt-4"></div>
</div>
<script src="<https://code.jquery.com/jquery-3.5.1.min.js>"></script>
<script>
$(document).ready(function() {
$('#news-form').on('submit', function(event) {
event.preventDefault();
const category = $('#category').val();
$.post('/get_news', { category: category }, function(data) {
let articlesHtml = '<div class="list-group">';
data.forEach(article => {
articlesHtml += `
<div class="list-group-item">
<h5 class="mb-1">${article.title}</h5>
<p class="mb-1">${article.description}</p>
<small>${article.source} - ${article.publishedAt}</small>
<button class="btn btn-secondary btn-sm mt-2" onclick="categorizeArticle('${article.content}')">Categorize</button>
<div class="category mt-2" id="category-${article.title}"></div>
<button class="btn btn-secondary btn-sm mt-2" onclick="summarizeArticle('${article.content}')">Summarize</button>
<div class="summary mt-2" id="summary-${article.title}"></div>
</div>
`;
});
articlesHtml += '</div>';
$('#news-articles').html(articlesHtml);
});
});
});
function summarizeArticle(article) {
const summaryType = 'extractive'; // or 'abstractive'
$.post('/summarize_article', { article: article, summary_type: summaryType }, function(data) {
$('#summary-' + article.title).html('<p>' + data.summary + '</p>');
});
}
function categorizeArticle(article) {
$.post('/categorize_article', { article: article }, function(data) {
$('#category-' + article.title).html('<p>' + data.category.join(', ') + '</p>');
});
}
</script>
</body>
</html>In this updated template, we add buttons for categorizing each article. The categorizeArticle function sends a request to the /categorize_article endpoint and displays the category within the page.
In this section, we built the user interface for the news aggregator chatbot. We used Flask to create a web-based interface, allowing users to request news updates, view summaries, and categorize news articles.
We created HTML templates with Bootstrap for a responsive and visually appealing design. The frontend interacts with the backend to fetch news articles, summarize them, and categorize them into different topics.
12.4 Building the User Interface
In this section, we will focus on building the user interface (UI) for the news aggregator chatbot. The UI is crucial as it allows users to interact with the chatbot and access the news articles. We will create a web-based interface using Flask, a lightweight web framework for Python. This interface will enable users to request news updates, view summaries, and explore categorized news articles.
12.4.1 Setting Up Flask
First, we need to install Flask if we haven't already:
pip install FlaskNext, let's set up the basic structure of our Flask application.
from flask import Flask, request, jsonify, render_template
from nlp_engine import preprocess_text, categorize_article
from summarizer import summarize_text, abstractive_summarize_text
from news_fetcher import fetch_news
app = Flask(__name__)
@app.route('/')
def home():
return render_template('index.html')
@app.route('/get_news', methods=['POST'])
def get_news():
category = request.form.get('category')
articles = fetch_news(category)
return jsonify(articles)
@app.route('/summarize', methods=['POST'])
def summarize():
article = request.form.get('article')
summary_type = request.form.get('summary_type')
if summary_type == 'extractive':
summary = summarize_text(article)
else:
summary = abstractive_summarize_text(article)
return jsonify({'summary': summary})
if __name__ == '__main__':
app.run(debug=True)In this script, we define the basic routes for our Flask application:
- The
homeroute renders the home page. - The
get_newsroute fetches news articles based on the selected category. - The
summarizeroute summarizes a given article using either extractive or abstractive summarization.
12.4.2 Creating HTML Templates
Next, we will create HTML templates to render the web pages. We will use Bootstrap for styling to create a responsive and visually appealing interface.
templates/index.html:
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>News Aggregator Chatbot</title>
<link rel="stylesheet" href="<https://stackpath.bootstrapcdn.com/bootstrap/4.5.2/css/bootstrap.min.css>">
</head>
<body>
<div class="container">
<h1 class="mt-5">News Aggregator Chatbot</h1>
<form id="news-form" class="mt-4">
<div class="form-group">
<label for="category">Select News Category</label>
<select class="form-control" id="category" name="category">
<option value="general">General</option>
<option value="business">Business</option>
<option value="technology">Technology</option>
<option value="sports">Sports</option>
<option value="entertainment">Entertainment</option>
</select>
</div>
<button type="submit" class="btn btn-primary">Get News</button>
</form>
<div id="news-articles" class="mt-4"></div>
</div>
<script src="<https://code.jquery.com/jquery-3.5.1.min.js>"></script>
<script>
$(document).ready(function() {
$('#news-form').on('submit', function(event) {
event.preventDefault();
const category = $('#category').val();
$.post('/get_news', { category: category }, function(data) {
let articlesHtml = '<div class="list-group">';
data.forEach(article => {
articlesHtml += `
<a href="${article.url}" target="_blank" class="list-group-item list-group-item-action">
<h5 class="mb-1">${article.title}</h5>
<p class="mb-1">${article.description}</p>
<small>${article.source} - ${article.publishedAt}</small>
</a>
`;
});
articlesHtml += '</div>';
$('#news-articles').html(articlesHtml);
});
});
});
</script>
</body>
</html>In this HTML template, we create a simple form to select the news category and a div to display the fetched news articles. We use jQuery to handle the form submission and dynamically update the page with the fetched news articles.
12.4.3 Integrating Summarization and Categorization
To enable users to view summaries and categorize articles, we need to add additional functionality to the frontend and backend.
app.py (continued):
@app.route('/categorize', methods=['POST'])
def categorize():
article = request.form.get('article')
category = categorize_article(article)
return jsonify({'category': category})
@app.route('/summarize_article', methods=['POST'])
def summarize_article():
article = request.form.get('article')
summary_type = request.form.get('summary_type')
if summary_type == 'extractive':
summary = summarize_text(article)
else:
summary = abstractive_summarize_text(article)
return jsonify({'summary': summary})
# Updated HTML template to include summarization and categorization
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>News Aggregator Chatbot</title>
<link rel="stylesheet" href="<https://stackpath.bootstrapcdn.com/bootstrap/4.5.2/css/bootstrap.min.css>">
</head>
<body>
<div class="container">
<h1 class="mt-5">News Aggregator Chatbot</h1>
<form id="news-form" class="mt-4">
<div class="form-group">
<label for="category">Select News Category</label>
<select class="form-control" id="category" name="category">
<option value="general">General</option>
<option value="business">Business</option>
<option value="technology">Technology</option>
<option value="sports">Sports</option>
<option value="entertainment">Entertainment</option>
</select>
</div>
<button type="submit" class="btn btn-primary">Get News</button>
</form>
<div id="news-articles" class="mt-4"></div>
</div>
<script src="<https://code.jquery.com/jquery-3.5.1.min.js>"></script>
<script>
$(document).ready(function() {
$('#news-form').on('submit', function(event) {
event.preventDefault();
const category = $('#category').val();
$.post('/get_news', { category: category }, function(data) {
let articlesHtml = '<div class="list-group">';
data.forEach(article => {
articlesHtml += `
<div class="list-group-item">
<h5 class="mb-1">${article.title}</h5>
<p class="mb-1">${article.description}</p>
<small>${article.source} - ${article.publishedAt}</small>
<button class="btn btn-secondary btn-sm mt-2" onclick="summarizeArticle('${article.content}')">Summarize</button>
<div class="summary mt-2" id="summary-${article.title}"></div>
</div>
`;
});
articlesHtml += '</div>';
$('#news-articles').html(articlesHtml);
});
});
});
function summarizeArticle(article) {
const summaryType = 'extractive'; // or 'abstractive'
$.post('/summarize_article', { article: article, summary_type: summaryType }, function(data) {
$('#summary-' + article.title).html('<p>' + data.summary + '</p>');
});
}
</script>
</body>
</html>In this updated template, we add buttons for summarizing each article. The summarizeArticle function sends a request to the /summarize_article endpoint and displays the summary within the page.
12.4.4 Categorizing Articles
We will integrate the categorization functionality to automatically categorize news articles into different topics.
nlp_engine.py (continued):
# Define function to categorize article
def categorize_article(article):
bow = dictionary.doc2bow(nltk.word_tokenize(article.lower()))
topics = lda_model.get_document_topics(bow)
sorted_topics = sorted(topics, key=lambda x: x[1], reverse=True)
topic_index = sorted_topics[0][0]
topic_terms = lda_model.show_topic(topic_index, topn=4)
topic_keywords = [term[0] for term in topic_terms]
return topic_keywordsThis function uses the trained LDA model to categorize the article into topics based on the most relevant keywords.
Updated Flask Application:
@app.route('/categorize_article', methods=['POST'])
def categorize_article_route():
article = request.form.get('article')
category = categorize_article(article)
return jsonify({'category': category})
# HTML template for categorizing articles
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width,
initial-scale=1.0">
<title>News Aggregator Chatbot</title>
<link rel="stylesheet" href="<https://stackpath.bootstrapcdn.com/bootstrap/4.5.2/css/bootstrap.min.css>">
</head>
<body>
<div class="container">
<h1 class="mt-5">News Aggregator Chatbot</h1>
<form id="news-form" class="mt-4">
<div class="form-group">
<label for="category">Select News Category</label>
<select class="form-control" id="category" name="category">
<option value="general">General</option>
<option value="business">Business</option>
<option value="technology">Technology</option>
<option value="sports">Sports</option>
<option value="entertainment">Entertainment</option>
</select>
</div>
<button type="submit" class="btn btn-primary">Get News</button>
</form>
<div id="news-articles" class="mt-4"></div>
</div>
<script src="<https://code.jquery.com/jquery-3.5.1.min.js>"></script>
<script>
$(document).ready(function() {
$('#news-form').on('submit', function(event) {
event.preventDefault();
const category = $('#category').val();
$.post('/get_news', { category: category }, function(data) {
let articlesHtml = '<div class="list-group">';
data.forEach(article => {
articlesHtml += `
<div class="list-group-item">
<h5 class="mb-1">${article.title}</h5>
<p class="mb-1">${article.description}</p>
<small>${article.source} - ${article.publishedAt}</small>
<button class="btn btn-secondary btn-sm mt-2" onclick="categorizeArticle('${article.content}')">Categorize</button>
<div class="category mt-2" id="category-${article.title}"></div>
<button class="btn btn-secondary btn-sm mt-2" onclick="summarizeArticle('${article.content}')">Summarize</button>
<div class="summary mt-2" id="summary-${article.title}"></div>
</div>
`;
});
articlesHtml += '</div>';
$('#news-articles').html(articlesHtml);
});
});
});
function summarizeArticle(article) {
const summaryType = 'extractive'; // or 'abstractive'
$.post('/summarize_article', { article: article, summary_type: summaryType }, function(data) {
$('#summary-' + article.title).html('<p>' + data.summary + '</p>');
});
}
function categorizeArticle(article) {
$.post('/categorize_article', { article: article }, function(data) {
$('#category-' + article.title).html('<p>' + data.category.join(', ') + '</p>');
});
}
</script>
</body>
</html>In this updated template, we add buttons for categorizing each article. The categorizeArticle function sends a request to the /categorize_article endpoint and displays the category within the page.
In this section, we built the user interface for the news aggregator chatbot. We used Flask to create a web-based interface, allowing users to request news updates, view summaries, and categorize news articles.
We created HTML templates with Bootstrap for a responsive and visually appealing design. The frontend interacts with the backend to fetch news articles, summarize them, and categorize them into different topics.
12.4 Building the User Interface
In this section, we will focus on building the user interface (UI) for the news aggregator chatbot. The UI is crucial as it allows users to interact with the chatbot and access the news articles. We will create a web-based interface using Flask, a lightweight web framework for Python. This interface will enable users to request news updates, view summaries, and explore categorized news articles.
12.4.1 Setting Up Flask
First, we need to install Flask if we haven't already:
pip install FlaskNext, let's set up the basic structure of our Flask application.
from flask import Flask, request, jsonify, render_template
from nlp_engine import preprocess_text, categorize_article
from summarizer import summarize_text, abstractive_summarize_text
from news_fetcher import fetch_news
app = Flask(__name__)
@app.route('/')
def home():
return render_template('index.html')
@app.route('/get_news', methods=['POST'])
def get_news():
category = request.form.get('category')
articles = fetch_news(category)
return jsonify(articles)
@app.route('/summarize', methods=['POST'])
def summarize():
article = request.form.get('article')
summary_type = request.form.get('summary_type')
if summary_type == 'extractive':
summary = summarize_text(article)
else:
summary = abstractive_summarize_text(article)
return jsonify({'summary': summary})
if __name__ == '__main__':
app.run(debug=True)In this script, we define the basic routes for our Flask application:
- The
homeroute renders the home page. - The
get_newsroute fetches news articles based on the selected category. - The
summarizeroute summarizes a given article using either extractive or abstractive summarization.
12.4.2 Creating HTML Templates
Next, we will create HTML templates to render the web pages. We will use Bootstrap for styling to create a responsive and visually appealing interface.
templates/index.html:
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>News Aggregator Chatbot</title>
<link rel="stylesheet" href="<https://stackpath.bootstrapcdn.com/bootstrap/4.5.2/css/bootstrap.min.css>">
</head>
<body>
<div class="container">
<h1 class="mt-5">News Aggregator Chatbot</h1>
<form id="news-form" class="mt-4">
<div class="form-group">
<label for="category">Select News Category</label>
<select class="form-control" id="category" name="category">
<option value="general">General</option>
<option value="business">Business</option>
<option value="technology">Technology</option>
<option value="sports">Sports</option>
<option value="entertainment">Entertainment</option>
</select>
</div>
<button type="submit" class="btn btn-primary">Get News</button>
</form>
<div id="news-articles" class="mt-4"></div>
</div>
<script src="<https://code.jquery.com/jquery-3.5.1.min.js>"></script>
<script>
$(document).ready(function() {
$('#news-form').on('submit', function(event) {
event.preventDefault();
const category = $('#category').val();
$.post('/get_news', { category: category }, function(data) {
let articlesHtml = '<div class="list-group">';
data.forEach(article => {
articlesHtml += `
<a href="${article.url}" target="_blank" class="list-group-item list-group-item-action">
<h5 class="mb-1">${article.title}</h5>
<p class="mb-1">${article.description}</p>
<small>${article.source} - ${article.publishedAt}</small>
</a>
`;
});
articlesHtml += '</div>';
$('#news-articles').html(articlesHtml);
});
});
});
</script>
</body>
</html>In this HTML template, we create a simple form to select the news category and a div to display the fetched news articles. We use jQuery to handle the form submission and dynamically update the page with the fetched news articles.
12.4.3 Integrating Summarization and Categorization
To enable users to view summaries and categorize articles, we need to add additional functionality to the frontend and backend.
app.py (continued):
@app.route('/categorize', methods=['POST'])
def categorize():
article = request.form.get('article')
category = categorize_article(article)
return jsonify({'category': category})
@app.route('/summarize_article', methods=['POST'])
def summarize_article():
article = request.form.get('article')
summary_type = request.form.get('summary_type')
if summary_type == 'extractive':
summary = summarize_text(article)
else:
summary = abstractive_summarize_text(article)
return jsonify({'summary': summary})
# Updated HTML template to include summarization and categorization
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>News Aggregator Chatbot</title>
<link rel="stylesheet" href="<https://stackpath.bootstrapcdn.com/bootstrap/4.5.2/css/bootstrap.min.css>">
</head>
<body>
<div class="container">
<h1 class="mt-5">News Aggregator Chatbot</h1>
<form id="news-form" class="mt-4">
<div class="form-group">
<label for="category">Select News Category</label>
<select class="form-control" id="category" name="category">
<option value="general">General</option>
<option value="business">Business</option>
<option value="technology">Technology</option>
<option value="sports">Sports</option>
<option value="entertainment">Entertainment</option>
</select>
</div>
<button type="submit" class="btn btn-primary">Get News</button>
</form>
<div id="news-articles" class="mt-4"></div>
</div>
<script src="<https://code.jquery.com/jquery-3.5.1.min.js>"></script>
<script>
$(document).ready(function() {
$('#news-form').on('submit', function(event) {
event.preventDefault();
const category = $('#category').val();
$.post('/get_news', { category: category }, function(data) {
let articlesHtml = '<div class="list-group">';
data.forEach(article => {
articlesHtml += `
<div class="list-group-item">
<h5 class="mb-1">${article.title}</h5>
<p class="mb-1">${article.description}</p>
<small>${article.source} - ${article.publishedAt}</small>
<button class="btn btn-secondary btn-sm mt-2" onclick="summarizeArticle('${article.content}')">Summarize</button>
<div class="summary mt-2" id="summary-${article.title}"></div>
</div>
`;
});
articlesHtml += '</div>';
$('#news-articles').html(articlesHtml);
});
});
});
function summarizeArticle(article) {
const summaryType = 'extractive'; // or 'abstractive'
$.post('/summarize_article', { article: article, summary_type: summaryType }, function(data) {
$('#summary-' + article.title).html('<p>' + data.summary + '</p>');
});
}
</script>
</body>
</html>In this updated template, we add buttons for summarizing each article. The summarizeArticle function sends a request to the /summarize_article endpoint and displays the summary within the page.
12.4.4 Categorizing Articles
We will integrate the categorization functionality to automatically categorize news articles into different topics.
nlp_engine.py (continued):
# Define function to categorize article
def categorize_article(article):
bow = dictionary.doc2bow(nltk.word_tokenize(article.lower()))
topics = lda_model.get_document_topics(bow)
sorted_topics = sorted(topics, key=lambda x: x[1], reverse=True)
topic_index = sorted_topics[0][0]
topic_terms = lda_model.show_topic(topic_index, topn=4)
topic_keywords = [term[0] for term in topic_terms]
return topic_keywordsThis function uses the trained LDA model to categorize the article into topics based on the most relevant keywords.
Updated Flask Application:
@app.route('/categorize_article', methods=['POST'])
def categorize_article_route():
article = request.form.get('article')
category = categorize_article(article)
return jsonify({'category': category})
# HTML template for categorizing articles
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width,
initial-scale=1.0">
<title>News Aggregator Chatbot</title>
<link rel="stylesheet" href="<https://stackpath.bootstrapcdn.com/bootstrap/4.5.2/css/bootstrap.min.css>">
</head>
<body>
<div class="container">
<h1 class="mt-5">News Aggregator Chatbot</h1>
<form id="news-form" class="mt-4">
<div class="form-group">
<label for="category">Select News Category</label>
<select class="form-control" id="category" name="category">
<option value="general">General</option>
<option value="business">Business</option>
<option value="technology">Technology</option>
<option value="sports">Sports</option>
<option value="entertainment">Entertainment</option>
</select>
</div>
<button type="submit" class="btn btn-primary">Get News</button>
</form>
<div id="news-articles" class="mt-4"></div>
</div>
<script src="<https://code.jquery.com/jquery-3.5.1.min.js>"></script>
<script>
$(document).ready(function() {
$('#news-form').on('submit', function(event) {
event.preventDefault();
const category = $('#category').val();
$.post('/get_news', { category: category }, function(data) {
let articlesHtml = '<div class="list-group">';
data.forEach(article => {
articlesHtml += `
<div class="list-group-item">
<h5 class="mb-1">${article.title}</h5>
<p class="mb-1">${article.description}</p>
<small>${article.source} - ${article.publishedAt}</small>
<button class="btn btn-secondary btn-sm mt-2" onclick="categorizeArticle('${article.content}')">Categorize</button>
<div class="category mt-2" id="category-${article.title}"></div>
<button class="btn btn-secondary btn-sm mt-2" onclick="summarizeArticle('${article.content}')">Summarize</button>
<div class="summary mt-2" id="summary-${article.title}"></div>
</div>
`;
});
articlesHtml += '</div>';
$('#news-articles').html(articlesHtml);
});
});
});
function summarizeArticle(article) {
const summaryType = 'extractive'; // or 'abstractive'
$.post('/summarize_article', { article: article, summary_type: summaryType }, function(data) {
$('#summary-' + article.title).html('<p>' + data.summary + '</p>');
});
}
function categorizeArticle(article) {
$.post('/categorize_article', { article: article }, function(data) {
$('#category-' + article.title).html('<p>' + data.category.join(', ') + '</p>');
});
}
</script>
</body>
</html>In this updated template, we add buttons for categorizing each article. The categorizeArticle function sends a request to the /categorize_article endpoint and displays the category within the page.
In this section, we built the user interface for the news aggregator chatbot. We used Flask to create a web-based interface, allowing users to request news updates, view summaries, and categorize news articles.
We created HTML templates with Bootstrap for a responsive and visually appealing design. The frontend interacts with the backend to fetch news articles, summarize them, and categorize them into different topics.
12.4 Building the User Interface
In this section, we will focus on building the user interface (UI) for the news aggregator chatbot. The UI is crucial as it allows users to interact with the chatbot and access the news articles. We will create a web-based interface using Flask, a lightweight web framework for Python. This interface will enable users to request news updates, view summaries, and explore categorized news articles.
12.4.1 Setting Up Flask
First, we need to install Flask if we haven't already:
pip install FlaskNext, let's set up the basic structure of our Flask application.
from flask import Flask, request, jsonify, render_template
from nlp_engine import preprocess_text, categorize_article
from summarizer import summarize_text, abstractive_summarize_text
from news_fetcher import fetch_news
app = Flask(__name__)
@app.route('/')
def home():
return render_template('index.html')
@app.route('/get_news', methods=['POST'])
def get_news():
category = request.form.get('category')
articles = fetch_news(category)
return jsonify(articles)
@app.route('/summarize', methods=['POST'])
def summarize():
article = request.form.get('article')
summary_type = request.form.get('summary_type')
if summary_type == 'extractive':
summary = summarize_text(article)
else:
summary = abstractive_summarize_text(article)
return jsonify({'summary': summary})
if __name__ == '__main__':
app.run(debug=True)In this script, we define the basic routes for our Flask application:
- The
homeroute renders the home page. - The
get_newsroute fetches news articles based on the selected category. - The
summarizeroute summarizes a given article using either extractive or abstractive summarization.
12.4.2 Creating HTML Templates
Next, we will create HTML templates to render the web pages. We will use Bootstrap for styling to create a responsive and visually appealing interface.
templates/index.html:
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>News Aggregator Chatbot</title>
<link rel="stylesheet" href="<https://stackpath.bootstrapcdn.com/bootstrap/4.5.2/css/bootstrap.min.css>">
</head>
<body>
<div class="container">
<h1 class="mt-5">News Aggregator Chatbot</h1>
<form id="news-form" class="mt-4">
<div class="form-group">
<label for="category">Select News Category</label>
<select class="form-control" id="category" name="category">
<option value="general">General</option>
<option value="business">Business</option>
<option value="technology">Technology</option>
<option value="sports">Sports</option>
<option value="entertainment">Entertainment</option>
</select>
</div>
<button type="submit" class="btn btn-primary">Get News</button>
</form>
<div id="news-articles" class="mt-4"></div>
</div>
<script src="<https://code.jquery.com/jquery-3.5.1.min.js>"></script>
<script>
$(document).ready(function() {
$('#news-form').on('submit', function(event) {
event.preventDefault();
const category = $('#category').val();
$.post('/get_news', { category: category }, function(data) {
let articlesHtml = '<div class="list-group">';
data.forEach(article => {
articlesHtml += `
<a href="${article.url}" target="_blank" class="list-group-item list-group-item-action">
<h5 class="mb-1">${article.title}</h5>
<p class="mb-1">${article.description}</p>
<small>${article.source} - ${article.publishedAt}</small>
</a>
`;
});
articlesHtml += '</div>';
$('#news-articles').html(articlesHtml);
});
});
});
</script>
</body>
</html>In this HTML template, we create a simple form to select the news category and a div to display the fetched news articles. We use jQuery to handle the form submission and dynamically update the page with the fetched news articles.
12.4.3 Integrating Summarization and Categorization
To enable users to view summaries and categorize articles, we need to add additional functionality to the frontend and backend.
app.py (continued):
@app.route('/categorize', methods=['POST'])
def categorize():
article = request.form.get('article')
category = categorize_article(article)
return jsonify({'category': category})
@app.route('/summarize_article', methods=['POST'])
def summarize_article():
article = request.form.get('article')
summary_type = request.form.get('summary_type')
if summary_type == 'extractive':
summary = summarize_text(article)
else:
summary = abstractive_summarize_text(article)
return jsonify({'summary': summary})
# Updated HTML template to include summarization and categorization
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>News Aggregator Chatbot</title>
<link rel="stylesheet" href="<https://stackpath.bootstrapcdn.com/bootstrap/4.5.2/css/bootstrap.min.css>">
</head>
<body>
<div class="container">
<h1 class="mt-5">News Aggregator Chatbot</h1>
<form id="news-form" class="mt-4">
<div class="form-group">
<label for="category">Select News Category</label>
<select class="form-control" id="category" name="category">
<option value="general">General</option>
<option value="business">Business</option>
<option value="technology">Technology</option>
<option value="sports">Sports</option>
<option value="entertainment">Entertainment</option>
</select>
</div>
<button type="submit" class="btn btn-primary">Get News</button>
</form>
<div id="news-articles" class="mt-4"></div>
</div>
<script src="<https://code.jquery.com/jquery-3.5.1.min.js>"></script>
<script>
$(document).ready(function() {
$('#news-form').on('submit', function(event) {
event.preventDefault();
const category = $('#category').val();
$.post('/get_news', { category: category }, function(data) {
let articlesHtml = '<div class="list-group">';
data.forEach(article => {
articlesHtml += `
<div class="list-group-item">
<h5 class="mb-1">${article.title}</h5>
<p class="mb-1">${article.description}</p>
<small>${article.source} - ${article.publishedAt}</small>
<button class="btn btn-secondary btn-sm mt-2" onclick="summarizeArticle('${article.content}')">Summarize</button>
<div class="summary mt-2" id="summary-${article.title}"></div>
</div>
`;
});
articlesHtml += '</div>';
$('#news-articles').html(articlesHtml);
});
});
});
function summarizeArticle(article) {
const summaryType = 'extractive'; // or 'abstractive'
$.post('/summarize_article', { article: article, summary_type: summaryType }, function(data) {
$('#summary-' + article.title).html('<p>' + data.summary + '</p>');
});
}
</script>
</body>
</html>In this updated template, we add buttons for summarizing each article. The summarizeArticle function sends a request to the /summarize_article endpoint and displays the summary within the page.
12.4.4 Categorizing Articles
We will integrate the categorization functionality to automatically categorize news articles into different topics.
nlp_engine.py (continued):
# Define function to categorize article
def categorize_article(article):
bow = dictionary.doc2bow(nltk.word_tokenize(article.lower()))
topics = lda_model.get_document_topics(bow)
sorted_topics = sorted(topics, key=lambda x: x[1], reverse=True)
topic_index = sorted_topics[0][0]
topic_terms = lda_model.show_topic(topic_index, topn=4)
topic_keywords = [term[0] for term in topic_terms]
return topic_keywordsThis function uses the trained LDA model to categorize the article into topics based on the most relevant keywords.
Updated Flask Application:
@app.route('/categorize_article', methods=['POST'])
def categorize_article_route():
article = request.form.get('article')
category = categorize_article(article)
return jsonify({'category': category})
# HTML template for categorizing articles
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width,
initial-scale=1.0">
<title>News Aggregator Chatbot</title>
<link rel="stylesheet" href="<https://stackpath.bootstrapcdn.com/bootstrap/4.5.2/css/bootstrap.min.css>">
</head>
<body>
<div class="container">
<h1 class="mt-5">News Aggregator Chatbot</h1>
<form id="news-form" class="mt-4">
<div class="form-group">
<label for="category">Select News Category</label>
<select class="form-control" id="category" name="category">
<option value="general">General</option>
<option value="business">Business</option>
<option value="technology">Technology</option>
<option value="sports">Sports</option>
<option value="entertainment">Entertainment</option>
</select>
</div>
<button type="submit" class="btn btn-primary">Get News</button>
</form>
<div id="news-articles" class="mt-4"></div>
</div>
<script src="<https://code.jquery.com/jquery-3.5.1.min.js>"></script>
<script>
$(document).ready(function() {
$('#news-form').on('submit', function(event) {
event.preventDefault();
const category = $('#category').val();
$.post('/get_news', { category: category }, function(data) {
let articlesHtml = '<div class="list-group">';
data.forEach(article => {
articlesHtml += `
<div class="list-group-item">
<h5 class="mb-1">${article.title}</h5>
<p class="mb-1">${article.description}</p>
<small>${article.source} - ${article.publishedAt}</small>
<button class="btn btn-secondary btn-sm mt-2" onclick="categorizeArticle('${article.content}')">Categorize</button>
<div class="category mt-2" id="category-${article.title}"></div>
<button class="btn btn-secondary btn-sm mt-2" onclick="summarizeArticle('${article.content}')">Summarize</button>
<div class="summary mt-2" id="summary-${article.title}"></div>
</div>
`;
});
articlesHtml += '</div>';
$('#news-articles').html(articlesHtml);
});
});
});
function summarizeArticle(article) {
const summaryType = 'extractive'; // or 'abstractive'
$.post('/summarize_article', { article: article, summary_type: summaryType }, function(data) {
$('#summary-' + article.title).html('<p>' + data.summary + '</p>');
});
}
function categorizeArticle(article) {
$.post('/categorize_article', { article: article }, function(data) {
$('#category-' + article.title).html('<p>' + data.category.join(', ') + '</p>');
});
}
</script>
</body>
</html>In this updated template, we add buttons for categorizing each article. The categorizeArticle function sends a request to the /categorize_article endpoint and displays the category within the page.
In this section, we built the user interface for the news aggregator chatbot. We used Flask to create a web-based interface, allowing users to request news updates, view summaries, and categorize news articles.
We created HTML templates with Bootstrap for a responsive and visually appealing design. The frontend interacts with the backend to fetch news articles, summarize them, and categorize them into different topics.