Click here to view the next lesson.

Chapter 4: Intermediate Level Exercises

Intermediate Level Exercises

Exercise 1: Web Scraping

Concepts:

HTTP requests using the requests module
HTML parsing using BeautifulSoup

Description: Web scraping is a common task for many Python programmers. In this exercise, you'll write a script that scrapes data from a website.

Solution:

import requests
from bs4 import BeautifulSoup

url = 'https://www.example.com'
response = requests.get(url)
soup = BeautifulSoup(response.text, 'html.parser')

# Find all links on the page
links = soup.find_all('a')
for link in links:
    print(link.get('href'))

Exercise 2: File I/O

Concepts:

Reading and writing files using Python's built-in open() function
String manipulation

Description: Working with files is an essential part of any programming language. In this exercise, you'll write a script that reads data from a file and writes data to a file.

Solution:

# Read data from a file
with open('input.txt', 'r') as f:
    data = f.read()

# Manipulate the data
data = data.upper()

# Write data to a file
with open('output.txt', 'w') as f:
    f.write(data)

Exercise 3: Data Analysis

Concepts:

Reading data from a CSV file using Pandas
Data manipulation using Pandas
Basic data analysis using Pandas

Description: Python has a vast ecosystem of libraries for data analysis. In this exercise, you'll use the Pandas library to read data from a CSV file and perform some basic data analysis.

Solution:

import pandas as pd

# Read data from a CSV file
data = pd.read_csv('data.csv')

# Display the first 5 rows of the data
print(data.head())

# Calculate some basic statistics
print(data.describe())

Exercise 4: Command-Line Interface

Concepts:

Command-line arguments using the argparse module
String manipulation

Description: Python can be used to create command-line tools. In this exercise, you'll write a script that accepts command-line arguments and performs a task based on those arguments.

Solution:

import argparse

# Create an argument parser
parser = argparse.ArgumentParser()
parser.add_argument('name', help='Your name')
parser.add_argument('age', type=int, help='Your age')
args = parser.parse_args()

# Print a greeting
greeting = f'Hello, {args.name}! You are {args.age} years old.'
print(greeting)

Exercise 5: API Integration

Concepts:

HTTP requests using the requests module
JSON parsing

Description: Many modern applications have APIs that can be accessed programmatically. In this exercise, you'll write a script that interacts with a RESTful API.

Solution:

import requests

# Make a GET request to an API
url = 'https://api.example.com/users'
response = requests.get(url)

# Parse the response as JSON
data = response.json()

# Display the data
for user in data:
    print(user['name'], user['email'])

Exercise 6: Regular Expressions

Concepts:

Regular expressions using the re module
User input validation

Description: Regular expressions are powerful tools for pattern matching and data validation. In this exercise, you'll write a script that uses regular expressions to validate user input.

Solution:

import re

# Define a regular expression pattern for email validation
pattern = r'^[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}$'

# Get user input
email = input('Enter your email address: ')

# Validate the input using the regular expression
if re.match(pattern, email):
    print('Valid email address')
else:
    print('Invalid email address')

Exercise 7: Object-Oriented Programming

Concepts:

Classes and objects
Encapsulation
Inheritance

Description: Object-oriented programming is a fundamental concept in Python. In this exercise, you'll write a simple class that demonstrates the principles of object-oriented programming.

Solution:

class Person:
    def __init__(self, name, age):
        self.name = name
        self.age = age

    def greet(self):
        print(f'Hello, my name is {self.name} and I am {self.age} years old.')

class Student(Person):
    def __init__(self, name, age, major):
        super().__init__(name, age)
        self.major = major

    def describe(self):
        print(f'I am a student majoring in {self.major}.')

# Create objects and call methods
person = Person('Alice', 25)
person.greet()

student = Student('Bob', 20, 'Computer Science')
student.greet()
student.describe()

Exercise 8: Concurrency

Concepts:

Threading using the threading module
Synchronization using locks

Description: Python has powerful concurrency features that allow you to write concurrent and parallel programs. In this exercise, you'll write a script that demonstrates concurrent programming using threads.

Solution:

import threading

counter = 0
lock = threading.Lock()

def increment():
    global counter
    with lock:
        counter += 1

threads = []
for i in range(10):
    t = threading.Thread(target=increment)
    threads.append(t)
    t.start()

for t in threads:
    t.join()

print(f'The counter is {counter}.')

Exercise 9: Testing

Concepts:

Unit testing using the unittest module
Test-driven development

Description: Testing is a crucial part of software development. In this exercise, you'll write unit tests for a simple function.

Solution:

import unittest

def is_palindrome(s):
    return s == s[::-1]

class TestPalindrome(unittest.TestCase):
    def test_is_palindrome(self):
        self.assertTrue(is_palindrome('racecar'))
        self.assertFalse(is_palindrome('hello'))

if __name__ == '__main__':
    unittest.main()

Exercise 10: Data Visualization

Concepts:

Data visualization using Matplotlib
Line charts

Description: Data visualization is an essential part of data analysis. In this exercise, you'll use the Matplotlib library to create a simple line chart.

Solution:

import matplotlib.pyplot as plt

# Create some data
x = [1, 2, 3, 4, 5]
y = [2, 4, 6, 8, 10]

# Create a line chart
plt.plot(x, y)

# Add labels and a title
plt.xlabel('X values')
plt.ylabel('Y values')
plt.title('Line chart')

# Display the chart
plt.show()

Exercise 11: Database Operations

Concepts:

Connecting to a database using the SQLite3 module
Creating tables and inserting data
Querying data using SQL

Description: Python has a wide variety of libraries for working with databases. In this exercise, you'll write a script that performs some common database operations.

Solution:

import sqlite3

# Connect to the database
conn = sqlite3.connect('example.db')

# Create a table
conn.execute('''CREATE TABLE IF NOT EXISTS users
                (id INTEGER PRIMARY KEY,
                 name TEXT,
                 email TEXT)''')

# Insert some data
conn.execute("INSERT INTO users (name, email) VALUES (?, ?)", ('Alice', 'alice@example.com'))
conn.execute("INSERT INTO users (name, email) VALUES (?, ?)", ('Bob', 'bob@example.com'))

# Query the data
cursor = conn.execute("SELECT id, name, email FROM users")
for row in cursor:
    print(f'{row[0]} - {row[1]} ({row[2]})')

# Close the database connection
conn.close()

Exercise 12: Networking

Concepts:

Socket programming using the socket module
Server-client architecture

Description: Python has excellent networking capabilities, allowing you to build network applications. In this exercise, you'll write a script that implements a simple server and client.

Solution:

# Server code
import socket

server_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server_socket.bind(('localhost', 8000))
server_socket.listen(1)

while True:
    client_socket, address = server_socket.accept()
    data = client_socket.recv(1024)
    response = data.upper()
    client_socket.sendall(response)
    client_socket.close()

# Client code
import socket

client_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
client_socket.connect(('localhost', 8000))
client_socket.sendall(b'Hello, server!')
response = client_socket.recv(1024)
print(response)
client_socket.close()

Exercise 13: Data Science

Concepts:

Data manipulation using NumPy and Pandas
Data analysis using Pandas

Description: Python is a popular language for data science due to its powerful libraries. In this exercise, you'll use the NumPy and Pandas libraries to perform some data analysis.

Solution:

import numpy as np
import pandas as pd

# Create some data
data = {'x': np.arange(10),
        'y': np.random.randn(10)}

# Create a DataFrame from the data
df = pd.DataFrame(data)

# Calculate some statistics
print(df.mean())
print(df.std())

# Create a new column based on a calculation
df['z'] = df['x'] * df['y']

# Display the DataFrame
print(df)

Exercise 14: Web Development

Concepts:

Web development using the Flask framework
HTTP methods and routing

Description: Python can be used to build web applications using a variety of web frameworks. In this exercise, you'll write a simple Flask web application.

Solution:

from flask import Flask, request

app = Flask(__name__)

@app.route('/', methods=['GET'])
def home():
    return 'Hello, world!'

@app.route('/greet', methods=['POST'])
def greet():
    name = request.form.get('name')
    return f'Hello, {name}!'

if __name__ == '__main__':
    app.run()

Exercise 15: Asynchronous Programming

Concepts:

Asynchronous programming using asyncio
Coroutines

Description: Python has powerful features for asynchronous programming, allowing you to write efficient and scalable programs. In this exercise, you'll write a script that demonstrates asynchronous programming using asyncio.

Solution:

import asyncio

async def coroutine():
    print('Coroutine started')
    await asyncio.sleep(1)
    print('Coroutine ended')

loop = asyncio.get_event_loop()
loop.run_until_complete(coroutine())
loop.close()

This script defines a coroutine using the async keyword. The coroutine sleeps for one second using the asyncio.sleep() function. The asyncio.get_event_loop() function is used to create an event loop, and the loop.run_until_complete() method is used to run the coroutine until it completes. Finally, the event loop is closed using the loop.close() method.

Note that the above code demonstrates only the basic concepts of asynchronous programming with asyncio. More complex programs may require additional concepts and techniques, such as event loops, futures, and callbacks.

Exercise 16: Image Processing

Concepts:

Image processing using the Pillow library
Image manipulation and conversion

Description: Python has many libraries for working with images. In this exercise, you'll use the Pillow library to perform some basic image processing tasks.

Solution:

from PIL import Image

# Open an image file
image = Image.open('image.jpg')

# Display information about the image
print(image.format)
print(image.size)
print(image.mode)

# Convert the image to grayscale
grayscale_image = image.convert('L')
grayscale_image.show()

# Resize the image
resized_image = image.resize((300, 300))
resized_image.show()

# Save the image in a different format
resized_image.save('image.png')

Exercise 17: Email Sending

Concepts:

Email sending using the smtplib library
SMTP servers and email authentication

Description: Python has libraries for sending email programmatically. In this exercise, you'll write a script that sends an email using the smtplib library.

Solution:

import smtplib

sender_email = 'your_email@example.com'
sender_password = 'your_password'
receiver_email = 'recipient_email@example.com'
message = 'Hello, world!'

with smtplib.SMTP('smtp.gmail.com', 587) as server:
    server.starttls()
    server.login(sender_email, sender_password)
    server.sendmail(sender_email, receiver_email, message)

Exercise 18: Web API Integration

Concepts:

RESTful APIs and HTTP requests
Web API integration using the requests library

Description: Python can be used to integrate with many different web APIs. In this exercise, you'll write a script that interacts with a RESTful API using the requests library.

Solution:

import requests

response = requests.get('https://api.example.com/users')
data = response.json()

for user in data:
    print(user['name'], user['email'])

Exercise 19: Data Encryption

Concepts:

Data encryption using the cryptography library
Symmetric and asymmetric encryption

Description: Python has built-in libraries for encrypting and decrypting data. In this exercise, you'll write a script that encrypts and decrypts some data using the cryptography library.

Solution:

from cryptography.fernet import Fernet
from cryptography.hazmat.primitives import hashes
from cryptography.hazmat.primitives.asymmetric import rsa, padding
from cryptography.hazmat.primitives.serialization import load_pem_private_key, load_pem_public_key

# Symmetric encryption
key = Fernet.generate_key()
fernet = Fernet(key)
plaintext = b'This is a secret message'
ciphertext = fernet.encrypt(plaintext)
decrypted_plaintext = fernet.decrypt(ciphertext)
print(decrypted_plaintext)

# Asymmetric encryption
private_key = rsa.generate_private_key(
    public_exponent=65537,
    key_size=2048
)
public_key = private_key.public_key()

plaintext = b'This is a secret message'
ciphertext = public_key.encrypt(
    plaintext,
    padding.OAEP(
        mgf=padding.MGF1(algorithm=hashes.SHA256()),
        algorithm=hashes.SHA256(),
        label=None
    )
)

decrypted_plaintext = private_key.decrypt(
    ciphertext,
    padding.OAEP(
        mgf=padding.MGF1(algorithm=hashes.SHA256()),
        algorithm=hashes.SHA256(),
        label=None
    )
)
print(decrypted_plaintext)

Exercise 20: GUI Programming

Concepts:

GUI programming using the Tkinter library
Widgets and event handling

Description: Python has libraries for building graphical user interfaces (GUIs). In this exercise, you'll write a script that uses the Tkinter library to create a simple GUI.

Solution:

import tkinter as tk

def button_clicked():
    label.config(text='Hello, world!')

root = tk.Tk()

label = tk.Label(root, text='Welcome to my GUI')
label.pack()

button = tk.Button(root, text='Click me!', command=button_clicked)
button.pack()

root.mainloop()

This script creates a window using the tkinter.Tk() method. It then creates a Label widget and a Button widget and packs them into the window using the pack() method. Finally, it enters the main event loop using the mainloop() method.

When the button is clicked, the button_clicked() function is called, which updates the text of the label widget using the config() method.

Exercise 21: File I/O

Concepts:

File I/O in Python
Reading and writing files

Description: Python has built-in support for reading and writing files. In this exercise, you'll write a script that reads from and writes to a file.

Solution:

# Write to a file
with open('example.txt', 'w') as file:
    file.write('Hello, world!\n')
    file.write('How are you today?\n')

# Read from a file
with open('example.txt', 'r') as file:
    for line in file:
        print(line.strip())

Exercise 22: Logging

Concepts:

Logging in Python
Log levels and handlers

Description: Python has built-in support for logging. In this exercise, you'll write a script that logs some messages to a file.

Solution:

import logging

logging.basicConfig(filename='example.log', level=logging.DEBUG)

logging.debug('This is a debug message')
logging.info('This is an info message')
logging.warning('This is a warning message')
logging.error('This is an error message')
logging.critical('This is a critical message')

Exercise 23: Web Scraping

Concepts:

Web scraping using BeautifulSoup
HTML parsing and navigation

Description: Python can be used for web scraping, which involves extracting data from websites. In this exercise, you'll use the BeautifulSoup library to scrape data from a website.

Solution:

import requests
from bs4 import BeautifulSoup

response = requests.get('https://en.wikipedia.org/wiki/Python_(programming_language)')
soup = BeautifulSoup(response.text, 'html.parser')
print(soup.title.string)

links = soup.find_all('a')
for link in links:
    print(link.get('href'))

Exercise 24: Concurrency with asyncio

Concepts:

Asynchronous programming using asyncio
Coroutines and event loops

Description: Python has powerful support for asynchronous programming using the asyncio library. In this exercise, you'll write a script that demonstrates the use of asyncio for concurrent programming.

Solution:

import asyncio

async def coroutine(id):
    print(f'Coroutine {id} started')
    await asyncio.sleep(1)
    print(f'Coroutine {id} ended')

async def main():
    tasks = []
    for i in range(10):
        tasks.append(asyncio.create_task(coroutine(i)))
    await asyncio.gather(*tasks)

asyncio.run(main())

Exercise 25: Data Analysis with Pandas

Concepts:

Data analysis with Pandas
Loading and manipulating data with DataFrames

Description: Pandas is a powerful library for data analysis and manipulation in Python. In this exercise, you'll use Pandas to perform some data analysis tasks.

Solution:

import pandas as pd

# Load data from a CSV file
data = pd.read_csv('example.csv')

# Print the first five rows of the data
print(data.head())

# Calculate some statistics on the data
print(data.describe())

# Group the data by a column and calculate some statistics
grouped_data = data.groupby('category')
print(grouped_data.mean())

In this script, we load data from a CSV file into a Pandas DataFrame. We then print the first five rows of the data using the head() method, and calculate some statistics on the data using the describe() method.

Finally, we group the data by the category column using the groupby() method and calculate some statistics on each group using the mean() method.

Exercise 26: Regular Expressions

Concepts:

Regular expressions in Python
Pattern matching and substitution

Description: Regular expressions are a powerful tool for searching and manipulating text. In this exercise, you'll write a script that uses regular expressions to search for patterns in text.

Solution:

import re

text = 'The quick brown fox jumps over the lazy dog'

# Search for a pattern in the text
pattern = r'\b\w{4}\b'
matches = re.findall(pattern, text)
print(matches)

# Substitute a pattern in the text
pattern = r'\bthe\b'
replaced_text = re.sub(pattern, 'a', text, flags=re.IGNORECASE)
print(replaced_text)

Exercise 27: Database ORM

Concepts:

Object-relational mapping using SQLAlchemy
Creating tables and querying data

Description: Python has many object-relational mapping (ORM) libraries for working with databases. In this exercise, you'll write a script that uses the SQLAlchemy ORM to interact with a database.

Solution:

from sqlalchemy import create_engine, Column, Integer, String
from sqlalchemy.orm import sessionmaker
from sqlalchemy.ext.declarative import declarative_base

# Define a database engine
engine = create_engine('sqlite:///example.db')

# Define a session factory
Session = sessionmaker(bind=engine)

# Define a declarative base
Base = declarative_base()

# Define a model class
class User(Base):
    __tablename__ = 'users'

    id = Column(Integer, primary_key=True)
    name = Column(String)
    email = Column(String)

# Create the table
Base.metadata.create_all(engine)

# Add some data
session = Session()
session.add(User(name='Alice', email='alice@example.com'))
session.add(User(name='Bob', email='bob@example.com'))
session.commit()

# Query the data
users = session.query(User).all()
for user in users:
    print(user.name, user.email)

Exercise 28: Web Scraping with Selenium

Concepts:

Web scraping with Selenium
Browser automation and DOM manipulation

Description: Selenium is a popular library for web scraping that allows you to control a web browser programmatically. In this exercise, you'll write a script that uses Selenium to scrape data from a website.

Solution:

from selenium import webdriver

driver = webdriver.Chrome()

driver.get('https://en.wikipedia.org/wiki/Python_(programming_language)')

heading = driver.find_element_by_tag_name('h1')
print(heading.text)

links = driver.find_elements_by_tag_name('a')
for link in links:
    print(link.get_attribute('href'))

driver.close()

Exercise 29: Natural Language Processing

Concepts:

Natural language processing using the NLTK library
Tokenization, stemming, and POS tagging

Description: Python has many libraries for natural language processing (NLP). In this exercise, you'll use the NLTK library to perform some basic NLP tasks.

Solution:

import nltk
from nltk.tokenize import word_tokenize
from nltk.stem import PorterStemmer
from nltk import pos_tag

nltk.download('punkt')
nltk.download('averaged_perceptron_tagger')

text = 'The quick brown fox jumped over the lazy dogs'

# Tokenize the text
tokens = word_tokenize(text)
print(tokens)

# Stem the tokens
stemmer = PorterStemmer()
stemmed_tokens = [stemmer.stem(token) for token in tokens]
print(stemmed_tokens)

# Perform part-of-speech tagging on the tokens
pos_tags = pos_tag(tokens)
print(pos_tags)

Exercise 30: Machine Learning

Concepts:

Machine learning using scikit-learn
Data preparation, model training, and prediction

Description: Python has many libraries for machine learning, including scikit-learn. In this exercise, you'll use scikit-learn to train a simple machine learning model.

Solution:

import pandas as pd
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LinearRegression

# Load data from a CSV file
data = pd.read_csv('example.csv')

# Prepare the data
X = data['x'].values.reshape(-1, 1)
y = data['y'].values.reshape(-1, 1)

# Split the data into training and testing sets
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=0)

# Train the model
model = LinearRegression()
model.fit(X_train, y_train)

# Make predictions on the test set
y_pred = model.predict(X_test)

# Evaluate the model
score = model.score(X_test, y_test)
print(score)

In this script, we load data from a CSV file into a Pandas DataFrame. We then prepare the data by splitting it into input (X) and output (y) variables, and splitting it into training and testing sets using the train_test_split() method.

We then train a linear regression model using the training data, and make predictions on the testing data using the predict() method. Finally, we evaluate the performance of the model using the score() method, which calculates the coefficient of determination (R^2) for the model.

Exercise 31: Image Recognition with Deep Learning

Concepts:

Deep learning with Keras
Convolutional neural networks and image recognition

Description: Deep learning is a powerful technique for image recognition. In this exercise, you'll use the Keras library to train a deep learning model to recognize images.

Solution:

import numpy as np
from keras.datasets import mnist
from keras.models import Sequential
from keras.layers import Dense, Flatten, Conv2D, MaxPooling2D
from keras.utils import to_categorical

# Load the MNIST dataset
(X_train, y_train), (X_test, y_test) = mnist.load_data()

# Preprocess the data
X_train = X_train.reshape(-1, 28, 28, 1) / 255.0
X_test = X_test.reshape(-1, 28, 28, 1) / 255.0
y_train = to_categorical(y_train)
y_test = to_categorical(y_test)

# Define the model architecture
model = Sequential()
model.add(Conv2D(32, (3, 3), activation='relu', input_shape=(28, 28, 1)))
model.add(MaxPooling2D((2, 2)))
model.add(Conv2D(64, (3, 3), activation='relu'))
model.add(MaxPooling2D((2, 2)))
model.add(Conv2D(64, (3, 3), activation='relu'))
model.add(Flatten())
model.add(Dense(64, activation='relu'))
model.add(Dense(10, activation='softmax'))

# Compile the model
model.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy'])

# Train the model
model.fit(X_train, y_train, epochs=5, batch_size=64)

# Evaluate the model
loss, accuracy = model.evaluate(X_test, y_test)
print(accuracy)

Exercise 32: Web APIs with Flask

Concepts:

Web APIs with Flask
RESTful architecture and HTTP methods

Description: Flask is a lightweight web framework for Python. In this exercise, you'll write a simple Flask application that exposes a web API.

Solution:

from flask import Flask, request

app = Flask(__name__)

@app.route('/hello', methods=['GET'])
def hello():
    name = request.args.get('name')
    if name:
        return f'Hello, {name}!'
    else:
        return 'Hello, world!'

if __name__ == '__main__':
    app.run()

Exercise 33: GUI Programming with PyQt

Concepts:

GUI programming with PyQt
Widgets and event handling

Description: PyQt is a powerful library for building graphical user interfaces (GUIs) with Python. In this exercise, you'll write a script that uses PyQt to create a simple GUI.

Solution:

import sys
from PyQt5.QtWidgets import QApplication, QWidget, QLabel, QLineEdit, QPushButton

class MyWindow(QWidget):
    def __init__(self):
        super().__init__()

        self.label = QLabel('Enter your name:', self)
        self.label.move(50, 50)

        self.textbox = QLineEdit(self)
        self.textbox.move(50, 80)

        self.button = QPushButton('Say hello', self)
        self.button.move(50, 110)
        self.button.clicked.connect(self.button_clicked)

        self.setWindowTitle('Hello, world!')
        self.setGeometry(100, 100, 200, 150)

    def button_clicked(self):
        name = self.textbox.text()
        self.label.setText(f'Hello, {name}!')

app = QApplication(sys.argv)
window = MyWindow()
window.show()
sys.exit(app.exec_())

Exercise 34: Web Scraping with Beautiful Soup and Requests

Concepts:

Web scraping with Beautiful Soup and Requests
HTML parsing and navigation

Description: Beautiful Soup and Requests are popular libraries for web scraping. In this exercise, you'll write a script that uses these libraries to scrape data from a website.

Solution:

import requests
from bs4 import BeautifulSoup

url = 'https://en.wikipedia.org/wiki/Python_(programming_language)'

response = requests.get(url)
soup = BeautifulSoup(response.text, 'html.parser')

title = soup.find('h1', {'id': 'firstHeading'}).text
print(title)

content = soup.find('div', {'id': 'mw-content-text'}).text
print(content[:100])

Exercise 35: Data Visualization with Matplotlib

Concepts:

Data visualization with Matplotlib
Line plots and labels

Description: Matplotlib is a popular library for data visualization in Python. In this exercise, you'll write a script that uses Matplotlib to create a simple line plot.

Solution:

import matplotlib.pyplot as plt

x = [1, 2, 3, 4, 5]
y = [1, 4, 9, 16, 25]

plt.plot(x, y)
plt.xlabel('x')
plt.ylabel('y')
plt.title('A Simple Line Plot')
plt.show()

Exercise 36: Data Analysis with NumPy

Concepts:

Data analysis with NumPy
Creating and manipulating arrays

Description: NumPy is a powerful library for numerical computing in Python. In this exercise, you'll write a script that uses NumPy to perform some basic data analysis tasks.

Solution:

import numpy as np

# Create a 2D array
data = np.array([[1, 2, 3], [4, 5, 6], [7, 8, 9]])

# Print the array
print(data)

# Calculate some statistics on the array
print(np.mean(data))
print(np.median(data))
print(np.std(data))

# Reshape the array
reshaped_data = data.reshape(1, 9)
print(reshaped_data)

Exercise 37: Object-Oriented Programming

Concepts:

Object-oriented programming in Python
Classes, objects, and inheritance

Description: Python is an object-oriented programming language. In this exercise, you'll write a script that demonstrates the use of object-oriented programming in Python.

Solution:

class Shape:
    def __init__(self, x, y):
        self.x = x
        self.y = y

    def move(self, dx, dy):
        self.x += dx
        self.y += dy

class Rectangle(Shape):
    def __init__(self, x, y, width, height):
        super().__init__(x, y)
        self.width = width
        self.height = height

    def area(self):
        return self.width * self.height

class Circle(Shape):
    def __init__(self, x, y, radius):
        super().__init__(x, y)
        self.radius = radius

    def area(self):
        return 3.14159 * self.radius ** 2

rect = Rectangle(0, 0, 10, 5)
print(rect.area())
rect.move(1, 1)
print(rect.x, rect.y)

circ = Circle(0, 0, 5)
print(circ.area())
circ.move(2, 2)
print(circ.x, circ.y)

In this script, we define a Shape class that has an x and y coordinate, and a move() method that moves the shape by a specified amount in the x and y directions.

We then define a Rectangle class that inherits from Shape, and adds a width and height attribute, as well as an area() method that calculates the area of the rectangle.

We also define a Circle class that inherits from Shape, and adds a radius attribute, as well as an area() method that calculates the area of the circle.

Finally, we create instances of the Rectangle and Circle classes, and call their area() and move() methods.

Exercise 38: Regular Expressions

Concepts:

Regular expressions in Python
Searching for patterns in text

Description: Regular expressions are a powerful tool for text processing in Python. In this exercise, you'll write a script that uses regular expressions to search for patterns in a text file.

Solution:

import re

with open('example.txt', 'r') as f:
    data = f.read()

pattern = r'\d{3}-\d{2}-\d{4}'
matches = re.findall(pattern, data)

for match in matches:
    print(match)

In this script, we open a text file and read its contents into a variable. We then define a regular expression pattern that matches a social security number in the format XXX-XX-XXXX.

We use the findall() function from the re module to find all occurrences of the pattern in the text, and print them out.

Exercise 39: File I/O

Concepts:

File I/O in Python
Reading and writing text files

Description: File I/O is a common task in Python programming. In this exercise, you'll write a script that reads data from a file, performs some processing, and writes the results to another file.

Solution:

with open('input.txt', 'r') as f:
    data = f.readlines()

# Process the data
output = []
for line in data:
    line = line.strip()
    words = line.split()
    words.reverse()
    output.append(' '.join(words))

# Write the results to a file
with open('output.txt', 'w') as f:
    for line in output:
        f.write(line + '\n')

In this script, we open an input file and read its contents into a list of strings. We then process the data by splitting each line into words, reversing the order of the words, and joining them back together into a single string.

We write the resulting strings to an output file, with each string on a separate line.

Exercise 40: Data Manipulation with Pandas

Concepts:

Data manipulation with Pandas
Reading and filtering data

Description: Pandas is a powerful library for data manipulation in Python. In this exercise, you'll write a script that uses Pandas to load, process, and analyze a dataset.

Solution:

import pandas as pd

# Load the data from a CSV file
data = pd.read_csv('example.csv')

# Filter the data to include only rows where the value is greater than 5
filtered_data = data[data['value'] > 5]

# Calculate the mean and standard deviation of the filtered data
mean = filtered_data['value'].mean()
std = filtered_data['value'].std()

print(f'Mean: {mean}')
print(f'Standard deviation: {std}')

In this script, we load a dataset from a CSV file into a Pandas DataFrame. We then filter the data to include only rows where the value is greater than 5.

We calculate the mean and standard deviation of the filtered data using the mean() and std() methods of the DataFrame, and print out the results.

Intermediate Level Exercises

Exercise 1: Web Scraping

Concepts:

HTTP requests using the requests module
HTML parsing using BeautifulSoup

Description: Web scraping is a common task for many Python programmers. In this exercise, you'll write a script that scrapes data from a website.

Solution:

import requests
from bs4 import BeautifulSoup

url = 'https://www.example.com'
response = requests.get(url)
soup = BeautifulSoup(response.text, 'html.parser')

# Find all links on the page
links = soup.find_all('a')
for link in links:
    print(link.get('href'))

Exercise 2: File I/O

Concepts:

Reading and writing files using Python's built-in open() function
String manipulation

Description: Working with files is an essential part of any programming language. In this exercise, you'll write a script that reads data from a file and writes data to a file.

Solution:

# Read data from a file
with open('input.txt', 'r') as f:
    data = f.read()

# Manipulate the data
data = data.upper()

# Write data to a file
with open('output.txt', 'w') as f:
    f.write(data)

Exercise 3: Data Analysis

Concepts:

Reading data from a CSV file using Pandas
Data manipulation using Pandas
Basic data analysis using Pandas

Description: Python has a vast ecosystem of libraries for data analysis. In this exercise, you'll use the Pandas library to read data from a CSV file and perform some basic data analysis.

Solution:

import pandas as pd

# Read data from a CSV file
data = pd.read_csv('data.csv')

# Display the first 5 rows of the data
print(data.head())

# Calculate some basic statistics
print(data.describe())

Exercise 4: Command-Line Interface

Concepts:

Command-line arguments using the argparse module
String manipulation

Description: Python can be used to create command-line tools. In this exercise, you'll write a script that accepts command-line arguments and performs a task based on those arguments.

Solution:

import argparse

# Create an argument parser
parser = argparse.ArgumentParser()
parser.add_argument('name', help='Your name')
parser.add_argument('age', type=int, help='Your age')
args = parser.parse_args()

# Print a greeting
greeting = f'Hello, {args.name}! You are {args.age} years old.'
print(greeting)

Exercise 5: API Integration

Concepts:

HTTP requests using the requests module
JSON parsing

Description: Many modern applications have APIs that can be accessed programmatically. In this exercise, you'll write a script that interacts with a RESTful API.

Solution:

import requests

# Make a GET request to an API
url = 'https://api.example.com/users'
response = requests.get(url)

# Parse the response as JSON
data = response.json()

# Display the data
for user in data:
    print(user['name'], user['email'])

Exercise 6: Regular Expressions

Concepts:

Regular expressions using the re module
User input validation

Description: Regular expressions are powerful tools for pattern matching and data validation. In this exercise, you'll write a script that uses regular expressions to validate user input.

Solution:

import re

# Define a regular expression pattern for email validation
pattern = r'^[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}$'

# Get user input
email = input('Enter your email address: ')

# Validate the input using the regular expression
if re.match(pattern, email):
    print('Valid email address')
else:
    print('Invalid email address')

Exercise 7: Object-Oriented Programming

Concepts:

Classes and objects
Encapsulation
Inheritance

Description: Object-oriented programming is a fundamental concept in Python. In this exercise, you'll write a simple class that demonstrates the principles of object-oriented programming.

Solution:

class Person:
    def __init__(self, name, age):
        self.name = name
        self.age = age

    def greet(self):
        print(f'Hello, my name is {self.name} and I am {self.age} years old.')

class Student(Person):
    def __init__(self, name, age, major):
        super().__init__(name, age)
        self.major = major

    def describe(self):
        print(f'I am a student majoring in {self.major}.')

# Create objects and call methods
person = Person('Alice', 25)
person.greet()

student = Student('Bob', 20, 'Computer Science')
student.greet()
student.describe()

Exercise 8: Concurrency

Concepts:

Threading using the threading module
Synchronization using locks

Solution:

import threading

counter = 0
lock = threading.Lock()

def increment():
    global counter
    with lock:
        counter += 1

threads = []
for i in range(10):
    t = threading.Thread(target=increment)
    threads.append(t)
    t.start()

for t in threads:
    t.join()

print(f'The counter is {counter}.')

Exercise 9: Testing

Concepts:

Unit testing using the unittest module
Test-driven development

Description: Testing is a crucial part of software development. In this exercise, you'll write unit tests for a simple function.

Solution:

import unittest

def is_palindrome(s):
    return s == s[::-1]

class TestPalindrome(unittest.TestCase):
    def test_is_palindrome(self):
        self.assertTrue(is_palindrome('racecar'))
        self.assertFalse(is_palindrome('hello'))

if __name__ == '__main__':
    unittest.main()

Exercise 10: Data Visualization

Concepts:

Data visualization using Matplotlib
Line charts

Description: Data visualization is an essential part of data analysis. In this exercise, you'll use the Matplotlib library to create a simple line chart.

Solution:

import matplotlib.pyplot as plt

# Create some data
x = [1, 2, 3, 4, 5]
y = [2, 4, 6, 8, 10]

# Create a line chart
plt.plot(x, y)

# Add labels and a title
plt.xlabel('X values')
plt.ylabel('Y values')
plt.title('Line chart')

# Display the chart
plt.show()

Exercise 11: Database Operations

Concepts:

Connecting to a database using the SQLite3 module
Creating tables and inserting data
Querying data using SQL

Description: Python has a wide variety of libraries for working with databases. In this exercise, you'll write a script that performs some common database operations.

Solution:

import sqlite3

# Connect to the database
conn = sqlite3.connect('example.db')

# Create a table
conn.execute('''CREATE TABLE IF NOT EXISTS users
                (id INTEGER PRIMARY KEY,
                 name TEXT,
                 email TEXT)''')

# Insert some data
conn.execute("INSERT INTO users (name, email) VALUES (?, ?)", ('Alice', 'alice@example.com'))
conn.execute("INSERT INTO users (name, email) VALUES (?, ?)", ('Bob', 'bob@example.com'))

# Query the data
cursor = conn.execute("SELECT id, name, email FROM users")
for row in cursor:
    print(f'{row[0]} - {row[1]} ({row[2]})')

# Close the database connection
conn.close()

Exercise 12: Networking

Concepts:

Socket programming using the socket module
Server-client architecture

Description: Python has excellent networking capabilities, allowing you to build network applications. In this exercise, you'll write a script that implements a simple server and client.

Solution:

# Server code
import socket

server_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server_socket.bind(('localhost', 8000))
server_socket.listen(1)

while True:
    client_socket, address = server_socket.accept()
    data = client_socket.recv(1024)
    response = data.upper()
    client_socket.sendall(response)
    client_socket.close()

# Client code
import socket

client_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
client_socket.connect(('localhost', 8000))
client_socket.sendall(b'Hello, server!')
response = client_socket.recv(1024)
print(response)
client_socket.close()

Exercise 13: Data Science

Concepts:

Data manipulation using NumPy and Pandas
Data analysis using Pandas

Description: Python is a popular language for data science due to its powerful libraries. In this exercise, you'll use the NumPy and Pandas libraries to perform some data analysis.

Solution:

import numpy as np
import pandas as pd

# Create some data
data = {'x': np.arange(10),
        'y': np.random.randn(10)}

# Create a DataFrame from the data
df = pd.DataFrame(data)

# Calculate some statistics
print(df.mean())
print(df.std())

# Create a new column based on a calculation
df['z'] = df['x'] * df['y']

# Display the DataFrame
print(df)

Exercise 14: Web Development

Concepts:

Web development using the Flask framework
HTTP methods and routing

Description: Python can be used to build web applications using a variety of web frameworks. In this exercise, you'll write a simple Flask web application.

Solution:

from flask import Flask, request

app = Flask(__name__)

@app.route('/', methods=['GET'])
def home():
    return 'Hello, world!'

@app.route('/greet', methods=['POST'])
def greet():
    name = request.form.get('name')
    return f'Hello, {name}!'

if __name__ == '__main__':
    app.run()

Exercise 15: Asynchronous Programming

Concepts:

Asynchronous programming using asyncio
Coroutines

Solution:

import asyncio

async def coroutine():
    print('Coroutine started')
    await asyncio.sleep(1)
    print('Coroutine ended')

loop = asyncio.get_event_loop()
loop.run_until_complete(coroutine())
loop.close()

Exercise 16: Image Processing

Concepts:

Image processing using the Pillow library
Image manipulation and conversion

Description: Python has many libraries for working with images. In this exercise, you'll use the Pillow library to perform some basic image processing tasks.

Solution:

from PIL import Image

# Open an image file
image = Image.open('image.jpg')

# Display information about the image
print(image.format)
print(image.size)
print(image.mode)

# Convert the image to grayscale
grayscale_image = image.convert('L')
grayscale_image.show()

# Resize the image
resized_image = image.resize((300, 300))
resized_image.show()

# Save the image in a different format
resized_image.save('image.png')

Exercise 17: Email Sending

Concepts:

Email sending using the smtplib library
SMTP servers and email authentication

Description: Python has libraries for sending email programmatically. In this exercise, you'll write a script that sends an email using the smtplib library.

Solution:

import smtplib

sender_email = 'your_email@example.com'
sender_password = 'your_password'
receiver_email = 'recipient_email@example.com'
message = 'Hello, world!'

with smtplib.SMTP('smtp.gmail.com', 587) as server:
    server.starttls()
    server.login(sender_email, sender_password)
    server.sendmail(sender_email, receiver_email, message)

Exercise 18: Web API Integration

Concepts:

RESTful APIs and HTTP requests
Web API integration using the requests library

Description: Python can be used to integrate with many different web APIs. In this exercise, you'll write a script that interacts with a RESTful API using the requests library.

Solution:

import requests

response = requests.get('https://api.example.com/users')
data = response.json()

for user in data:
    print(user['name'], user['email'])

Exercise 19: Data Encryption

Concepts:

Data encryption using the cryptography library
Symmetric and asymmetric encryption

Description: Python has built-in libraries for encrypting and decrypting data. In this exercise, you'll write a script that encrypts and decrypts some data using the cryptography library.

Solution:

from cryptography.fernet import Fernet
from cryptography.hazmat.primitives import hashes
from cryptography.hazmat.primitives.asymmetric import rsa, padding
from cryptography.hazmat.primitives.serialization import load_pem_private_key, load_pem_public_key

# Symmetric encryption
key = Fernet.generate_key()
fernet = Fernet(key)
plaintext = b'This is a secret message'
ciphertext = fernet.encrypt(plaintext)
decrypted_plaintext = fernet.decrypt(ciphertext)
print(decrypted_plaintext)

# Asymmetric encryption
private_key = rsa.generate_private_key(
    public_exponent=65537,
    key_size=2048
)
public_key = private_key.public_key()

plaintext = b'This is a secret message'
ciphertext = public_key.encrypt(
    plaintext,
    padding.OAEP(
        mgf=padding.MGF1(algorithm=hashes.SHA256()),
        algorithm=hashes.SHA256(),
        label=None
    )
)

decrypted_plaintext = private_key.decrypt(
    ciphertext,
    padding.OAEP(
        mgf=padding.MGF1(algorithm=hashes.SHA256()),
        algorithm=hashes.SHA256(),
        label=None
    )
)
print(decrypted_plaintext)

Exercise 20: GUI Programming

Concepts:

GUI programming using the Tkinter library
Widgets and event handling

Description: Python has libraries for building graphical user interfaces (GUIs). In this exercise, you'll write a script that uses the Tkinter library to create a simple GUI.

Solution:

import tkinter as tk

def button_clicked():
    label.config(text='Hello, world!')

root = tk.Tk()

label = tk.Label(root, text='Welcome to my GUI')
label.pack()

button = tk.Button(root, text='Click me!', command=button_clicked)
button.pack()

root.mainloop()

When the button is clicked, the button_clicked() function is called, which updates the text of the label widget using the config() method.

Exercise 21: File I/O

Concepts:

File I/O in Python
Reading and writing files

Description: Python has built-in support for reading and writing files. In this exercise, you'll write a script that reads from and writes to a file.

Solution:

# Write to a file
with open('example.txt', 'w') as file:
    file.write('Hello, world!\n')
    file.write('How are you today?\n')

# Read from a file
with open('example.txt', 'r') as file:
    for line in file:
        print(line.strip())

Exercise 22: Logging

Concepts:

Logging in Python
Log levels and handlers

Description: Python has built-in support for logging. In this exercise, you'll write a script that logs some messages to a file.

Solution:

import logging

logging.basicConfig(filename='example.log', level=logging.DEBUG)

logging.debug('This is a debug message')
logging.info('This is an info message')
logging.warning('This is a warning message')
logging.error('This is an error message')
logging.critical('This is a critical message')

Exercise 23: Web Scraping

Concepts:

Web scraping using BeautifulSoup
HTML parsing and navigation

Description: Python can be used for web scraping, which involves extracting data from websites. In this exercise, you'll use the BeautifulSoup library to scrape data from a website.

Solution:

import requests
from bs4 import BeautifulSoup

response = requests.get('https://en.wikipedia.org/wiki/Python_(programming_language)')
soup = BeautifulSoup(response.text, 'html.parser')
print(soup.title.string)

links = soup.find_all('a')
for link in links:
    print(link.get('href'))

Exercise 24: Concurrency with asyncio

Concepts:

Asynchronous programming using asyncio
Coroutines and event loops

Solution:

import asyncio

async def coroutine(id):
    print(f'Coroutine {id} started')
    await asyncio.sleep(1)
    print(f'Coroutine {id} ended')

async def main():
    tasks = []
    for i in range(10):
        tasks.append(asyncio.create_task(coroutine(i)))
    await asyncio.gather(*tasks)

asyncio.run(main())

Exercise 25: Data Analysis with Pandas

Concepts:

Data analysis with Pandas
Loading and manipulating data with DataFrames

Description: Pandas is a powerful library for data analysis and manipulation in Python. In this exercise, you'll use Pandas to perform some data analysis tasks.

Solution:

import pandas as pd

# Load data from a CSV file
data = pd.read_csv('example.csv')

# Print the first five rows of the data
print(data.head())

# Calculate some statistics on the data
print(data.describe())

# Group the data by a column and calculate some statistics
grouped_data = data.groupby('category')
print(grouped_data.mean())

Finally, we group the data by the category column using the groupby() method and calculate some statistics on each group using the mean() method.

Exercise 26: Regular Expressions

Concepts:

Regular expressions in Python
Pattern matching and substitution

Description: Regular expressions are a powerful tool for searching and manipulating text. In this exercise, you'll write a script that uses regular expressions to search for patterns in text.

Solution:

import re

text = 'The quick brown fox jumps over the lazy dog'

# Search for a pattern in the text
pattern = r'\b\w{4}\b'
matches = re.findall(pattern, text)
print(matches)

# Substitute a pattern in the text
pattern = r'\bthe\b'
replaced_text = re.sub(pattern, 'a', text, flags=re.IGNORECASE)
print(replaced_text)

Exercise 27: Database ORM

Concepts:

Object-relational mapping using SQLAlchemy
Creating tables and querying data

Description: Python has many object-relational mapping (ORM) libraries for working with databases. In this exercise, you'll write a script that uses the SQLAlchemy ORM to interact with a database.

Solution:

from sqlalchemy import create_engine, Column, Integer, String
from sqlalchemy.orm import sessionmaker
from sqlalchemy.ext.declarative import declarative_base

# Define a database engine
engine = create_engine('sqlite:///example.db')

# Define a session factory
Session = sessionmaker(bind=engine)

# Define a declarative base
Base = declarative_base()

# Define a model class
class User(Base):
    __tablename__ = 'users'

    id = Column(Integer, primary_key=True)
    name = Column(String)
    email = Column(String)

# Create the table
Base.metadata.create_all(engine)

# Add some data
session = Session()
session.add(User(name='Alice', email='alice@example.com'))
session.add(User(name='Bob', email='bob@example.com'))
session.commit()

# Query the data
users = session.query(User).all()
for user in users:
    print(user.name, user.email)

Exercise 28: Web Scraping with Selenium

Concepts:

Web scraping with Selenium
Browser automation and DOM manipulation

Solution:

from selenium import webdriver

driver = webdriver.Chrome()

driver.get('https://en.wikipedia.org/wiki/Python_(programming_language)')

heading = driver.find_element_by_tag_name('h1')
print(heading.text)

links = driver.find_elements_by_tag_name('a')
for link in links:
    print(link.get_attribute('href'))

driver.close()

Exercise 29: Natural Language Processing

Concepts:

Natural language processing using the NLTK library
Tokenization, stemming, and POS tagging

Description: Python has many libraries for natural language processing (NLP). In this exercise, you'll use the NLTK library to perform some basic NLP tasks.

Solution:

import nltk
from nltk.tokenize import word_tokenize
from nltk.stem import PorterStemmer
from nltk import pos_tag

nltk.download('punkt')
nltk.download('averaged_perceptron_tagger')

text = 'The quick brown fox jumped over the lazy dogs'

# Tokenize the text
tokens = word_tokenize(text)
print(tokens)

# Stem the tokens
stemmer = PorterStemmer()
stemmed_tokens = [stemmer.stem(token) for token in tokens]
print(stemmed_tokens)

# Perform part-of-speech tagging on the tokens
pos_tags = pos_tag(tokens)
print(pos_tags)

Exercise 30: Machine Learning

Concepts:

Machine learning using scikit-learn
Data preparation, model training, and prediction

Description: Python has many libraries for machine learning, including scikit-learn. In this exercise, you'll use scikit-learn to train a simple machine learning model.

Solution:

import pandas as pd
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LinearRegression

# Load data from a CSV file
data = pd.read_csv('example.csv')

# Prepare the data
X = data['x'].values.reshape(-1, 1)
y = data['y'].values.reshape(-1, 1)

# Split the data into training and testing sets
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=0)

# Train the model
model = LinearRegression()
model.fit(X_train, y_train)

# Make predictions on the test set
y_pred = model.predict(X_test)

# Evaluate the model
score = model.score(X_test, y_test)
print(score)

Exercise 31: Image Recognition with Deep Learning

Concepts:

Deep learning with Keras
Convolutional neural networks and image recognition

Description: Deep learning is a powerful technique for image recognition. In this exercise, you'll use the Keras library to train a deep learning model to recognize images.

Solution:

import numpy as np
from keras.datasets import mnist
from keras.models import Sequential
from keras.layers import Dense, Flatten, Conv2D, MaxPooling2D
from keras.utils import to_categorical

# Load the MNIST dataset
(X_train, y_train), (X_test, y_test) = mnist.load_data()

# Preprocess the data
X_train = X_train.reshape(-1, 28, 28, 1) / 255.0
X_test = X_test.reshape(-1, 28, 28, 1) / 255.0
y_train = to_categorical(y_train)
y_test = to_categorical(y_test)

# Define the model architecture
model = Sequential()
model.add(Conv2D(32, (3, 3), activation='relu', input_shape=(28, 28, 1)))
model.add(MaxPooling2D((2, 2)))
model.add(Conv2D(64, (3, 3), activation='relu'))
model.add(MaxPooling2D((2, 2)))
model.add(Conv2D(64, (3, 3), activation='relu'))
model.add(Flatten())
model.add(Dense(64, activation='relu'))
model.add(Dense(10, activation='softmax'))

# Compile the model
model.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy'])

# Train the model
model.fit(X_train, y_train, epochs=5, batch_size=64)

# Evaluate the model
loss, accuracy = model.evaluate(X_test, y_test)
print(accuracy)

Exercise 32: Web APIs with Flask

Concepts:

Web APIs with Flask
RESTful architecture and HTTP methods

Description: Flask is a lightweight web framework for Python. In this exercise, you'll write a simple Flask application that exposes a web API.

Solution:

from flask import Flask, request

app = Flask(__name__)

@app.route('/hello', methods=['GET'])
def hello():
    name = request.args.get('name')
    if name:
        return f'Hello, {name}!'
    else:
        return 'Hello, world!'

if __name__ == '__main__':
    app.run()

Exercise 33: GUI Programming with PyQt

Concepts:

GUI programming with PyQt
Widgets and event handling

Description: PyQt is a powerful library for building graphical user interfaces (GUIs) with Python. In this exercise, you'll write a script that uses PyQt to create a simple GUI.

Solution:

import sys
from PyQt5.QtWidgets import QApplication, QWidget, QLabel, QLineEdit, QPushButton

class MyWindow(QWidget):
    def __init__(self):
        super().__init__()

        self.label = QLabel('Enter your name:', self)
        self.label.move(50, 50)

        self.textbox = QLineEdit(self)
        self.textbox.move(50, 80)

        self.button = QPushButton('Say hello', self)
        self.button.move(50, 110)
        self.button.clicked.connect(self.button_clicked)

        self.setWindowTitle('Hello, world!')
        self.setGeometry(100, 100, 200, 150)

    def button_clicked(self):
        name = self.textbox.text()
        self.label.setText(f'Hello, {name}!')

app = QApplication(sys.argv)
window = MyWindow()
window.show()
sys.exit(app.exec_())

Exercise 34: Web Scraping with Beautiful Soup and Requests

Concepts:

Web scraping with Beautiful Soup and Requests
HTML parsing and navigation

Description: Beautiful Soup and Requests are popular libraries for web scraping. In this exercise, you'll write a script that uses these libraries to scrape data from a website.

Solution:

import requests
from bs4 import BeautifulSoup

url = 'https://en.wikipedia.org/wiki/Python_(programming_language)'

response = requests.get(url)
soup = BeautifulSoup(response.text, 'html.parser')

title = soup.find('h1', {'id': 'firstHeading'}).text
print(title)

content = soup.find('div', {'id': 'mw-content-text'}).text
print(content[:100])

Exercise 35: Data Visualization with Matplotlib

Concepts:

Data visualization with Matplotlib
Line plots and labels

Description: Matplotlib is a popular library for data visualization in Python. In this exercise, you'll write a script that uses Matplotlib to create a simple line plot.

Solution:

import matplotlib.pyplot as plt

x = [1, 2, 3, 4, 5]
y = [1, 4, 9, 16, 25]

plt.plot(x, y)
plt.xlabel('x')
plt.ylabel('y')
plt.title('A Simple Line Plot')
plt.show()

Exercise 36: Data Analysis with NumPy

Concepts:

Data analysis with NumPy
Creating and manipulating arrays

Description: NumPy is a powerful library for numerical computing in Python. In this exercise, you'll write a script that uses NumPy to perform some basic data analysis tasks.

Solution:

import numpy as np

# Create a 2D array
data = np.array([[1, 2, 3], [4, 5, 6], [7, 8, 9]])

# Print the array
print(data)

# Calculate some statistics on the array
print(np.mean(data))
print(np.median(data))
print(np.std(data))

# Reshape the array
reshaped_data = data.reshape(1, 9)
print(reshaped_data)

Exercise 37: Object-Oriented Programming

Concepts:

Object-oriented programming in Python
Classes, objects, and inheritance

Description: Python is an object-oriented programming language. In this exercise, you'll write a script that demonstrates the use of object-oriented programming in Python.

Solution:

class Shape:
    def __init__(self, x, y):
        self.x = x
        self.y = y

    def move(self, dx, dy):
        self.x += dx
        self.y += dy

class Rectangle(Shape):
    def __init__(self, x, y, width, height):
        super().__init__(x, y)
        self.width = width
        self.height = height

    def area(self):
        return self.width * self.height

class Circle(Shape):
    def __init__(self, x, y, radius):
        super().__init__(x, y)
        self.radius = radius

    def area(self):
        return 3.14159 * self.radius ** 2

rect = Rectangle(0, 0, 10, 5)
print(rect.area())
rect.move(1, 1)
print(rect.x, rect.y)

circ = Circle(0, 0, 5)
print(circ.area())
circ.move(2, 2)
print(circ.x, circ.y)

In this script, we define a Shape class that has an x and y coordinate, and a move() method that moves the shape by a specified amount in the x and y directions.

We then define a Rectangle class that inherits from Shape, and adds a width and height attribute, as well as an area() method that calculates the area of the rectangle.

We also define a Circle class that inherits from Shape, and adds a radius attribute, as well as an area() method that calculates the area of the circle.

Finally, we create instances of the Rectangle and Circle classes, and call their area() and move() methods.

Exercise 38: Regular Expressions

Concepts:

Regular expressions in Python
Searching for patterns in text

Description: Regular expressions are a powerful tool for text processing in Python. In this exercise, you'll write a script that uses regular expressions to search for patterns in a text file.

Solution:

import re

with open('example.txt', 'r') as f:
    data = f.read()

pattern = r'\d{3}-\d{2}-\d{4}'
matches = re.findall(pattern, data)

for match in matches:
    print(match)

In this script, we open a text file and read its contents into a variable. We then define a regular expression pattern that matches a social security number in the format XXX-XX-XXXX.

We use the findall() function from the re module to find all occurrences of the pattern in the text, and print them out.

Exercise 39: File I/O

Concepts:

File I/O in Python
Reading and writing text files

Description: File I/O is a common task in Python programming. In this exercise, you'll write a script that reads data from a file, performs some processing, and writes the results to another file.

Solution:

with open('input.txt', 'r') as f:
    data = f.readlines()

# Process the data
output = []
for line in data:
    line = line.strip()
    words = line.split()
    words.reverse()
    output.append(' '.join(words))

# Write the results to a file
with open('output.txt', 'w') as f:
    for line in output:
        f.write(line + '\n')

We write the resulting strings to an output file, with each string on a separate line.

Exercise 40: Data Manipulation with Pandas

Concepts:

Data manipulation with Pandas
Reading and filtering data

Description: Pandas is a powerful library for data manipulation in Python. In this exercise, you'll write a script that uses Pandas to load, process, and analyze a dataset.

Solution:

import pandas as pd

# Load the data from a CSV file
data = pd.read_csv('example.csv')

# Filter the data to include only rows where the value is greater than 5
filtered_data = data[data['value'] > 5]

# Calculate the mean and standard deviation of the filtered data
mean = filtered_data['value'].mean()
std = filtered_data['value'].std()

print(f'Mean: {mean}')
print(f'Standard deviation: {std}')

In this script, we load a dataset from a CSV file into a Pandas DataFrame. We then filter the data to include only rows where the value is greater than 5.

We calculate the mean and standard deviation of the filtered data using the mean() and std() methods of the DataFrame, and print out the results.

Intermediate Level Exercises

Exercise 1: Web Scraping

Concepts:

HTTP requests using the requests module
HTML parsing using BeautifulSoup

Description: Web scraping is a common task for many Python programmers. In this exercise, you'll write a script that scrapes data from a website.

Solution:

import requests
from bs4 import BeautifulSoup

url = 'https://www.example.com'
response = requests.get(url)
soup = BeautifulSoup(response.text, 'html.parser')

# Find all links on the page
links = soup.find_all('a')
for link in links:
    print(link.get('href'))

Exercise 2: File I/O

Concepts:

Reading and writing files using Python's built-in open() function
String manipulation

Description: Working with files is an essential part of any programming language. In this exercise, you'll write a script that reads data from a file and writes data to a file.

Solution:

# Read data from a file
with open('input.txt', 'r') as f:
    data = f.read()

# Manipulate the data
data = data.upper()

# Write data to a file
with open('output.txt', 'w') as f:
    f.write(data)

Exercise 3: Data Analysis

Concepts:

Reading data from a CSV file using Pandas
Data manipulation using Pandas
Basic data analysis using Pandas

Description: Python has a vast ecosystem of libraries for data analysis. In this exercise, you'll use the Pandas library to read data from a CSV file and perform some basic data analysis.

Solution:

import pandas as pd

# Read data from a CSV file
data = pd.read_csv('data.csv')

# Display the first 5 rows of the data
print(data.head())

# Calculate some basic statistics
print(data.describe())

Exercise 4: Command-Line Interface

Concepts:

Command-line arguments using the argparse module
String manipulation

Description: Python can be used to create command-line tools. In this exercise, you'll write a script that accepts command-line arguments and performs a task based on those arguments.

Solution:

import argparse

# Create an argument parser
parser = argparse.ArgumentParser()
parser.add_argument('name', help='Your name')
parser.add_argument('age', type=int, help='Your age')
args = parser.parse_args()

# Print a greeting
greeting = f'Hello, {args.name}! You are {args.age} years old.'
print(greeting)

Exercise 5: API Integration

Concepts:

HTTP requests using the requests module
JSON parsing

Description: Many modern applications have APIs that can be accessed programmatically. In this exercise, you'll write a script that interacts with a RESTful API.

Solution:

import requests

# Make a GET request to an API
url = 'https://api.example.com/users'
response = requests.get(url)

# Parse the response as JSON
data = response.json()

# Display the data
for user in data:
    print(user['name'], user['email'])

Exercise 6: Regular Expressions

Concepts:

Regular expressions using the re module
User input validation

Description: Regular expressions are powerful tools for pattern matching and data validation. In this exercise, you'll write a script that uses regular expressions to validate user input.

Solution:

import re

# Define a regular expression pattern for email validation
pattern = r'^[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}$'

# Get user input
email = input('Enter your email address: ')

# Validate the input using the regular expression
if re.match(pattern, email):
    print('Valid email address')
else:
    print('Invalid email address')

Exercise 7: Object-Oriented Programming

Concepts:

Classes and objects
Encapsulation
Inheritance

Description: Object-oriented programming is a fundamental concept in Python. In this exercise, you'll write a simple class that demonstrates the principles of object-oriented programming.

Solution:

class Person:
    def __init__(self, name, age):
        self.name = name
        self.age = age

    def greet(self):
        print(f'Hello, my name is {self.name} and I am {self.age} years old.')

class Student(Person):
    def __init__(self, name, age, major):
        super().__init__(name, age)
        self.major = major

    def describe(self):
        print(f'I am a student majoring in {self.major}.')

# Create objects and call methods
person = Person('Alice', 25)
person.greet()

student = Student('Bob', 20, 'Computer Science')
student.greet()
student.describe()

Exercise 8: Concurrency

Concepts:

Threading using the threading module
Synchronization using locks

Solution:

import threading

counter = 0
lock = threading.Lock()

def increment():
    global counter
    with lock:
        counter += 1

threads = []
for i in range(10):
    t = threading.Thread(target=increment)
    threads.append(t)
    t.start()

for t in threads:
    t.join()

print(f'The counter is {counter}.')

Exercise 9: Testing

Concepts:

Unit testing using the unittest module
Test-driven development

Description: Testing is a crucial part of software development. In this exercise, you'll write unit tests for a simple function.

Solution:

import unittest

def is_palindrome(s):
    return s == s[::-1]

class TestPalindrome(unittest.TestCase):
    def test_is_palindrome(self):
        self.assertTrue(is_palindrome('racecar'))
        self.assertFalse(is_palindrome('hello'))

if __name__ == '__main__':
    unittest.main()

Exercise 10: Data Visualization

Concepts:

Data visualization using Matplotlib
Line charts

Description: Data visualization is an essential part of data analysis. In this exercise, you'll use the Matplotlib library to create a simple line chart.

Solution:

import matplotlib.pyplot as plt

# Create some data
x = [1, 2, 3, 4, 5]
y = [2, 4, 6, 8, 10]

# Create a line chart
plt.plot(x, y)

# Add labels and a title
plt.xlabel('X values')
plt.ylabel('Y values')
plt.title('Line chart')

# Display the chart
plt.show()

Exercise 11: Database Operations

Concepts:

Connecting to a database using the SQLite3 module
Creating tables and inserting data
Querying data using SQL

Description: Python has a wide variety of libraries for working with databases. In this exercise, you'll write a script that performs some common database operations.

Solution:

import sqlite3

# Connect to the database
conn = sqlite3.connect('example.db')

# Create a table
conn.execute('''CREATE TABLE IF NOT EXISTS users
                (id INTEGER PRIMARY KEY,
                 name TEXT,
                 email TEXT)''')

# Insert some data
conn.execute("INSERT INTO users (name, email) VALUES (?, ?)", ('Alice', 'alice@example.com'))
conn.execute("INSERT INTO users (name, email) VALUES (?, ?)", ('Bob', 'bob@example.com'))

# Query the data
cursor = conn.execute("SELECT id, name, email FROM users")
for row in cursor:
    print(f'{row[0]} - {row[1]} ({row[2]})')

# Close the database connection
conn.close()

Exercise 12: Networking

Concepts:

Socket programming using the socket module
Server-client architecture

Description: Python has excellent networking capabilities, allowing you to build network applications. In this exercise, you'll write a script that implements a simple server and client.

Solution:

# Server code
import socket

server_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server_socket.bind(('localhost', 8000))
server_socket.listen(1)

while True:
    client_socket, address = server_socket.accept()
    data = client_socket.recv(1024)
    response = data.upper()
    client_socket.sendall(response)
    client_socket.close()

# Client code
import socket

client_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
client_socket.connect(('localhost', 8000))
client_socket.sendall(b'Hello, server!')
response = client_socket.recv(1024)
print(response)
client_socket.close()

Exercise 13: Data Science

Concepts:

Data manipulation using NumPy and Pandas
Data analysis using Pandas

Description: Python is a popular language for data science due to its powerful libraries. In this exercise, you'll use the NumPy and Pandas libraries to perform some data analysis.

Solution:

import numpy as np
import pandas as pd

# Create some data
data = {'x': np.arange(10),
        'y': np.random.randn(10)}

# Create a DataFrame from the data
df = pd.DataFrame(data)

# Calculate some statistics
print(df.mean())
print(df.std())

# Create a new column based on a calculation
df['z'] = df['x'] * df['y']

# Display the DataFrame
print(df)

Exercise 14: Web Development

Concepts:

Web development using the Flask framework
HTTP methods and routing

Description: Python can be used to build web applications using a variety of web frameworks. In this exercise, you'll write a simple Flask web application.

Solution:

from flask import Flask, request

app = Flask(__name__)

@app.route('/', methods=['GET'])
def home():
    return 'Hello, world!'

@app.route('/greet', methods=['POST'])
def greet():
    name = request.form.get('name')
    return f'Hello, {name}!'

if __name__ == '__main__':
    app.run()

Exercise 15: Asynchronous Programming

Concepts:

Asynchronous programming using asyncio
Coroutines

Solution:

import asyncio

async def coroutine():
    print('Coroutine started')
    await asyncio.sleep(1)
    print('Coroutine ended')

loop = asyncio.get_event_loop()
loop.run_until_complete(coroutine())
loop.close()

Exercise 16: Image Processing

Concepts:

Image processing using the Pillow library
Image manipulation and conversion

Description: Python has many libraries for working with images. In this exercise, you'll use the Pillow library to perform some basic image processing tasks.

Solution:

from PIL import Image

# Open an image file
image = Image.open('image.jpg')

# Display information about the image
print(image.format)
print(image.size)
print(image.mode)

# Convert the image to grayscale
grayscale_image = image.convert('L')
grayscale_image.show()

# Resize the image
resized_image = image.resize((300, 300))
resized_image.show()

# Save the image in a different format
resized_image.save('image.png')

Exercise 17: Email Sending

Concepts:

Email sending using the smtplib library
SMTP servers and email authentication

Description: Python has libraries for sending email programmatically. In this exercise, you'll write a script that sends an email using the smtplib library.

Solution:

import smtplib

sender_email = 'your_email@example.com'
sender_password = 'your_password'
receiver_email = 'recipient_email@example.com'
message = 'Hello, world!'

with smtplib.SMTP('smtp.gmail.com', 587) as server:
    server.starttls()
    server.login(sender_email, sender_password)
    server.sendmail(sender_email, receiver_email, message)

Exercise 18: Web API Integration

Concepts:

RESTful APIs and HTTP requests
Web API integration using the requests library

Description: Python can be used to integrate with many different web APIs. In this exercise, you'll write a script that interacts with a RESTful API using the requests library.

Solution:

import requests

response = requests.get('https://api.example.com/users')
data = response.json()

for user in data:
    print(user['name'], user['email'])

Exercise 19: Data Encryption

Concepts:

Data encryption using the cryptography library
Symmetric and asymmetric encryption

Description: Python has built-in libraries for encrypting and decrypting data. In this exercise, you'll write a script that encrypts and decrypts some data using the cryptography library.

Solution:

from cryptography.fernet import Fernet
from cryptography.hazmat.primitives import hashes
from cryptography.hazmat.primitives.asymmetric import rsa, padding
from cryptography.hazmat.primitives.serialization import load_pem_private_key, load_pem_public_key

# Symmetric encryption
key = Fernet.generate_key()
fernet = Fernet(key)
plaintext = b'This is a secret message'
ciphertext = fernet.encrypt(plaintext)
decrypted_plaintext = fernet.decrypt(ciphertext)
print(decrypted_plaintext)

# Asymmetric encryption
private_key = rsa.generate_private_key(
    public_exponent=65537,
    key_size=2048
)
public_key = private_key.public_key()

plaintext = b'This is a secret message'
ciphertext = public_key.encrypt(
    plaintext,
    padding.OAEP(
        mgf=padding.MGF1(algorithm=hashes.SHA256()),
        algorithm=hashes.SHA256(),
        label=None
    )
)

decrypted_plaintext = private_key.decrypt(
    ciphertext,
    padding.OAEP(
        mgf=padding.MGF1(algorithm=hashes.SHA256()),
        algorithm=hashes.SHA256(),
        label=None
    )
)
print(decrypted_plaintext)

Exercise 20: GUI Programming

Concepts:

GUI programming using the Tkinter library
Widgets and event handling

Description: Python has libraries for building graphical user interfaces (GUIs). In this exercise, you'll write a script that uses the Tkinter library to create a simple GUI.

Solution:

import tkinter as tk

def button_clicked():
    label.config(text='Hello, world!')

root = tk.Tk()

label = tk.Label(root, text='Welcome to my GUI')
label.pack()

button = tk.Button(root, text='Click me!', command=button_clicked)
button.pack()

root.mainloop()

When the button is clicked, the button_clicked() function is called, which updates the text of the label widget using the config() method.

Exercise 21: File I/O

Concepts:

File I/O in Python
Reading and writing files

Description: Python has built-in support for reading and writing files. In this exercise, you'll write a script that reads from and writes to a file.

Solution:

# Write to a file
with open('example.txt', 'w') as file:
    file.write('Hello, world!\n')
    file.write('How are you today?\n')

# Read from a file
with open('example.txt', 'r') as file:
    for line in file:
        print(line.strip())

Exercise 22: Logging

Concepts:

Logging in Python
Log levels and handlers

Description: Python has built-in support for logging. In this exercise, you'll write a script that logs some messages to a file.

Solution:

import logging

logging.basicConfig(filename='example.log', level=logging.DEBUG)

logging.debug('This is a debug message')
logging.info('This is an info message')
logging.warning('This is a warning message')
logging.error('This is an error message')
logging.critical('This is a critical message')

Exercise 23: Web Scraping

Concepts:

Web scraping using BeautifulSoup
HTML parsing and navigation

Description: Python can be used for web scraping, which involves extracting data from websites. In this exercise, you'll use the BeautifulSoup library to scrape data from a website.

Solution:

import requests
from bs4 import BeautifulSoup

response = requests.get('https://en.wikipedia.org/wiki/Python_(programming_language)')
soup = BeautifulSoup(response.text, 'html.parser')
print(soup.title.string)

links = soup.find_all('a')
for link in links:
    print(link.get('href'))

Exercise 24: Concurrency with asyncio

Concepts:

Asynchronous programming using asyncio
Coroutines and event loops

Solution:

import asyncio

async def coroutine(id):
    print(f'Coroutine {id} started')
    await asyncio.sleep(1)
    print(f'Coroutine {id} ended')

async def main():
    tasks = []
    for i in range(10):
        tasks.append(asyncio.create_task(coroutine(i)))
    await asyncio.gather(*tasks)

asyncio.run(main())

Exercise 25: Data Analysis with Pandas

Concepts:

Data analysis with Pandas
Loading and manipulating data with DataFrames

Description: Pandas is a powerful library for data analysis and manipulation in Python. In this exercise, you'll use Pandas to perform some data analysis tasks.

Solution:

import pandas as pd

# Load data from a CSV file
data = pd.read_csv('example.csv')

# Print the first five rows of the data
print(data.head())

# Calculate some statistics on the data
print(data.describe())

# Group the data by a column and calculate some statistics
grouped_data = data.groupby('category')
print(grouped_data.mean())

Finally, we group the data by the category column using the groupby() method and calculate some statistics on each group using the mean() method.

Exercise 26: Regular Expressions

Concepts:

Regular expressions in Python
Pattern matching and substitution

Description: Regular expressions are a powerful tool for searching and manipulating text. In this exercise, you'll write a script that uses regular expressions to search for patterns in text.

Solution:

import re

text = 'The quick brown fox jumps over the lazy dog'

# Search for a pattern in the text
pattern = r'\b\w{4}\b'
matches = re.findall(pattern, text)
print(matches)

# Substitute a pattern in the text
pattern = r'\bthe\b'
replaced_text = re.sub(pattern, 'a', text, flags=re.IGNORECASE)
print(replaced_text)

Exercise 27: Database ORM

Concepts:

Object-relational mapping using SQLAlchemy
Creating tables and querying data

Description: Python has many object-relational mapping (ORM) libraries for working with databases. In this exercise, you'll write a script that uses the SQLAlchemy ORM to interact with a database.

Solution:

from sqlalchemy import create_engine, Column, Integer, String
from sqlalchemy.orm import sessionmaker
from sqlalchemy.ext.declarative import declarative_base

# Define a database engine
engine = create_engine('sqlite:///example.db')

# Define a session factory
Session = sessionmaker(bind=engine)

# Define a declarative base
Base = declarative_base()

# Define a model class
class User(Base):
    __tablename__ = 'users'

    id = Column(Integer, primary_key=True)
    name = Column(String)
    email = Column(String)

# Create the table
Base.metadata.create_all(engine)

# Add some data
session = Session()
session.add(User(name='Alice', email='alice@example.com'))
session.add(User(name='Bob', email='bob@example.com'))
session.commit()

# Query the data
users = session.query(User).all()
for user in users:
    print(user.name, user.email)

Exercise 28: Web Scraping with Selenium

Concepts:

Web scraping with Selenium
Browser automation and DOM manipulation

Solution:

from selenium import webdriver

driver = webdriver.Chrome()

driver.get('https://en.wikipedia.org/wiki/Python_(programming_language)')

heading = driver.find_element_by_tag_name('h1')
print(heading.text)

links = driver.find_elements_by_tag_name('a')
for link in links:
    print(link.get_attribute('href'))

driver.close()

Exercise 29: Natural Language Processing

Concepts:

Natural language processing using the NLTK library
Tokenization, stemming, and POS tagging

Description: Python has many libraries for natural language processing (NLP). In this exercise, you'll use the NLTK library to perform some basic NLP tasks.

Solution:

import nltk
from nltk.tokenize import word_tokenize
from nltk.stem import PorterStemmer
from nltk import pos_tag

nltk.download('punkt')
nltk.download('averaged_perceptron_tagger')

text = 'The quick brown fox jumped over the lazy dogs'

# Tokenize the text
tokens = word_tokenize(text)
print(tokens)

# Stem the tokens
stemmer = PorterStemmer()
stemmed_tokens = [stemmer.stem(token) for token in tokens]
print(stemmed_tokens)

# Perform part-of-speech tagging on the tokens
pos_tags = pos_tag(tokens)
print(pos_tags)

Exercise 30: Machine Learning

Concepts:

Machine learning using scikit-learn
Data preparation, model training, and prediction

Description: Python has many libraries for machine learning, including scikit-learn. In this exercise, you'll use scikit-learn to train a simple machine learning model.

Solution:

import pandas as pd
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LinearRegression

# Load data from a CSV file
data = pd.read_csv('example.csv')

# Prepare the data
X = data['x'].values.reshape(-1, 1)
y = data['y'].values.reshape(-1, 1)

# Split the data into training and testing sets
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=0)

# Train the model
model = LinearRegression()
model.fit(X_train, y_train)

# Make predictions on the test set
y_pred = model.predict(X_test)

# Evaluate the model
score = model.score(X_test, y_test)
print(score)

Exercise 31: Image Recognition with Deep Learning

Concepts:

Deep learning with Keras
Convolutional neural networks and image recognition

Description: Deep learning is a powerful technique for image recognition. In this exercise, you'll use the Keras library to train a deep learning model to recognize images.

Solution:

import numpy as np
from keras.datasets import mnist
from keras.models import Sequential
from keras.layers import Dense, Flatten, Conv2D, MaxPooling2D
from keras.utils import to_categorical

# Load the MNIST dataset
(X_train, y_train), (X_test, y_test) = mnist.load_data()

# Preprocess the data
X_train = X_train.reshape(-1, 28, 28, 1) / 255.0
X_test = X_test.reshape(-1, 28, 28, 1) / 255.0
y_train = to_categorical(y_train)
y_test = to_categorical(y_test)

# Define the model architecture
model = Sequential()
model.add(Conv2D(32, (3, 3), activation='relu', input_shape=(28, 28, 1)))
model.add(MaxPooling2D((2, 2)))
model.add(Conv2D(64, (3, 3), activation='relu'))
model.add(MaxPooling2D((2, 2)))
model.add(Conv2D(64, (3, 3), activation='relu'))
model.add(Flatten())
model.add(Dense(64, activation='relu'))
model.add(Dense(10, activation='softmax'))

# Compile the model
model.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy'])

# Train the model
model.fit(X_train, y_train, epochs=5, batch_size=64)

# Evaluate the model
loss, accuracy = model.evaluate(X_test, y_test)
print(accuracy)

Exercise 32: Web APIs with Flask

Concepts:

Web APIs with Flask
RESTful architecture and HTTP methods

Description: Flask is a lightweight web framework for Python. In this exercise, you'll write a simple Flask application that exposes a web API.

Solution:

from flask import Flask, request

app = Flask(__name__)

@app.route('/hello', methods=['GET'])
def hello():
    name = request.args.get('name')
    if name:
        return f'Hello, {name}!'
    else:
        return 'Hello, world!'

if __name__ == '__main__':
    app.run()

Exercise 33: GUI Programming with PyQt

Concepts:

GUI programming with PyQt
Widgets and event handling

Description: PyQt is a powerful library for building graphical user interfaces (GUIs) with Python. In this exercise, you'll write a script that uses PyQt to create a simple GUI.

Solution:

import sys
from PyQt5.QtWidgets import QApplication, QWidget, QLabel, QLineEdit, QPushButton

class MyWindow(QWidget):
    def __init__(self):
        super().__init__()

        self.label = QLabel('Enter your name:', self)
        self.label.move(50, 50)

        self.textbox = QLineEdit(self)
        self.textbox.move(50, 80)

        self.button = QPushButton('Say hello', self)
        self.button.move(50, 110)
        self.button.clicked.connect(self.button_clicked)

        self.setWindowTitle('Hello, world!')
        self.setGeometry(100, 100, 200, 150)

    def button_clicked(self):
        name = self.textbox.text()
        self.label.setText(f'Hello, {name}!')

app = QApplication(sys.argv)
window = MyWindow()
window.show()
sys.exit(app.exec_())

Exercise 34: Web Scraping with Beautiful Soup and Requests

Concepts:

Web scraping with Beautiful Soup and Requests
HTML parsing and navigation

Description: Beautiful Soup and Requests are popular libraries for web scraping. In this exercise, you'll write a script that uses these libraries to scrape data from a website.

Solution:

import requests
from bs4 import BeautifulSoup

url = 'https://en.wikipedia.org/wiki/Python_(programming_language)'

response = requests.get(url)
soup = BeautifulSoup(response.text, 'html.parser')

title = soup.find('h1', {'id': 'firstHeading'}).text
print(title)

content = soup.find('div', {'id': 'mw-content-text'}).text
print(content[:100])

Exercise 35: Data Visualization with Matplotlib

Concepts:

Data visualization with Matplotlib
Line plots and labels

Description: Matplotlib is a popular library for data visualization in Python. In this exercise, you'll write a script that uses Matplotlib to create a simple line plot.

Solution:

import matplotlib.pyplot as plt

x = [1, 2, 3, 4, 5]
y = [1, 4, 9, 16, 25]

plt.plot(x, y)
plt.xlabel('x')
plt.ylabel('y')
plt.title('A Simple Line Plot')
plt.show()

Exercise 36: Data Analysis with NumPy

Concepts:

Data analysis with NumPy
Creating and manipulating arrays

Description: NumPy is a powerful library for numerical computing in Python. In this exercise, you'll write a script that uses NumPy to perform some basic data analysis tasks.

Solution:

import numpy as np

# Create a 2D array
data = np.array([[1, 2, 3], [4, 5, 6], [7, 8, 9]])

# Print the array
print(data)

# Calculate some statistics on the array
print(np.mean(data))
print(np.median(data))
print(np.std(data))

# Reshape the array
reshaped_data = data.reshape(1, 9)
print(reshaped_data)

Exercise 37: Object-Oriented Programming

Concepts:

Object-oriented programming in Python
Classes, objects, and inheritance

Description: Python is an object-oriented programming language. In this exercise, you'll write a script that demonstrates the use of object-oriented programming in Python.

Solution:

class Shape:
    def __init__(self, x, y):
        self.x = x
        self.y = y

    def move(self, dx, dy):
        self.x += dx
        self.y += dy

class Rectangle(Shape):
    def __init__(self, x, y, width, height):
        super().__init__(x, y)
        self.width = width
        self.height = height

    def area(self):
        return self.width * self.height

class Circle(Shape):
    def __init__(self, x, y, radius):
        super().__init__(x, y)
        self.radius = radius

    def area(self):
        return 3.14159 * self.radius ** 2

rect = Rectangle(0, 0, 10, 5)
print(rect.area())
rect.move(1, 1)
print(rect.x, rect.y)

circ = Circle(0, 0, 5)
print(circ.area())
circ.move(2, 2)
print(circ.x, circ.y)

In this script, we define a Shape class that has an x and y coordinate, and a move() method that moves the shape by a specified amount in the x and y directions.

We then define a Rectangle class that inherits from Shape, and adds a width and height attribute, as well as an area() method that calculates the area of the rectangle.

We also define a Circle class that inherits from Shape, and adds a radius attribute, as well as an area() method that calculates the area of the circle.

Finally, we create instances of the Rectangle and Circle classes, and call their area() and move() methods.

Exercise 38: Regular Expressions

Concepts:

Regular expressions in Python
Searching for patterns in text

Description: Regular expressions are a powerful tool for text processing in Python. In this exercise, you'll write a script that uses regular expressions to search for patterns in a text file.

Solution:

import re

with open('example.txt', 'r') as f:
    data = f.read()

pattern = r'\d{3}-\d{2}-\d{4}'
matches = re.findall(pattern, data)

for match in matches:
    print(match)

In this script, we open a text file and read its contents into a variable. We then define a regular expression pattern that matches a social security number in the format XXX-XX-XXXX.

We use the findall() function from the re module to find all occurrences of the pattern in the text, and print them out.

Exercise 39: File I/O

Concepts:

File I/O in Python
Reading and writing text files

Description: File I/O is a common task in Python programming. In this exercise, you'll write a script that reads data from a file, performs some processing, and writes the results to another file.

Solution:

with open('input.txt', 'r') as f:
    data = f.readlines()

# Process the data
output = []
for line in data:
    line = line.strip()
    words = line.split()
    words.reverse()
    output.append(' '.join(words))

# Write the results to a file
with open('output.txt', 'w') as f:
    for line in output:
        f.write(line + '\n')

We write the resulting strings to an output file, with each string on a separate line.

Exercise 40: Data Manipulation with Pandas

Concepts:

Data manipulation with Pandas
Reading and filtering data

Description: Pandas is a powerful library for data manipulation in Python. In this exercise, you'll write a script that uses Pandas to load, process, and analyze a dataset.

Solution:

import pandas as pd

# Load the data from a CSV file
data = pd.read_csv('example.csv')

# Filter the data to include only rows where the value is greater than 5
filtered_data = data[data['value'] > 5]

# Calculate the mean and standard deviation of the filtered data
mean = filtered_data['value'].mean()
std = filtered_data['value'].std()

print(f'Mean: {mean}')
print(f'Standard deviation: {std}')

In this script, we load a dataset from a CSV file into a Pandas DataFrame. We then filter the data to include only rows where the value is greater than 5.

We calculate the mean and standard deviation of the filtered data using the mean() and std() methods of the DataFrame, and print out the results.

Intermediate Level Exercises

Exercise 1: Web Scraping

Concepts:

HTTP requests using the requests module
HTML parsing using BeautifulSoup

Description: Web scraping is a common task for many Python programmers. In this exercise, you'll write a script that scrapes data from a website.

Solution:

import requests
from bs4 import BeautifulSoup

url = 'https://www.example.com'
response = requests.get(url)
soup = BeautifulSoup(response.text, 'html.parser')

# Find all links on the page
links = soup.find_all('a')
for link in links:
    print(link.get('href'))

Exercise 2: File I/O

Concepts:

Reading and writing files using Python's built-in open() function
String manipulation

Description: Working with files is an essential part of any programming language. In this exercise, you'll write a script that reads data from a file and writes data to a file.

Solution:

# Read data from a file
with open('input.txt', 'r') as f:
    data = f.read()

# Manipulate the data
data = data.upper()

# Write data to a file
with open('output.txt', 'w') as f:
    f.write(data)

Exercise 3: Data Analysis

Concepts:

Reading data from a CSV file using Pandas
Data manipulation using Pandas
Basic data analysis using Pandas

Description: Python has a vast ecosystem of libraries for data analysis. In this exercise, you'll use the Pandas library to read data from a CSV file and perform some basic data analysis.

Solution:

import pandas as pd

# Read data from a CSV file
data = pd.read_csv('data.csv')

# Display the first 5 rows of the data
print(data.head())

# Calculate some basic statistics
print(data.describe())

Exercise 4: Command-Line Interface

Concepts:

Command-line arguments using the argparse module
String manipulation

Description: Python can be used to create command-line tools. In this exercise, you'll write a script that accepts command-line arguments and performs a task based on those arguments.

Solution:

import argparse

# Create an argument parser
parser = argparse.ArgumentParser()
parser.add_argument('name', help='Your name')
parser.add_argument('age', type=int, help='Your age')
args = parser.parse_args()

# Print a greeting
greeting = f'Hello, {args.name}! You are {args.age} years old.'
print(greeting)

Exercise 5: API Integration

Concepts:

HTTP requests using the requests module
JSON parsing

Description: Many modern applications have APIs that can be accessed programmatically. In this exercise, you'll write a script that interacts with a RESTful API.

Solution:

import requests

# Make a GET request to an API
url = 'https://api.example.com/users'
response = requests.get(url)

# Parse the response as JSON
data = response.json()

# Display the data
for user in data:
    print(user['name'], user['email'])

Exercise 6: Regular Expressions

Concepts:

Regular expressions using the re module
User input validation

Description: Regular expressions are powerful tools for pattern matching and data validation. In this exercise, you'll write a script that uses regular expressions to validate user input.

Solution:

import re

# Define a regular expression pattern for email validation
pattern = r'^[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}$'

# Get user input
email = input('Enter your email address: ')

# Validate the input using the regular expression
if re.match(pattern, email):
    print('Valid email address')
else:
    print('Invalid email address')

Exercise 7: Object-Oriented Programming

Concepts:

Classes and objects
Encapsulation
Inheritance

Description: Object-oriented programming is a fundamental concept in Python. In this exercise, you'll write a simple class that demonstrates the principles of object-oriented programming.

Solution:

class Person:
    def __init__(self, name, age):
        self.name = name
        self.age = age

    def greet(self):
        print(f'Hello, my name is {self.name} and I am {self.age} years old.')

class Student(Person):
    def __init__(self, name, age, major):
        super().__init__(name, age)
        self.major = major

    def describe(self):
        print(f'I am a student majoring in {self.major}.')

# Create objects and call methods
person = Person('Alice', 25)
person.greet()

student = Student('Bob', 20, 'Computer Science')
student.greet()
student.describe()

Exercise 8: Concurrency

Concepts:

Threading using the threading module
Synchronization using locks

Solution:

import threading

counter = 0
lock = threading.Lock()

def increment():
    global counter
    with lock:
        counter += 1

threads = []
for i in range(10):
    t = threading.Thread(target=increment)
    threads.append(t)
    t.start()

for t in threads:
    t.join()

print(f'The counter is {counter}.')

Exercise 9: Testing

Concepts:

Unit testing using the unittest module
Test-driven development

Description: Testing is a crucial part of software development. In this exercise, you'll write unit tests for a simple function.

Solution:

import unittest

def is_palindrome(s):
    return s == s[::-1]

class TestPalindrome(unittest.TestCase):
    def test_is_palindrome(self):
        self.assertTrue(is_palindrome('racecar'))
        self.assertFalse(is_palindrome('hello'))

if __name__ == '__main__':
    unittest.main()

Exercise 10: Data Visualization

Concepts:

Data visualization using Matplotlib
Line charts

Description: Data visualization is an essential part of data analysis. In this exercise, you'll use the Matplotlib library to create a simple line chart.

Solution:

import matplotlib.pyplot as plt

# Create some data
x = [1, 2, 3, 4, 5]
y = [2, 4, 6, 8, 10]

# Create a line chart
plt.plot(x, y)

# Add labels and a title
plt.xlabel('X values')
plt.ylabel('Y values')
plt.title('Line chart')

# Display the chart
plt.show()

Exercise 11: Database Operations

Concepts:

Connecting to a database using the SQLite3 module
Creating tables and inserting data
Querying data using SQL

Description: Python has a wide variety of libraries for working with databases. In this exercise, you'll write a script that performs some common database operations.

Solution:

import sqlite3

# Connect to the database
conn = sqlite3.connect('example.db')

# Create a table
conn.execute('''CREATE TABLE IF NOT EXISTS users
                (id INTEGER PRIMARY KEY,
                 name TEXT,
                 email TEXT)''')

# Insert some data
conn.execute("INSERT INTO users (name, email) VALUES (?, ?)", ('Alice', 'alice@example.com'))
conn.execute("INSERT INTO users (name, email) VALUES (?, ?)", ('Bob', 'bob@example.com'))

# Query the data
cursor = conn.execute("SELECT id, name, email FROM users")
for row in cursor:
    print(f'{row[0]} - {row[1]} ({row[2]})')

# Close the database connection
conn.close()

Exercise 12: Networking

Concepts:

Socket programming using the socket module
Server-client architecture

Description: Python has excellent networking capabilities, allowing you to build network applications. In this exercise, you'll write a script that implements a simple server and client.

Solution:

# Server code
import socket

server_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server_socket.bind(('localhost', 8000))
server_socket.listen(1)

while True:
    client_socket, address = server_socket.accept()
    data = client_socket.recv(1024)
    response = data.upper()
    client_socket.sendall(response)
    client_socket.close()

# Client code
import socket

client_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
client_socket.connect(('localhost', 8000))
client_socket.sendall(b'Hello, server!')
response = client_socket.recv(1024)
print(response)
client_socket.close()

Exercise 13: Data Science

Concepts:

Data manipulation using NumPy and Pandas
Data analysis using Pandas

Description: Python is a popular language for data science due to its powerful libraries. In this exercise, you'll use the NumPy and Pandas libraries to perform some data analysis.

Solution:

import numpy as np
import pandas as pd

# Create some data
data = {'x': np.arange(10),
        'y': np.random.randn(10)}

# Create a DataFrame from the data
df = pd.DataFrame(data)

# Calculate some statistics
print(df.mean())
print(df.std())

# Create a new column based on a calculation
df['z'] = df['x'] * df['y']

# Display the DataFrame
print(df)

Exercise 14: Web Development

Concepts:

Web development using the Flask framework
HTTP methods and routing

Description: Python can be used to build web applications using a variety of web frameworks. In this exercise, you'll write a simple Flask web application.

Solution:

from flask import Flask, request

app = Flask(__name__)

@app.route('/', methods=['GET'])
def home():
    return 'Hello, world!'

@app.route('/greet', methods=['POST'])
def greet():
    name = request.form.get('name')
    return f'Hello, {name}!'

if __name__ == '__main__':
    app.run()

Exercise 15: Asynchronous Programming

Concepts:

Asynchronous programming using asyncio
Coroutines

Solution:

import asyncio

async def coroutine():
    print('Coroutine started')
    await asyncio.sleep(1)
    print('Coroutine ended')

loop = asyncio.get_event_loop()
loop.run_until_complete(coroutine())
loop.close()

Exercise 16: Image Processing

Concepts:

Image processing using the Pillow library
Image manipulation and conversion

Description: Python has many libraries for working with images. In this exercise, you'll use the Pillow library to perform some basic image processing tasks.

Solution:

from PIL import Image

# Open an image file
image = Image.open('image.jpg')

# Display information about the image
print(image.format)
print(image.size)
print(image.mode)

# Convert the image to grayscale
grayscale_image = image.convert('L')
grayscale_image.show()

# Resize the image
resized_image = image.resize((300, 300))
resized_image.show()

# Save the image in a different format
resized_image.save('image.png')

Exercise 17: Email Sending

Concepts:

Email sending using the smtplib library
SMTP servers and email authentication

Description: Python has libraries for sending email programmatically. In this exercise, you'll write a script that sends an email using the smtplib library.

Solution:

import smtplib

sender_email = 'your_email@example.com'
sender_password = 'your_password'
receiver_email = 'recipient_email@example.com'
message = 'Hello, world!'

with smtplib.SMTP('smtp.gmail.com', 587) as server:
    server.starttls()
    server.login(sender_email, sender_password)
    server.sendmail(sender_email, receiver_email, message)

Exercise 18: Web API Integration

Concepts:

RESTful APIs and HTTP requests
Web API integration using the requests library

Description: Python can be used to integrate with many different web APIs. In this exercise, you'll write a script that interacts with a RESTful API using the requests library.

Solution:

import requests

response = requests.get('https://api.example.com/users')
data = response.json()

for user in data:
    print(user['name'], user['email'])

Exercise 19: Data Encryption

Concepts:

Data encryption using the cryptography library
Symmetric and asymmetric encryption

Description: Python has built-in libraries for encrypting and decrypting data. In this exercise, you'll write a script that encrypts and decrypts some data using the cryptography library.

Solution:

from cryptography.fernet import Fernet
from cryptography.hazmat.primitives import hashes
from cryptography.hazmat.primitives.asymmetric import rsa, padding
from cryptography.hazmat.primitives.serialization import load_pem_private_key, load_pem_public_key

# Symmetric encryption
key = Fernet.generate_key()
fernet = Fernet(key)
plaintext = b'This is a secret message'
ciphertext = fernet.encrypt(plaintext)
decrypted_plaintext = fernet.decrypt(ciphertext)
print(decrypted_plaintext)

# Asymmetric encryption
private_key = rsa.generate_private_key(
    public_exponent=65537,
    key_size=2048
)
public_key = private_key.public_key()

plaintext = b'This is a secret message'
ciphertext = public_key.encrypt(
    plaintext,
    padding.OAEP(
        mgf=padding.MGF1(algorithm=hashes.SHA256()),
        algorithm=hashes.SHA256(),
        label=None
    )
)

decrypted_plaintext = private_key.decrypt(
    ciphertext,
    padding.OAEP(
        mgf=padding.MGF1(algorithm=hashes.SHA256()),
        algorithm=hashes.SHA256(),
        label=None
    )
)
print(decrypted_plaintext)

Exercise 20: GUI Programming

Concepts:

GUI programming using the Tkinter library
Widgets and event handling

Description: Python has libraries for building graphical user interfaces (GUIs). In this exercise, you'll write a script that uses the Tkinter library to create a simple GUI.

Solution:

import tkinter as tk

def button_clicked():
    label.config(text='Hello, world!')

root = tk.Tk()

label = tk.Label(root, text='Welcome to my GUI')
label.pack()

button = tk.Button(root, text='Click me!', command=button_clicked)
button.pack()

root.mainloop()

When the button is clicked, the button_clicked() function is called, which updates the text of the label widget using the config() method.

Exercise 21: File I/O

Concepts:

File I/O in Python
Reading and writing files

Description: Python has built-in support for reading and writing files. In this exercise, you'll write a script that reads from and writes to a file.

Solution:

# Write to a file
with open('example.txt', 'w') as file:
    file.write('Hello, world!\n')
    file.write('How are you today?\n')

# Read from a file
with open('example.txt', 'r') as file:
    for line in file:
        print(line.strip())

Exercise 22: Logging

Concepts:

Logging in Python
Log levels and handlers

Description: Python has built-in support for logging. In this exercise, you'll write a script that logs some messages to a file.

Solution:

import logging

logging.basicConfig(filename='example.log', level=logging.DEBUG)

logging.debug('This is a debug message')
logging.info('This is an info message')
logging.warning('This is a warning message')
logging.error('This is an error message')
logging.critical('This is a critical message')

Exercise 23: Web Scraping

Concepts:

Web scraping using BeautifulSoup
HTML parsing and navigation

Description: Python can be used for web scraping, which involves extracting data from websites. In this exercise, you'll use the BeautifulSoup library to scrape data from a website.

Solution:

import requests
from bs4 import BeautifulSoup

response = requests.get('https://en.wikipedia.org/wiki/Python_(programming_language)')
soup = BeautifulSoup(response.text, 'html.parser')
print(soup.title.string)

links = soup.find_all('a')
for link in links:
    print(link.get('href'))

Exercise 24: Concurrency with asyncio

Concepts:

Asynchronous programming using asyncio
Coroutines and event loops

Solution:

import asyncio

async def coroutine(id):
    print(f'Coroutine {id} started')
    await asyncio.sleep(1)
    print(f'Coroutine {id} ended')

async def main():
    tasks = []
    for i in range(10):
        tasks.append(asyncio.create_task(coroutine(i)))
    await asyncio.gather(*tasks)

asyncio.run(main())

Exercise 25: Data Analysis with Pandas

Concepts:

Data analysis with Pandas
Loading and manipulating data with DataFrames

Description: Pandas is a powerful library for data analysis and manipulation in Python. In this exercise, you'll use Pandas to perform some data analysis tasks.

Solution:

import pandas as pd

# Load data from a CSV file
data = pd.read_csv('example.csv')

# Print the first five rows of the data
print(data.head())

# Calculate some statistics on the data
print(data.describe())

# Group the data by a column and calculate some statistics
grouped_data = data.groupby('category')
print(grouped_data.mean())

Finally, we group the data by the category column using the groupby() method and calculate some statistics on each group using the mean() method.

Exercise 26: Regular Expressions

Concepts:

Regular expressions in Python
Pattern matching and substitution

Description: Regular expressions are a powerful tool for searching and manipulating text. In this exercise, you'll write a script that uses regular expressions to search for patterns in text.

Solution:

import re

text = 'The quick brown fox jumps over the lazy dog'

# Search for a pattern in the text
pattern = r'\b\w{4}\b'
matches = re.findall(pattern, text)
print(matches)

# Substitute a pattern in the text
pattern = r'\bthe\b'
replaced_text = re.sub(pattern, 'a', text, flags=re.IGNORECASE)
print(replaced_text)

Exercise 27: Database ORM

Concepts:

Object-relational mapping using SQLAlchemy
Creating tables and querying data

Description: Python has many object-relational mapping (ORM) libraries for working with databases. In this exercise, you'll write a script that uses the SQLAlchemy ORM to interact with a database.

Solution:

from sqlalchemy import create_engine, Column, Integer, String
from sqlalchemy.orm import sessionmaker
from sqlalchemy.ext.declarative import declarative_base

# Define a database engine
engine = create_engine('sqlite:///example.db')

# Define a session factory
Session = sessionmaker(bind=engine)

# Define a declarative base
Base = declarative_base()

# Define a model class
class User(Base):
    __tablename__ = 'users'

    id = Column(Integer, primary_key=True)
    name = Column(String)
    email = Column(String)

# Create the table
Base.metadata.create_all(engine)

# Add some data
session = Session()
session.add(User(name='Alice', email='alice@example.com'))
session.add(User(name='Bob', email='bob@example.com'))
session.commit()

# Query the data
users = session.query(User).all()
for user in users:
    print(user.name, user.email)

Exercise 28: Web Scraping with Selenium

Concepts:

Web scraping with Selenium
Browser automation and DOM manipulation

Solution:

from selenium import webdriver

driver = webdriver.Chrome()

driver.get('https://en.wikipedia.org/wiki/Python_(programming_language)')

heading = driver.find_element_by_tag_name('h1')
print(heading.text)

links = driver.find_elements_by_tag_name('a')
for link in links:
    print(link.get_attribute('href'))

driver.close()

Exercise 29: Natural Language Processing

Concepts:

Natural language processing using the NLTK library
Tokenization, stemming, and POS tagging

Description: Python has many libraries for natural language processing (NLP). In this exercise, you'll use the NLTK library to perform some basic NLP tasks.

Solution:

import nltk
from nltk.tokenize import word_tokenize
from nltk.stem import PorterStemmer
from nltk import pos_tag

nltk.download('punkt')
nltk.download('averaged_perceptron_tagger')

text = 'The quick brown fox jumped over the lazy dogs'

# Tokenize the text
tokens = word_tokenize(text)
print(tokens)

# Stem the tokens
stemmer = PorterStemmer()
stemmed_tokens = [stemmer.stem(token) for token in tokens]
print(stemmed_tokens)

# Perform part-of-speech tagging on the tokens
pos_tags = pos_tag(tokens)
print(pos_tags)

Exercise 30: Machine Learning

Concepts:

Machine learning using scikit-learn
Data preparation, model training, and prediction

Description: Python has many libraries for machine learning, including scikit-learn. In this exercise, you'll use scikit-learn to train a simple machine learning model.

Solution:

import pandas as pd
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LinearRegression

# Load data from a CSV file
data = pd.read_csv('example.csv')

# Prepare the data
X = data['x'].values.reshape(-1, 1)
y = data['y'].values.reshape(-1, 1)

# Split the data into training and testing sets
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=0)

# Train the model
model = LinearRegression()
model.fit(X_train, y_train)

# Make predictions on the test set
y_pred = model.predict(X_test)

# Evaluate the model
score = model.score(X_test, y_test)
print(score)

Exercise 31: Image Recognition with Deep Learning

Concepts:

Deep learning with Keras
Convolutional neural networks and image recognition

Description: Deep learning is a powerful technique for image recognition. In this exercise, you'll use the Keras library to train a deep learning model to recognize images.

Solution:

import numpy as np
from keras.datasets import mnist
from keras.models import Sequential
from keras.layers import Dense, Flatten, Conv2D, MaxPooling2D
from keras.utils import to_categorical

# Load the MNIST dataset
(X_train, y_train), (X_test, y_test) = mnist.load_data()

# Preprocess the data
X_train = X_train.reshape(-1, 28, 28, 1) / 255.0
X_test = X_test.reshape(-1, 28, 28, 1) / 255.0
y_train = to_categorical(y_train)
y_test = to_categorical(y_test)

# Define the model architecture
model = Sequential()
model.add(Conv2D(32, (3, 3), activation='relu', input_shape=(28, 28, 1)))
model.add(MaxPooling2D((2, 2)))
model.add(Conv2D(64, (3, 3), activation='relu'))
model.add(MaxPooling2D((2, 2)))
model.add(Conv2D(64, (3, 3), activation='relu'))
model.add(Flatten())
model.add(Dense(64, activation='relu'))
model.add(Dense(10, activation='softmax'))

# Compile the model
model.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy'])

# Train the model
model.fit(X_train, y_train, epochs=5, batch_size=64)

# Evaluate the model
loss, accuracy = model.evaluate(X_test, y_test)
print(accuracy)

Exercise 32: Web APIs with Flask

Concepts:

Web APIs with Flask
RESTful architecture and HTTP methods

Description: Flask is a lightweight web framework for Python. In this exercise, you'll write a simple Flask application that exposes a web API.

Solution:

from flask import Flask, request

app = Flask(__name__)

@app.route('/hello', methods=['GET'])
def hello():
    name = request.args.get('name')
    if name:
        return f'Hello, {name}!'
    else:
        return 'Hello, world!'

if __name__ == '__main__':
    app.run()

Exercise 33: GUI Programming with PyQt

Concepts:

GUI programming with PyQt
Widgets and event handling

Description: PyQt is a powerful library for building graphical user interfaces (GUIs) with Python. In this exercise, you'll write a script that uses PyQt to create a simple GUI.

Solution:

import sys
from PyQt5.QtWidgets import QApplication, QWidget, QLabel, QLineEdit, QPushButton

class MyWindow(QWidget):
    def __init__(self):
        super().__init__()

        self.label = QLabel('Enter your name:', self)
        self.label.move(50, 50)

        self.textbox = QLineEdit(self)
        self.textbox.move(50, 80)

        self.button = QPushButton('Say hello', self)
        self.button.move(50, 110)
        self.button.clicked.connect(self.button_clicked)

        self.setWindowTitle('Hello, world!')
        self.setGeometry(100, 100, 200, 150)

    def button_clicked(self):
        name = self.textbox.text()
        self.label.setText(f'Hello, {name}!')

app = QApplication(sys.argv)
window = MyWindow()
window.show()
sys.exit(app.exec_())

Exercise 34: Web Scraping with Beautiful Soup and Requests

Concepts:

Web scraping with Beautiful Soup and Requests
HTML parsing and navigation

Description: Beautiful Soup and Requests are popular libraries for web scraping. In this exercise, you'll write a script that uses these libraries to scrape data from a website.

Solution:

import requests
from bs4 import BeautifulSoup

url = 'https://en.wikipedia.org/wiki/Python_(programming_language)'

response = requests.get(url)
soup = BeautifulSoup(response.text, 'html.parser')

title = soup.find('h1', {'id': 'firstHeading'}).text
print(title)

content = soup.find('div', {'id': 'mw-content-text'}).text
print(content[:100])

Exercise 35: Data Visualization with Matplotlib

Concepts:

Data visualization with Matplotlib
Line plots and labels

Description: Matplotlib is a popular library for data visualization in Python. In this exercise, you'll write a script that uses Matplotlib to create a simple line plot.

Solution:

import matplotlib.pyplot as plt

x = [1, 2, 3, 4, 5]
y = [1, 4, 9, 16, 25]

plt.plot(x, y)
plt.xlabel('x')
plt.ylabel('y')
plt.title('A Simple Line Plot')
plt.show()

Exercise 36: Data Analysis with NumPy

Concepts:

Data analysis with NumPy
Creating and manipulating arrays

Description: NumPy is a powerful library for numerical computing in Python. In this exercise, you'll write a script that uses NumPy to perform some basic data analysis tasks.

Solution:

import numpy as np

# Create a 2D array
data = np.array([[1, 2, 3], [4, 5, 6], [7, 8, 9]])

# Print the array
print(data)

# Calculate some statistics on the array
print(np.mean(data))
print(np.median(data))
print(np.std(data))

# Reshape the array
reshaped_data = data.reshape(1, 9)
print(reshaped_data)

Exercise 37: Object-Oriented Programming

Concepts:

Object-oriented programming in Python
Classes, objects, and inheritance

Description: Python is an object-oriented programming language. In this exercise, you'll write a script that demonstrates the use of object-oriented programming in Python.

Solution:

class Shape:
    def __init__(self, x, y):
        self.x = x
        self.y = y

    def move(self, dx, dy):
        self.x += dx
        self.y += dy

class Rectangle(Shape):
    def __init__(self, x, y, width, height):
        super().__init__(x, y)
        self.width = width
        self.height = height

    def area(self):
        return self.width * self.height

class Circle(Shape):
    def __init__(self, x, y, radius):
        super().__init__(x, y)
        self.radius = radius

    def area(self):
        return 3.14159 * self.radius ** 2

rect = Rectangle(0, 0, 10, 5)
print(rect.area())
rect.move(1, 1)
print(rect.x, rect.y)

circ = Circle(0, 0, 5)
print(circ.area())
circ.move(2, 2)
print(circ.x, circ.y)

In this script, we define a Shape class that has an x and y coordinate, and a move() method that moves the shape by a specified amount in the x and y directions.

We then define a Rectangle class that inherits from Shape, and adds a width and height attribute, as well as an area() method that calculates the area of the rectangle.

We also define a Circle class that inherits from Shape, and adds a radius attribute, as well as an area() method that calculates the area of the circle.

Finally, we create instances of the Rectangle and Circle classes, and call their area() and move() methods.

Exercise 38: Regular Expressions

Concepts:

Regular expressions in Python
Searching for patterns in text

Description: Regular expressions are a powerful tool for text processing in Python. In this exercise, you'll write a script that uses regular expressions to search for patterns in a text file.

Solution:

import re

with open('example.txt', 'r') as f:
    data = f.read()

pattern = r'\d{3}-\d{2}-\d{4}'
matches = re.findall(pattern, data)

for match in matches:
    print(match)

In this script, we open a text file and read its contents into a variable. We then define a regular expression pattern that matches a social security number in the format XXX-XX-XXXX.

We use the findall() function from the re module to find all occurrences of the pattern in the text, and print them out.

Exercise 39: File I/O

Concepts:

File I/O in Python
Reading and writing text files

Description: File I/O is a common task in Python programming. In this exercise, you'll write a script that reads data from a file, performs some processing, and writes the results to another file.

Solution:

with open('input.txt', 'r') as f:
    data = f.readlines()

# Process the data
output = []
for line in data:
    line = line.strip()
    words = line.split()
    words.reverse()
    output.append(' '.join(words))

# Write the results to a file
with open('output.txt', 'w') as f:
    for line in output:
        f.write(line + '\n')

We write the resulting strings to an output file, with each string on a separate line.

Exercise 40: Data Manipulation with Pandas

Concepts:

Data manipulation with Pandas
Reading and filtering data

Description: Pandas is a powerful library for data manipulation in Python. In this exercise, you'll write a script that uses Pandas to load, process, and analyze a dataset.

Solution:

import pandas as pd

# Load the data from a CSV file
data = pd.read_csv('example.csv')

# Filter the data to include only rows where the value is greater than 5
filtered_data = data[data['value'] > 5]

# Calculate the mean and standard deviation of the filtered data
mean = filtered_data['value'].mean()
std = filtered_data['value'].std()

print(f'Mean: {mean}')
print(f'Standard deviation: {std}')

In this script, we load a dataset from a CSV file into a Pandas DataFrame. We then filter the data to include only rows where the value is greater than 5.

We calculate the mean and standard deviation of the filtered data using the mean() and std() methods of the DataFrame, and print out the results.

Purchase this book

The App is Under a Quick Maintenance

We apologize for the inconvenience. Please come back later

Chapter 4: Intermediate Level Exercises

Intermediate Level Exercises

Exercise 1: Web Scraping

Exercise 2: File I/O

Exercise 3: Data Analysis

Exercise 4: Command-Line Interface

Exercise 5: API Integration

Exercise 6: Regular Expressions

Exercise 7: Object-Oriented Programming

Exercise 8: Concurrency

Exercise 9: Testing

Exercise 10: Data Visualization

Exercise 11: Database Operations

Exercise 12: Networking

Exercise 13: Data Science

Exercise 14: Web Development

Exercise 15: Asynchronous Programming

Exercise 16: Image Processing

Exercise 17: Email Sending

Exercise 18: Web API Integration

Exercise 19: Data Encryption

Exercise 20: GUI Programming

Exercise 21: File I/O

Exercise 22: Logging

Exercise 23: Web Scraping

Exercise 24: Concurrency with asyncio

Exercise 25: Data Analysis with Pandas

Exercise 26: Regular Expressions

Exercise 27: Database ORM

Exercise 28: Web Scraping with Selenium

Exercise 29: Natural Language Processing

Exercise 30: Machine Learning

Exercise 31: Image Recognition with Deep Learning

Exercise 32: Web APIs with Flask

Exercise 33: GUI Programming with PyQt

Exercise 34: Web Scraping with Beautiful Soup and Requests

Exercise 35: Data Visualization with Matplotlib

Exercise 36: Data Analysis with NumPy

Exercise 37: Object-Oriented Programming

Exercise 38: Regular Expressions

Exercise 39: File I/O

Exercise 40: Data Manipulation with Pandas

Intermediate Level Exercises

Exercise 1: Web Scraping

Exercise 2: File I/O

Exercise 3: Data Analysis

Exercise 4: Command-Line Interface

Exercise 5: API Integration

Exercise 6: Regular Expressions

Exercise 7: Object-Oriented Programming

Exercise 8: Concurrency

Exercise 9: Testing

Exercise 10: Data Visualization

Exercise 11: Database Operations

Exercise 12: Networking

Exercise 13: Data Science

Exercise 14: Web Development

Exercise 15: Asynchronous Programming

Exercise 16: Image Processing

Exercise 17: Email Sending

Exercise 18: Web API Integration

Exercise 19: Data Encryption

Exercise 20: GUI Programming

Exercise 21: File I/O

Exercise 22: Logging

Exercise 23: Web Scraping

Exercise 24: Concurrency with asyncio

Exercise 25: Data Analysis with Pandas

Exercise 26: Regular Expressions

Exercise 27: Database ORM

Exercise 28: Web Scraping with Selenium

Exercise 29: Natural Language Processing

Exercise 30: Machine Learning

Exercise 31: Image Recognition with Deep Learning

Exercise 32: Web APIs with Flask

Exercise 33: GUI Programming with PyQt

Exercise 34: Web Scraping with Beautiful Soup and Requests

Exercise 35: Data Visualization with Matplotlib