Color maps in matplotlib

To create a scatter chart that uses a selected color palette, enter the name of the selected color scheme as the cmap argument.

The colors are grouped (see documentation). For instance: Sequential – different intensity of one color, Diverging – different intensity of two contrasting colors, Qualitative – different contrasting colors, but matching the palette, e.g Pastel1 – containing different pastel colors.

import matplotlib.pyplot as plt

fig, ax = plt.subplots()

ax.set_title('Chart title')
ax.set_xlabel('X axis label')
ax.set_ylabel('Y axis label')

# x - list with X axis values, e.g. product name
# y - list with Y axis values, e.g. product price
# intensivity - a list with numbers corresponding to the intensity of the attribute
# cm - name of the selected color palette

ax.scatter(x, y, c=intensivity,  cmap=cm)  
  # e.g: plt.scatter(x, y, c=intensivity, s=50,  cmap='plasma')

mappable = ax.collections[0]
cbar = fig.colorbar(mappable=mappable)
cbar.set_label('intensivity')

plt.show()

The s parameter is the size of drawn characters.

To display a colorbar that shows the intensity of a given feature for the displayed point, the mappable object should be specified as a parameter, which for the scatter plot is stored as an element of the AxesSubplot collections list.

As a result, on the chart, which may show, for example, the name of the product and the price, you can add additional information using the color, e.g. popularity among buyers or the quantity of goods in the warehouse (e.g. green – the product is available without any problems, up to red – no goods in stock) )

Decorators with parameters – Fizz Buzz game

First, I will define a numbers function that returns a dictionary, where the keys are integers in the range given as the argument to the function (from start to end inclusive).

numbers() function source code:

def numbers(start, end):
    dict = {}
    for number in range(start, end + 1):
        dict[number] = ''
    return dict

The function can be modified using the decorators with parameters.

My decorator function that modifies dictionary values:

def fizzbuzz(number, message):
    def decor(func):
        def wrapper(*args):
            dict = func(*args)
            for key in dict:
                if not (key % number):
                    dict[key] += message
            return dict
        return wrapper
    return decor

Function numbers() definition with the decorator:

@fizzbuzz(5, 'BUZZ')
@fizzbuzz(3, 'FIZZ')
def numbers(start, end):
    dict = {}
    for number in range(start, end + 1):
        dict[number] = ''
    return dict

Calling the numbers function will not change, e.g. to display dictionary values ​​with keys from 0 to 15:

print(numbers(0, 15))

Modification by using a decorator is done behind the scenes.

Python and SQLite

Python includes the sqlite3 module in the standard library. To present the capabilities of the sqlite3 library, I’ll make a simple CRUD application for vehicle service and repair registration.

First, I’ll create two support files: car.py and cars_sql_scheme.py. The first one will define the Car class – describing the auto, and the second one will define the database schema (or in fact, queries creating two tables in the database).

car.py file listing

class Car():
    """Represents a sample car.

    Arguments:
    make - car make e.g. Honda
    model - car model e.g. Civic
    year - year of production
    vrn - vehicle registration number
    vin - VIN number
    sold - if car is still our property 

    """

    def __init__(self, make, model, year, vrn, vin, sold=False):
        self.make = make
        self.model = model
        self.year = year
        self.vrn = vrn
        self.vin = vin
        self.sold = sold

cars_sql_scheme.py file listing:

create_table_cars = """CREATE TABLE IF NOT EXISTS cars(
	make TEXT,
	model TEXT,
	year TEXT,
	vrn TEXT,
	vin TEXT,
	sold INTEGER
	)"""

create_table_repairs = """CREATE TABLE IF NOT EXISTS repairs(
	date TEXT,
	car INTEGER,
	description TEXT,
	FOREIGN KEY(car) REFERENCES cars(ROWID)
	)"""

In the third file, sql_helper.py, I define the Helper class, which will contain the methods:

  • add_car() – adding the car to the database
  • del_car() – removes the car from the database
  • search_by_vrn() – searches for a car by the registration number
  • show_all_cars() – returns all cars in the base
  • set_sold() – updates car data when the car has been sold
  • add_repair() – adds an annotation about the repair of the car
  • show_repairs() – displays all car repairs

The script imports the sqlite3 module, the Car class from car.py and the variables create_table_cars and create_table_repairs from cars_sql_scheme.py, i.e.

import sqlite3
from car import Car
from cars_sql_scheme import create_table_cars, create_table_repairs

class Helper():
    def __init__(self):
        self.conn = sqlite3.connect('cars.db')
        self.c = self.conn.cursor()
        self.c.execute(create_table_cars)
        self.c.execute(create_table_repairs)

The init () method creates a connection to the database named cars.db, creates a query cursor, and creates the cars and repairs tables (if not already created).

The add_car () method takes the vehicle make, model, year of construction, registration number, and VIN as arguments. The function returns a new instance of the Car class. Adding a new car to the database is carried out using the context manager, so the transaction is automatically committed (no need to commit () after each database change), i.e.

def add_car(self, make, model, year, vrn, vin):
    	"""Adds new car to database.

    	Arguments:
    	make - car make e.g. Honda
    	model - car model e.g. Civic
    	year - year of production
    	vrn - vehicle registration number
    	vin - VIN number

    	Returns:
    	new Car instance
    	"""
        with self.conn:
            self.c.execute("INSERT INTO cars VALUES (:make, :model, :year, :vrn, :vin, :sold)", {
                'make': make, 'model': model, 'year': year, 'vrn': vrn, 'vin': vin, 'sold': False})
            return Car(make, model, year, vrn, vin)

Similarly, the del_car () method looks like this:

def del_car(self, car):
    	"""Deletes car from database.

    	Arguments:
    	car  - car instance

    	Returns:
    	None
    	"""
        with self.conn:
            self.c.execute("DELETE FROM cars WHERE vin=:vin", {'vin': car.vin})

The search function does not require to commit transaction, so I do not use the context manager, i.e.

def search_by_vrn(self, vrn):
        """Search car by vehicle registration number.

        Arguments:
        vrn  - vehicle registration number

        Returns:
        search result tuple
        """
        self.c.execute("SELECT * FROM CARS WHERE vrn=:vrn", {'vrn': vrn})
        return self.c.fetchone()

    def show_all_cars(self):
    	"""Search availale cars.

        Returns:
        search result  - list of tuples
        """
        self.c.execute("SELECT * FROM CARS")
        return self.c.fetchall()

The set_sold () function sets the sold column in the database to True, represented by a value of 1.

def set_sold(self, car):
    	"""Mark car as sold.

        Arguments:
        car  - Car instance

        Returns:
        None
        """
        car.sold = True
        with self.conn:
            self.c.execute("UPDATE cars SET sold=True WHERE 
              vin=:vin",'vin': car.vin})

The last two methods allow you to add a note about the car’s repair and display the repair list. ie.

def add_repair(self, car, date, description):
    	"""Adds repair note.

        Arguments:
        car  - Car instance
        date  - repair date
        description  - repair description

        Returns:
        None
        """
        self.c.execute("SELECT ROWID FROM cars WHERE vin=:vin",
                       {'vin': car.vin})
        car_id = self.c.fetchone()[0]
        with self.conn:
            self.c.execute("INSERT INTO repairs VALUES (:date, :car, :description)", {
                'date': date, 'car': car_id, 'description': description})
def show_repairs(self, car):
        """Shows car repairs notes.

        Arguments:
        car  - Car instance

        Returns:
        search result  - list of tuples
        """
        self.c.execute("SELECT ROWID FROM cars WHERE vin=:vin",
                       {'vin': car.vin})
        car_id = self.c.fetchone()
        self.c.execute("SELECT * FROM repairs WHERE car=?",
                       (car_id))
        return self.c.fetchall()

Venv – how to use

The venv library present as a built-in module starting with python 3.3 allows you to create virtual environments.

The virtual environment allows the use of specific versions of libraries, regardless of those that are installed in the default system locations. This prevents errors when the new, updated version of the library does not work well with the rest of the project (the project requires an earlier version).

To create a virtual environment for the current project, type:

python -m venv project_name/env_name

A env_name directory is created in the project directory project_name that contains a virtual environment.

The virtual environment directory contains several subdirectories, including / bin directory containing a link to the python interpreter, as well as an activate file to start the environment. In addition, the env_name directory also has a /lib directory containing installed modules (site-packages).

To run the created environment named env_name, enter in the shell:

source project_name/env_name/bin/activate

You can close the environment with the deactivate command.

When we have a virtual environment running, the newly installed libraries are available only in the selected virtual environment. You can save the library required for the project in a text file with:

pip freeze > requirements.txt

If you have a requirements.txt file describing the project requirements for libraries and their versions, you can install them with:

pip install -r requirements.txt

Unit Testing in Python – using unittest module

In order to check the correctness of the written code, we can write verification tests using the unittest module.

Once written tests also help with subsequent changes to the code – if the changes did not cause errors.

We can also use the TDD (Test Driven Development) technique, in which writing tests precedes the creation of the actual program.

I’m going to show a class that tests the correctness of the methods present in the Employee class of the employee.py module. A simple Employee class will contain, in addition to the init () method, an alternative constructor (class method) and methods: set_bonus () – setting a bonus for an employee, get_full_salary () – returns salary (basis + bonus), get_email () – returns employee’s email address.

For this purpose, I’m writing a script containing a class that inherits from unittest.TestCase. Of course, the script has to import the unittest module and the tested module, i.e. the listing of test_employee.py will look like this:

import unittest
from employee import Employee

class EmployeeTestCase(unittest.TestCase):
    def setUp(self):
        self.john_doe = Employee('John', 5000)
        self.jane_doe = Employee.from_full_name('Jane', 'Doe', 6000)

    def test_get_email(self):
        self.assertEqual(self.john_doe.get_email(), 'John@company.com')
        self.assertEqual(self.jane_doe.get_email(), 'Jane.Doe@company.com')

    def test_get_full_salary(self):
        self.assertEqual(self.john_doe.get_full_salary(), 5000)
        self.assertEqual(self.jane_doe.get_full_salary(), 6000)
        self.john_doe.set_bonus(10)
        self.jane_doe.set_bonus(15)
        self.assertEqual(self.john_doe.get_full_salary(), 5500)
        self.assertEqual(self.jane_doe.get_full_salary(), 6900)

To run the above test script correctly, it must be loaded as a module, i.e.

python -m unittest test_employee.py

or attach the call to unittest.main () to the test script, i.e.

if __name__ == '__main__':
    unittest.main()

Based on the written test script, we can write the file employee.py, i.e.

class Employee():
    bonus = 0

    def __init__(self, name, salary):
        self.name = name
        self.salary = salary
    
    def get_email(self):
        pass

    def set_bonus(self, bonus):
        pass

    def get_full_salary(self):
        pass

    @classmethod
    def from_full_name(cls, first_name, last_name, salary):
        name = '{}.{}'.format(first_name, last_name)
        return cls(name, salary)

The stub of the Employee class created in this way, after running the tests, generates 2 failures in the performed tests (despite the presence of several assertions in one test).

Writing the Employee class methods will result in the correct passing of 2 tests, i.e.

class Employee():
    bonus = 0

    def __init__(self, name, salary):
        self.name = name
        self.salary = salary

    def get_email(self):
        return '{}@company.com'.format(self.name)

    def set_bonus(self, bonus):
        self.bonus = bonus

    def get_full_salary(self):
        return self.salary + self.salary * self.bonus / 100

    @classmethod
    def from_full_name(cls, first_name, last_name, salary):
        name = '{}.{}'.format(first_name, last_name)
        return cls(name, salary)

In addition to checking the equality of assertEqual (x, y), we can check various types of assertions, e.g.

  • assertNotEqual(x, y) # checks if x != y
  • assertIsNone(x) # checks if x is None
  • assertTrue(x) # checks if bool (x) is True

The setUp () method from the unittest module is run each time before the next test is performed. Likewise, there is a tearDown () method that executes after each test completes.

There are also setUpClass () and tearDownClass () class methods, which are run before starting and after completing all tests, eg to initiate and then close the database connection.

Running the program from a script

To run another program in Python, we can use the system () function from the os module. The problem is when we want to read the result of the running program, i.e.

import os
result = os.system('df -h')
print(result)

The above program will display the result of the df program in the console, but the value of the result variable is the error code of the os.system () function execution.

If we want the result variable to store the result returned by the program, then instead of the system () function we should use the popen () function or its newer equivalent – the Popen class from the subprocess module. ie.

# 1 first option - function popen()
import os
result = os.popen('df -h').read()
print(result)

# 2 second option - Popen class
import subprocess
command = subprocess.Popen('df -h', shell=True, stdout=subprocess.PIPE)
result = command.stdout.read().decode('utf-8')
print(result)

Parsing the CSV file

CSV files are text files in which each line represents one data record, and the individual data in the line is separated by a delimiter, usually a comma.

In the example below, we are parsing a refueling report from a gas station. The first line is the header and contains the data: Contractor’s data; Name; Surname; Correction number; WZ number; Date; Time; Counter; Station; Registration number; Card number; Product name; VAT percentage; Price at the station; Net price; Gross price; Discount value; Quantity; Net; VAT; Gross.

In this particular case, the delimiter is the semicolon character. The following lines will contain entries about the next refueling. We want to obtain from the source file data on the date of refueling, the registration number of the car and the number of liters of fuel taken.

import csv

with open('report.csv') as csv_file:
    csv_reader = csv.DictReader(csv_file, delimiter=';')
    total = 0
    for line in csv_reader:
        print('{}  {}  {} ltr'.format(
            line['Data'], line['Registration number'], line['Quantity']))

        total += float(line['Quantity'])
    print('Total: ', total, 'ltr')

    with open('new-report.csv', 'w') as new_csv_file:
        field_names = ['Date', 'Auto', 'Refueling']
        csv_writer = csv.DictWriter(
            new_csv_file, fieldnames=field_names, delimiter=';')
        csv_writer.writeheader()
        csv_file.seek(0)
        next(csv_reader)
        for line in csv_reader:
            dict = {}
            dict['Date'] = line['Date']
            dict['Auto'] = line['Registration number']
            dict['Refueling] = line['Quantity']
            csv_writer.writerow(dict)

We perform parsing using the csv module. Then, using the context manager, open the report.csv file for reading. We use the DictReader object for reading, thanks to which it will be possible to refer to the value by specifying the keys from the csv file header.
Then the total value is calculated – the total amount of fuel taken.

We save the obtained data on refueling in the new-report.csv file. In this case, we’re using a DictWriter object. To use the iterator again, set the file content pointer to the beginning of the file – csv_file.seek (0). We replace the default headers with the new field_names contained in the list, so it is necessary to jump to the new iterator value by executing next (csv_reader). We save the new line in the file with the writerow () method of the csv.DictWriter object.

PySimpleGUI – faster GUI creation

PySimpleGUI is a wrapper that facilitates and speeds up the creation of Python window applications. There are 4 ports, based on the following libraries: tkinter, Qt, WxPython, Remi.

Changing the port eg. from PySimpleGUI (based on tkinter) to PySimpleGUIQt (using Qt) does not require any further code changes! More details can be found >>here<<.

A sample program that collects data from the user and displays the data in a second window:

import PySimpleGUIQt as sg

layout = [[sg.Text('Please enter some sample text')],
          [sg.InputText()],
          [sg.Submit('Apply'), sg.Cancel('Cancel')]]

window = sg.Window('Data source window', layout)

event, values = window.Read()

window.Close()

if event == 'Apply':
    text_input = values[0]
    sg.Popup('Text entered:', text_input, title='Data display window')

The first line of the script imports the wrapper module (it must be installed in the system e.g. via pip, as well as the framework that is used by the port – in this case Qt).

The next line defines the layout, which is a list of lists. Each subsequent list defines the next line of the template – the Text element on the first line that writes ‘Please enter some sample text’ , the InputText field in the second, and the Apply and Cancel buttons on the third line.

Then we create a window and assign the previously created layout as the second argument. Pressing any button or closing the window will switch to the window status reading – a tuple consisting of: the event element (e.g. button name or None when the window closing button is pressed) and a dictionary containing keys describing the window input fields and the values ​​entered in these fields is returned.

When the key argument is not given for an input field when creating a layout, by default the dictionary keys are consecutive integer values. In the above example, the text entered, which is saved in the values ​​dictionary as the key value 0, is passed as an element of the string displayed in the popup window (Window displaying data).

Window applications in Python and PyQt5 – using UI files

In Python, we can use .ui files describing the appearance of the interface generated in the Qt Designer tool. To run a window application that uses the xml ui file containing the interface appearance and uses the Qt library, install the PyQt5 overlay, i.e.

pip install PyQt5

Then we can use the following code which loads the ui file and sets the title of the application window.

import sys
from PyQt5.QtWidgets import QApplication, QMainWindow
from PyQt5 import uic

class MyApp(QMainWindow):
    def __init__(self):
        QMainWindow.__init__(self)
        QMainWindow.setWindowTitle(self, 'App Window Title')
        uic.loadUi('app.ui', self)

if __name__ == "__main__":
    app = QApplication(sys.argv)
    window = MyApp()
    window.show()
    sys.exit(app.exec_())

The above code does nothing, except to display a window with selected buttons, labels, etc. In order for these elements to respond to e.g. a click, you must define signals and slots for the GUI element e.g. for a button named calculateButton we create a method which will be the slot corresponding to the clicked signal. A list of signals that a given element can respond to can be found in the Qt library documentation. We place the assignment of the method responding to the signal in the init () method, i.e.

self.calculateButton.clicked.connect(self.my_calculate_function)

and then define the method that will be run when the button is pressed (in this case, the pass statement):

def my_calculate_function(self):
        pass   

Basics of object-oriented programming in Python #3 instance, class and static methods

As in the case of class attributes and variables, the same division can be observed for methods, i.e. functions included in the class.

Instance methods in the definition as the first argument must have a reference to the class instance – self, e.g.

class Employee:
    bonus = 10
    def set_bonus(self, bonus):
        self.bonus = bonus

The set_bonus method above sets an instance variable that overrides the class variable bonus.

However, when we want to create a class method, we must use the argument cls instead of self, which is a reference to the class and not to an instance. In addition, you must use the @classmethod decorator, i.e.

@classmethod
    def set_base_bonus(cls, bonus):
        cls.bonus = bonus

The above method will set the base value of the variable bonus in the Employee class and all its instances.

Class methods can also be a way to create alternative constructors with a different number of arguments (constructor overloading). For example, considering the example Employee class, which takes the employee’s name and the salary value as arguments, we can create a class method that takes 3 arguments: first name, last name and salary value and returns an instance of the Employee class.

class Employee:
    def __init__(self, name, salary):
        self.name = name
        self.salary = salary
    @classmethod
    def from_full_name(cls, first, last, salary):
        name = '{} {}'.format(first, last)
        return cls(name, salary)

john_doe = Employee.from_full_name('John', 'Doe', 8000)

In addition to class methods, there are also static methods that are preceded by the @staticmethod decorator. Static methods take neither a self instance reference nor cls to a class as an argument. They can take explicit arguments that are parameters to these functions. Static functions are thematically related to the class in which they are contained, but should not operate on class or instance variables.