Skip to main content

Introduction to Computer Programming with Python: Chapter 8. Modules and Packages

Introduction to Computer Programming with Python
Chapter 8. Modules and Packages
  • Show the following:

    Annotations
    Resources
  • Adjust appearance:

    Font
    Font style
    Color Scheme
    Light
    Dark
    Annotation contrast
    Low
    High
    Margins
  • Search within:
    • Notifications
    • Privacy
  • Project HomeIntroduction to Computer Programming with Python
  • Learn more about Manifold

Notes

table of contents
  1. Cover
  2. Title Page
  3. Copyright Page
  4. Contents
  5. Chapter 1 Introduction
    1. Learning Objectives
    2. 1.1 A Brief History of Computers
    3. 1.2 Fundamentals of Computing and Modern Computers
      1. Number Systems and the Foundation of Computing
      2. Computability and Computational Complexity
      3. The Construction of Modern Computers
        1. Analog Computers
        2. Digital Computers
        3. Mechanic-Based Components
        4. Vacuum Tube–Based Components
        5. Transistors
        6. Integrated Circuits and Very Large-Scale Integrated Circuits
    4. 1.3 Programming and Programming Languages
    5. 1.4 Python Programming Language
      1. The Development and Implementation of Python
      2. Advantages of Python
      3. Resources for Python and Python Education
    6. 1.5 Getting Ready to Learn Programming in Python
      1. Installing and Setting Up the Python Programming Environment
        1. Installing Python
        2. Setting Up a Virtual Environment for a Python Project
        3. Installing Jupyter Notebook
        4. Installing Visual Studio Code
      2. Additional Tools Supporting Software Development in Python
        1. Buildbot
        2. Trac
        3. Roundup
    7. 1.6 Getting a Taste of Programming with Python
      1. Program Interactively with Python Interactive Shell
      2. Program with VS Code IDE
      3. Use Jupyter Notebook Within VS Code to Program Interactively
      4. Write Documentation in Markdown
        1. Headings
        2. Paragraphs
        3. New Lines
        4. Italic, Bold, and Strikethrough Texts
        5. Horizontal Rules
        6. Keyboard Keys
        7. Unordered Lists
        8. Ordered Lists
        9. Definition Lists
        10. Links
        11. Links to Internal Sections
        12. Images
        13. Blockquotes
        14. Tables
        15. Inline Program / Script Code
        16. Code Block
        17. Mathematical Formulas and Expressions
        18. To-Do List
        19. Escape Sequence for Special Characters
      5. Programming Interactively with Jupyter Notebook Within VS Code
      6. Run Python Programs Outside IDE
      7. Make the Python Program File Executable
      8. Errors in Programs
    8. 1.7 Essentials of Problem Solving and Software Development
      1. Design Algorithms to Solve Problems
      2. Phases of Software System Development
        1. Phase 1. Understand the Project
        2. Phase 2. Analyze the Requirements to Identify Computer-Solvable Problems and Tasks
        3. Phase 3. Design the System
        4. Phase 4. Implement the System
        5. Phase 5. Test the System
        6. Phase 6. Maintain the System
    9. 1.8 Manage Your Working Files for Software Development Projects
      1. Set Up Git on Your Computer and Version-Control Locally
      2. Set Up an Account on GitHub and Version-Control with Remote Repositories
    10. Chapter Summary
    11. Exercises
    12. Projects
  6. Chapter 2 Essential Building Blocks of Computer Programs
    1. Learning Objectives
    2. 2.1 Primary Constructs of Computer Programs in Python
      1. Vocabulary of the Programming Language
        1. Rules of Naming Identifiers
        2. Python Naming Conventions
        3. Names with Leading and/or Trailing Underscores
        4. Rules of Scope Resolution for Identifiers
      2. Simple Data Types
        1. Signed Integers (int)
        2. Float (float)
        3. Boolean (bool)
        4. Complex (complex)
      3. Compound Data Types
        1. String (str)
        2. List
        3. Tuple
        4. Set
        5. Dictionary
        6. Object
      4. Variables and Constants
        1. Variables
        2. Built-In Constants
      5. Operators
        1. Arithmetic Operators
        2. Comparison Operators
        3. Logical Operators
        4. Bitwise Operators
        5. Assignment Operators
        6. Identity Operators
        7. Sequence Operators
        8. Membership Operator
      6. Built-In Functions
      7. Expressions
    3. 2.2 Higher-Level Constructs of Python Programs
      1. Structure of Python Programs
      2. Documentation and Comments
      3. Simple Statements
        1. Expression Statement
        2. Assignment Statement
        3. print Statement
        4. input Statement
        5. assert Statement
        6. pass Statement
        7. del Statement
        8. return Statement
        9. open Statement
        10. yield Statement
        11. raise Statement
        12. break Statement
        13. continue Statement
        14. import Statement
        15. global Statement
        16. nonlocal Statement
        17. help Statement
      4. Compound Statements
        1. Code Blocks
        2. Rules of Indentation
        3. Rules of Spacing
        4. if Statement
        5. if-else Statement
        6. if-elif Statement
        7. if-elif-else Statement
        8. while Statement
        9. for Statement
        10. def Statement
        11. class Statement
        12. try-except Statement
        13. with Statement
    4. Chapter Summary
    5. Exercises
    6. Projects
  7. Chapter 3 Flow Control of Statements
    1. Learning Objectives
    2. 3.1 Selective with the if Statement
    3. 3.2 Single-branch selective with if Statement
    4. 3.3 Multiple-Branch Selective with if-elif-… and if-elif-…-else Statements
    5. 3.4 Iterate with for Statement
      1. Using break and continue Statements and an else Clause Within Loops
      2. Common Coding Mistakes with the for Loop
    6. 3.5 Iterate with the while Statement
      1. Common Coding Mistakes with a while Loop
    7. 3.6 Iterate with for Versus while
    8. Chapter Summary
    9. Exercises
    10. Projects
  8. Chapter 4 Handle Errors and Exceptions in Programs
    1. Learning Objectives
    2. 4.1 Errors in Your Programs
      1. Exception
      2. ArithmeticError
      3. OverflowError
      4. ZeroDivisionError
      5. FloatingPointError
      6. AssertionError
      7. AttributeError
      8. BufferError
      9. EOFError
      10. GeneratorExit
      11. ImportError
      12. IndexError
      13. KeyError
      14. KeyboardInterrupt
      15. MemoryError
      16. ModuleNotFoundError
      17. NameError
      18. NotImplementedError
      19. OSError
      20. BlockingIOError
      21. ChildProcessError
      22. ConnectionError
      23. BrokenPipeError
      24. ConnectionAbortedError
      25. ConnectionRefusedError
      26. ConnectionResetError
      27. FileExistsError
      28. FileNotFoundError
      29. IsADirectoryError
      30. NotADirectoryError
      31. PermissionError
      32. ProcessLookupError
      33. TimeoutError
      34. RecursionError
      35. ReferenceError
      36. RuntimeError
      37. StopIteration
      38. StopAsyncIteration
      39. SyntaxError
      40. IndentationError
      41. TabError
      42. SystemError
      43. SystemExit
      44. TypeError
      45. UnboundLocalError
      46. UnicodeError
      47. UnicodeEncodeError
      48. UnicodeDecodeError
      49. UnicodeTranslateError
      50. ValueError
    3. 4.2 Handling Runtime Errors and Exceptions
    4. Chapter Summary
    5. Exercises
  9. Chapter 5 Use Sequences, Sets, Dictionaries, and Text Files
    1. Learning Objectives
    2. 5.1 Strings
      1. Methods of Built-In Class str
      2. Built-In Functions and Operators for Strings
      3. Constructing and Formatting Strings
      4. Regular Expressions
    3. 5.2 Lists
    4. 5.3 Tuples
    5. 5.4 Sets
    6. 5.5 Dictionaries
    7. 5.6 List, Set, and Dictionary Comprehension
      1. List Comprehension
      2. Set Comprehension
      3. Dictionary Comprehension
    8. 5.7 Text Files
      1. Opening and Closing a File
      2. Write or Append to a File
      3. Reading from a File
      4. Update Existing Content of a Text File
      5. Deleting Portion of a Text File
    9. Chapter Summary
    10. Exercises
    11. Projects
  10. Chapter 6 Define and Use Functions
    1. Learning Objectives
    2. 6.1 Defining and Using Functions in Python
    3. 6.2 Parameters and Arguments in Functions
    4. 6.3 Recursive Functions
    5. 6.4 Anonymous Functions: lambda Expressions
    6. 6.5 Special Functions: Mapping, Filtering, and Reducing
      1. Mapping
      2. Filtering
      3. Reducing
    7. 6.6 Generators: Turning a Function into a Generator of Iterables
    8. 6.7 Closures: Turning a Function into a Closure
    9. 6.8 Decorators: Using Function as a Decorator in Python
    10. 6.9 Properties of Functions
    11. Chapter Summary
    12. Exercises
    13. Projects
  11. Chapter 7 Object-Oriented Programming with Python
    1. Learning Objectives
    2. 7.1 Introduction to Object-Oriented Programming (OOP)
      1. Abstraction
      2. Information Hiding or Data Encapsulation
      3. Inheritance
    3. 7.2 Defining and Using Classes in Python
      1. Inheritance: Subclass and Superclass
      2. Public, Private, and Protected Members of a Class
      3. Class Methods
      4. Static Methods
      5. Class Attributes
    4. 7.3 Advanced Topics in OOP with Python
      1. Dunder Methods in Class Definition
      2. Using Class as Decorator
      3. Built-In Property() Function and Property Decorator
      4. Creating a New Class Dynamically and Modify a Defined Class or Instance
      5. Keeping Objects in Permanent Storage
    5. Chapter Summary
    6. Exercises
    7. Project
  12. Chapter 8 Modules and Packages
    1. Learning Objectives
    2. 8.1 Creating Modules and Packages
    3. 8.2 Using Modules and Packages
    4. 8.3 Install and Learn About Modules Developed by Others
    5. 8.4 Module for Generating Random Numbers
      1. Functions for Bookkeeping
      2. Functions for Generating Random Integers
      3. Functions for Randomly Generating Float Numbers
      4. Functions for Randomly Selected Item(s) from Sequences
    6. 8.5 Module for Mathematical Operations
    7. 8.6 Modules for Time, Date, and Calendar
      1. The Datetime Module
      2. The Time Module
      3. The Calendar Module
    8. 8.7 Modules for Data Representation and Exchange
    9. 8.8 Modules for Interfacing Operating Systems and Python Interpreter
      1. OS Module for Interacting with the Operating System
      2. The path Submodule from os for Manipulating File Paths
      3. The sys Module for Interaction Between the Python and Python Interpreter or Python Virtual Machine (PVM)
    10. 8.9 Module for Logging Events During Program Runtime
    11. 8.10 Modules for Playing and Manipulating Audio and Video Files
      1. winsound
      2. PyGame
    12. 8.11 Modules for Creating and Manipulating Graphics and Images
      1. Create Graphics with Tkinter
      2. Manipulate Images with Pillow
    13. 8.12 Modules for Data Analytics
    14. Chapter Summary
    15. Exercises
    16. Projects
  13. Chapter 9 Develop GUI-Based Applications
    1. Learning Objectives
    2. 9.1 Terminal-Based Applications Versus GUI-Based Applications
    3. 9.2 Designing and Developing GUI-Based Applications in Python
      1. Tkinter Module
      2. tkinter.ttk—Tk-Themed Widgets
    4. Chapter Summary
    5. Exercises
    6. Projects

Chapter 8 Modules and Packages

Divide-and-conquer is a fundamental but effective strategy in problem solving as well as system design and development. This is because big problems can often be divided into smaller problems that can be easily solved or have already been solved, and large systems can often be made of smaller ones that can be easily created or are already readily available. Not only that, but the divide-and-conquer method also results in easier system maintenance, better quality insurance, quicker error detection and correction, and better reusability.

In Chapter 6, we learned how functions can be used in problem solving, and in Chapter 7, we studied object-oriented programming and learned how classes and objects can be used in system development and to solve problems. In computing, both functions and objects are common programming technologies that implement divide-and-conquer strategies.

In this chapter, we will study modules and packages that can also be used to implement the divide-and-conquer strategy in programming. In this way, we will learn how to create and use modules in programming and software development. We will also study some Python modules and packages that are readily available and often needed to solve problems and develop computer applications.

Learning Objectives

After completing this chapter, you should be able to

  • • describe modules.
  • • explain what packages are, what files are required, and how they are structured in a file system.
  • • import and use modules already in the Python programming/development environment.
  • • import and use specific parts from a module or specific modules from a package.
  • • explain the functionalities of some standard and widely used modules, and use them comfortably in programming.
  • • write and use your own modules and packages.

8.1 Creating Modules and Packages

As mentioned in previous sections, a module can be just a Python script file defining functions, classes, and other program constructs such as variables and constants. The following is an example of a Python module:

Code sample in VS Code IDE

1

"""

2

The module in this file defines some functions often used in calculations related to circles

3

4

Author: John Doe

5

Date: March 30, 2019

6

Version: 1.0

7

"""

8

9

10

PAI = 3.1415926

11

12

13

def area(r):

14

  """Calculate the area of a circle with radius r."""

15

  return PAI * r ** 2   # calculate the area

16

17

18

def circumference(r):

19

  """Calculate the circumference of a circle with radius r."""

20

  return 2 * PAI * r

21

22

The file can then be imported into a Python program and used as a module, as shown below:

In [ ]:

import circle

print(f'The area of a circle with a radius of 12 is {circle.area(12)}')

Out [ ]:

The area of a circle with a radius of 12 is 452.3893344

How do you create a module or package and publish it at https://pypi.org/ for the Python community at large so that it can be found and installed with the pip command?

Suppose you want to create a package for developing e-learning applications. You would need to take the following steps to develop the package to make it available for the others in the Python community:

  1. 1. First, create a directory called mypackages, which will contain all the packages and modules that you will develop for yourself or the Python community at large.
  2. 2. Suppose you would like the package to be called elearn. You first need to check to see if the name has been used by others to name any top-level module or package published at http://pypi.python.org.
  3. 3. Under the mypackages directory, create a directory named elearn, which will contain everything for your elearn package or module.
  4. 4. Under this elearn directory, you can create Python script files (.py) and other subdirectories under which you may have other packages or modules.
  5. 5. To distinguish this elearn directory from other ordinary directories of the file system so that it can be searchable through a Python interpreter, create a special file called __init__.py right under the elearn directory. The __init__.py file should contain all the names and objects defined in the package or module, either directly or indirectly (by importing individual names and objects from each of the other files). The must-have __init__.py file is the entry point of each package or module.
  6. 6. Under the mypackages directory, write a special Python script file called setup.py, which imports a special module called setuptools and calls the setup function to prepare the package to be sent to the repository. The following is an example of the setup.py file.

    import setuptools

    setup(name = 'elearn',

    version = '1.0beta',

    description = 'A package for developing elearn applications',

    url = 'http://github.com/AU.CA/elearn',

    author = 'SCIS Athabasca',

    author_email = 'scis@athabascau.ca',

    license = 'FSF free software foundation',

    packages = ['elearn'],

    zip_safe = False)

  7. 7. If you are not really ready to submit the package to the repository but would rather test it or just want to use it by yourself, you can install the package locally so that it can be found with the import statement. The following is the command to be used to install the package when your current working directory is mypackages:

    $ pip install .

  8. 8. To publish your elearn package, you need to
    1. a. Register the package with PyPi so that the online repository will know about your package and create an entry-point link to the elearn package at GitHub, as specified in the setup.py file.
    2. b. Create a single zip file that contains all Python script files.
    3. c. Upload the zip file to PYPI repository.

    By running the following command also under mypackages directory:

    $ python setup.py register sdist upload

Upon completing the above steps, anyone on the internet will be able to install and use your elearn library to develop elearn applications using the following command:

$ pip install elearn

Please note that for your package to be available in a GitHub repository so that the link used in your entry at PYPI is valid, you will need to create or sign into an account with GitHub and do the following:

  1. 1. Create the project within your account at GitHub
  2. 2. Install Git on your computer and use the commands to manually synchronize your work on your computer with GitHub

VS Code IDE can work with Git and GitHub through respective GitHub extensions that can be downloaded and installed in VS Code so that your project in VS can be easily synchronized with GitHub.

8.2 Using Modules and Packages

To use a module, we first import it, as shown in the following example:

Code sample in Python interactive mode

1

2

import circle

3

4

radius = float(input("Tell me the radius:"))

5

6

print(f"The area of a circle with the radius {radius} is {circle.area(radius)}")

7

print(f"The circumference of a circle with the radius {radius} is {circle.circumference(radius)}")

The result

Tell me the radius: 12.3

The area of a circle with the radius 12.3 is 475.2915444540001

The circumference of a circle with the radius 12.3 is 77.28317796

When importing a module, you can also give the module an alias to use, especially if the original name is too long or hard to remember. In the above code sample, the import statement import circle can be changed to ci, as in the following:

>>> import circle as ci

Then in the program file, you can use ci in place of circle, as shown below:

>>> print(f"The area of a circle with the radius {radius} is {ci.area(radius)}")

When designing and coding a system, you may create and use as many modules as needed, but do so wisely and do not make a simple system too complicated. Managing too many unnecessary modules and files will consume time and resources as well, especially since many Python modules have already been developed by the great Python community for almost every application domain you can think of. In most cases, all you need is to know is what modules are available out there and what each module does, even if you do not want to learn about the details now.

In general, the import statement may take one of the following forms.

import <the name of the module>

This is the simplest form, although you do need to know what the module has defined and know exactly what the name is, which is just the file name without the extension py.

import <the name of the module> as <simpler alias for the module name>

Sometimes, the name of a module can be long and hard to remember, and giving the module an alias would make programming more efficient. For example, there is a module for mathematical plotting called matplotlib if you give an alias to the module when importing it as follows:

import matplotlib as mpl

A name such as o defined in the module can then be referred to using mpl.o, which is much simpler than matplotlib.o.

from <module/package name> import <name of object or module>

When importing from a module, you can import a specific name you want to use instead of the entire module. For example, a mathematical module may have defined a number of mathematical constants such as PI and E. You may import only the one you want to use in your program.

As previously mentioned, a package usually contains other packages and modules, which can often be depicted as a tree. You can import just the package, module or even the name you want to use by using the import statement above. Again, the important thing is that you need to know where the thing you want to import is located within the tree. Assume from the root r the module m is located at r.n.m; then the import statement can be written as follows:

from r.n import m

This dot notation can also be used to import a module from a package without using from, as shown in the following example:

import matplotlib.pyplot as ppl

which imports the pyplot module from the matplotlib package and assigns an alias to the module.

from <module/package name> import <name of object or module> as <alias>

This is the last form an import statement can take. It gives an alias to the module/name imported from a package or module.

8.3 Install and Learn About Modules Developed by Others

To learn about modules already developed by others, your starting point can be a website called Python Package Index (PyPi) at https://pypi.org, where you can find, install, and publish Python packages and modules. By browsing this site, you will quickly get an idea what packages and modules have been already developed and published and are ready for use.

To see what modules and packages have already been installed on your computer, you can simply run pip list on a shell terminal such as PowerShell, as shown below:

PS C:\> pip list

Package             Version

------------------  -----------

appdirs             1.4.4

argon2-cffi         20.1.0

asgiref             3.3.1

async-generator     1.10

attrs               20.3.0

backcall            0.2.0

bleach              3.2.1

certifi             2020.11.8

cffi                1.14.3

chardet             3.0.4

colorama            0.4.4

decorator           4.4.2

defusedxml          0.6.0

distlib             0.3.1

Django              3.1.5

entrypoints         0.3

filelock            3.0.12

idna                2.10

ipykernel           5.3.4

ipython             7.19.0

ipython-genutils    0.2.0

ipywidgets          7.5.1

jedi                0.17.2

Jinja2              2.11.2

json5               0.9.5

jsonschema          3.2.0

jupyter             1.0.0

jupyter-client      6.1.7

jupyter-console     6.2.0

jupyter-core        4.6.3

jupyterlab          2.2.9

jupyterlab-pygments 0.1.2

jupyterlab-server   1.2.0

MarkupSafe          1.1.1

mistune             0.8.4

nbclient            0.5.1

nbconvert           6.0.7

nbformat            5.0.8

nest-asyncio        1.4.2

notebook            6.1.5

packaging           20.4

pandocfilters       1.4.3

parso               0.7.1

pickleshare         0.7.5

Pip                 21.1.3

pipenv              2020.11.15

prometheus-client   0.8.0

To find out what a particular installed module or package does, you can import the package or module into a Python interactive shell or a Jupyter Notebook cell, then run the dir command/statement to find out the names defined in the module or package and use the help command/statement to see more detailed information for a particular name.

PS C:\> python

Python 3.9.0 (tags/v3.9.0:9cf6752, Oct 5 2020, 15:34:40) [MSC v.1927 64 bit (AMD64)] on win32

Type "help", "copyright", "credits" or "license" for more information.

>>> import math

>>> dir(math)

['__doc__', '__loader__', '__name__', '__package__', '__spec__', 'acos', 'acosh', 'asin , asinh , atan', 'atan2', 'ata

nh', 'ceil', 'comb', 'copysign', 'cos', 'cosh', 'degrees', 'dist', 'e', 'erf', 'erfc', 'exp', 'expmr, 'fabs', 'factoria

l', 'floor', 'fmod', 'frexp', 'fsum', 'gamma', 'gcd', 'hypot', 'inf', 'isclose', 'isfinite', 'isinf', 'isnan', 'isqrt',

'1cm', 'Idexp', 'lgamma', 'log', 'log10', 'Ioglp', 'log2', 'modf', 'nan', 'nextafter', 'perm', 'pi', 'pow', 'prod', 'rad

ians', 'remainder', 'sin', 'sinh', 'sqrt', 'tan', 'tanh', 'tau', 'trunc', 'ulp']

>>> help(math)

Help on built-in module math:

NAME

    math

DESCRIPTION

    This module provides access to the mathematical functions

    defined by the C standard.

FUNCTIONS

    acos(x, /)

        Return the arc cosine (measured in radians) of x.

        The result is between 0 and pi.

    acosh(x, /)

        Return the inverse hyperbolic cosine of x.

    asin(x, /)

        Return the arc sine (measured in radians) of x.

        The result is between -pi/2 and pi/2.

    asinh(x, /)

        Return the inverse hyperbolic sine of x.

    atan(x, /)

        Return the arc tangent (measured in radians) of x.

        The result is between -pi/2 and pi/2.

    atan2(y, x, /)

        Return the arc tangent (measured in radians) of y/x.

        Unlike atan(y/x), the signs of both x and y are considered.

    atanh(x, /)

        Return the inverse hyperbolic tangent of x.

Another way to learn about and navigate through the available modules and packages is to use a module called pydoc. However, this module is usually run from a command shell or PowerShell terminal as shown below:

(base) PS C:\Users\james> python pydoc

pydoc - the Python documentation tool

pydoc <name> …

    Show text documentation on something. <name> may be the name of a

    Python keyword, topic, function, module, or package, or a dotted

    reference to a class or function within a module or module in a

    package. If <name> contains a '\', it is used as the path to a

    Python source file to document. If name is 'keywords', 'topics',

    or 'modules', a listing of these things is displayed.

pydoc -k <keyword>

    Search for a keyword in the synopsis lines of all available modules.

pydoc -n <hostname>

    Start an HTTP server with the given hostname (default: localhost).

pydoc -p <port>

    Start an HTTP server on the given port on the local machine. Port

    number 0 can be used to get an arbitrary unused port.

pydoc -b

    Start an HTTP server on an arbitrary unused port and open a Web browser

    to interactively browse documentation. This option can be used in

    combination with -n and/or -p.

pydoc -w <name> …

    Write out the HTML documentation for a module to a file in the current

    directory. If <name> contains a '\', it is treated as a filename; if

    it names a directory, documentation is written for all the contents.

(base) PS C:\Users\james>

Note that when you want to run a Python module as a normal script, run Python with the -m switch before the module name.

As shown above, if you want to get documentation on something, just add the something behind pydoc. For example, if you want to see the documentation on the math module, run the following command in the command shell:

python -m pydoc math

The resulting documentation is shown below:

Help on built-in module math:

NAME

  math

DESCRIPTION

  This module provides access to the mathematical functions

  defined by the C standard.

FUNCTIONS

  acos(x, /)

   Return the arc cosine (measured in radians) of x.

  acosh(x, /)

   Return the inverse hyperbolic cosine of x.

  asin(x, /)

   Return the arc sine (measured in radians) of x.

  asinh(x, /)

   Return the inverse hyperbolic sine of x.

  atan(x, /)

   Return the arc tangent (measured in radians) of x.

  atan2(y, x, /)

   Return the arc tangent (measured in radians) of y/x.

   Unlike atan(y/x), the signs of both x and y are considered.

  atanh(x, /)

   Return the inverse hyperbolic tangent of x.

With some installations of Python, such as those installed with Anaconda, pydoc has been made available as executable directly from command shell, as shown here:

(base) PS C:\Users\james> pydoc math

Help on built-in module math:

NAME

  math

DESCRIPTION

  This module provides access to the mathematical functions

  defined by the C standard.

FUNCTIONS

  acos(x, /)

   Return the arc cosine (measured in radians) of x.

  acosh(x, /)

   Return the inverse hyperbolic cosine of x.

  asin(x, /)

   Return the arc sine (measured in radians) of x.

  asinh(x, /)

   Return the inverse hyperbolic sine of x.

  atan(x, /)

   Return the arc tangent (measured in radians) of x.

  atan2(y, x, /)

   Return the arc tangent (measured in radians) of y/x.

   Unlike atan(y/x), the signs of both x and y are considered.

  atanh(x, /)

   Return the inverse hyperbolic tangent of x.

pydoc has some switches that can be used when running it as a Python module or an executable directly from the command shell. These switches include the following.

-k <keyword>

Used to search for a keyword in the synopsis lines of all available modules installed. The following example searches for all modules that has the word “hash” in the synopsis.

PS C:\Users\james> python -m pydoc -k hash

-n <hostname>

Used to start an HTTP server with the given <hostname>. When no hostname is given, localhost is default.

-p <port>

Used to start an HTTP server on the given <port> on the local machine. Port number 0 can be used to get an arbitrary unused port.

-b

Used to start an HTTP server on an arbitrary unused port and open a web browser to interactively browse documentation. This option can be used in combination with -n and/or -p.

The following example starts a server on the local machine at an available port and launches a web browser to browse all available modules and packages.

(base) PS S:\Dev\learn-python> pydoc -b

Server ready at http://localhost:30128/

Server commands: [b]rowser, [q]uit

server>

Please note that if you type q to quit from the program, the server will be down and no longer available.

The browser opened by the example above will look like this:

Index of Modules

Built-In Modules

  • _abc
  • _imp
  • _stat
  • _ast
  • _io
  • _statistics
  • builtins
  • _bisect
  • _json
  • _string
  • cmath
  • _blake2
  • _locale
  • _struct
  • errno
  • _codecs
  • _lsprof
  • _symtable
  • faulthandler
  • _codecs_cn
  • _md5
  • _thread
  • gc
  • _codecs_hk
  • _multibvtecodec
  • _tokenize
  • itertools
  • _codecs_iso2022
  • _opcode
  • _tracemalloc
  • marshal
  • _codecs_jp
  • _operator
  • _typing
  • math
  • _codecs_kr
  • _pickle
  • _warnings
  • mmap
  • _codecs_tw
  • _random
  • _weakref
  • msvcrt
  • _collections
  • _shal
  • _winapi
  • nt
  • _contextvars
  • _sha256
  • _xxsubinterpreters
  • sys
  • _csv
  • _sha3
  • array
  • time
  • _datetime
  • _sha512
  • atexit
  • winreg
  • _functools
  • _signal
  • audioop
  • xxsubtype
  • _heapq
  • _sre
  • binascii
  • zlib

Please note that the above only shows the built-in modules. There is more about nonbuilt-in modules if you scroll down to read further.

-w <name>…

Used to write out the HTML documentation for a module to a file in the current directory. If the name contains a backslash \, it is treated as a filename; if it names a directory, documentation is written for all the contents. The following example generates documentation in HTML for a module called timeit:

(base) PS S:\Dev\learn-python > pydoc -w timeit

wrote timeit.html

As can be seen, using the pydoc -b in command shell can easily access a server and start to browse documentation for all the available modules and packages installed on your computer. For example, if we want to learn more about the timeit module, we can browse or search for timeit within the first page of the browser window launched by the pydoc -b command, then click the link to see the details of the documentation.

The remainder of this chapter introduces some Python modules that you may need to use in the development of different computer applications.

8.4 Module for Generating Random Numbers

In certain computer applications, you often need to generate random numbers. For example, you need random numbers to automatically generate quizzes. In computer security, big random numbers play very important roles.

The random module comes with the standard Python distribution library. It provides functions and class definitions related to generating pseudorandom numbers, though they are not good enough to be used for security purposes. To see what names are defined at the top level of the module, run the dir(random) statement after importing the module to get a list of the names, as shown below:

>>> import random

>>> dir(random)

['BPF', 'LOG4', 'NV_MAGICCONST', 'RECIP_BPF', 'Random', 'SG_MAGICCONST', 'SystemRandom', 'TWOPI', '_BuiltinMethodType', '_MethodType', '_Sequence', '_Set', '__all__', '__builtins__', '__cached__', '__doc__', '__file__', '__loader__', '__name__', '__package__', '__spec__', '_acos', '_bisect', '_ceil', '_cos', '_e', '_exp', '_inst', '_itertools', '_log', '_os', '_pi', '_random', '_sha512', '_sin', '_sqrt', '_test', '_test_generator', '_urandom', '_warn', 'betavariate', 'choice', 'choices', 'expovariate', 'gammavariate', 'gauss', 'getrandbits', 'getstate', 'lognormvariate', 'normalvariate', 'paretovariate', 'randint', 'random', 'randrange', 'sample', 'seed', 'setstate', 'shuffle', 'triangular', 'uniform', 'vonmisesvariate', 'weibullvariate']

In the list, names with leading underscores are often intended to be hidden. It’s important to know only what those without leading underscores are and what they do. You can use the help statement to find out what randint is, for example:

>>> help(random.randint)

Running help on method randint in the random module outputs the following:

  • randint(a, b) method of random.Random instance

This returns a random integer within the range [a, b], including both end points.

As you can see with the help statement, all these names defined in the random module are functions and methods. These functions and methods can be categorized as bookkeeping methods, random integer generating methods, random real/float number generating methods, random item generating methods, or items from a sequence. The module also provides alternate random number generators such as the one provided by the operating system.

In the following, you will run through the functions provided in the random module. These functions are categorized into four groups: bookkeeping functions, functions randomly generating integers, functions randomly generating float/real numbers, and functions randomly selecting items from a sequence.

Functions for Bookkeeping

seed(a = None, version = 2)

This initializes the random number generator by seeding the generator. If there is no argument for a, the current time is used to seed the generator. Version 2 is the default version of the algorithm.

>>> import random

>>> random.seed(a = 3)

getstate()

This gets and returns the current internal state of the random number generator.

>>> s1 = random.getstate()

setstate(state)

This sets the internal state of the random number generator to state.

>>> random.setstate(s1)

getrandbits(k)

This generates and returns an integer with k random bits.

>>> bin(random.getrandbits(9))

'0b100101111'

>>> bin(random.getrandbits(9))   # will get a different number

'0b10111101'

Functions for Generating Random Integers

randrange(start, stop=None, step = 1)

This generates and returns a random number within the given range.

>>> random.randrange(99)

69

>>> random.randrange(99)

randint(a, b)

This generates and returns a random integer within the given range.

>>> random.randint(1, 100)

42

>>> random.randint(1, 100)

85

Functions for Randomly Generating Float Numbers

random()

This randomly generates and returns a random float number between 0 and 1, including 0 but excluding 1.

>>> random.random()

0.4084252811341471

uniform(a, b)

This randomly generates and returns a float number between a and b.

>>> random.uniform(2, 99)

73.1658416986511

>>> random.uniform(2, 99)

92.92610150048253

triangular(low = 0.0, high = 1.0, mode = None)

This randomly generates and returns a random float number between low and high. The third argument for mode parameter can be used to indicate the preference for the outcome. If it is closer to low, it is more likely to get a random float number on the low end, for example. Internally, the default value for mode is the midpoint between low and high.

>>> random.triangular(2, 99)

84.02716580051677

>>> random.triangular(2,99,35)

50.303535641546

betavariate(alpha, beta)

This randomly generates and returns a random float number between 0 and 1 based on the beta distribution of statistics. Parameters alpha and beta (both > 0) are used to set the conditions of the distribution, as used in the beta distribution function.

>>> random.betavariate(2, 99)

0.011368344795580798

>>> random.betavariate(2, 99)

0.019428131869773747

expovariate(lambda)

This randomly generates and returns a random float number between 0 and 1, or between 0 and −1 if lambda is negative, based on the exponential distribution of statistics.

>>> random.expovariate(2)

0.379317249922913

gammavariate(alpha, beta)

This randomly generates and returns a random float number between 0 and 1 based on the gamma distribution of statistics. Parameters alpha and beta (both > 0) are used to set the conditions of the distribution, as used in the gamma distribution function.

>>> random.gammavariate(2,99)

43.06391063895096

gauss(mu, sigma)

This randomly generates and returns a random float number between 0 and 1 based on the Gaussian distribution of probability theories. Parameter mu is the mean, and sigma is the standard deviation, as used in the distribution function.

>>> random.gauss(2, 99)

38.05513497609059

lognormvariate(mu, sigma)

This generates and returns a random float number between 0 and 1 based on a log-normal distribution of probability theories. Parameter mu is the mean, and sigma is the standard deviation, as used in the log normal distribution function.

>>> random.lognormvariate(2, 99)

9.252497191266324e-41

normalvariate(mu, sigma)

This generates and returns a random float number between 0 and 1 based on the normal distribution of probability theories. Parameter mu is the mean, and sigma is the standard deviation, as used in the normal distribution function.

>>> random.normalvariate(2, 99)

155.45854862650918

vonmisesvariate(mu, kappa)

This generates and returns a random float number between 0 and 1 based on the von Mises distribution of directional statistics. Parameter mu is the mean angle, expressed in radians between 0 and 2 * pi, whereas kappa is the concentration.

>>> random.vonmisesvariate(2, 99)

1.9289474404869416

paretovariate(alpha)

This generates and returns a random float number between 0 and 1 based on the Pareto distribution of probability theories. Parameter alpha is used to indicate the shape.

>>> random.paretovariate(2)

1.7794461337233882

weibullvariate(alpha,beta)

This generates and returns a random float number between 0 and 1 based on the Weibull distribution of statistics. Parameter alpha is the scale, and beta is the shape, as in its mathematical function.

>>> random.weibullvariate(2, 99)

2.0164248554211417

Functions for Randomly Selected Item(s) from Sequences

Functions in this group are often used in statistics.

choice(population)

This generates and returns a random element from the given population (in the form of a Python sequence).

>>> random.choice(range(1, 1000))

536

>>> random.choice(list("The message will look like this but"))

'l'

>>> random.choice(list("The message will look like this but"))

't'

choices(population, weights = None, *, cum_weights = None, k = 1)

This generates and returns a list with k randomly selected items from the given sequence, with weights or cumulative weights considered if they are given. An optional argument for weights should be a list of integers specifying how likely the corresponding items are to be selected. The number of integers in weights must match the number of items in the population; optional argument for cum_weights is also a list. A value in the list is shown as cumulation of weights so far. Argument weights and cum_weights are just different ways of representing the same preferences.

>>> random.choices(range(1, 1000), k = 5)

[456, 79, 57, 51, 110]

>>> random.choices(range(1, 10), weights = [2, 3, 5, 8, 2, 3, 3, 2, 10], k = 5)

[9, 4, 6, 3, 8]

This function can be very useful when you want to generate a quiz by randomly selecting questions from a question bank. Assume the question bank is called q_bank, and each quiz will have 10 questions. The random choices can be easily made by calling the function as follows:

>>> random.choices(q_bank, k = 10)

shuffle(population, random = None)

This takes a sequence, shuffles the members, then returns the sequence with the members randomly shuffled.

>>> l = list(range(10))

>>> l

[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]

>>> random.shuffle(l)

>>> l

[2, 9, 0, 4, 1, 6, 5, 3, 7, 8]

sample(population, k)

This generates and returns a sample for a given population. It seems similar to choices(), but internally the algorithm should check whether the choices would make a sample of the population according to some sampling criteria in statistics.

>>> random.sample(range(1, 1000), k = 5)

[251, 337, 822, 499, 853]

In the following programming problem in Table 8-1, you will study how well the random generator works. You will randomly generate 1000 integer numbers within the range of 0 to 999 and visualize the randomness of these numbers.

Table 8-1: Case study: Random number generator quality

The problem

This case study will find out how good the random number generator is.

The analysis and design

Random numbers generated by computers are not completely random. Rather, they are pseudorandom sequences. To study how good a generator is, you need to see if the numbers generated by the generator look random. We will show that in a two-dimensional chart to visualize it.

The code

import random

import matplotlib.pyplot as pyp

data = [random.randint(0,100) for i in range(1000)]

pyp.plot(list(range(1000)), data)

pyp.xlabel('Run #')

pyp.ylabel('Random Integer')

pyp.title('Visualization of Randomness')

pyp.show()

The result

Figure 8-1 below is plotted by the program. It seems the random generator works very well.

A graph where the x-axis is labeled “run number” 0 to 1,000 and the y-axis is labeled “Random Integer” 0 to 100.

Figure 8-1: Visualization of randomness

8.5 Module for Mathematical Operations

The math module also comes with the standard Python distribution library and provides many useful mathematical functions that you may need. To learn what the module has for you, type the following statement in a Jupyter Notebook cell and click run or press Shift+Enter at the same time. You will see the documentation for all these functions as well as the constants defined in the math module.

To learn what is defined in the module, run the dir statement on math after import, as we did before with the random module:

>>> import math

>>> dir(math)

['__doc__', '__loader__', '__name__', '__package__', '__spec__', 'acos', 'acosh', 'asin', 'asinh', 'atan', 'atan2', 'atanh', 'ceil', 'copysign', 'cos', 'cosh', 'degrees', 'e', 'erf', 'erfc', 'exp', 'expm1', 'fabs', 'factorial', 'floor', 'fmod', 'frexp', 'fsum', 'gamma', 'gcd', 'hypot', 'inf', 'isclose', 'isfinite', 'isinf', 'isnan', 'ldexp', 'lgamma', 'log', 'log10', 'log1p', 'log2', 'modf', 'nan', 'pi', 'pow', 'radians', 'remainder', 'sin', 'sinh', 'sqrt', 'tan', 'tanh', 'tau', 'trunc']

Then run help on a particular name to find out what that name, such as ceil, does:

>>> help(math.ceil)

Running help on the built-in function ceil in the math module outputs the following:

  • ceil(x, /)
  •     Return the ceiling of x as an Integral.

This returns the smallest integer >= x.

In the following, we run through the functions available in the math module.

acos(x)

This calculates and returns the arc cosine (measured in radians) of x, where 0 <= x < = 1.

>>> import math

>>> math.acos(0.5)

1.0471975511965979

acosh(x)

This calculates and returns the inverse hyperbolic cosine of x. (Google “hyperbolic functions” to learn more.)

>>> math.acosh(1000)

7.600902209541989

asin(x)

This calculates and returns the arc sine (measured in radians) of x.

>>> math.asin(0.5)

0.5235987755982989

math.asinh(x)

This calculates and returns the inverse hyperbolic sine of x.

>>> math.asinh(5)

2.3124383412727525

math.atan(x)

This calculates and returns the arc tangent (measured in radians) of x.

>>> math.atan(0.5)

0.4636476090008061

math.atan2(y, x)

This calculates and returns the arc tangent (measured in radians) of y/x. Unlike atan(y/x), the signs of both x and y are considered.

>>> math.atan2(3, -5)

2.601173153319209

math.atanh(x)

This calculates and returns the inverse hyperbolic tangent of x.

>>> math.atanh(0.5)

0.5493061443340549

math.ceil(x)

This calculates and returns the ceiling of x as an Integral—the smallest integer >= x.

>>> math.ceil(0.5)

1

math.copysign(x, y)

This calculates and returns a float with the magnitude (absolute value) of x but the sign of y. On platforms that support signed zeros, copysign(1.0, −0.0) returns −1.0.

>>> math.copysign(0.5, -1.2)

-0.5

math.cos(x)

This calculates and returns the cosine of x (measured in radians).

>>> math.cos(0.5)

0.8775825618903728

math.cosh(x)

This calculates and returns the hyperbolic cosine of x.

>>> math.cosh(50)

2.592352764293536e+21

math.degrees(x)

This converts angle x from radians to degrees and returns the result.

>>> math.degrees(0.5)

28.64788975654116

math.erf(x)

This is the error function at x, as defined in statistics.

>>> math.erf(0.5)

0.5204998778130465

math.erfc(x)

This is the complementary error function at x, so that erf(x) + erfc(x) = 1.

>>> math.erfc(0.5)

0.4795001221869534

math.exp(x)

This calculates and returns e raised to the power of x. This is the same as math.e**x, or math.pow(math.e, x).

>>> math.exp(5)

148.4131591025766

math.expm1(x)

This calculates and returns exp(x)−1. This function avoids the loss of precision involved in the direct evaluation of exp(x)−1 for small x.

>>> math.expm1(5)

147.4131591025766

math.fabs(x)

This calculates and returns the absolute value of the float x.

>>> math.fabs(-23.6)

23.6

math.factorial(x)

This calculates and returns x!. It will raise a ValueError if x is negative or nonintegral.

>>> math.factorial(23)

25852016738884976640000

math.floor(x)

This calculates and returns the floor of x as an integer—that is, the return value is the largest integer < = x.

>>> math.floor(2.3)

2

math.fmod(x, y)

This calculates and returns fmod(x, y), according to platform C. x % y may differ.

>>> math.fmod(2.3, 1.5)

0.7999999999999998

math.frexp(x)

This calculates the mantissa and exponent of x and returns a pair (m, e). m is a float, and e is an integer, such that x = m * 2** e. If x is 0, m and e are both 0. Otherwise, 0.5 <= abs(m) < 1.0.

>>> math.frexp(2.3)

(0.575, 2)

math.fsum(seq)

This calculates and returns an accurate floating-point sum of values in the iterable seq. It assumes IEEE-754 floating-point arithmetic. It is a lossless sum.

>>> math.fsum([2.3, 2, 53454, 6.71232])

53465.01232

math.gamma(x)

This returns the value of the gamma function at x.

>>> math.gamma(2.3)

1.16671190519816

math.gcd(x, y)

This calculates and returns the greatest common divisor of x and y.

>>> math.gcd(222, 780)

6

math.hypot(x, y)

This calculates and returns the Euclidean distance—that is, the value of sqrt(x * x + y * y).

>>> math.hypot(3, 4)

5.0

math.isclose(a, b, *, rel_tol = 1e-09, abs_tol = 0.0)

Determine whether two floating-point numbers are close in value. The rel_tol argument sets the maximum difference for being considered “close” relative to the magnitude of the input values, whereas abs_tol argument sets the maximum difference for being considered “close” regardless of the magnitude of the input values. Return True if a is close in value to b, and False otherwise. For the values to be considered close, the difference between them must be smaller than at least one of the tolerances set by rel_tol and abs_tol. -inf, inf, and NaN behave similarly to the IEEE 754 Standard. That is, NaN is not close to anything, even itself, and inf and -inf are only close to themselves.

>>> math.isclose(3.5, 3.51)

False

>>> math.isclose(3.5, 3.500000001)

True

>>> math.isclose(3.5, 3.50000001)

False

math.isfinite(x)

This returns True if x is neither an infinity nor a NaN, and False otherwise.

>>> math.isfinite(3.5)

True

math.isinf(x)

This returns True if x is a positive or negative infinity, and False otherwise.

>>> math.isinf(3.5)

False

math.isnan(x)

This returns True if x is a NaN (not a number), and False otherwise.

>>> math.isnan(3.5)

False

math.isqrt(n)

This returns the integer square root of the nonnegative integer n, which is the floor of the exact square root of n, or equivalently the greatest integer is such that a2 ≤ n. This function is only available in Python 3.8.0 or later.

>>> math.isqrt(43)

6

math.ldexp(x, i)

This calculates and returns x* (2 ** i). The function is essentially the inverse of frexp().

>>> math.ldexp(3, 12)

12288.0

math.lgamma(x)

This calculates and returns the natural logarithm of the absolute value of the gamma function at x.

>>> math.lgamma(3)

0.693147180559945

math.log(x, base = math.e)

This calculates and returns the logarithm of x to the given base. If the base is not specified, it returns the natural logarithm (base-e) of x.

>>> math.log(3)

1.0986122886681098

>>> math.log(3,3)

1.0

>>> math.log(3,5)

0.6826061944859854

math.log10(x)

This calculates and returns the base-10 logarithm of x.

>>> math.log10(3)

0.47712125471966244

math.log1p(x)

This calculates and returns the natural logarithm of 1 + x (base-e). The result is computed in a way that is accurate for x near 0.

>>> math.log1p(3)

1.3862943611198906

math.log2(x)

This calculates and returns the base-2 logarithm of x.

>>> math.log2(3)

1.584962500721156

math.modf(x)

This calculates and returns the fractional and integer parts of x. Both results carry the sign of x and are floats.

>>> math.modf(32.6)

(0.6000000000000014, 32.0)

math.pow(x, y)

This calculates and returns x ** y (x to the power of y).

>>> math.pow(32,6)

1073741824.0

math.radians(x)

This converts angle x from degrees to radians and returns the result.

>>> math.radians(32)

0.5585053606381855

math.remainder(x, y)

This calculates and returns the difference between x and the closest integer multiple of y, which is x − n * y, where n * y is the closest integer multiple of y. In the case where x is exactly halfway between two multiples of y, the nearest even value of n is used. The result is always exact.

>>> math.remainder(32,7)

-3.0

>>> math.remainder(30,7)

2.0

>>> math.remainder(31,7)

3.0

math.sin(x)

This calculates and returns the sine of x (measured in radians).

>>> math.sin(0.31)

0.3050586364434435

math.sinh(x)

This calculates and returns the hyperbolic sine of x.

>>> math.sinh(31)

14524424832623.713

math.sqrt(x)

This calculates and returns the square root of x.

>>> math.sqrt(31)

5.5677643628300215

math.tan(x)

This calculates and returns the tangent of x (measured in radians).

>>> math.tan(31)

-0.441695568020698

math.tanh(x)

This calculates and returns the hyperbolic tangent of x.

>>> math.tanh(31)

1.0

math.trunc(x)

This truncates the float number x to the nearest Integral toward 0. It uses the __trunc__ magic method.

>>> math.trunc(3.561)

3

In addition, the module also defines the following constants used in math:

math.e = 2.718281828459045

math.inf = inf

math.nan = nan

math.pi = 3.141592653589793

math.tau = 6.283185307179586


Note that to use these functions and constants, you will have to use the dot notation shown in the sample code above to indicate that it is from the math module.

If you only need to use one or some functions from the math module, you may import the particular functions from it to save computer memory and use the functions without using the dot notation. The following is an example, generating a table of square roots for numbers from 1 to 100, in which only the sqrt function has been imported from the math module:

In [ ]:

from math import sqrt

for i in range(100):

  print('{:<5.3}'.format(sqrt(i + 1)), end = ' ')

  if (i + 1) % 10 == 0:

    print('\n')

Out [ ]:

1.0 1.41 1.73 2.0 2.24 2.45 2.65 2.83 3.0 3.16

3.32 3.46 3.61 3.74 3.87 4.0 4.12 4.24 4.36 4.47

4.58 4.69 4.8 4.9 5.0 5.1 5.2 5.29 5.39 5.48

5.57 5.66 5.74 5.83 5.92 6.0 6.08 6.16 6.24 6.32

6.4 6.48 6.56 6.63 6.71 6.78 6.86 6.93 7.0 7.07

7.14 7.21 7.28 7.35 7.42 7.48 7.55 7.62 7.68 7.75

7.81 7.87 7.94 8.0 8.06 8.12 8.19 8.25 8.31 8.37

8.43 8.49 8.54 8.6 8.66 8.72 8.77 8.83 8.89 8.94

9.0 9.06 9.11 9.17 9.22 9.27 9.33 9.38 9.43 9.49

9.54 9.59 9.64 9.7 9.75 9.8 9.85 9.9 9.95 10.0

8.6 Modules for Time, Date, and Calendar

Date and time are often used and referred to in many applications. You may also want to include a calendar in an application. Python has modules in its standard distribution that allow you to import these modules right away.

The Datetime Module

The first of these modules is the datetime module that comes in the standard Python library. To use the module, simply import it as shown below:

>>> import datetime

To find out what is defined and available in the module, run the following dir statement:

>>> dir(datetime)

['MAXYEAR', 'MINYEAR', '__builtins__', '__cached__', '__doc__', '__file__', '__loader__', '__name__', '__package__', '__spec__', 'date', 'datetime', 'datetime_CAPI', 'sys', 'time', 'timedelta', 'timezone', 'tzinfo']

To further find out what each name is defined for, use the help statement on each. As usual, we will go through some of the important names defined in the module, with examples.

datetime.date(<year, month, day>)

This is the constructor of the date class defined in the datetime module and used to construct and return a date object for the day of the month of the year.

>>> d1 = datetime.date(2020, 7, 1)   # create a date object

>>> d1.year   # get the year attribute of the date object

2020

>>> d1.month   # get the month attribute of the date object

7

>>> d1.day   # get the day attribute of the date object

1


A date object has the following methods defined.

datetime.date.ctime()

This returns a ctime()-style string.

>>> d1.ctime()

'Wed Jul 1 00:00:00 2020'

datetime.date.isocalendar()

This returns a three-tuple containing an ISO year, week number of the year, and day number of the week. In the datetime module, Monday is 1, Tuesday is 2,…, Sunday is 7.

>>> d1.isocalendar()

(2020, 27, 3)

datetime.date.isoformat()

This returns a date string in ISO 8601 format, YYYY-MM-DD.

>>> d1.isoformat()

'2020-07-01'

datetime.date.isoweekday()

This returns an integer from 1 to 7 as the day of the week represented by the date.

>>> d1.isoweekday()

3

datetime.date.replace(…)

This returns the date with new specified fields.

datetime.date.strftime(…)

This changes the date format and returns a strftime()-style string.

datetime.date.timetuple(…)

This returns a time-tuple that is compatible with time.localtime().

datetime.date.toordinal(…)

This returns a proleptic Gregorian ordinal. January 1 of year 1 is day 1.

datetime.date.weekday(…)

This returns the day of the week represented by the date: Monday is 0…Sunday is 6.


The following are all class methods of the date class defined in the datetime module, which means they can be called from the class name date.

datetime.date.fromisoformat(<ISO_date_format string>)

This will construct a date object from an ISO date format string, which is YYYY-MM-DD.

>>> d2 = datetime.date.fromisoformat('2020-07-01')

>>> d2.ctime()

'Wed Jul 1 00:00:00 2020'

datetime.date.fromordinal(<days in relation to a proleptic Gregorian ordinal>)

This constructs a date object from an integer >= 1 representing the days after the proleptic Gregorian ordinal, which is January 1 of year 1, with ordinal 1.

>>> from datetime import date   # after this, you don't need to have datetime in the reference to date class

>>> d3 = date.fromordinal(1235)

>>> d3.ctime()

'Wed May 19 00:00:00 0004'

datetime.date.fromtimestamp(<timestamp>)

This constructs a local date object from a POSIX timestamp (a big positive float number), such as returned time.time(), which will be explained shortly.

>>> import time

>>> time.time()   # it returns a timestamp for now

1593745988.6121984

>>> date.fromtimestamp(1593745988.6121984)

datetime.date(2020, 7, 2)

datetime.date.today()

This returns a date object of for the current date.

>>> from datetime import date

>>> print(date.today())

2020-07-05


The datetime module also has a class called time. The following is the constructor of the time class.

datetime.time(hour = 0, minute = 0, second = 0, microsecond = 0, tzinfo = None)

This returns a time object. All arguments with 0 as their default value must be in their reasonable range or the program will raise a value error. If no argument is provided, they are all 0, except tzinfo (for time zone information, which needs to be an instance of tzinfo class if given). The default value of tzinfo is None.

>>> from datetime import time

>>> print(time(hour = 23))

23:00:00

>>> t1 = time(11, 25, 5)

time is 11:25:05

>>> print(f'time is {t1}')

>>> print(f'hour is {t1.hour}')   # this is to get the hour of a time

hour is 11

>>> print(f'minute is {t1.minute}')   # this is to get the minute

minute is 25

>>> print(f'second is {t1.second}')   # this is to get second

second is 5


The following is the only class method of the date class in the datetime module.

time.fromisoformat(…)

This class method will construct a date object from a string passed in the parameter.

>> import datetime

>> canada_day_str = "2022-07-01"

>> canada_day_object = datetime.date.fromisoformat(canada_day_str)

>> print(f"{canada_day_object} as {type(canada_day_object)}")

2022-07-01 as <class 'datetime.date'>


The datetime module also has a class called datetime, which is a combination of date and time. The following is the constructor of the datetime objects.

datetime.datetime(year, month, day, hour = 0, minute = 0, second = 0, microsecond = 0, tzinfo = None, *, fold = 0)

This returns a datetime object for the date and time given in the arguments. If no time is given, the default is the beginning of the day of the month of the year.


The following are the methods defined in the datetime class.

datetime.ctime(…)

This returns a ctime()-style time string.

>>> from datetime import datetime

>>> dt1 = datetime.now()

>>> print(f'now it is {dt1.ctime()}')

now it is Mon Jul 6 14:07:01 2020

datetime.astimezone(tz)

This converts to the local time with the time zone set to <tz> or local.

>>> from datetime import datetime

>>> dt1 = datetime.now()

>>> print(dt1)

2020-07-06 14:07:01.046202

>>> print(dt1.astimezone())

2020-07-06 14:07:01.046202-06:00

>>> import pytz

>>> asiachina=pytz.timezone('Asia/Chongqing')

>>> print(dt1)

2020-07-06 14:07:01.046202

>>> print(dt1.astimezone(asiachina)) # print time in China

2020-07-07 04:07:01.046202+08:00

A complete list of time zone names can be found at https://en.wikipedia.org/wiki/List_of_tz_database_time_zones.

datetime.date()

This returns a date object of the date portion of the datetime object with the same year, month, and day.

>>> from datetime import datetime

>>> tm1 = datetime.now()

>>> print(f'now is {dtm1.ctime()}')

now is Tue Jul 7 08:50:25 2020

>>> dt1 = dtm1.date()

>>> print(f'the date is {dt1}')

the date is 2020-07-07

datetime.dst()

This returns the DST (daylight saving time) status of a given tzinfo.

>>> print(f'the date is {dt1}')

the date is 2020-07-07

>>> print(f'the dst status is {dtm1.dst()}')

None

datetime.isoformat(sep = 'T')

This returns a date and time string in ISO 8601 format, YYYY-MM-DDT[HH[:MM[:SS[.mmm[uuu]]]]][+HH:MM]. sep is a single character used to separate the year from the time, and defaults to T. timespec specifies what components of the time to include. The allowed values include the following: auto, hours, minutes, seconds, milliseconds, and microseconds.

>>> from datetime import datetime

>>> dt1 = datetime.now()

>>> print(dt1.isoformat(sep='@'))

2023-03-13@18:51:29.324588

datetime.replace(<field>=<value>)

This returns a datetime object with the named field(s) replaced.

>>> dt2 = dt1.replace(year=2025)

>>> print(f'new datetime becomes {dt}')

new datetime becomes 2021-07-07 09:00:37.138388

datetime.time()

This returns a time object for the time portion of the datetime object but with tzinfo = None.

>>> dt1 = dtm1.date()

>>> tm1=dtm1.time()

>>> print(f'the date is {dt1}')

the date is 2020-07-07

>>> print(f'the time is {tm1}')

the time is 09:17:06.055195

datetime.timestamp()

This returns POSIX timestamp as a float number.

>>> tmstamp = dtm1.timestamp()

>>> print(f'The timestamp of {dtm1} is {tmstamp}')

datetime.timetuple()

This returns a time-tuple compatible with time.localtime().

>>> tmtuple = dtm1.timetuple()

>>> print(f'The time-tuple of {dtm1} is {tmtuple}')

datetime.timetz()

This returns a time object with the same time and tzinfo. Note the difference between timetz() and time().

>>> tminfo = dtm1.timetz()

>>> print(f'The time of {dtm1} is {tminfo}')

The timezone info of 2020-07-07 09:24:39.517213 is 09:24:39.517213

datetime.tzname(…)

This returns the tzname of tzinfo.

>>> import pytz

>>> tz = pytz.timezone('Canada/Mountain')

>>> dtm = datetime.fromisoformat('2020-07-05T21:05:33')

>>> ndtm = dtm.replace(tzinfo = tz)

>>> tmzname = ndtm.tzname()

>>> print(f'The timezone for {ndtm} is {tmzname}')

The timezone for 2020-07-05 21:05:33-07:34 is LMT

datetime.utcoffset(…)

This returns utcoffset of tzinfo.

>>> tmzutcoffset = ndtm.utcoffset()

>>> print(f'The timezone utc offset of {ndtm} is {tmzutcoffset}')

The timezone utc offset of 2020-07-05 21:05:33-07:34 is -1 day, 16:26:00


The following are some class methods defined in the datetime class.

datetime.combine(dt, tm)

This combines the date dt and time tm into a datetime object and returns the datetime object.

>>> dt = datetime.date.today()

>>> tm = datetime.time(20,59,12)

>>> dtm = datetime.datetime.combine(dt, tm)

>>> print(f'date is {dt}, time is {tm}, datetime is {dtm}')

date is 2020-07-05, time is 20:59:12, datetime is 2020-07-05 20:59:12

datetime.fromisoformat(dtmstr)

This constructs the datetime object from a date and time string in ISO format and returns the converted datetimeobject. Remember that the ISO time string format is YYYY-MM-DDTHH:MM:SS:mmm:uuu.

>>> dtm = datetime.datetime.fromisoformat('2020-07-05T21:05:33')

>>> print(dtm)

2020-07-05 21:05:33

datetime.fromtimestamp(…)

This constructs a datetime object from a POSIX timestamp.

>>> tmstamp1 = ndtm.timestamp()

>>> print(f'The time stamp of {ndtm} is {tmstamp1}')

The time stamp of 2020-07-05 21:05:33-07:34 is 1594010373.0

>>> redtmobj = datetime.fromtimestamp(tmstamp1)

>>> print(f'It is a different object and the time value has changed to {redtmobj}')

It is a different object, and the time value has changed to 2020-07-05 22:39:33.

datetime.now(tz = None)

This returns a datetime object representing the current time local to tz, which should be a Timezone object if given. If no tz is specified, the local timezone is used.

>>> from datetime import datetime   # import the datetime class from the datetime module

>>> dt1 = datetime.now()

>>> print(f'it is {dt1}')

it is 2020-07-05 11:22:48.876825

datetime.strptime(<date_string, format>)

This returns a datetime object by parsing a date_string, based on a given format.

>>> dtstring = "7 July, 2020"

>>> dtobj = datetime.strptime(dtstring, "%d %B, %Y")   # note the date formatting string

>>> print("date object = ", dtobj)

date object = 2020-07-07 00:00:00

datetime.today()

This returns a datetime object for today.

>>> dt2 = datetime.today()

>>> print(f'Today is {dt2}')

Today is 11:34:09.228618

datetime.utcfromtimestamp()

This constructs a naive UTC datetime from a POSIX timestamp.

>>> redtmobj = datetime.utcfromtimestamp(tmstamp1)

>>> print(f'{redtmobj} is a UTC datetime from a POSIX timestamp {redtmobj}')

2020-07-06 04:39:33 is a UTC datetime from a POSIX timestamp 1594010373.0

datetime.utcnow()

This returns a new datetime representing the UTC day and time.

>>> dt2 = datetime.today()

>>> dt3 = datetime.utcnow()

>>> print(f'Today is {dt2}, and the UTC time is {dt3}')

Today is 2020-07-07 11:37:02.862356, and the UTC time is 2020-07-07 17:37:02.862356


Sometimes, you need to deal with time intervals such as how long has passed since the last time you saw your best friend. That is what the timedelta class is defined for in the datetime module. The following is the constructor of the class.

datetime.timedelta(days = 0, seconds = 0, microseconds = 0, milliseconds = 0, minutes = 0, hours = 0, weeks = 0)

This constructs and returns a timedelta object. Note that all arguments are optional, and all default to 0 if not provided.

>>> from datetime import timedelta

>>> ndlt = timedelta(days = 31)

timedelta.total_seconds(…)

This returns the total number of seconds in the duration.

>>> print(f'the total number of seconds in 31 days is {ndlt.total_seconds()}')

the total number of seconds in 31 days is 2678400.0

>>> ndlt = timedelta(31, 25, hours = -3)

>>> print(f'the total number of seconds in 31 days is {ndlt.total_seconds()}')

the total number of seconds in 31 days and 25 seconds minus 3 hours is 2667625.0

The Time Module

The second of these modules is the time module. It comes with the standard Python distribution, so there is no need for you to install anything in particular. This module can be imported and used directly within your program when needed. The following statements get us a list of the names defined in the module:

>>> import time

>>> dir(time)

['_STRUCT_TM_ITEMS', '__doc__', '__loader__', '__name__', '__package__', '__spec__', 'altzone', 'asctime', 'clock', 'ctime', 'daylight', 'get_clock_info', 'gmtime', 'localtime', 'mktime', 'monotonic', 'monotonic_ns', 'perf_counter', 'perf_counter_ns', 'process_time', 'process_time_ns', 'sleep', 'strftime', 'strptime', 'struct_time', 'thread_time', 'thread_time_ns', 'time', 'time_ns', 'timezone', 'tzname']

In the following, we explain the names, including attributes and functions, available in the time module on a Windows platform. The code samples are all shown as they would appear in a Python interactive shell. If you wish to see all functions defined in the time module, please read the documentation at https://docs.python.org/3/library/time.html.

time.altzone

This is an attribute that contains the offset of the local DST timezone in seconds west of UTC, if one is defined. The value is negative if the local DST timezone is east of UTC (as in Western Europe, including the UK). Only use this if daylight is nonzero.

>>> import time

>>> print("local time zone is %d " % (time.altzone/3600))

local time zone is 6

time.asctime(tupletime)

This accepts a time-tuple and returns a readable 24-character string such as Tue Dec 11 18:07:14 2008. A time-tuple has nine elements, as returned by gmtime() or localtime().

>>> import time   # you only need to import a module once, so this is just in case

>>> print("local time is %s " % (time.asctime()))

local time is Tue Nov 12 15:10:50 2019

>>> time.asctime(tuple(time.localtime()))

'Tue Nov 12 15:24:05 2019'

time.clock()

This returns a floating-point number for the CPU time or real time since the start of the process or since the first call to clock(). It is very useful, especially when measuring the computational cost of a code block.

>>> time.clock()

428446.1717301

time.ctime([secs])

This returns a time in seconds since the epoch to a string in the local time. Remember that the argument in [] is optional.

This has the same result as asctime(localtime(secs)), and simply a call of asctime(), which will use the current local time in seconds.

>>> time.asctime()

'Tue Nov 12 15:24:49 2019'

>>> time.ctime()

'Tue Nov 12 15:24:55 2019'

time.get_clock_info(name)

This returns information on the specified clock as a namespace object. Supported clock names and the corresponding functions to read their value are the following: monotonic, perf_counter, process_time, thread_time, and time.

>>> time.get_clock_info('monotonic')

namespace(adjustable = False, implementation='GetTickCount64()', monotonic=True, resolution = 0.015625)

>>> time.get_clock_info('time')

namespace(adjustable = True, implementation = 'GetSystemTimeAsFileTime()', monotonic=False, resolution=0.015625)

time.gmtime([secs])

This accepts an instant expressed in seconds since the epoch and returns a time-tuple t with the UTC time. Note: t.tm_isdst is always 0.

>>> time.gmtime()

time.struct_time(tm_year = 2019, tm_mon = 11, tm_mday = 12, tm_hour = 22, tm_min = 21, tm_sec = 31, tm_wday = 1, tm_yday = 316, tm_isdst = 0)

>>> tuple(time.gmtime())

(2019, 11, 12, 22, 22, 2, 1, 316, 0)

time.localtime([secs])

This accepts an instant expressed in seconds since the epoch and returns a time-tuple t with the local time (t.tm_isdst is 0 or 1, depending on whether DST applies to instant secs by local rules).

>>> time.localtime()

time.struct_time(tm_year = 2019, tm_mon = 11, tm_mday = 12, tm_hour = 15, tm_min = 19, tm_sec = 0, tm_wday = 1, tm_yday = 316, tm_isdst = 0)

>>> tuple(time.localtime())

(2019, 11, 12, 15, 22, 32, 1, 316, 0)

time.mktime(tupletime)

This accepts a time instant expressed as a time-tuple in the local time and returns a floating-point value, with the instant expressed in seconds since the epoch.

>>> time.mktime((2019, 11, 12, 22, 22, 2, 1, 316, 0))

1573622522.0

time.monotonic()

This returns the value of a monotonic clock as a float number, the number of seconds since the previous call. The clock is not affected by system clock updates. The reference point of the returned value is undefined, so that only the difference between the results of consecutive calls is valid.

>>> time.monotonic()

1557979.093

time.monotonic_ns()

This is similar to monotonic() but returns time as nanoseconds.

>>> time.monotonic_ns()

1557954406000000

time.perf_counter()

This returns the value of a performance counter as a float number since the previous call. A performance counter is a clock with the highest available resolution to measure a short duration. It includes time elapsed during sleep and is system-wide. The reference point of the returned value is undefined, so that only the difference between the results of consecutive calls is valid.

>>> time.perf_counter()

429437.6389873

time.perf_counter_ns()

This is similar to perf_counter() but returns time as nanoseconds.

>>> time.perf_counter_ns()

429556266018100

time.process_time()

This returns the value (in fractional seconds) of the sum of the system and the user CPU time of the current process. It does not include time elapsed during sleep. It is process-wide by definition. The reference point of the returned value is undefined so that only the difference between the results of consecutive calls is valid.

>>> time.process_time()

6.71875

time.process_time_ns()

This is similar to process_time() but returns time as nanoseconds.

>>> time.process_time_ns()

6687500000

time.sleep(secs)

This suspends the calling thread for secs (seconds). It can be used to delay programs.

>>> time.sleep(6)   # sleep 6 seconds

time.strftime(fmt[,tupletime])

This accepts an instant expressed as a time-tuple in the local time and returns a string representing the instant as specified by string fmt.

>>> t = (2019, 11, 17, 17, 3, 38, 1, 48, 0)

>>> t = time.mktime(t)

>>> print(time.strftime("%b %d %Y %H:%M:%S", time.gmtime(t)))

Nov 18 2019 00:03:38

time.strptime(stringtime[, fmt])

This parses str according to format string fmt and returns the instant in time-tuple format.

>>> time.strptime('Tue Nov 12 15:24:05 2019','%a %b %d %H:%M:%S %Y')

time.struct_time(tm_year = 2019, tm_mon = 11, tm_mday = 12, tm_hour = 15, tm_min = 24, tm_sec = 5, tm_wday = 1, tm_yday = 316, tm_isdst = -1)

>>> tuple(time.strptime('Tue Nov 12 15:24:05 2019','%a %b %d %H:%M:%S %Y'))

(2019, 11, 12, 15, 24, 5, 1, 316, -1)

time.time()

This returns the current time instant, a floating-point number of seconds since the epoch.

>>> time.time()

1573607220.4043384

>>> time.asctime(time.localtime(time.time()))   # it is the same as time.asctime()

'Mon Jun 8 13:59:35 2020'

>>> time.asctime()

'Mon Jun 8 13:59:45 2020'

The Calendar Module

If you prefer a simple and more direct module to handle time and date, you can use the calendar module, as detailed below.

calendar.calendar(year, w = 2, l = 1, c = 6)

This returns a formatted calendar for year—a multiline string formatted into three columns separated by c spaces. w is the width in characters of each date; each line has length 21 * w + 18 + 2 * c. l is the number of lines for each week.

>>> import calendar as cl

>>> print(cl.calendar(2021))

2021

January
MoTuWeThFrSaSu
123
45678910
11121314151617
18192021222324
25262728293031
February
MoTuWeThFrSaSu
1234567
891011121314
15161718192021
22232425262728
March
MoTuWeThFrSaSu
1234567
891011121314
15161718192021
22232425262728
293031
April
MoTuWeThFrSaSu
1234
567891011
12131415161718
19202122232425
2627282930
May
MoTuWeThFrSaSu
12
3456789
10111213141516
17181920212223
24252627282930
31
June
MoTuWeThFrSaSu
123456
78910111213
14151617181920
21222324252627
282930
July
MoTuWeThFrSaSu
1234
567891011
12131415161718
19202122232425
262728293031
August
MoTuWeThFrSaSu
1
2345678
9101112131415
16171819202122
23242526272829
3031
September
MoTuWeThFrSaSu
12345
6789101112
13141516171819
20212223242526
27282930
October
MoTuWeThFrSaSu
123
45678910
11121314151617
18192021222324
25262728293031
November
MoTuWeThFrSaSu
1234567
891011121314
15161718192021
22232425262728
2930
December
MoTuWeThFrSaSu
12345
6789101112
13141516171819
20212223242526
2728293031

calendar.firstweekday()

This returns an integer that is the current setting for the weekday that starts each week. By default, when the calendar module is first imported, it is 0 for Monday.

>>> import calendar as cl

>>> cl.firstweekday()

0

calendar.isleap(year)

This tests if a year is a leap year. It returns True if it is; it returns False otherwise.

>>> cl.isleap(2022)

False

calendar.leapdays(y1, y2)

This returns the total number of leap days in the years within range(y1, y2).

>>> cl.leapdays(2020, 2030)

3

calendar.month(year, month, w = 2, l = 1)

This returns a multiline string with a calendar for month of year, one line per week plus two header lines. w is the width in characters of each date; each line has length 7 * w + 6. l is the number of lines for each week.

>>> print(cl.month(2021, 3))

March 2021
MoTuWeThFrSaSu
1234567
891011121314
15161718192021
22232425262728
293031

calendar.monthcalendar(year, month)

This returns a list of sublists of integers. Each sublist denotes a week starting from Monday. Days outside month of year are set to 0; days within the month are set to their day-of-month, 1 and up. The result as a list of sublists can be conveniently used in applications. For example, you can easily tell what date it is for Monday of the third week of a month.

>>> print(cl.monthcalendar(2020, 6))

[[1, 2, 3, 4, 5, 6, 7], [8, 9, 10, 11, 12, 13, 14], [15, 16, 17, 18, 19, 20, 21], [22, 23, 24, 25, 26, 27, 28], [29, 30, 0, 0, 0, 0, 0]]

calendar.monthrange(year, month)

This returns two integers. The first one is the code of the weekday for the first day of the month in year; the second one is the number of days in the month. Weekday codes are 0 (Monday) to 6 (Sunday); month numbers are 1 (January) to 12 (December). This is useful if you want to print a calendar that begins on a specific day of the week.

>>> print(cl.monthrange(2020, 6))

(0, 30)

calendar.prcal(year, w = 2, l = 1, c = 6)

This prints a well-formatted calendar of a given year. It is the same as calendar.calendar(year, w, l, c). Remember that w is the width of each date in number of characters and l is the number of lines for each week.

>>> cl.prcal(2020, w = 2, l = 1, c = 6)

2020

January
MoTuWeThFrSaSu
12345
6789101112
13141516171819
20212223242526
2728293031
February
MoTuWeThFrSaSu
12
3456789
10111213141516
17181920212223
242526272829
March
MoTuWeThFrSaSu
1
2345678
9101112131415
16171819202122
23242526272829
3031
April
MoTuWeThFrSaSu
12345
6789101112
13141516171819
20212223242526
27282930
May
MoTuWeThFrSaSu
123
45678910
11121314151617
18192021222324
25262728293031
June
MoTuWeThFrSaSu
1234567
891011121314
15161718192021
22232425262728
2930
July
MoTuWeThFrSaSu
12345
6789101112
13141516171819
20212223242526
2728293031
August
MoTuWeThFrSaSu
12
3456789
10111213141516
17181920212223
24252627282930
31
September
MoTuWeThFrSaSu
123456
78910111213
14151617181920
21222324252627
282930
October
MoTuWeThFrSaSu
1234
567891011
12131415161718
19202122232425
262728293031
November
MoTuWeThFrSaSu
1
2345678
9101112131415
16171819202122
23242526272829
30
December
MoTuWeThFrSaSu
123456
78910111213
14151617181920
21222324252627
28293031

calendar.prmonth(year, month, w = 2, l = 1)

This prints a well-formatted calendar month, the same as the one created by calendar.month(year, month, w, l).

>>> cl.prmonth(2020, 6, w = 2, l = 1)

June 2020
MoTuWeThFrSaSu
1234567
891011121314
15161718192021
22232425262728
2930

calendar.setfirstweekday(weekday)

This sets the first day of each week. Weekday codes are 0 (Monday by default) to 6 (Sunday by default), so if you change this, you will see the days in the calendar shift.

>>> cl.setfirstweekday(6)   # set to start from Sunday

>>> cl.prmonth(2020, 6, w = 2, l = 1)

June 2020
SuMoTuWeThFrSa
123456
78910111213
14151617181920
21222324252627
282930

calendar.timegm(tupletime)

This is the inverse of time.gmtime. It accepts a time instant in time-tuple form and returns the same instant as a floating-point number of seconds since the epoch.

>>> cl.timegm((2020, 6, 19, 11, 35, 56))

1592566556

calendar.weekday(year, month, day)

This returns the weekday code for the given date. Weekday codes are 0 (Monday) to 6 (Sunday); month numbers are 1 (January) to 12 (December).

>>> import calendar as cl

>>> cl.weekday(2020, 6, 19)   # it is a Friday

4

Our last example in Jupyter Lab is to display a calendar for March of 1961.

import calendar

cld = calendar.month(1961, 3)

print(cld)

March 1961
MoTuWeThFrSaSu
12345
6789101112
13141516171819
20212223242526
2728293031

With the calendar module, you will be able to produce a calendar of any year you want.

8.7 Modules for Data Representation and Exchange

JavaScript Object Notation (JSON) is a lightweight data interchange format widely used today. JSON can be used to represent different types of data, though the most used are objects or associate arrays made of key-value pairs. When used for data interchanges between applications, JSON data are represented in a string so that they can be stored, transmitted, and parsed by different applications.

Python has a built-in module called json to handle JSON data. The following two statements will get us a list of the few names defined in the json module:

>>> import json

>>> dir(json)

['JSONDecodeError', 'JSONDecoder', 'JSONEncoder', '__all__', '__author__', '__builtins__', '__cached__', '__doc__', '__file__', '__loader__', '__name__', '__package__', '__path__', '__spec__', '__version__', '_default_decoder', '_default_encoder', 'codecs', 'decoder', 'detect_encoding', 'dump', 'dumps', 'encoder', 'load', 'loads', 'scanner']

In the following, we explain two pairs of important functions provided in the json module. The first pair of functions is used to convert JSON data to Python data, which is called deserialization or decoding. The other is used to convert Python data to JSON data, which is called serialization.

json.loads(s, *, encoding=None, cls=None, object_hook=None, parse_float=None, parse_int=None, parse_constant=None, object_pairs_hook=None, **kw)

This deserializes json data in s, which can be a str, byte, or bytearray instance containing a JSON document and returns a Python object of s. Or, in plainer terms, it converts a string of JSON data into a Python object and returns the converted Python object.

>>> import json

>>> sj = '{"firstname": "Jone", "lastname": "Doe"}'

>>> pdict = json.loads(sj)

>>> print(pdict)

{'firstname': 'Jone', 'lastname': 'Doe'}

json.load(fp, *, cls = None, object_hook = None, parse_float=None, parse_int = None, parse_constant = None, object_pairs_hook = None, **kw)

This deserializes data in a file object or file-like object, such as a socket, referred to as fp that contains a JSON document, making it a Python object. Note the difference between loads and load function: loads convert JSON data in a string, whereas load converts JSON data in a file.

>>> import json

>>> from io import StringIO

>>> sio = StringIO('{"firstname": "Jone", "lastname": "Doe"}')

>>> json.load(sio)

{'firstname': 'Jone', 'lastname': 'Doe'}


The next pair of functions are used to convert Python objects into JSON data. This process is called serialization or encoding.

json.dumps(obj, *, skipkeys=False, ensure_ascii=True, check_circular=True, allow_nan = True, cls = None, indent = None, separators = None, default = None, sort_keys=False, **kw)

This serializes a Python object obj and returns a JSON-formatted string of the object.

>>> print(pdict)

{'firstname': 'Jone', 'lastname': 'Doe'}

>>> js = json.dumps(pdict)

>>> print(js)   # please note the double quotation marks used in JSON-formatted data

{"firstname": "Jone", "lastname": "Doe"}

json.dump(obj, fp, *, skipkeys = False, ensure_ascii = True, check_circular = True, allow_nan = True, cls = None, indent = None, separators = None, default = None, sort_keys = False, **kw)

This serializes the Python object obj and writes it as a JSON-formatted stream to a writable file or file-like object, such as a socket, referred to as fp.

>>> from io import StringIO

>>> io = StringIO()

>>> json.dump({'firstname': 'Jone', 'lastname': 'Doe'}, io)

>>> io.getvalue()

'{"firstname": "Jone", "lastname": "Doe"}'

Conversions can be made between JSON and Python on different types of data. Table 8-2 shows those conversions.

Table 8-2: Conversion between JSON and Python data

Python

Sample Python data

JSON

Sample JSON data

dict

{'firstname': 'Jone', 'lastname': 'Doe'}

Object

{"firstname": "Jone", "lastname": "Doe"}

list

['James', 'Jone', 'Smith', 'Doe']

Array

["James", "Jone", "Smith", "Doe"]

tuple

('James', 'Jone', 'Smith', 'Doe')

Array

["James", "Jone", "Smith", "Doe"]

str

"James, Jone, Smith, Doe"

String

"James, Jone, Smith, Doe"

int

98765321

Number

98765321

float

98765.321

Number

98765.321

True

True

true

true

False

False

false

false

None

None

null

null

As you can see, all basic Python data can be converted/serialized into JSON data, and vice versa. However, trying to serialize multiple Python objects by repeatedly calling dump() using the same file handle will result in an invalid JSON file because when doing deserialization with load() from the file, load has no way to find out the boundary between different JSON data. As such, there will be only one serialized Python object per file.

8.8 Modules for Interfacing Operating Systems and Python Interpreter

On a computer, all applications run on top of an operating system (OS) such as Windows, MacOS, or Linux. Therefore, quite often when developing applications, you will need to interact with an OS, file paths, and the Python interpreter. In Python, the built-in os module, path submodule of os, and sys module provide powerful and convenient ways to interact with the OS, the file path, the Python interpreter, and the runtime environment.

OS Module for Interacting with the Operating System

Since it is a built-in module, all you need to do to use the os module is import it, as shown below:

import os

If you use the dir(os) statement, you can get a rather big list of names defined in the module. Note that because the os module is operating-system dependent, you may get a different list of names available depending on your platform (e.g., Windows or Linux).

The following are some functions provided in the os module. You are encouraged to test these functions with your own examples on your own machine.

os.access(path, mode)

This tests if access to path is in mode, which is an integer such as 777 (111111111) representing the global, group, and user’s executable, write, and read rights.

>>> import os

>>> p = os.path.abspath(".")

>>> p

'd:\\workshop'

>>> os.access(p,763)

True

os.chdir(path)

This changes the current working directory to path.

>>> os.chdir('c:\\workbench')

>>> os.getcwd()

' c:\\workbench'

>>> os.listdir()

['myprimes.txt', ' news_01.txt', ' personal']

os.chmod(path, mode)

This changes the mode of path to the numeric mode.

>>> os.chmod('c:\\workbench', 477)

os.chown(path, uid, gid)

This changes the owner and group id of path to the numeric uid and gid. Please note that these operations are more similar to what you would do on a Unix/Linux system, all subject to permission by the operating system.

os.close(fd)

This closes the file descriptor fd. A file descriptor is returned by the os.open() function.

>>> fd = os.open('/home/james/testing.txt')

>>> os.close(fd)

os.cpu_count()

This returns the number of CPUs in the system; it will return None if the number is indeterminable.

>>> os.cpu_count()

8

os.get_exec_path(env=None)

This returns the sequence of directories that will be searched for by the named executable.

>>> import os

>>> print(os.get_exec_path())

['/opt/tljh/user/bin', '/usr/local/sbin', '/usr/local/bin', '/usr/sbin', '/usr/bin', '/sbin', '/bin', '/snap/bin']

os.getcwd()

This returns a Unicode string representing the current working directory.

>>> import os

>>> print(os.getcwd())

/home/jupyter-kevin

os.getcwdb()

This returns a byte string representing the current working directory.

>>> import os

>>> print(os.getcwdb())

b'/home/jupyter-kevin'

os.getenv(key, default=None)

This returns an environment variable and returns None if it does not exist. The optional second argument can specify an alternate default.

os.getlogin()

This returns the actual login name.

>>> os.getlogin()

kevin

os.link(src, dst)

This creates a hard link pointing to src named dst.

os.listdir(path)

This returns a list containing the names of the entries in the directory given by path.

>>> os.listdir('.')

['backups', 'Pipfile', 'snap']

os.mkdir(path, mode=511, dir_fd=None)

This function is used to create a directory, the mode argument only used on Unix-like systems, and will be ignored on Windows. The path argument is required; the mode argument is optional and takes an integer representing permission for the path to be created. Argument _dir_fd is a file descriptor referring to a directory that the new directory will be under; the path is not an absolute path. The default value is None.

>>> import os

>>> os.mkdir('/home/james/learn_python')

>>> os.listdir('.')

['backups', 'learn_python', 'Pipfile', 'snap']

os.makedirs(path, mode = 511, exist_ok = False)

This recursively makes directories. For example, if the path is ./comp218/assignment1, it will first make a directory named comp218 under the current working directory, if it doesn’t exist, then make assignment1 under comp218. The optional mode argument is the same as the one in os.mkdir(). The optional exist_ok argument tells if the operation will continue if the leaf directory already exists. The default is False, meaning that a FileExistsError will be raised if the leaf directory already exists.

os.open(path, flags, mode = 511, *, dir_fd = None)

This opens the file path and sets various flags for low-level IO and returns a file descriptor to be used by other functions in the os module. Argument dir_fd should be a file descriptor open to a directory (if not default None) and can be used to provide a directory that the file path is relative to. The flags argument tells what the path is opened for. It can take one or some of the following values joined with or |: os.O_RDONLY, os.O_WRONLY, os.O_RDWR, os.O_APPEND, os.O_CREAT, os.O_EXCL¶, os.O_TRUNC. These values are available on both Windows and Unix platforms.

os.putenv(name, value)

This changes or adds an environment variable if name doesn’t exist yet.

>>> os.times()

nt.times_result(user = 4.125, system = 1.890625, children_user = 0.0, children_system = 0.0, elapsed = 0.0)

os.read(fd, n)

This reads at most n bytes from file descriptor fd and returns a string containing the bytes read. If the end of the file referred to by fd has been reached, an empty string is returned.

os.umask(mask)

This sets the current numeric umask to mask and returns the previous umask. umask is used by operating systems to determine the default permission for newly created files.

>>> import os

>>> os.umask(666)   # from now on all new files created will have umask 666 till next change

256

os.urandom(size)

This returns a bytes object containing random bytes suitable for cryptographic use.

>>> os.urandom(5)

b'-\x8e\xeb\xf1\x7f'

os.utime(path, times = None)

This sets the access and modified times of path, such as on a file.

os.walk(top)

This is a directory-tree generator and returns a walk object. For each directory in the directory tree rooted at top, it will yield a three-tuple (dirpath, dirnames, filenames), in which dirpath is a string, the path to the directory; dirnames is a list of the names of the subdirectories in dirpath; and filenames is a list of the names of the nondirectory files in dirpath.

Note that the names in the lists are just names, with no path components. To get a full path (which begins with top) to a file or directory in dirpath, use os.path.join(dirpath, name).

In [ ]:

import os

# loop through all the directories and files

for dirName, subdirList, fileList in os.walk('.'):

    print('Found directory: %s' % dirName)

    for fname in fileList:

        print('\t%s' % fname)

Out [ ]:

Found directory: ./samples

        average marks.py

        brutal attack on cipher.py

        circle-v2.py

Found directory: ./samples/chapter3

    randomness.py

    regex1.py

    scratch-v3.py

    sieve prime.py

os.walk(top, topdown = True, onerror = None, followlinks = False)

This generates the file names in a directory tree by walking the tree either from the top down or from the bottom up. The os.walk function will be very useful in completing one of the projects in the textbook.

os.write(fd, str)

This writes the string str to the file descriptor fd and returns the number of bytes actually written.

The path Submodule from os for Manipulating File Paths

When dealing with files and file systems, we quite often need to manipulate file paths. For that reason, the os module has a submodule called path. To use the path module, run the following statement:

>>> from os import path

The path module provides functions for joining and splitting paths, getting information about a path or file such as its size and timestamp, and testing whether a path is a file, a directory, a real path, or just a link.

path.abspath(p)

This returns the absolute version of p.

>>> path.abspath('.')

'd:\\workshop\\comp218'

path.basename(p)

This returns the final component of a pathname.

>>> os.path.basename(p)

'comp218'

path.commonpath(paths)

This returns the longest common subpath for a given sequence of pathnames.

>>> os.path.commonpath(['d:/workshop/comp218','d:/workshop/comp369'])

'd:\\workshop'

path.commonprefix(paths)

This returns the longest common leading component of a given list of pathnames.

>>> os.path.commonprefix(['d:/workshop/comp218','d:/workshop/comp369'])

'd:/workshop/comp'

path.dirname(p)

This returns the directory component of a pathname.

>>> os.path.dirname('d:/workshop/comp218/test.py')

'd:/workshop/comp218'

path.exists(p)

This tests whether a path exists. It returns False for broken symbolic links.

>>> os.path.exists('d:/workshop/comp218/test.py')

False

path.expanduser(p)

This expands ~ and ~user constructs, mostly for Unix/Linux systems. If user or $HOME is unknown, it does nothing.

>>> os.path.expanduser('~/workshop/comp218/test.py')

'C:\\Users\\kevin/workshop/comp218/test.py'

path.expandvars(p)

This expands shell variables of the forms $var, ${var}, and %var%. Unknown variables will be left unchanged.

path.getatime(filename)

This returns the time a file was last accessed, as reported by os.stat().

path.getctime(filename)

This returns the time a file’s metadata was last changed, as reported by os.stat().

path.getmtime(filename)

This returns the time a file was last modified, as reported by os.stat().

path.getsize(filename)

This returns the size of a file, as reported by os.stat().

path.isabs(s)

This tests whether a path is absolute.

path.isdir(p)

path._isdir(p)

These return True if the pathname refers to an existing directory.

>>> from os import path

>>> path.isdir('.')

True

path.isfile(p)

This tests whether a path is a regular file.

>>> from os import path

>>> path.isfile('.')

False

path.islink(p)

This tests whether a path is a symbolic link. It will always return False for Windows prior to 6.0.

path.ismount(p)

This tests whether a path is a mount point (a drive root, the root of a share, or a mounted volume).

path.join(p1, p2)

This is used to join two paths or a path with a file.

>>> from os import path

>>> fullpath = path.join('/comp218/', 'testfile')

>>> print(fullpath)

/comp218/testfile

path.lexists(p)

This tests whether a path exists. It will return True for broken symbolic links.

path.normcase(s)

This normalizes the case of a pathname. That is, it makes all characters lower case and all slashes backslashes.

path.normpath(p)

This normalizes the path, eliminating double slashes, etc.

path.abspath(p)

This returns the absolute version of a path.

path.relpath(p, start=None)

This returns a relative version of a path.

path.samefile(f1, f2)

This tests whether two pathnames reference the same actual file or directory.

path.sameopenfile(fp1, fp2)

This tests whether two open file objects reference the same file.

path.samestat(s1, s2)

This tests whether two stat buffers reference the same file.

path.split(p)

This splits a pathname and returns tuple (head, tail), where tail is everything after the final slash.

path.splitdrive(p)

This splits a pathname into a drive/UNC sharepoint and relative path specifiers and returns a two-tuple (drive_or_unc, path); either part may be empty.

path.splitext(p)

This splits the extension from a pathname. An extension is everything from the last dot to the end, ignoring leading dots. For some paths without a dot, the extension part will be empty.

The sys Module for Interaction Between the Python and Python Interpreter or Python Virtual Machine (PVM)

The os and path modules we studied above provide programmers with ways to interact with the operating system and to access the underlying interface of the operating system. The sys module we are going to study below allows programs to interact with Python interpreter.

The following are the objects defined in the sys module and maintained by Python interpreter. These objects are put into two groups: dynamic objects and static objects.

The following are the dynamic objects defined in the sys module. Dynamic means the values can be changed.

sys.argv

This holds command-line arguments; argv[0] is the script pathname if known. The following example shows what happens when we test it in Jupyter Lab:

# factorial.py

def fac(n):

    if n == 0:

        return 1

    else:

        return n * fac(n-1)

n = 9

print(f"{n}! = {fac(n)}")

import sys

print(f'argv = {sys.argv}')

python -u "d:\workshop\research\books\COMP218\samples\factorial.py"

9! = 362880

argv = ['d:\\workshop\\research\\books\\COMP218\\samples\\factorial.py']

sys.path

This holds the module search path; path[0] is the script directory. The sys.path for the above Python program/script will be

sys.path = ['d:\\workshop\\research\\books\\COMP218\\samples', 's:\\python\\python311\\python311.zip', 's:\\python\\python311\\Lib', 's:\\python\\python311\\DLLs', 'C:\\Users\\james\\AppData\\Roaming\\Python\\Python311\\site-packages', 'C:\\Users\\james\\AppData\\Roaming\\Python\\Python311\\site-packages\\win32', 'C:\\Users\\james\\AppData\\Roaming\\Python\\Python311\\site-packages\\win32\\lib', 'C:\\Users\\james\\AppData\\Roaming\\Python\\Python311\\site-packages\\Pythonwin', 's:\\python\\python311', 's:\\python\\python311\\Lib\\site-packages']

sys.modules

This is a dictionary of all loaded modules. It will provide a long list of modules it is using.

sys.displayhook

This contains an executable object and can be called to show the results in an interactive session.

>>> sys.displayhook

<ipykernel.displayhook.ZMQShellDisplayHook at 0x15a70b56b48>

sys.excepthook

This contains an executable object and can be called to handle any uncaught exception other than SystemExit.

sys.stdin

This contains the standard input file object; it is used by input().

sys.stdout

It contains the standard output file object; it is used by print().

sys.stderr

This contains the standard error object; it is used for error messages.

sys.last_type

This contains the type of the last uncaught exception.

>>> sys.last_type

AttributeError

sys.last_value

This contains the value of the last uncaught exception.

>>> sys.last_value

AttributeError("module 'os' has no attribute 'chroot'")

sys.last_traceback

This contains the traceback of the last uncaught exception.

>>> sys.last_traceback

<traceback at 0x15a70ca9388>

The above three objects are only available in an interactive session after a traceback has been printed.


The next group of objects available from the sys module are called static objects, which means the values do not change for the given Python interpreter being used.

sys.builtin_module_names

This contains a tuple of built-in module names.

sys.copyright

This contains the copyright notice pertaining to the interpreter in use. sys.copyright in our case will produce the following, as an example:

Copyright (c) 2001–2022 Python Software Foundation.

All Rights Reserved.

Copyright (c) 2000 BeOpen.com.

All Rights Reserved.

Copyright (c) 1995–2001 Corporation for National Research Initiatives.

All Rights Reserved.

Copyright (c) 1991–1995 Stichting Mathematisch Centrum, Amsterdam.

All Rights Reserved.

sys.exec_prefix

This contains the prefix used to find the machine-specific Python library.

sys.executable

This contains the absolute path of the executable binary of the Python interpreter.

sys.float_info

This contains a named tuple with information about the float implementation.

sys.float_repr_style

This contains a string indicating the style of repr() output for floats.

sys.hash_info

This contains a named tuple with information about the hash algorithm.

>>> print(sys.hash_info)

sys.hash_info(width = 64, modulus = 2305843009213693951, inf = 314159, nan = 0, imag = 1000003, algorithm = 'siphash24', hash_bits = 64, seed_bits = 128, cutoff = 0)

sys.hexversion

This contains version information encoded as a single integer.

sys.implementation

This contains Python implementation information.

>>> print(sys.implementation)

namespace(cache_tag = 'cpython-37', hexversion = 50792432, name = 'cpython', version =s ys.version_info(major = 3, minor = 7, micro = 7, releaselevel = 'final', serial = 0))

sys.int_info

This contains a named tuple with information about the int implementation.

>>> print(sys.int_info)

sys.int_info(bits_per_digit = 30, sizeof_digit = 4)

sys.maxsize

This contains the largest supported length of containers.

>>> print(sys.maxsize)

9223372036854775807

sys.maxunicode

This contains the value of the largest Unicode code point.

>>> print(sys.maxunicode)

1114111

>>> print(chr(1114111))

􏿿

>>> print(chr(1114112))   # this is out of range and will cause an error

----------------------------------

ValueError   Traceback (most recent call last)

<ipython-input-81-1965bd6642f9> in <module>

----> 1< print(chr<(1114112))

ValueError: chr() arg not in range(0x110000)

sys.platform

This contains the platform identifier.

>>> print(sys.platform)

win32

sys.prefix

This contains the prefix used to find the Python library.

>>> print(sys.prefix)

C:\ProgramData\Anaconda3

sys.thread_info

This contains a named tuple with information about the thread implementation.

>>> print(sys.thread_info)

sys.thread_info(name = 'nt', lock = None, version = None)

sys.version

This contains the version of this interpreter as a string.

sys.version_info

This contains the version information as a named tuple.

sys.dllhandle

This is the integer handle of the Python DLL (Windows only).

sys.winver

This contains the version number of the Python DLL (Windows only).

sys.__stdin__

This is the original stdin.

sys.__stdout__

This is the original stdout.

sys.__stderr__

This is the original stderr.

sys.__displayhook__

This is the original displayhook.

sys.__excepthook__

This is the original excepthook.


The following are functions also defined in the sys module.

sys.displayhook()

This function prints an object to the screen and saves it in builtins.

sys.excepthook()

This function prints an exception and its traceback to sys.stderr.

sys.exc_info()

This function returns thread-safe information about the current exception.

sys.exit()

This function exits the interpreter by raising SystemExit.

sys.getdlopenflags()

This function returns flags to be used for dlopen() calls.

sys.getprofile()

This function returns the global profiling function.

sys.getrefcount()

This function returns the reference count for an object.

sys.getrecursionlimit()

This function returns the max recursion depth for the interpreter.

sys.getsizeof()

This function returns the size of an object in bytes.

>>> from datetime import datetime

>>> import sys

>>> dt1 = datetime.now()

>>> print(sys.getsizeof(dt1))

48

sys.gettrace()

This function gets the global debug tracing function.

sys.setcheckinterval()

This function controls how often the interpreter checks for events.

sys.setdlopenflags()

This function sets the flags to be used for dlopen() calls.

sys.setprofile()

This function sets the global profiling function.

sys.setrecursionlimit()

This function sets the max recursion depth for the interpreter.

sys.settrace()

This function sets the global debug tracing function.


As can be seen, the sys module gives programmers a way to find out information about the Python interpreter and the runtime environment in particular.

8.9 Module for Logging Events During Program Runtime

In some applications, sometimes it’s necessary or desirable to keep a record of what happened with the program for performance improvement, error debugging, security, and audit purposes. Examples of such events/data include, but are not limited to, how many times a function/method has been called, how long a function call takes, how much memory it used, and so on. In Python, a package called logging is available for logging within its standard distribution.

Due to the variety of purposes stated above, logged events can be classified into the following five categories based on the nature or severity of the events, in the view of the users of the application, as shown in Table 8-3.

Table 8-3: Levels of logging

Category of logs

Description

Numeric value of the level

NOTSET

The level hasn't been set.

0

DEBUG

Events are useful for error debugging. This is the lowest level of severity.

10

INFO

Information can be useful for improving the performance of the application/program or for assurance of security and auditing.

20

WARNING

Something needs checking.

30

ERROR

These errors are often logical and should be checked.

40

CRITICAL

The event is critical for the program to perform correctly and should be checked and resolved. This is highest level of severity.

50

The logging library defines several classes and module-level functions, and it is the latter that you would be using directly in your programs and applications. The basicConfig() function is used to set up the logging file and other parameters for the logger. The logging() function is for logging messages describing events in each category, as shown in the following code sample:

In [ ]:

import logging

logging.debug('Everything logged with logging.debug is labelled as debug')

logging.info('Everything logged with logging.info is labelled as info')

logging.warning('Everything logged with logging.warning is labelled as warning')

logging.error('Everything logged with logging.error is labelled as error')

logging.critical('Everything logged with logging.critical is labelled as critical')

The output of the code above is shown below:

Out [ ]:

WARNING:root:Everything logged with logging.warning is labelled as warning ERROR:root:Everything logged with logging.error is labelled as error CRITICAL:root:Everything logged with logging.critical is labelled as critical

You may have noted that output from the debug and info logging functions are missing from the output. This is because the default configuration of the logging module only logs events at warning level or higher. To change the default log-ging level, you can call a function of the logging module named basicConfig(), as shown in the following example:

1

import logging

2

3

logging.basicConfig(level=logging.DEBUG,

4

filename='c:\\users\\james\\myapp.log',

5

filemode='w', format='%(name)s - %(levelname)s - %(message)s')   # this line belongs to the basicConfig call as well

6

logging.debug('Everything logged with logging.debug is labelled as debug')

7

logging.info('Everything logged with logging.info is labelled as info')

8

logging.warning('Everything logged with logging.warning is labelled as warning')

9

logging.error('Everything logged with logging.error is labelled as error')

10

logging.critical('Everything logged with logging.critical is labelled as critical')

Instead of directly printing out to the terminal, this code writes the logs to a file named myapp.log, and the content of the generated logging file myapp.log is as follows:

root - DEBUG - Everything logged with logging.debug is labelled as debug

root - INFO - Everything logged with logging.info is labelled as info

root - WARNING - Everything logged with logging.warning is labelled as warning

root - ERROR - Everything logged with logging.error is labelled as error

root - CRITICAL - Everything logged with logging.critical is labelled as critical

Now as can be seen, DEBUG and INFO are recorded in the logging file because we changed the logging level to DEBUG.

Please note that for logging configuration to take full effect, all should be configured in a single basicConfig() method call. If the statement becomes too long to fit in a single line, the statement can take two or more lines, as long as newline is not within the string or word, as shown above.

In the basicConfig() function call shown above, keyword arguments are used to set the level of logging to DEBUG, the logging file name to c:\users\james\myapp.log, and the log file mode to w for write, which means that everything in the log file will be overwritten by new logging messages. If you want to keep the old logs and add the new logs to the end of the old logs, you need to set the file mode to a for append, which is the default set by the logging mode.

It has been noted that the basicConfig() function for logging is not fully functional within Jupyter Notebook. To change the logging level within Jupyter Notebook, you can use the logging.getLogger().setLevel() method. However, you cannot set the logging file name and logging file mode within Jupyter Notebook.

8.10 Modules for Playing and Manipulating Audio and Video Files

This section covers how to develop sound- and music-related applications with Python.

winsound

To play WAV files in your Windows applications, you can use the winsound module included in the standard Python distribution. You can import the module and use the functions defined in it without installing the module. Using the following statements, you can get a list of names defined in the module:

>>> import winsound

>>> dir(winsound)

['Beep', 'MB_ICONASTERISK', 'MB_ICONEXCLAMATION', 'MB_ICONHAND', 'MB_ICONQUESTION', 'MB_OK', 'MessageBeep', 'PlaySound', 'SND_ALIAS', 'SND_APPLICATION', 'SND_ASYNC', 'SND_FILENAME', 'SND_LOOP', 'SND_MEMORY', 'SND_NODEFAULT', 'SND_NOSTOP', 'SND_NOWAIT', 'SND_PURGE', '__doc__', '__file__', '__loader__', '__name__', '__package__', '__spec__']

For more details about the module and functionalities provided, run help(winsound) in Python interactive mode, as shown below:

>>> import winsound

>>> help(winsound)

 Help on module winsound:

  • NAME
  • winsound
  • DESCRIPTION
  • PlaySound(sound, flags)—play a sound
  • SND_FILENAME—sound is a wav file name
  • SND_ALIAS—sound is a registry sound association name
  • SND_LOOP—play the sound repeatedly; must also specify SND_ASYNC
  • SND_MEMORY—sound is a memory image of a wav file
  • SND_PURGE—stop all instances of the specified sound
  • SND_ASYNC—PlaySound returns immediately
  • SND_NODEFAULT—Do not play a default beep if the sound cannot be found
  • SND_NOSTOP—Do not interrupt any sounds currently playing
  • SND_NOWAIT—Return immediately if the sound driver is busy
  • Beep(frequency, duration)—Make a beep through the PC speaker.
  • MessageBeep(type)—Call Windows MessageBeep.
  • FUNCTIONS
  • eep(frequency, duration)
  • A wrapper around the Windows Beep API.
  • frequency
  • Frequency of the sound in hertz.
  • Must be in the range 37 through 32,767.
  • duration
  • How long the sound should play, in milliseconds.
  • MessageBeep(type = 0)
  • Call Windows MessageBeep(x).
  • x defaults to MB_OK.
  • PlaySound(sound, flags)
  • A wrapper around the Windows PlaySound API.
  • sound
  • The sound to play; a filename, data, or None.
  • flags
  • Flag values, ORed together. See module documentation.
  • DATA
  • MB_ICONASTERISK = 64
  • MB_ICONEXCLAMATION = 48
  • MB_ICONHAND = 16
  • MB_ICONQUESTION = 32
  • MB_OK = 0
  • SND_ALIAS = 65536
  • SND_APPLICATION = 128
  • SND_ASYNC = 1
  • SND_FILENAME = 131072
  • SND_LOOP = 8
  • SND_MEMORY = 4
  • SND_NODEFAULT = 2
  • SND_NOSTOP = 16
  • SND_NOWAIT = 8192
  • SND_PURGE = 64
  • FILE
  • s:\python\python311\dlls\winsound.pyd

Among the functions defined in the module, PlaySound is an important one for playing sound or music files. The following statement will play a WAV file named dj.wav.

>>> import winsound

>>> winsound.PlaySound("c:/users/comp218/dj.wav",winsound.SND_FILENAME)

When using the PlaySound function to a play sound file, you must make sure the WAV file exists in the default or specified path. In the example above, an absolute path has been given. You can also use a relative path that makes use of two special notations, a single dot (.) representing the current directory and a double dot (..) representing the parent directory; or you don’t need to specify the path at all if the WAV file is in the current directory. In any case, the rule is that you must be clearly aware of where the file is located. This rule is applicable whenever the file is used.

PyGame

The PlaySound function in the standard winsound module can play only WAV files. To play the popular MP3 music files in your Python applications, use the module called mixer in the PyGame package. Because the package is usually included in the standard Python distribution, you can install the package into your Python programming environment using the pip command, as shown below:

pip install pygame

Then you can import and use the mixer module to load and play MP3 files, as shown below:

>>> from pygame import mixer # load the required library

Hello from the pygame community. https://www.pygame.org/contribute.html

>>> mixer.init()

>>> mixer.music.load("../I_Will_Remember_You.mp3")

>>> mixer.music.play()

To learn more about how to use the mixer and mixer.music module, you can run the following commands in Python interactive mode as shown below, after the module has been imported:

>>> help(mixer)

You can then see the functions defined within the module, as further detailed below.

Channel(id)

This is used to create and return a Channel object for controlling playback.

fadeout(time)

This sets the time to fade out the volume on all sounds before stopping.

find_channel(force = False)

This finds and returns an unused channel.

get_busy()

This tests if any sound is being mixed and returns a Boolean value.

get_init()

This tests if the mixer is initialized and returns a tuple (frequency, format, channels) representing the channel.

get_num_channels()

This can be used to check and return the total number of playback channels.

init(frequency = 22050, size = −16, channels = 2, buffer = 4096, devicename = None, allowedchanges = AUDIO_ALLOW_FREQUENCY_CHANGE | AUDIO_ALLOW_CHANNELS_CHANGE)

This can be used to initialize the mixer module.

pause() -> None, temporarily stop playback of all sound channels

pre_init(frequency=22050, size = −16, channels = 2, buffersize = 4096, devicename = None)

These can be used to preset the mixer init arguments.

quit()

This can be used to uninitialize the mixer.

set_num_channels(count)

This can be used to set the total number of playback channels.

set_reserved(count)

This can be used to keep channels from being automatically used.

stop()

This can be used to stop playback on all sound channels.

unpause()

This can be used to resume playback on sound channels after it has been paused.


The mixer module has a submodule named music. To learn what functions are available in the mixer.music submodule module, run the following statement:

>>> help(mixer.music)

You will then see the following information about the related functions.

fadeout(time)

This can be used to stop music playback after fading out.

get_busy()

This can be used to check if the music stream is playing. It will return True or False.

get_endevent()

This can be used to get the event a channel sends when playback stops.

get_pos()

This can be used to get the music playtime.

get_volume() -> value

This can be used to get the music volume.

load(filename) -> None, or load(object) -> None,

This can be used to load a music file/object for playback.

pause() -> None

This can be used to temporarily stop music playback.

play(loops = 0, start = 0.0) -> None

This can be used to start the music stream playback.

queue(filename) -> None

This can be used to queue a music file to follow the currently playing file.

rewind() -> None

This can be used to restart the music.

set_endevent() -> None

set_endevent(type) -> None

These can be used to have the mixer send events when playback stops.

set_pos(pos) -> None

This can be used to set the position in the music file when starting playback.

set_volume(value) -> None

This can be used to set the music volume.

stop() -> None

This can be used to stop the music playback.

unpause() -> None

This can be used to resume paused music.


These functions in mixer.music are the ones used directly to handle music files. These functions are sufficient for you to develop a high-quality music player with what you will learn in Chapter 9 on developing GUI-based applications in Python.

Please note that the mixer module from the PyGame package can also play other types of music files including WAV, as shown below:

>>> from pygame import mixer   # import mixer module from PyGame

>>> mixer.init()   # initialize the mixer

>>> mixer.music.load("../I_will_Remember_you.mp3")   # load the mp3 file

>>> mixer.music.play(loops = 2)   # play the most recent loaded file twice

The functions listed above are needed if you are developing a music player with PyGame. For details on these functions, please refer to the official documentation on the PyGame mixer at https://www.pygame.org/docs/ref/mixer.html.

8.11 Modules for Creating and Manipulating Graphics and Images

In computing and information processing, graphics are an important part of data and information. In this section, we learn how to create and manipulate graphics and images with Python.

A blank widget window with the label “tk” in the header.

Figure 8-2: An example of TK Canvas

Create Graphics with Tkinter

The module built into the standard Python distribution for creating graphics is the Tkinter module, which is commonly used to develop graphical user interface (GUI) applications. However, Tkinter also provides a widget called Canvas for graphics and images. The following statements in Python interactive mode will produce a window containing a Canvas ready for drawing graphic objects—Canvas items:

>>> from tkinter import *

>>> d_board = Canvas()

Table 8-4 is a list of Canvas items we can draw on a Canvas.

Table 8-4: A list of functions that can be used to draw on a Canvas

Graphic object

Canvas method to create

Code sample

arc or arc-shaped region (such as a chord or pie slice)

create_arc(bbox, **options)

>>> d_board = Canvas()

>>> d_board.pack()

>>> d_board.create_arc(30, 50, 100, 200)

1

bitmap (built-in or read from an XBM file)

create_bitmap(position, **options)

>>> d_board.create_bitmap(30, 50)

image (a BitmapImage or PhotoImage instance)

create_image(position, **options)

>>> d_board.create_image(30, 50)

line

create_line(coords, **options)

>>> d_board.create_line(30, 50, 100, 200)

2

oval (a circle or an ellipse)

create_oval(bbox, **options)

>>> d_board.create_oval(30, 50, 80, 80, fill = "RED")

polygon

create_polygon(coords, **options)

>>> d_board.create_polygon(30, 50, 80, 80, 70, 90, fill = "RED")

rectangle

create_rectangle(bbox, **options)

>>> d_board.create_rectangle(30, 50, 80, 80, fill = "RED")

text

create_text(position, **options)

>>> d_board.create_text(130, 150, text = "Hello!")

window

create_window(position, **options)

Every method listed in Table 8-4 returns a unique ID for the created graphic object, which can be used later to manipulate the object.

Note that graphic objects created by the above methods will be stacked on the Canvas and will remain until being moved, lifted, lowered, or deleted, with the methods in Table 8-5.

Table 8-5: Other methods of Canvas within Tkinter

Method

Operation

Code sample

dchars(item, from, to = None)

Deletes text from an editable graphic item such as text: from is where to start deleting text, to is where to stop deleting text. If to is omitted, only a single character is deleted.

>>> d_board.create_text(130, 150, text = "Hello Python!")

>>> d_board.dchars('text', 1,2)

delete(item)

Deletes all matching items.

>>> h3 = d_board.create_text(230, 150, text="Hello!")

>>> d_board.delete(h3)

>>> d_board.delete(3)

>>> d_board.delete(10)

>>> d_board.delete(11)

>>> d_board.delete(9)

coords(item, *coords)

Returns or sets the coordinates of matching items.

>>> d_board.coords(o1, 30, 150, 80, 250)

move(item, dx, dy)

Moves matching items by an offset.

>>> d_board.move(o1, 10, 15)

Canvas has many other methods for accessing and manipulating graphic objects. Running the following statements in Python interactive mode will give you a list of names defined within the Canvas class.

>>> from tkinter import *

>>> dir(Canvas)

['_Misc__winfo_getint', '_Misc__winfo_parseitem', '__class__', '__delattr__', '__dict__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__getitem__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__le__', '__lt__', '__module__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__setitem__', '__sizeof__', '__str__', '__subclasshook__', '__weakref__', '_bind', '_configure', '_create', '_displayof', '_do', '_getboolean', '_getconfigure', '_getconfigure1', '_getdoubles', '_getints', '_grid_configure', '_gridconvvalue', '_last_child_ids', '_nametowidget', '_noarg_', '_options', '_register', '_report_exception', '_root', '_setup', '_subst_format', '_subst_format_str', '_substitute', '_tclCommands', '_windowingsystem', 'addtag', 'addtag_above', 'addtag_all', 'addtag_below', 'addtag_closest', 'addtag_enclosed', 'addtag_overlapping', 'addtag_withtag', 'after', 'after_cancel', 'after_idle', 'anchor', 'bbox', 'bell', 'bind', 'bind_all', 'bind_class', 'bindtags', 'canvasx', 'canvasy', 'cget', 'clipboard_append', 'clipboard_clear', 'clipboard_get', 'columnconfigure', 'config', 'configure', 'coords', 'create_arc', 'create_bitmap', 'create_image', 'create_line', 'create_oval', 'create_polygon', 'create_rectangle', 'create_text', 'create_window', 'dchars', 'delete', 'deletecommand', 'destroy', 'dtag', 'event_add', 'event_delete', 'event_generate', 'event_info', 'find', 'find_above', 'find_all', 'find_below', 'find_closest', 'find_enclosed', 'find_overlapping', 'find_withtag', 'focus', 'focus_displayof', 'focus_force', 'focus_get', 'focus_lastfor', 'focus_set', 'forget', 'getboolean', 'getdouble', 'getint', 'gettags', 'getvar', 'grab_current', 'grab_release', 'grab_set', 'grab_set_global', 'grab_status', 'grid', 'grid_anchor', 'grid_bbox', 'grid_columnconfigure', 'grid_configure', 'grid_forget', 'grid_info', 'grid_location', 'grid_propagate', 'grid_remove', 'grid_rowconfigure', 'grid_size', 'grid_slaves', 'icursor', 'image_names', 'image_types', 'index', 'info', 'insert', 'itemcget', 'itemconfig', 'itemconfigure', 'keys', 'lift', 'location', 'lower', 'mainloop', 'move', 'nametowidget', 'option_add', 'option_clear', 'option_get', 'option_readfile', 'pack', 'pack_configure', 'pack_forget', 'pack_info', 'pack_propagate', 'pack_slaves', 'place', 'place_configure', 'place_forget', 'place_info', 'place_slaves', 'postscript', 'propagate', 'quit', 'register', 'rowconfigure', 'scale', 'scan_dragto', 'scan_mark', 'select_adjust', 'select_clear', 'select_from', 'select_item', 'select_to', 'selection_clear', 'selection_get', 'selection_handle', 'selection_own', 'selection_own_get', 'send', 'setvar', 'size', 'slaves', 'tag_bind', 'tag_lower', 'tag_raise', 'tag_unbind', 'tk_bisque', 'tk_focusFollowsMouse', 'tk_focusNext', 'tk_focusPrev', 'tk_setPalette', 'tk_strictMotif', 'tkraise', 'type', 'unbind', 'unbind_all', 'unbind_class', 'update', 'update_idletasks', 'wait_variable', 'wait_visibility', 'wait_window', 'waitvar', 'winfo_atom', 'winfo_atomname', 'winfo_cells', 'winfo_children', 'winfo_class', 'winfo_colormapfull', 'winfo_containing', 'winfo_depth', 'winfo_exists', 'winfo_fpixels', 'winfo_geometry', 'winfo_height', 'winfo_id', 'winfo_interps', 'winfo_ismapped', 'winfo_manager', 'winfo_name', 'winfo_parent', 'winfo_pathname', 'winfo_pixels', 'winfo_pointerx', 'winfo_pointerxy', 'winfo_pointery', 'winfo_reqheight', 'winfo_reqwidth', 'winfo_rgb', 'winfo_rootx', 'winfo_rooty', 'winfo_screen', 'winfo_screencells', 'winfo_screendepth', 'winfo_screenheight', 'winfo_screenmmheight', 'winfo_screenmmwidth', 'winfo_screenvisual', 'winfo_screenwidth', 'winfo_server', 'winfo_toplevel', 'winfo_viewable', 'winfo_visual', 'winfo_visualid', 'winfo_visualsavailable', 'winfo_vrootheight', 'winfo_vrootwidth', 'winfo_vrootx', 'winfo_vrooty', 'winfo_width', 'winfo_x', 'winfo_y', 'xview', 'xview_moveto', 'xview_scroll', 'yview', 'yview_moveto', 'yview_scroll']

You can then call help on each of the names in the list to learn more about the name defined. The following are just two examples:

>>> help(Canvas.addtag)

Running help on the function addtag in module tkinter outputs the following:

  • addtag(self, *args)
  •     Internal function.

>>> help(Canvas.after)

Running help on the function after in module tkinter outputs the following:

  • after(self, ms, func=None, *args)
  •     Call function once after given time.
  •     MS specifies the time in milliseconds. FUNC gives the
  •     function, which shall be called. Additional parameters
  •     are given as parameters to the function call. Returns
  •     identifier to cancel scheduling with after_cancel.

>>> help(Canvas.create_image)

Running help on the function create_image in module tkinter outputs the following:

  • create_image(self, *args, **kw)
  •     Create image item with coordinates x1, y1.

The following coding example will draw a line on a Canvas:

import tkinter

from tkinter.constants import *

tk = tkinter.Tk()

canvas = tkinter.Canvas(tk, relief = RIDGE, borderwidth = 2)

canvas.pack(fill = BOTH, expand=1)

ln1 = canvas.create_line(100, 100, 300, 300, width = 6)

tk.mainloop()

A “tk” widget window with a straight red line that starts in the center-left area and slants down to the right to touch the bottom of the window.

Figure 8-3: Drawing on TK Canvas

Manipulate Images with Pillow

Another way you can work with visual objects in Python is to manipulate images stored in files. These manipulations include the following:

  • • rotating
  • • converting from colour to grey-scale
  • • applying colour filtering
  • • highlighting a specific area of an image
  • • blurring an image or part of it
  • • sharpening an image or part of it
  • • changing the brightness of an image
  • • detecting the edge on an image
  • • scaling an image
  • • applying colour inversion to an image
  • • morphing one image into another image

How can all these manipulations be done within your computer? First, an image is made of pixels, which can be stored in an m × n matrix, or two-dimensional array, mapped to a rectangular area of the computer screen. The value of each cell of the matrix represents a pixel and contains all the information about it. All manipulations to the image can be done by manipulating the matrix or its values.

To manipulate an image with Python, you can use a package called Pillow (available from https://pypi.org/project/Pillow/2.2.1/ or https://github.com/python-pillow/Pillow). Because it is not a standard part of the Python library, you will need to install it with the following statement:

  • pip install Pillow
  • Collecting Pillow
  • Downloading
  • wordhttps://files.pythonhosted.org/packages/70/21/04723e78916eff8e09901dbb7dc9705f4de8a0dfe7882a9ed56982bd128e/Pillow-6.0.0-cp37-cp37m-win32.whl (1.7MB)
  • |████████████████████████████████| 1.7MB 1.3MB/s
  • Installing collected packages: Pillow
  • Successfully installed Pillow-6.0.0
  • Once this is done, you can import and then use the following two modules:
  • Image,
  • ImageFilter

The following is a coding sample:

from PIL import Image, ImageFilter

# read the image

im = Image.open('./resort.jpg')

im_sharp = im.filter( ImageFilter.SHARPEN )

# save the filtered image to a new file

im_sharp.save( './resort-sharp.jpg', 'JPEG' )

# display the image

im_sharp.show()

The sharpened image is shown in Figure 8-4.

Note that the Image module has a class with the same name as the module, Image, although it is more convenient to construct an object of Image with the open statement rather than the constructor of Image class.

Once an image object has been generated with the open statement, we can check its format, size, and mode by looking at the format, size, and mode attributes and using the following methods of the Image class to manipulate the image object:

  • • Image.convert(self, mode = None, matrix = None, dither = None, palette = 0, colors = 256) makes various conversions to the image object.

A landscape with a blue sky and a slim beach area with palm trees in the background and green grass and plants with two palm trees in the foreground.

Figure 8-4: Picture sharpened with Pillow (©Harris Wang, Athabasca University)

  • • Image.copy(self) makes a copy and retains the original image object.
  • • Image.crop(self, box=None) returns a rectangular region of the image, defined by box.
  • • Image.draft(self, mode, size) returns a draft version of the image, such as a grey-scale version.
  • • Image.effect_spread(self, distance) returns an image with pixels randomly spread throughout the image.
  • • Image.filter(self, filter) filters this image using the given filter specified in the ImageFilter module.
  • • Image.paste(self, im, box=None, mask=None) pastes another image (im) into this image.
  • • Image.putalpha(self, alpha) adds or replaces the alpha layer in this image.
  • • Image.putdata(self, data, scale = 1.0, offset = 0.0) copies a sequence of pixel data to this image.
  • • Image.putpalette(self, data, rawmode = 'RGB') attaches a palette to this image.
  • • Image.putpixel(self, xy, value) modifies the pixel at the given position.
  • • Image.quantize(self, colors = 256, method = None, kmeans = 0, palette = None, dither = 1) converts the image to P mode with the specified number of colours.
  • • Image.emap_palette(self, dest_map, source_palette = None) rewrites the image to reorder the palette.
  • • Image.resize(self, size, resample = 0, box = None) returns a resized copy of this image.
  • • Image.rotate(self, angle, resample = 0, expand = 0, center = None, translate = None, fillcolor = None) returns a rotated copy of this image.
  • • Image.split(self), splits the image into individual bands, such as R, G, B.
  • • Image.tobitmap(self, name='image') converts the image to an X11 bitmap.
  • • Image.tobytes(self, encoder_name = 'raw', *args) returns the image as a bytes-object.
  • • Image.toqimage(self) returns a QImage copy of this image.
  • • Image.toqpixmap(self) returns a QPixmap copy of this image.
  • • Image.transform(self, size, method, data=None, resample = 0, fill = 1, fillcolor = None) transforms this image to a given size but in the same mode as the original.
  • • Image.transpose(self, method) transposes the image (flips or rotates in 90-degree steps).

There are other methods defined within the Image class for other purposes. You can find out more info about the Image class by running the following statement in Python interactive mode:

>>> from PIL import Image, ImageFilter

>>> help(Image.Image)

As we have seen from the above list, the Image class has provided a good set of methods to manipulate an image.

The ImageFilter module provides some filtering operations on images, as the name implies. These filtering operations include blurring, box blurring, contouring, colour transformation, detailing, edge enhancing, embossing, sharpening, smoothing, and more.

8.12 Modules for Data Analytics

The modules often used for data analytics include pandas, NumPy, SciPy, and matplotlib. Among the four modules, pandas is mostly used to prepare data; NumPy and SciPy are used for numeric analysis and science computing, respectively; and matplotlib is for visualization.

Since data analytics is a big topic, we will only give some examples of how the modules can be used.

The first example is to show how to use matplotlib to visualise the square root function in math.

import math

import matplotlib.pyplot as mpl

sqrt_data = {i+1 : math.sqrt(i+1) for i in range(100)}

x1 = list(sqrt_data.keys())

y1 = list(sqrt_data.values())

mpl.plot(x1, y1)

mpl.title("visualization of square root")

mpl.xlabel('x')

mpl.ylabel('y')

mpl.show()

The plot produced by the program is shown in Figure 8-5.

A graph where the x-axis goes to 100 and the y-axis goes to 10. A blue line starts at 0 and curves upward until it hits x-100, y-10.

Figure 8-5: Visualization of square root

Our next example is to visualize the world population changes in some regions as well as world total since 1960. The program code is as follows:

import pandas as pd

import matplotlib.pyplot as mplt

content = pd.read_excel("world-population.xls")

years = [1960, 1970, 1980, 1990, 2000, 2010, 2017]

mplt.plot(years, content.iloc[0, 1:])

mplt.plot(years, content.iloc[1, 1:])

mplt.plot(years, content.iloc[7, 1:])

mplt.plot(years, content.iloc[11, 1:])

mplt.plot(years, content.iloc[12, 1:])

mplt.plot(years, content.iloc[17, 1:])

mplt.plot(years, content.iloc[22, 1:])

mplt.title("Population - World Total and Region Total")

mplt.xlabel("years")

mplt.ylabel("Populations (in millions)")

mplt.legend(["World Total", "Africa", "Latin America", "North America", "Asia", "Europe", "Oceana"])

mplt.show()

In the program, the pandas module is used to read and prepare the data. For details on how it works, please read the complete documentation at https://pandas.pydata.org/docs/—the user guide at https://pandas.pydata.org/docs/user_guide/index.html#user-guide in particular.

The rendered result of the program is shown in Figure 8-6.

NumPy allows you to do math manipulations on a data set, most often manipulations on a matrix. Here is an example:

import numpy as np # import numpy

a = np.array(range(21,25)) # create an array from a python list

print(a, a.shape) # print the array and its shape

# create a 2-D array from nested lists

b = np.array([[1, 2, 3, 4], [5, 6, 7, 8],

        [11, 12, 13, 14], [15, 16, 17, 18]])

print(b, b.shape)  # print the array and its shape

A graph of the population growth in millions of different world regions and the world total from the year 1960 to 2010.

Figure 8-6: Visualization of world population changes

c = a + b # perform element-wise addition on two arrays

print(c, c.shape)  # print the result and the shape

d = np.dot(a, b)  # perform matrix multiplication

print(d)  # print the result

e = np.cos(a) # apply a mathematical function to an array

print(e)  # print the result

f = np.sort(b, axis=0) # sort an array along a given axis

print(f)  # print the result

The output from the program is as follows:

[21 22 23 24] (4,)

[[ 1 2 3 4]

 [ 5 6 7 8]

 [11 12 13 14]

 [15 16 17 18]] (4, 4)

[[22 24 26 28]

 [26 28 30 32]

 [32 34 36 38]

 [36 38 40 42]] (4, 4)

[ 744 834 924 1014]

[-0.54772926 -0.99996083 -0.53283302 0.42417901]

[[ 1 2 3 4]

 [ 5 6 7 8]

 [11 12 13 14]

 [15 16 17 18]]

More details on NumPy can be found at https://numpy.org/doc/stable/.

SciPy is built upon NumPy, including various functions often needed for scientific computing. The following example shows how integration function in SciPy is used to integrate a function.

import numpy as np # import numpy and scipy.integrate

from scipy.integrate import quad  # import quad integral function

# define a function to be integrated

def f(x):

  return np.exp(-x**2)  # use exp function from NumPy

# integrate the function from 0 to 1 using quad

result, _ = quad(f, 0, 1)

# print the result and the estimated error

print(result)

The result is 0.7468241328124271.

Chapter Summary

  • • In addition to functions and classes, modules and packages are important constructs that can be used to do modular programming in Python.
  • • A module can be any file with a py extension that has legitimate Python program codes or scripts in it.
  • • A package is a file directory containing several Python modules.
  • • To identify the directories in which the Python packages reside, each such directory must contain a file named __init__.py.
  • • The __init__.py file defines what can be imported from the package. These may include variables, functions, and classes.
  • • Packages can be nested to any depth, just like file directories are nested. However, each directory for a package must have its own __init__.py file.
  • • Dotted notation is used to refer to a package or module within a deep package hierarchy.
  • • To use variables, functions, and classes defined in a module, you have to import the module or the specific variables, functions, and/or classes, using the import statement or from … import statement.
  • • A large number of Python modules and packages have been developed and made available on the internet for Python programmers. Many of these modules and packages have been already installed with the standard Python distribution, such as the Anaconda package.
  • • A good Python programmer or software developer should have knowledge of existing modules and packages, including what they are developed for and what they do.
  • • Programmers can develop their own modules for their own applications and development.

Exercises

  1. 1. Open VS Code, create or open a Jupyter Notebook file (.ipynb), and select a Python virtual environment for the notebook file. Open a terminal and run the pip list command to see what library modules have been installed in the virtual environment.

    In the output of your VS Code shell terminal, identify some interesting packages and modules, and write a summary of those packages and modules in a Word document, including what each is developed for, where it may be used, and so on.

  2. 2. In VS Code, open the Jupyter Notebook file named chapter-8.ipynb. Create a new cell and import the math module, run the help command to study each of the functions defined in the math module, and do some hands-on coding with the function.
  3. 3. Search the internet for tutorials or other web documents related to web scraping with Python and choose some to watch or read. Take some coding samples to run in your Jupyter Notebook within VS Code, then develop your own application based on the code samples.

Projects

  1. 1. Rational numbers are those real numbers that can be represented as a quotient of two integers such as a/b, which can then be represented by the pair of integers a and b. For this project, define a module that contains the definition of a class named Rational, within which dunder methods for print, addition, subtraction, division, multiplication, and various comparisons are defined.
  2. 2. Develop an application using the math module to calculate and display a table of squares for integers from 0 to 99. The layout is illustrated in Table 8-6.
    Table 8-6: The layout of the table to be produced

    0

    1

    2

    3

    4

    5

    6

    7

    8

    9

    0

    0

    1

    4

    9

    26

    25

    36

    49

    8

    9

    1

    100

    121

    144

    169

    196

    225

    256

    289

    324

    361

    2

    …

    …

    …

    3

    4

    5

    6

    7

    8

    9

  3. 3. Develop an application to calculate and display a table showing the square roots of integers from 0 to 99 (similar to what you did for Project 2).

Annotate

Next Chapter
Chapter 9. Develop GUI-Based Applications
PreviousNext
Powered by Manifold Scholarship. Learn more at
Opens in new tab or windowmanifoldapp.org