Chapter 7 Object-Oriented Programming with Python

Abstraction

Abstraction is a very fundamental concept of object-oriented programming. The concept is rather simple. Because an object in the real word can be very complicated, containing many parts and with many attributes, it would be practical to consider only those parts and attributes that are relevant to the programming tasks at hand. This simplified model of a real-world object is an abstraction of it.

Information Hiding or Data Encapsulation

The concept of information hiding is very simple and straightforward. There are two reasons for hiding certain kinds of information: one is to protect the information, and the other is to make things easier and safer by hiding the details. An example of information hiding that you have already seen is defining and using functions. The code block of a function can be very lengthy and hard to understand. After a function is defined, however, a programmer only needs to know what the function does and how to use it, without considering the lengthy code block of how the function works.

In OOP, information hiding is further down within classes. Some OOP languages strictly limit direct access to class members (variables declared within a class), and all access must be done through the setter and getter methods. Python, however, has no such restriction, but you still need to remember the benefit of information hiding: avoiding direct access to the internal members of objects. Python also provides a way to hide, if you want to, by using double underscored names, such as __init__, __str__, and __repr__.

Inheritance

Inheritance is a very important concept of object-oriented programming, and inheriting is an important mechanism in problem solving and system development with the object-oriented approach. The underlying philosophy of inheritance is how programmers describe and understand the world. Most often, things are categorized and put in a tree-like hierarchy with the root on the top, as shown in Figure 7-1 below. In such a tree-like hierarchy, the root of the tree is the most generic class or concept, and the leaves are the most specific and often refer to specific objects. From the root down to the leaves, nodes on a lower level will inherit the properties of all the connected nodes at higher levels.

Within such a hierarchy, it is more convenient to have a model that captures general attributes shared by desktop, laptop, and tablet computers than to have models capturing the specifics of desktop, laptop, and tablet computers, respectively, with these specific models inheriting the common attributes captured by the generic model for computers. In such a hierarchy, the computer is the superclass of the desktop, laptop, and tablet, whereas the desktop, laptop, and tablet are a subclass of the computer.

Inheritance: Subclass and Superclass

If you want to inherit from other base classes when defining a new class in Python, you need to add all the base classes to the list of inheritances enclosed in a pair of parentheses, as shown below:

The new class will inherit all the attributes and methods from base_1, base_2, and base_3 and override the methods defined in the base classes.

In Python, all classes are a subclass of the built-in base object class and inherit all the properties and methods of object, even if object is not explicitly included in the inheritance list. So the following two statements will have the same effects:

As you can see, myClassA and myClassB both inherit from the base class object. However, in the list, some of the dunder names such as __le__ and __ge__ inherited from base class object are merely wrapper descriptors. If you want to make real use of these wrapper descriptors in your class, you will need to override them. Some inherited dunder methods can be used, but the values returned are not so useful. In particular, you will need to override the dunder methods __init__ __str__, and __repr__. The method __init__ is a dunder method used as constructor called internally by PVM whenever a new instance of a class needs to be created. The method __str__ is a dunder method called internally whenever an object of a class needs to be converted to a string, such as for printout. Finally, __repr__ is a dunder method called internally when an object needs to be converted to a representation for serialization—a process that converts a Python object into a byte stream that can be stored or transmitted.

Public, Private, and Protected Members of a Class

People familiar with other OO programming languages such as C++ and Java may be wondering how to declare and use public, private, and protected attributes in Python, as they are used to doing in other OO programming languages. Python, however, doesn’t differentiate attributes between public, private, and protected members. Instead, Python treats all attributes of a class as public. It is up to the programmers to decide whether a member should be public, private, or protected. To ensure that everyone reading the code understands the intention of the original coder/programmer, Python has a convention for naming protected members, private members, and public members: a name with a leading underscore _ is a protected member of the class in which it is defined, a name with a double underscore __ is a private member of the class in which it is defined, and all other names will be public.

According to the common principles of object-oriented programming, public members of a class can be seen and accessed from outside of the class or the instance of the class; protected members of a class can only be seen and accessible within the class or a subclass of the class; private members can only be accessed within the class. In Python, however, the rule for protected members is not strictly enforced. As such, the following code will not raise any error or exception.

In our example about shapes, _width and _length are protected attributes of the class, which should only be accessible within the class or its subclasses and should be hidden from the outside.

In contrast, the rule on private members, those with names prefixed with a double underscore __, is strictly enforced in Python. So if we change firstname and lastname to private members, it will raise exceptions, as shown in the following example:

If you want to access the private members of a class from outside, the built-in functions setattr and getattr can be used to access both private and protected members of a class or its instance, as shown below:

Class Methods

As we have seen, in Python, special method __init__ is used as a constructor of the class. People familiar with C++ or Java might be wondering if __init__ can be overloaded or defined multiple times in Python to make multiple constructors, because a class in C++ and Java can have two or more constructors with different signature. In Python, however, a class cannot have more than one __init__ effectively defined. If you define two or more __init__ within a class definition, only the last one will take effect.

So how can you create an instance of a class differently in Python if there can be only one constructor, the special __init__ method, in a class definition? The solution is to use the class method, as shown in the next example:

In the example above, decorator @classmethod is used to declare that method newGraduate() is a class method, which means that the first parameter refers to the class instead of an instance of the class, and the method is bound to the class and can be called through the class. A class method can be used to directly modify the structure of the class. For example, we can have a class method for Graduate class that sets the value of the class attribute at the beginning of a new year, say, 20210101. In this particular example above, however, class method newGraduate() is used as an alternate constructor of class Graduate, which takes first name and last name separately to instantiate an instance of Graduate.

When defining a class method, it is a convention to use cls as the name of the first parameter to refer to the class. Technically, however, it can be anything unique in the scope of a class definition, as long as you know it refers to the class. This is similar to “self” in definitions of regular methods, which is only the conventional name referring to the instance itself being instantiated.

Static Methods

Similar to the class method of a class, a static method can be called directly through the class. The difference is that in the definition of a static method, no parameter refers to the class nor to the instance itself. Its usefulness may be demonstrated by the example below, where we define a class called Convertor, and within the class definition, we define a number of static methods, each of which convert values from one unit to another:

As you can see from the above example, the static methods can be called directly through the class Convertor, without an instance. Defining a static method within a class is a way to add utility functions to the class so that it can be used without instantiating an object.

Class Attributes

As previously mentioned, in Python, you can define a class without explicitly declaring attributes of the class. However, Python does allow the explicit declaration of attributes within a class definition. These attributes are called class attributes. Explicit declaration of an attribute within a class definition means that the attribute is declared outside of the __init__ method. In the following example, student_id is declared as a class attribute.

From this example, you can see how a class attribute is used and shared among all the instances of the class. You can create student_id as a class attribute to dynamically track the student IDs allocated to new students—a new instance of the class Graduate. It starts from 20201195 and increases by one every time a new student is enrolled.

Note the dunder name __class__ used in the example. In Python, __class__ refers to the class of an object. So in the example above, self.__class__ refers to the class of object self—that is, Graduate. If you do not have __class__ between self and student_id but simply use self.student_id, student_id would become an attribute of each instance of the class, different from the class attribute, as demonstrated in the following example:

As you can see from the example above, although value 20201195 of class attribute student_id is used to calculate the student_id (20201195 + 1) when instantiating each instance of the class, the result 20201196 has no effect on the class attribute whose value remains to be 20201195. How did that work? Remember that assignment self.student_id += 1 is short for self.student_id = self.student_id. In this example, self.student_id on the right side of the assignment statement is resolved to be the class attribute according to the rule, whereas self.student_id on the left side of the assignment statement is an attribute of the object being instantiated. That is, the two self.student_id are two different variables.

Class attributes have different behaviours from static attributes in C++ or Java, although they do have certain things in common. In Python, class attributes are shared among all the objects of the class and can also be accessed directly from the class, without an instance of the class, as shown above.

Dunder Methods in Class Definition

Python has a list of names reserved for some special methods called dunder methods or magic methods. These special names have two prefix and suffix underscores in the name. The word dunder is short for “double under (underscores).” Most of these dunder methods are used for operator overloading. Examples of the dunder/magic methods that can be used for overloading operators are __add__ for addition, __sub__ for subtraction, __mul__ for multiplication, and __div__ for division.

Some dunder methods have specific meanings and effects when overloaded (defined in a user-defined class). These dunder methods include __init__, __new__, __call__, __len__, __repr__, __str__, and __getitem__.

In the previous section you saw how __init__ method is used as a constructor of class. The following sheds some light on the others.

__call__

Dunder method __call__ can be used to make an object, an instance of class, callable, like a function. In the following example, a class is defined as Home, which has two attributes: size and value. In addition to __init__ as a constructor, a dunder function __call__ is defined, which turns object h1, an instance of class Home, into a callable function.

What can be done through dunder method __call__ can also be done through a normal method, say checking. The obvious benefit of using the dunder method __call__ is cleaner and neater code.

__new__

Dunder method __new__ is a method already defined in the object base class. It can be overridden to do something extra before the creation of a new instance of a class. Dunder method __ new__ itself doesn’t create or initialize an object for a class. Instead, it can check, for example, if a new object can be created. If the answer is yes, then it will call __init__ to do the actual work of creation and initialization, as shown in the following example:

The example above makes a travel planning system by making a list of places to be visited. The __new__ method is defined to control the addition of a new city to the list. It checks whether the city has already been added to the plan and will not add the city if it is already on the list.

Note that we also defined and used two protected class attributes in the definition of class TravelPlan. One is a dictionary storing the places in the plan and their order. We also used a static method so we can print out the entire plan when it is needed. Because the class attributes are shared among the instances of the class, changes to the class attributes will be retained, and at any time before the application is stopped, one can use the static method to print out the plan.

__str__

Dunder method __str__can be used to implement the string representation of objects for a class so that it can be printed out by using the print statement. The method will be invoked when str() function is called to convert an object of the class to a string. In our definition of class Graduate in our earlier example, the __str__ has been implemented to just return the full name of a student. You may, of course, choose to return whatever string you think would better represent the object for a given purpose.

__len__

Dunder method __len__ can be used to return the length of an object for a class. It is invoked when function len(o) is called on object o of the class. It is up to the programmer to decide how to measure the length, though. In our definition of class Graduate, we simply use the sum of the first name, last name, and id length.

__repr__

Dunder method __repr__ can be used to return the object representation of a class instance so that, for example, the object can be saved to and retrieved from a file or database. An object representation can be in the form of a list, tuple, or dictionary, but it has to be returned as a string in order to be written to and read from a file or database. The __repr__ method will be invoked when function repr() is called on an object of the class. The following example extended from the definition of class Graduate shows how dunder methods __str__, __len__, and __repr__ can be defined and used.

__getitem__ and __setitem__

These two methods are used to turn a class into an indexable container object. __getitem__ is called to implement the evaluation of self[key] whereas __setitem__ is called to implement the assignment to self[key].

__delitem__

This is called to implement the deletion of self[key].

__missing__

This is called by dict.__getitem__() to implement self[key] for dict subclasses when the key is not in the dictionary.

__iter__

This is called when an iterator is required for a container.

__reversed__

This can be implemented and called by the reversed() built-in function to implement a reverse iteration.

__contain__

This is called to implement membership test operators. It should return True if the item is in self and False if it is not.

__delete__

This is called to delete the attribute on an instance of the owner class.

Tables 7-4, 7-5, 7-6, and 7-7 show a comprehensive list of dunder methods and their respective operators that can be overridden by programmers when defining new classes.

Using Class as Decorator

In Chapter 6, we saw how decorators can be used as powerful and useful tools that wrap one function with another to change the behaviour of the wrapped function without modifying the wrapped function. In this section, we show how to use classes as decorators to modify the behaviour of an existing function.

As we know, a class has two types of members. One is attributes and the other is methods. Our first example uses the methods of a class as a decorator, as shown below:

As shown in the example above, the __init__ method is used to pass a function to be decorated or wrapped to the class, and the __call__ method is used to actually wrap the real function passed to the class.

We learned in Chapter 6 how to calculate the actual execution time of a program using a function as a decorator. The same can be done with a class as a decorator when we time finding all the primes within a given range using an algorithm called “sieve of Eratosthenes.”

In the above examples, we used class to decorate a function. Within the class, a special dunder function __call__ is used to execute the decorated function and calculate the time spent on the execution of the function that finds all the prime numbers between 2 and 32768 (2 ** 15).

Built-In Property() Function and Property Decorator

Python has a special built-in function named property, as mentioned in Chapter 2. It can be called directly to create an attribute of a class with potentially added setter, getter, and deleter methods and documentation. In this case, the function can either be called within a class definition or outside a class definition while adding the created property to a class or object of a class.

A call of the property function may take none or any combination of the four optional keyword arguments, as shown below:

If property() function is called with no argument, you can add a setter function, getter function, and/or deleter function later by calling the setter, getter. and deleter methods.

The following example demonstrates how the built-in property function is used to add a property/attribute to a class:

A built-in property function can also be used as a decorator within a class definition to make a method name to be used as an attribute/property of the class.

Similar to the example above, suppose we want to define a class named Student for a management system at a university. The attributes should include the first name, last name, full name, email address, and some other information. We know that a full name is made of a first name and a last name and that the university often has a rule of assigning an email address based on the first name and last name. If we define full name and email address as ordinary attributes, the dependencies would not be reflected because a change to the first name or last name of a student will not automatically result in the change to the full name and email address. Using property() function as a decorator can nicely solve the problem, as shown in the following example:

In the example above, we use built-in property() function as a decorator to decorate method fullname() and emailaddress(). By doing that, the function name can be used as a property or attribute of the object, but access to the attribute invokes a call to the function so that an updated full name or email address is retrieved. In the example, when the last name is changed to Smith, the full name is automatically changed to John Smith, and the email address is automatically changed to John.Smith@globalemail.com.

Using decorator, we can also further add setter, getter, and deleter methods to the property fullname, as shown below:

Note that using a property decorator can make a method to be used like an attribute/property without the need to allocate memory space to keep the value of the attribute. As such, you cannot assign a value directly to such an attribute.

Creating a New Class Dynamically and Modify a Defined Class or Instance

In Python, you have the freedom to make changes to an already defined class or an instance of the class, and you can even create a new class dynamically.

To create a new class dynamically in your Python program, a built-in function type is used, as shown below:

In the example above, we dynamically create a new class called Shape and set one attribute called points, which is a list of points initially assigned one point (1, 1). We then create two instances of the Shape class, s0 and s1, in the same way that we do with other classes defined with the class statement. We then even added one more point to s0. Because attributes added to the class with built-in function type() are automatically treated as class attributes and shared by all instances of the class, changes to points of s0 are also reflected in the value of points of s1. However, the third-to-last statement above adds an attribute with the same name as the class attribute to object s1, which only belongs to s1. As a result, a change to this attribute of s1 has no effect on s0.

As we have already seen above, with a defined class or an instance of a defined class, you can modify the class or instance by adding or deleting attributes to/from the class or instance. Moreover, you can even add or delete a new method dynamically to or from a class or instance of a class.

One way to add an attribute to an object, either a class or an instance of a class, is to directly name a new attribute and assign a value in the same way that we introduce a new variable to a program. For example, we can add a new attribute called shape_type to s0 instantiated above with the first statement of code below:

The second statement has proved that a new attribute shape_type has been added to object s0, and now this particular instance of Shape is called line.

A new attribute can also be added by using the built-in function setattr(). To add the shape_type attribute to s0 with setattr(), run the following statement:

An attribute can also be deleted from an object using the built-in function delattr(). When you want to add or delete an attribute, but you are not sure if the attribute exits, you can use built-in function hasattr() to check, as shown below:

Remember, attributes added to an instance of a class will not be seen by other instances of the same class. If we want to make the shape_type attribute visible to all instances of the Shape class because every shape should have a shape type, we need to add the shape_type attribute to the class to make it a class attribute. This is done with the following statement:

From now on, all instances of the Shape class will have an attribute called shape_type, as shown in the examples below:

As you may have noted, the attribute shape_type and its value, added to the Shape class, have been propagated to both s0 and s1 because the shape_type attribute was added as a class attribute. By comparison, the attribute later added to an individual instance of the class is the attribute of that instance only. This is shown in the following code sample:

The example shows that the new attribute weight was only added to object s0, and s1 does not have the attribute. Again, if you want the weight attribute and its value to be shared by all instances of the class, you have to add the attribute to the class directly, as shown below:

How can you add a new method to an already defined class? You can do this in almost the same way as you would add new attributes to a class or instance of a class, as shown in the following example:

This provides programmers with so much power, as they can create new classes and modify all aspects of classes dynamically. For example, we know we can only define one __init__() in a class definition, so we only have one constructor to use when creating new instances of the class. By attaching a different properly defined method to the __init__ attribute, we are now able to construct a new instance of a class in whatever way we want, as shown in the following example:

In this example, we defined three methods to use as constructors or initiators for the shape class we previously defined. Constructor c1 is for creating point objects, c2 is for creating line objects, and c3 is for creating triangle objects. We then attach each method to the __init__ attribute of the shape class to create the shape object we want. We also defined a method called print_shape() for the class, just to show the results of the three different constructors.

As you may imagine, however, the consequence of modifying instances of a class is that different instances of the same class may have totally different attributes. In an extreme case, two instances of the same class can have totally different attributes. This is something you need to keep in mind when enjoying the freedom offered by Python.

Keeping Objects in Permanent Storage

As mentioned previously, classes and objects are a very powerful way to model the world. Hence, in a program, classes and objects can be used to represent information and knowledge for real-world application. You do not want to lose that information and knowledge whenever you shut down the computer. Instead, you want to keep this information and knowledge in permanent storage and reuse it when you need it. For example, you may have developed a management system using the Student class, defined earlier in this section, and created a number of objects of the Student class containing information about these students. You need to reuse the information about these students contained in those student objects next time you turn on the computer and run the system. How can you do that?

Previously, we discussed defining the __repr__ dunder method, which returns a string representation of an object that can be in the form of list, tuple, or dictionary, as shown in the following example:

With the __repr__() method for a class, you can save the representation of these objects into a file. To use the object representations stored in a file, you will need to write another function/method to pick up the information from each object representation and add it to its respective object. The two processes must work together to ensure that objects can be correctly restored from files. Neither process is easy. Fortunately, there is a Python module called pickle already developed just for this purpose.

In formal terms, saving data, especially complex data such as objects in OOP, in permanent storage is called serialization, whereas restoring data from permanent storage back to its original form in programs is called deserialization. The pickle module is a library implemented for serializing and deserializing objects in Python. The pickling/serializing process converts objects with hierarchical structure into a byte stream ready to save on a binary file, send across a network such as the internet, or store in a database, whereas the unpickling/deserializing process does the reverse: it converts a byte stream from a binary file or database, or received from a network, back into the object hierarchy.

There are some good sections and documents on the internet that explain how to use the pickle module for your Python programming needs. When you do need to use it, be aware of the following:

1. 1. Unpickling with the pickle module is not secure. Unpickling objects from unknown and untrusted sources can be dangerous because harmful executable code may be deserialized into your computer memory.
2. 2. Not all objects can be pickled. You need to know what data types and objects can be pickled before you pickle them. Pickling unpicklable objects will raise exception.