Chapter 14. Exceptions – The Quick Python Book

Chapter 14. Exceptions

This chapter covers

  • Understanding exceptions
  • Handling exceptions in Python
  • Using the with keyword

This chapter discusses exceptions, which are a language feature specifically aimed at handling unusual circumstances during the execution of a program. The most common use for exceptions is to handle errors that arise during the execution of a program, but they can also be used effectively for many other purposes. Python provides a comprehensive set of exceptions, and new ones can be defined by users for their own purposes.

The concept of exceptions as an error-handling mechanism has been around for some time. C and Perl, the most commonly used systems and scripting languages, don’t provide any exception capabilities, and even programmers who use languages such as C++, which do include exceptions, are often unfamiliar with them. This chapter doesn’t assume familiarity with exceptions on your part but instead provides detailed explanations. If you’re already familiar with exceptions, you can skip directly to “Exceptions in Python” (section 14.2).

14.1. Introduction to exceptions

The following sections provide an introduction to exceptions and how they’re used. Feel free to skip them if you’re already familiar with exceptions from other languages.

14.1.1. General philosophy of errors and exception handling

Any program may encounter errors during its execution. For the purposes of illustrating exceptions, we’ll look at the case of a word processor that writes files to disk and that therefore may run out of disk space before all of its data is written. There are various ways of coming to grips with this problem.

Solution 1: don’t handle the problem

The simplest way of handling this disk-space problem is to assume that there will always be adequate disk space for whatever files we write, and we needn’t worry about it. Unfortunately, this seems to be the most commonly used option. It’s usually tolerable for small programs dealing with small amounts of data, but it’s completely unsatisfactory for more mission-critical programs. For several months while I was writing this book, my officemate spent hours every day cleaning up files that had become corrupt when a program written by someone else ran out of disk space and crashed. Eventually, he went into the code and put in some checks, which took care of most of the problem; even now, he still has to do occasional disk cleanups. Because the original program wasn’t written cleanly with exceptions, the checks he put in could do only a partial job.

Solution 2: all functions return success/failure status

The next level of sophistication in error handling is to realize that errors will occur and to define a methodology using standard language mechanisms for detecting and handling them. There are various ways of doing this, but a typical one is to have each function or procedure return a status value that indicates if that function or procedure call executed successfully. Normal results can be passed back in a call-by-reference parameter.

Let’s look at how this might work with our hypothetical word-processing program. We’ll assume that the program invokes a single high-level function, save_to_file, to save the current document to file. This will call various subfunctions to save different parts of the entire document to the file: for example, save_text_to_file to save the actual document text, save_prefs_to_file to save user preferences for that document, save_formats_to_file to save user-defined formats for the document, and so forth. Any of these may in turn call their own subfunctions, which save smaller pieces to the file. At the bottom will be built-in system functions, which write primitive data to the file and report on the success or failure of the file-writing operations.

We could put error-handling code into every function that might get a disk-space error, but that makes little sense. The only thing the error handler will be able to do is to put up a dialog box telling the user that there’s no more disk space and asking that the user remove some files and save again. It wouldn’t make sense to duplicate this code everywhere we do a disk write. Instead, we’ll put one piece of error-handling code into the main disk-writing function, save_to_file.

Unfortunately, for save_to_file to be able to determine when to call this error-handling code, every function it calls that writes to disk must itself check for disk space errors and return a status value indicating success or failure of the disk write. In addition, the save_to_file function must explicitly check every call to a function that writes to disk, even though it doesn’t care about which function fails. The code, using a C-like syntax, looks something like this:

const ERROR = 1;
const OK = 0;
int save_to_file(filename) {
int status;
status = save_prefs_to_file(filename);
if (status == ERROR) {
...handle the error...
status = save_text_to_file(filename);
if (status == ERROR) {
...handle the error...
status = save_formats_to_file(filename);
if (status == ERROR) {
...handle the error...
int save_text_to_file(filename) {
int status;
status = ...lower-level call to write size of text...
if (status == ERROR) {
status = ...lower-level call to write actual text data...
if (status == ERROR) {

And so on for save_prefs_to_file, save_formats_to_file, and all other functions that either write to filename directly or (in any way) call functions that write to filename.

Under this methodology, code to detect and handle errors can become a significant portion of the entire program, because every function and procedure containing calls that might result in an error need to contain code to check for an error. Often, programmers don’t have the time or the energy put in this type of complete error checking, and programs end up being unreliable and crash prone.

Solution 3: the exception mechanism

It’s obvious that most of the error-checking code in the previous type of program is largely repetitive: it checks for errors on each attempted file write and passes an error status message back up to the calling procedure if an error is detected. The disk space error is handled in only one place, the top-level save_to_file. In other words, most of the error-handling code is plumbing code, which connects the place where an error is generated with the place where it’s handled. What we really want to do is to get rid of this plumbing and write code that looks something like this:

def save_to_file(filename)
try to execute the following block
except that, if the disk runs out of space while
executing the above block, do this
...handle the error...

def save_text_to_file(filename)
...lower-level call to write size of text...
...lower-level call to write actual text data...

The error-handling code is completely removed from the lower-level functions; an error (if it occurs) will be generated by the built-in file writing routines and will propagate directly to the save_to_file routine, where our error-handling code will (presumably) take care of it. Although youcan’t write this code in C, languages that offer exceptions permit exactly this sort of behavior; and, of course, Python is one such language. Exceptions let you write clearer code and handle error conditions better.

14.1.2. A more formal definition of exceptions

The act of generating an exception is called raising or throwing an exception. In the previous example, all exceptions are raised by the disk-writing functions, but exceptions can also be raised by any other functions or can be explicitly raised by your own code. We’ll discuss this in more detail shortly. In the previous example, the low-level disk-writing functions (not seen in the code) would throw an exception if the disk were to run out of space.

The act of responding to an exception is called catching an exception, and the code that handles an exception is called exception-handling code, or just an exception handler. In the example, the except that... line catches the disk-write exception, and the code that would be in place of the ...handle the error... line would be an exception handler for disk-write (out of space) exceptions. There may be other exception handlers for other types of exceptions or even other exception handlers for the same type of exception but at another place in your code.

14.1.3. User-defined exceptions

Depending on exactly what event causes an exception, a program may need to take different actions. For example, an exception raised when disk space is exhausted needs to be handled quite differently from an exception that is raised if we run out of memory, and both are completely different from an exception that arises when a divide-by-zero error occurs. One way of handling these different types of exceptions would be to globally record an error message indicating the cause of the exception and to have all exception handlers examine this error message and take appropriate action. In practice, a different method has proven to be much more flexible.

Rather than defining a single kind of exception, Python, like most modern languages that implement exceptions, defines different types of exceptions, corresponding to various problems that may occur. Depending on the underlying event, different types of exceptions may be raised. In addition, the code that catches exceptions may be told to catch only certain types. This feature was used in the earlier pseudocode when we said, except that, if the disk runs out of space . . ., do this; we were specifying that this particular exception-handling code is interested only in disk-space exceptions. Another type of exception wouldn’t be caught by that exception-handling code. It would either be caught by an exception handler that was looking for numeric exceptions, or, if there were no such exception handler, it would cause the program to exit prematurely with an error.

14.2. Exceptions in Python

The remaining sections of this chapter talk specifically about the exception mechanisms built into Python. The entire Python exception mechanism is built around an object-oriented paradigm, which makes it both flexible and expandable. If you aren’t familiar with OOP, you don’t need to learn OO techniques in order to use exceptions.

Like everything else in Python, an exception is an object. It’s generated automatically by Python functions with a raise statement. After it’s generated, the raise statement, which raises an exception, causes execution of the Python program to proceed in a manner different than would normally occur. Instead of proceeding with the next statement after the raise, or whatever generated the exception, the current call chain is searched for a handler that can handle the generated exception. If such a handler is found, it’s invoked and may access the exception object for more information. If no suitable exception handler is found, the program aborts with an error message.

14.2.1. Types of Python exceptions

It’s possible to generate different types of exceptions to reflect the actual cause of the error or exceptional circumstance being reported. Python provides a number of different exception types:

WindowsError (Windows)
VMSError (VMS)


The Python exception set is hierarchically structured, as reflected by the indentation in this list of exceptions. As you saw in a previous chapter, you can obtain an alphabetized list from the __builtin__ module.

Each type of exception is a Python class, which inherits from its parent exception type. But if you’re not into OOP yet, don’t worry about that. For example, an IndexError is also a LookupError and by inheritance an Exception and also a BaseException.

This hierarchy is deliberate: most exceptions inherit from Exception, and it’s strongly recommended that any user-defined exceptions also subclass Exception, not BaseException. The reason is that if you have code set up like this

# do stuff
except Exception:
# handle exceptions

you could still interrupt the code in the try block with Ctrl-C without triggering the exception-handling code, because the KeyboardInterrupt exception is not a subclass of Exception.

You can find an explanation of the meaning of each type of exception in the documentation, but you’ll rapidly become acquainted with the most common types as you program!

14.2.2. Raising exceptions

Exceptions are raised by many of the Python built-in functions. For example:

>>> alist = [1, 2, 3]
>>> element = alist[7]
Traceback (innermost last):
File "<stdin>", line 1, in ?
IndexError: list index out of range

Error-checking code built into Python detects that the second input line requests an element at a list index that doesn’t exist and raises an IndexError exception. This exception propagates all the way back to the top level (the interactive Python interpreter), which handles it by printing out a message stating that the exception has occurred.

Exceptions may also be raised explicitly in your own code, through the use of the raise statement. The most basic form of this statement is

raise exception(args)

The exception(args) part of the code creates an exception. The arguments to the new exception are typically values that aid you in determining what happened, something we’ll discuss shortly. After the exception has been created, raise takes it and throws it upward along the stack of Python functions that were invoked in getting to the line containing the raise statement. The new exception is thrown up to the nearest (on the stack) exception catcher looking for that type of exception. If no catcher is found on the way to the top level of the program, this will either cause the program to terminate with an error or, in an interactive session, cause an error message to be printed to the console.

Try the following:

>>> raise IndexError("Just kidding")
Traceback (innermost last):
File "<stdin>", line 1, in ?
IndexError: Just kidding

The use of raise here generates what at first glance looks similar to all the Python list-index error messages you’ve seen so far. Closer inspection reveals this not to be the case. The actual error reported isn’t as serious as those other ones.

The use of a string argument when creating exceptions is common. Most of the built-in Python exceptions, if given a first argument, assume it’s a message to be shown to you as an explanation of what happened. This isn’t always the case, though, because each exception type is its own class, and the arguments expected when a new exception of that class is created are determined entirely by the class definition. Also, programmer-defined exceptions, created by you or by other programmers, are often used for reasons other than error handling and, as such, may not take a text message.

14.2.3. Catching and handling exceptions

The important thing about exceptions isn’t that they cause a program to halt with an error message. Achieving that in a program is never much of a problem. What’s special about exceptions is that they don’t have to cause the program to halt. By defining appropriate exception handlers, you can ensure that commonly encountered exceptional circumstances don’t cause the program to fail; perhaps they display an error message to the user or do something else, even fix the problem, but they don’t crash the program.

The basic Python syntax for exception catching and handling is as follows, using the try and except and sometimes the else keywords:

except exception_type1 as var1:
except exception_type2 as var2:

A try statement is executed by first executing the code in the body part of the statement. If this is successful (that is, no exceptions are thrown to be caught by the try statement), then the else_body is executed and the try statement is finished. Nothing else occurs. If an exception is thrown to the try, then the except clauses are searched sequentially for one whose associated exception type matches that which was thrown. If a matching except clause is found, the thrown exception is assigned to the variable named after the associated exception type, and the exception code body associated with the matching exception is executed. If the line except exception_type, var: matches some thrown expression exc, the variable var will be created, and exc will be assigned as the value of var, before the exception-handling code of the except statement is executed. You don’t need to put in var; you can say something like except exception_type:, which will still catch exceptions of the given type but won’t assign them to any variable.

If no matching except clause is found, then the thrown exception can’t be handled by that try statement, and the exception is thrown further up the call chain in hope that some enclosing try will be able to handle it.

The last except clause of a try statement can optionally refer to no exception types at all, in which case it will handle all types of exceptions. This can be convenient for some debugging and extremely rapid prototyping but generally isn’t a good idea: all errors are hidden by the except clause, which can lead to some confusing behavior on the part of your program.

The else clause of a try statement is optional and is rarely used. It’s executed if and only if the body of the try statement executes without throwing any errors.

The finally clause of a try statement is also optional and executes after the try, except, and else sections have executed. If an exception is raised in the try block and isn’t handled by any of the except blocks, that exception is reraised after the finally block executes. Because the finally block always executes, it gives you a chance to include code to clean up after any exception handling by closing files, resetting variables, and so on.

14.2.4. Defining new exceptions

You can easily define your own exception. The following two lines will do this for you:

class MyError(Exception):

This creates a class that inherits everything from the base Exception class. But you don’t have to worry about that if you don’t want to.

You can raise, catch, and handle it like any other exception. If you give it a single argument (and you don’t catch and handle it), this will be printed at the end of the traceback:

>>> raise MyError("Some information about what went wrong")
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
__main__.MyError: Some information about what went wrong

This argument will, of course, be available to a handler you write as well:

raise MyError("Some information about what went wrong")
except MyError as error:
print("Situation:", error)

The result here is

Situation: Some information about what went wrong

If you raise your exception with multiple arguments, these will be delivered to your handler as a tuple, which you can access through the args variable of the error:

raise MyError("Some information", "my_filename", 3)
except MyError as error:
print("Situation: problem {0} with file {1}: {2}".format(
error.args[1], error.args[0]))

This gives the result

Situation: problem 3 with file my_filename: Some information

Because an exception type is a regular class in Python and happens to inherit from the root Exception class, it’s a simple matter to create you own subhierarchy of exception types for use by your own code. You don’t have to worry about this on a first read of the book. You can always come back to it after you’ve read chapter 15, “Classes and object-oriented programming.” Exactly how you create your own exceptions depends on your particular needs. If you’re writing a small program that may generate only a few unique errors or exceptions, subclass the main Exception class as we’ve done here. If, on the other hand, you’re writing a large, multifile code library with a special goal in mind—say, weather forecasting—you may decide to define a unique class called WeatherLibraryException and then define all the unique exceptions of the library as subclasses of WeatherLibraryException.

14.2.5. Debugging programs with the assert statement

The assert statement is a specialized form of the raise statement:

assert expression, argument

The AssertionError exception with the optional argument is raised if the expression evaluates to False and the system variable __debug__ is True. The __debug__ variable defaults to True. It’s turned off by either starting up the Python interpreter with the -O or -OO option or by setting the system variable PYTHONOPTIMIZE to True.

The code generator creates no code for assertion statements if __debug__ is false. You can use assert statements to instrument your code with debug statements during development and leave them in the code for possible future use with no runtime cost during regular use:

>>> x = (1, 2, 3)
>>> assert len(x) > 5
Traceback (most recent call last):
File "<stdin>", line 1, in <module>

14.2.6. The exception inheritance hierarchy

Now, let’s expand on an earlier notion that Python exceptions are hierarchically structured and on what it means in terms of how except clauses catch exceptions.

The following code

except LookupError as error:
exception code
except IndexError as error:
exception code

catches two different types of exceptions: IndexError and LookupError. It just so happens that IndexError is a subclass of LookupError. If body throws an IndexError, that error is first examined by the except LookupError as error: line, and because an IndexError is a LookupError by inheritance, the first except will succeed. The second except clause will never be used because it’s subsumed by the first except clause.

On the other hand, flipping the order of the two except clauses could potentially be useful; the first clause would then handle IndexError exceptions, and the second clause would handle any LookupError exceptions that aren’t IndexError errors.

14.2.7. Example: a disk-writing program in Python

Let’s revisit our example of a word-processing program that needs to check for disk out-of-space conditions as it writes a document to disk:

def save_to_file(filename) :
except IOError:
...handle the error...
def save_text_to_file(filename):
...lower-level call to write size of text...
...lower-level call to write actual text data...

Notice how unobtrusive the error-handling code is; it’s wrapped around the main sequence of disk-writing calls in the save_to_file function. None of the subsidiary disk-writing functions need any error-handling code. It would be easy to develop the program first and to add error-handling code later. That’s often what is done, although this isn’t the optimal ordering of events.

As another note of interest, this code doesn’t respond specifically to disk-full errors; rather, it responds to IOError exceptions, which Python built-in functions raise automatically whenever they can’t complete an I/O request, for whatever reason. That’s probably satisfactory for your needs; but if you need to identify disk-full conditions, you can do a couple of different things. The except body can check to see how much room is available on disk. If the disk is out of space, clearly it’s a disk-full problem and should be handled in this except body; otherwise, the code in the except body can throw the IOError further up the call chain to be handled by some other except. If that isn’t sufficient, you can do something more extreme, like going into the C source for the Python disk-writing functions and raising your own DiskFull exceptions as necessary. I wouldn’t recommend this latter option, but it’s nice to know the possibility exists if you need to make use of it.

14.2.8. Example: exceptions in normal evaluation

Exceptions are most often used in error handling but can also be remarkably useful in certain situations involving what we would think of as normal evaluation. An example I encountered involved a spreadsheet-like program I was implementing. Like most spreadsheets, it permits arithmetic operations involving cells, and it permits cells to contain values other than numbers. For my application, I wanted blank cells used in a numerical calculation to be considered as containing the value 0, and cells containing any other nonnumeric string to be considered as invalid, which I represented as the Python value None. Any calculation involving an invalid value should return an invalid value.

The first step was to write a function that would evaluate a string from a cell of the spreadsheet and return an appropriate value:

def cell_value(string):
return float(string)
except ValueError:
if string == "":
return 0
return None

Python’s exception-handling ability made this a simple function to write. I tried to convert the string from the cell into a number and return it, in a try block using the float built-in function. float raises the ValueError exception if it can’t convert its string argument to a number, so I caught that and returned either 0 or None depending on whether the argument string was empty or non-empty.

The next step was to handle the fact that some of my arithmetic might have to deal with a value of None. In a language without exceptions, the normal way to do this would be to define a custom set of arithmetic functions, which check their arguments for None, and then to use those functions rather than the built-in arithmetic functions to perform all of the spreadsheet arithmetic. This is time consuming and error prone, and it leads to slow execution because you’re effectively building an interpreter in your spreadsheet. I took a different approach. All of my spreadsheet formulas were actually Python functions that took as arguments the x and y coordinates of the cell being evaluated and the spreadsheet itself and calculated the result for the given cell using standard Python arithmetic operators, using cell_value to extract the necessary values from the spreadsheet. I defined a function called safe_apply, which took one of these formulas, applied it to the appropriate arguments in a try block, and returned either the formula’s result or None, depending on whether the formula evaluated successfully:

def safe_apply(function, x, y, spreadsheet):
return function, (x, y, spreadsheet)
except TypeError:
return None

These two changes were enough to integrate the idea of an empty (None) value into the semantics of my spreadsheet. Trying to develop this ability without the use of exceptions is a highly educational exercise.

14.2.9. Where to use exceptions

Exceptions are a natural choice for handling almost any error condition. It’s an unfortunate fact that error handling is often added after the rest of the program is largely complete, but exceptions are particularly good at intelligibly managing this sort of after-the-fact error-handling code (or, more optimistically, for the case where you’re adding more of them after the fact).

Exceptions are also highly useful in circumstances where a large amount of processing may need to be discarded after it has become obvious that a computational branch in your program has become untenable. The spreadsheet example was one such case; others are branch-and-bound algorithms and parsing algorithms.

14.3. Using with

Some situations, such as reading files, follow a predictable pattern with a set beginning and end. In the case of reading from a file, quite often the file needs to be open only one time, while data is being read, and then it can be closed. In terms of exceptions, you can code this kind of file access like this:

infile = open(filename)
data =

In Python 3, there’s a more generic way of handling situations like this: context managers. Context managers wrap a block and manage requirements on entry and departure from the block and are marked by the with keyword. File objects are context managers, and you can read files using that capability:

with open(filename) as infile:
data =

These two lines of code are equivalent to the five previous lines. In both cases, we know that the file will be closed immediately after the last read, whether the operation was successful or not. In the second case, closure of the file is also assured, because it’s part of the file object’s context management, so we don’t need to write the code. In other words, by using with combined with a context management (in this case a file object), we don’t need to worry about the routine cleanup.

Context managers are intended for things like locking and unlocking resources, closing files, committing database transactions, and so on. This is still a relatively new feature in Python, and its use will continue to be refined.

14.4. Summary

Python’s exception-handling mechanism and exception classes provide a rich system to handle runtime errors in your code. Python’s philosophy is that errors should not pass silently unless explicitly silenced, and by using try, except, else, and finally blocks and by selecting and even creating the types of exceptions caught, you can have very fine-grained control over how exceptions are handled and even ignored, when that makes sense.

You’ve seen that in Python exception types are organized in a hierarchy. That’s possible because exceptions, like all objects in Python, are based on classes, and you’ll see how Python uses classes and objects in the next chapter.