Let's condense the basics of the logging Python module.

tldr

• If you don't configure any logger, only the log levels WARN and up will show up.
• In your program (not a library), set up the root logger as one of the first things you do. Calling logging.basicConfig does that for example.
• In each module, have a global logger = loging.getLogger(__name__) variable.
• If you write a library, never add a handler to your loggers, because this is the consumer's job. Either don't add any handlers, or use a NullHandler - see here

Logging module basics

In a nutshell:

• A logger can have:

  • handlers: a log handler is responible for the output of a log record. A logger can have several handlers: e.g., it can log to console and a file. Each handler can be configured differently, e.g. regarding the log level. If you define a custom handler, you need to implement the emit(record) method that outputs a log record.
  • formatters: a handler can have a formatter that defines how a log record should be formatted.
  • filters: a handler (or a logger) can filter messages and only emits log records that pass the filter. You can simply add a filter function that takes a record (string) and returns zero or one.

• Loggers follow a hierachy. When a child logger receives a log record it first passes it to its own handlers. Per default, log records are then passed to any parent loggers. This is important for configuring the loggers. The hierarchy can be established in two ways:

  1. By configuring the root logger. This is done by e.g. calling logging.basicConfig. Any logger that you create with logging.getLogger('my_name') will be a child of the root logger. Note: calling logging.basicConfig if the root logger is already configured will have no effect.
  2. By establishing a logger hierarchy via dot notation: main.child, where the main logger would be the parent of the child logger. This way, we could define a main logger in our main module and configure it there: main_logger = logging.getLogger("main") And in each submodule we explicitly create a child logger: library_logger = logging.getLogger("main.library"). Now, in tutorials and the Python docs you always see the recommendation to use logging.getLogger(__name__) in all sub modules. This makes a lot of sense, if you have a package structure, because then the logging hierarchy is automatically configured:

     my_package
     my_package/__init__.py -> configure the root logger here
     my_package/some_module.py -> Use `logger = logging.getLogger(__name__)`
     

     If we use a module as an entry point (i.e., we call it as something like python my_package/start.py, set the logger manually using the file name so that it is not just logged as __main__: logger = logging.getLogger("my_package.start").
• Use a logger per module, not one per class. If you need a logger in a class, initialize it automatically as self.logger = logging.getLogger(f"{__name__}.{self.__class__.__name__}") or similar. Don't pass loggers around as parameters.

Tidbits

• You can pass additional information to logging calls. See LoggerAdapters and the extra keyword that allows to use additional parameters in the format string.
• In larger projects, configure loggers via text configuration. See the logging.config documentation. Either use logging.config.fileConfig (which uses the configparser module) with ini-style config files, or use a library to de-serialize a text format to a dictionary and then initialize the logger with logging.config.dictConfig.

Resources

• Hitchhiker's Guide to Python
• Python logging cookbook
• Python logging module docs
• Python logging config docs