How to Define Plugins - Packaging in Python

The Problem¶

When you’re developing an application, your might want to make it easy for users to add new features. For example, a new fitting routine for your GUI application. You want this to be something that users can do without your intervention; we are all bottlenecks on our own projects, and we don’t want to get in the way of our users!

This means that we can’t import users code in our application, i.e. we can’t do:

import some_users_module

def my_cool_application():
    some_users_module.do_something()

as we do not know where the user addition(s) comes from when writing our package.

Figure 1:How you might be feeling at this point.

How do we solve this problem? There’s an established pattern for this kind of task, called Dependency injection.

Figure 2:Watch out, I know several different ways to inject plugins into your code.

What is Dependency Injection?¶

Dependency injection: Receiving dependencies (functions, classes, etc.) as inputs instead of importing them directly.

In simple terms, this can be as simple as passing in a function as a functional argument, transforming from this:

1
2
3
4
import plugin

def application():
    plugin.compute()

to this:

1
2
def application(plugin):
    plugin.compute()

Uses for Plugins¶

As indicated above, the main reason to create a plugin is to allow users to define new features that you can request ahead of time:

Adding new features to a framework or app.
- Jupyter server extensions
- GlueViz fitting routines
Changing the implementation of a framework or app.

How to Create a Plugin¶

Your plugin implementation can do very little...

And the pyproject.toml entry is also small:

This is all that is required to define a plugin and expose it from your package. Once your package has been installed, other packages in the same Python environment can request the plugin as part of the plugin group.

Loading a Plugin¶

To load plugins as a consumer, you can use the importlib.metadata package. Since Python 3.10 this is no-longer provisional, and ships with Python.

To load all plugins, it’s as simple as:

import importlib.metadata

entry_points = importlib.metadata.entry_points()

We can filter this result to select only our plugin group of interest. To select only the history_of_war group:

plugins = entry_points.select(group="history_of_war")

The corresponding list of plugins contains plugin information objects. These contain details about where the plugin came from, and how to load it. We can load the plugin from the entry point object using load(), and then call the plugin-

for plugin in plugins:
    impl = plugin.load()
    impl()

Ow!!! My kneeeeeeeeeeeeeeeeeeeeeeeee. Ow.

Introduction to Packaging

Apps vs Libraries

Advanced Packaging

How Editable Installs Work