How Editable Installs Work

Install your Package!¶

Installing a package takes time and manual effort, but it’s an important part of using someone elses software; Python packages are not just Python code that we can add to the PYTHONPATH, they’re meant to be installed to be used. But, it’s not convenient to keep re-installing it when you make changes; seeing the result of the edits to your code goes from a simple file-save to running a shell command. How can you see your changes in real-time?

Install your package editably¶

The solution, as ever, can be found in pip. pip can build and install an Editable wheel for your package, using -e:

pip install -e .

import os
import pathlib
import tempfile
path = os.getcwd()
tmp_path = pathlib.Path(tempfile.mkdtemp())
link_src = pathlib.Path(path).parents[3] / "arrow-to-knee"
package_path = (tmp_path / link_src.name)
package_path.symlink_to(link_src)
os.chdir(package_path)

os.environ['HATCH_INTERACTIVE'] = '0'

---------------------------------------------------------------------------
FileNotFoundError                         Traceback (most recent call last)
Cell In[1], line 9
      7 package_path = (tmp_path / link_src.name)
      8 package_path.symlink_to(link_src)
----> 9 os.chdir(package_path)
     11 os.environ['HATCH_INTERACTIVE'] = '0'

FileNotFoundError: [Errno 2] No such file or directory: '/tmp/tmp11yfca27/arrow-to-knee'

We can build this wheel by hand:

!hatch build -t wheel:editable .

It contains very little!

!zipinfo ./arrow_to_knee*.whl

What do we see here that’s not familiar to us thus far? It’s the /_arrow_to_knee.pth file. More on that in a moment.

Python is a detective¶

To go any further, we are going to need to understand a little bit about how Python import statements actually work. We’re not going to dig into the line-by-line account of what Python is doing under the hood, but we are going to appreciate how extensible and powerful this part of the interpreter is.

At startup, Python runs some code that prepares sys.path. This includes finding .pth files in known directories, and processing them according to some rules:

• .pth files can contain paths that are added to sys.path.
• They can also contain import statements, which are exec by the interpreter.

The latter feature is somewhat incredible — one can use .pth files (slightly in poor form) to write code that runs at Python startup!

Editable wheel packages are designed around this .pth mechanism — they are designed to ship a .pth file that either exposes the source directory, or runs some code that handles this slightly more explicitly.