Skip to content

Working with PEP 582#

PEP 582 has been rejected

This is a rejected PEP. However, due to the fact that this feature is the reason for PDM's birth, PDM will retain the support. We recommend using virtual environments instead.

With PEP 582, dependencies will be installed into __pypackages__ directory under the project root. With PEP 582 enabled globally, you can also use the project interpreter to run scripts directly.

When the project interpreter is a normal Python, this mode is enabled.

Besides, on a project you work with for the first time on your machine, if it contains an empty __pypackages__ directory, PEP 582 is enabled automatically, and virtualenv won't be created.

Enable PEP 582 globally#

To make the Python interpreters aware of PEP 582 packages, one need to add the pdm/pep582/sitecustomize.py to the Python library search path.

One just needs to execute pdm --pep582, then environment variable will be changed automatically. Don't forget to restart the terminal session to take effect.

The command to change the environment variables can be printed by pdm --pep582 [<SHELL>]. If <SHELL> isn't given, PDM will pick one based on some guesses. You can run eval "$(pdm --pep582)" to execute the command.

You may want to write a line in your .bash_profile(or similar profiles) to make it effective when logging in. For example, in bash you can do this:

1
pdm --pep582 >> ~/.bash_profile

Once again, Don't forget to restart the terminal session to take effect.

How is it done?

Thanks to the site packages loading on Python startup. It is possible to patch the sys.path by executing the sitecustomize.py shipped with PDM. The interpreter can search the directories for the nearest __pypackage__ folder and append it to the sys.path variable.

Configure IDE to support PEP 582#

Now there are no built-in support or plugins for PEP 582 in most IDEs, you have to configure your tools manually.

PyCharm#

Mark __pypackages__/<major.minor>/lib as Sources Root. Then, select as Python interpreter a Python installation with the same <major.minor> version.

Additionally, if you want to use tools from the environment (e.g. pytest), you have to add the __pypackages__/<major.minor>/bin directory to the PATH variable in the corresponding run/debug configuration.

VSCode#

Add the following two entries to the top-level dict in .vscode/settings.json:

1
2
3
4
{
  "python.autoComplete.extraPaths": ["__pypackages__/<major.minor>/lib"],
  "python.analysis.extraPaths": ["__pypackages__/<major.minor>/lib"]
}

This file can be auto-generated with plugin pdm-vscode.

Enable PEP582 globally, and make sure VSCode runs using the same user and shell you enabled PEP582 for.

Cannot enable PEP582 globally?

If for some reason you cannot enable PEP582 globally, you can still configure each "launch" in each project: set the PYTHONPATH environment variable in your launch configuration, in .vscode/launch.json. For example, to debug your pytest run:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
{
    "version": "0.2.0",
    "configurations": [
        {
            "name": "pytest",
            "type": "python",
            "request": "launch",
            "module": "pytest",
            "args": ["tests"],
            "justMyCode": false,
            "env": {"PYTHONPATH": "__pypackages__/<major.minor>/lib"}
        }
    ]
}

If your package resides in a src directory, add it to PYTHONPATH as well:

1
"env": {"PYTHONPATH": "src:__pypackages__/<major.minor>/lib"}
Using Pylance/Pyright?

If you have configured "python.analysis.diagnosticMode": "workspace", and you see a ton of errors/warnings as a result. you may need to create pyrightconfig.json in the workspace directory, and fill in the following fields:

1
2
3
{
    "exclude": ["__pypackages__"]
}

Then restart the language server or VS Code and you're good to go. In the future (microsoft/pylance-release#1150), maybe the problem will be solved.

Using Jupyter Notebook?

If you wish to use pdm to install jupyter notebook and use it in vscode in conjunction with the python extension:

  1. Use pdm add notebook or so to install notebook
  2. Add a .env file inside of your project directory with contents like the following:
1
PYTHONPATH=/your-workspace-path/__pypackages__/<major>.<minor>/lib

If the above still doesn't work, it's most likely because the environment variable is not properly loaded when the Notebook starts. There are two workarounds.

  1. Run code . in Terminal. It will open a new VSCode window in the current directory with the path set correctly. Use the Jupyter Notebook in the new window
  2. If you prefer not to open a new window, run the following at the beginning of your Jupyter Notebook to explicitly set the path:
1
2
import sys
sys.path.append('/your-workspace-path/__pypackages__/<major>.<minor>/lib')

Reference Issue

PDM Task Provider

In addition, there is a VSCode Task Provider extension available for download.

This makes it possible for VSCode to automatically detect pdm scripts so they can be run natively as VSCode Tasks.

Neovim#

If using neovim-lsp with pyright and want your __pypackages__ directory to be added to the path, you can add this to your project's pyproject.toml.

1
2
[tool.pyright]
extraPaths = ["__pypackages__/<major.minor>/lib/"]

Emacs#

You have a few options, but basically you'll want to tell an LSP client to add __pypackages__ to the paths it looks at. Here are a few options that are available:

Using pyproject.toml and pyright#

Add this to your project's pyproject.toml:

1
2
[tool.pyright]
extraPaths = ["__pypackages__/<major.minor>/lib/"]

eglot + pyright#

Using pyright and eglot (included in Emacs 29), add the following to your config:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
(defun get-pdm-packages-path ()
  "For the current PDM project, find the path to the packages."
  (let ((packages-path (string-trim (shell-command-to-string "pdm info --packages"))))
    (concat packages-path "/lib")))

(defun my/eglot-workspace-config (server)
  "For the current PDM project, dynamically generate a python lsp config."
  `(:python\.analysis (:extraPaths ,(vector (get-pdm-packages-path)))))

(setq-default eglot-workspace-configuration #'my/eglot-workspace-config)

You'll want pyright installed either globally, or in your project (probably as a dev dependency). You can add this with, for example:

1
pdm add --dev --group devel pyright

LSP-Mode + lsp-python-ms#

Below is a sample code snippet showing how to make PDM work with lsp-python-ms in Emacs. Contributed by @linw1995.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
  ;; TODO: Cache result
  (defun linw1995/pdm-get-python-executable (&optional dir)
    (let ((pdm-get-python-cmd "pdm info --python"))
      (string-trim
       (shell-command-to-string
        (if dir
            (concat "cd "
                    dir
                    " && "
                    pdm-get-python-cmd)
          pdm-get-python-cmd)))))

  (defun linw1995/pdm-get-packages-path (&optional dir)
    (let ((pdm-get-packages-cmd "pdm info --packages"))
      (concat (string-trim
               (shell-command-to-string
                (if dir
                    (concat "cd "
                            dir
                            " && "
                            pdm-get-packages-cmd)
                  pdm-get-packages-cmd)))
              "/lib")))

  (use-package lsp-python-ms
    :ensure t
    :init (setq lsp-python-ms-auto-install-server t)
    :hook (python-mode
           . (lambda ()
               (setq lsp-python-ms-python-executable (linw1995/pdm-get-python-executable))
               (setq lsp-python-ms-extra-paths (vector (linw1995/pdm-get-packages-path)))
               (require 'lsp-python-ms)
               (lsp))))  ; or lsp-deferred