Improving QGIS plugin developer experience
At the National Land Survey of Finland (NLS) we are developing multiple QGIS plugins, and we needed a way to share the common code and break the components to smaller independent plugins while still providing a good developer experience.
One of the main issues when sharing library code between different QGIS plugins is the runtime environment uncertainty. Since Python import machinery is not easily configurable to support multiple versions of dependencies (like nested node_modules in nodejs-world), the runtime is limited by default to a single version of a library, and later access to the same module is cached. This limits the version available to all plugins in a single QGIS session to the code that is first run, which makes sharing code difficult, especially when breaking API changes are necessary to the dependency library code.
At NLS we developed tooling to work around these limitations, which improves the developer experience and allows sharing of common QGIS plugin code easily via standard Python libraries. Tool provides a streamlined developer workflow and necessities like typing and IDE helpers, and a way to package a plugin that depends on other standard Python libraries.
Development environment for a QGIS plugin can be initialized simply by using a virtual environment, installing the dependencies and launching QGIS with the plugin and its dependencies fully setup. This works with bootstrap code passed on the command line, which will provide QGIS access to the virtual environment, setups the plugin from the environment with access to any library dependencies. Tool also provides a debugger session and could also provide for example hot reload signals for the plugin when code is changed. This provides a quicker and easier feedback cycle for the developer and simplifies the workflows when developing QGIS plugins.
Runtime dependencies are reorganized at build-time to be imported for a sub-package of the plugin, so only the exact packaged version of a dependency is used at runtime. This works by rewriting external library dependency import statements in the source code. Tool also generates the metadata.txt file in a way that is compatible with standard Python packaging tools, for example setuptools. This allows easily sharing the same code both as Python library and as a QGIS plugin.