How to create your own Python packages

This blog post provides a summary on how one creates one’s own Python package. The information is out there in the documentation of setuptools but since it took me a few moments to figure this out I thought it was a good idea to provide a short write up.

• Directory structure and file content
  • Pyproject.toml
  • The setup configuration
  • README and LICENSE
  • The init.py file
  • Your source files
• Building the package
• Distributing the package

Directory structure and file content

The first thing to honor is the directory structure of one’s project. Inside the repository one should have a src directory that includes all of the Python source files as well as an (usually empty) __init__.py. In addition there will be a pyproject.toml that configures the build system and a setup.cfg that provides information about the package. It’s also a good idea to include a README.md and a LICENSE.md directly in your repositories root directory. Thus one has the following directory layout to start off:

|- src
|    |- PACKAGENAME
|    |            |- __init__.py
|
|- LICENSE.md
|- README.md
|- pyproject.toml
|- setup.cfg

Pyproject.toml

The pyproject.toml file specifies build time requirements and the used build backend:

[build-system]
requires = [
    "setuptools>=42",
    "wheel"
]
build-backend = "setuptools.build_meta"

The setup configuration

The setup.cfg contains configurations of the package itself. First let’s look at an example of my gammaionctl project:

[metadata]
name = gammaionctl-tspspi
version = 0.0.1
author = Thomas Spielauer
author_email = pypipackages01@tspi.at
description = Gamma ion pump ethernet control CLI utility and library
long_description = file: README.md
long_description_content_type = text/markdown
url = https://github.com/tspspi/gammacli
classifiers =
    Programming Language :: Python :: 3
    License :: OSI Approved :: BSD License
    Operating System :: OS Independent

[options]
package_dir =
    = src
packages = find:
python_requires = >=3.6

[options.packages.find]
where = src

[options.entry_points]
console_scripts =
    gammaioncli = gammaionctl.gammaioncli:gammaioncli

As one can see there are multiple sections and attributes in an INI style format:

• Metadata:
  • name specifies the name of the package. This has to be unique in the PyPi repository. The username should be appended as last argument to the package name in case one does some small hobby or educational projects.
  • The version field works as usual and uses the typical major.minor.revision layout.
  • author and author_email specify the usual author information of the package. This is also the contact information of the current responsible maintainer
  • description includes a short English description of the package content.
  • long_description in this case references a markdown file (this is specified via the long_description_content_type attribute that is set to the text/markdown MIME type) that’s also used as README.md
  • url points to a project page. In this sample this is the GitHub repository of the project.
  • The classifiers section contains a list of potential classifiers for the repositories index. A list of all classifiers can be found at the PyPi page for classifiers.
• Inside the options section some generic options like the package directory (see package_dir) as well as the required Python version is specified.
• options.packages.find again specifies which location should be scanned for the package content
• The entry_points section is required if one wants to install executable console scripts. In this case a program is installed that can later be executed by the command gammaioncli. This invokes the function def gammaioncli() inside the file src/gammaionctl/gammaioncli.py

README and LICENSE

The files README.md and LICENSE.md work the same as for every other project. Inside README.md there should be a short Markdown formatted description about the project and some additional information that should be read by any user of the package itself.

I consider LICENSE or LICENSE.md mandatory - there one should specify the license that one wants to give the software away under. I personally prefer one of the BSD licenses as one can see from my repositories since they offer the most freedom from point of view what one’s allowed to do with software and it includes the typical liability exclusion:

My typical license file looks somewhat like the following content:

Copyright <YEAR>, <COPYRIGHTHOLDERS NAME>

Redistribution and use in source and binary forms, with or without modification,
are permitted provided that the following conditions are met:

* Redistributions of source code must retain this list of conditions
  and the following disclaimer.
* Redistributions in binary form must reproduce this list of conditions
  and the following disclaimer in the documentation and/or other materials
  provided with the distribution.
* Neither the name of the copyright holder nor the names of its contributors
  may be used to endorse or promote products derived from this software without
  specific prior written permission.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT,
INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING,
BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE
OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF
ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

The __init__.py file

Under most circumstances this file is empty

Your source files

This is the most crucial part of course. I won’t go into any details - but I should mention how the imports will work based on my example.

Let’s say for example we have a src/gammaionctl/gammaioncli.py file that will include an executable command line utility as well as the src/gammaionctl/gammaionctl.py file that includes a single class definition for class GammaIonPump. The import inside gammaioncli as well as the entry point that we have defined in setup.cfg would for example look like the following:

[options.entry_points] console_scripts = gammaioncli = gammaionctl.gammaioncli:gammaioncli

from gammaionctl import gammaionctl

def gammaioncli():
    # Anything here
    with GammaIonPump(host) as pump:
        # Next code ...

Building the package

Building the package is simple. I assume setuptools and build is already installed. If not they can be installed using:

python -m pip install --upgrade build

Now one can change into the root directory of ones repository and execute

python -m build

This will create a clean build virtual environment and create the package files for the module described in setup.cfg. This will create the files

dist
   |- PACKAGENAME.tar.gz
   |- PACKAGENAME.whl

The tar.gz is a source archive whereas the whl file is a built distribution. Older versions of pip exclusively used the source archive, newer ones prefer the built distribution with a fallback to the source archive.

Distributing the package

After one has built and thoroughly tested the package one can upload them to the public repositories. For this PyPi offers first a test repository and second the live repository. These are reachable via https://test.pypi.org/ and https://www.pypi.org respectively. One should always first use the test repository. To upload anything one’s required to create an account first using the registration forms:

• Test repository registration
• Live repository registration

To upload any packages one requires an API token from the given repository. These can be created when you select your username on the right top of the page, go into Account settings and move down to API tokens. Then select Add API token there. After you’ve supplied name and scope copy the token immediately. There is no way to recover it later on, it’s not stored in clear text on PyPi’s side. The page tells you how to configure so that twine uses your user and token by editing ~/.pypirc.

Make sure twine is installed and up to date:

python -m pip install --upgrade twine

If everything turns out to work correctly you can upload your source archive and built distribution to the given repository (first always try on the test repository though):

python -m twine upload --repository testpypi dist/*

To install from the test repository one can then use:

python -m pip install --index-url https://test.pypi.org/simple/ --no-deps PACKAGENAME

After extensive testing (again) one can uninstall using

python -m pip uninstall PACKAGENAME

In case everything turned out to work perfectly well one can deploy the package on the live repository:

python -m twine upload dist/*

After that anyone can install the package using the standard PyPi commands:

python -m pip install PACKAGENAME

This article is tagged:

• Computer
• Python
• Programming
• System administration