We will briefly cover the steps you need to take to publish your package on the Python Package Index (PyPI). You can use these steps to help you publish plugins for the CrossCompute Analytics Automation Framework. These steps are up-to-date as of September 2021.
- Test with
- Publish on PyPI with
pyproject.toml configuration file provides a hook so that you can specify a different package build system. For example, JupyterLab uses a custom build system.
pyproject.toml configuration file can also contain configuration options for other development tools such as
coverage. However, we decided to put the these other configuration options into
setup.cfg instead, as it is more widely supported.
Here is the default configuration that uses the standard
# pyproject.toml [build-system] requires = [ 'setuptools >= 40.9.0', 'wheel', ] build-backend = 'setuptools.build_meta'
You might notice during
pip install -e . that the build process is slower than
setup.py. If you want to continue using
setup.py with all the benefits of
setup.cfg, you can opt for the following instead of
# setup.py from setuptools import setup setup()
The Python Packaging User Guide now recommends replacing
setup.cfg . Here are some advantages:
attr:to include a value from a module.
file:to include content from a file.
- Include configuration options from other development tools like
toxand GitHub Actions.
Here are other points:
- You can write
classifiersto help people find your package.
project_urlsfor related links to display next to your package on PyPI.
# setup.cfg [metadata] name = crosscompute version = attr: crosscompute.__version__ description = Automate your Jupyter notebooks and scripts as web-based reports, tools, widgets, dashboards, wizards. long_description = file: README.md long_description_content_type = text/markdown url = https://crosscompute.com author = CrossCompute Inc. author_email = email@example.com license = MIT license_file = LICENSE.md platforms = any classifiers = Development Status :: 3 - Alpha Framework :: CrossCompute Framework :: Pyramid Intended Audience :: Analysts Intended Audience :: Developers Intended Audience :: Information Technology Intended Audience :: Science/Research Intended Audience :: System Administrators License :: OSI Approved :: MIT License Operating System :: OS Independent Programming Language :: Python :: 3 Topic :: Internet :: WWW/HTTP :: Dynamic Content Topic :: Internet :: WWW/HTTP :: WSGI Topic :: Internet :: WWW/HTTP :: WSGI :: Application Topic :: Software Development :: Libraries :: Application Frameworks project_urls = Bug Tracker = https://github.com/crosscompute/crosscompute/issues Documentation = https://github.com/crosscompute/crosscompute-docs Source Code = https://github.com/crosscompute/crosscompute [options] packages = find: python_requires = >=3.6 install_requires = pyramid zip_safe = True [options.entry_points] console_scripts = crosscompute = crosscompute.scripts:launch [options.extras_require] test = # check-manifest hypothesis # pytest-check-links pytest-console-scripts pytest-cov pytest-mock pytest-xtest requests-cache [flake8] max-line-length = 79 select = B,C,E,F,W ignore = W503 [coverage:run] parallel = true branch = true source = crosscompute [coverage:paths] source = crosscompute [tox] envlist = lint,py36,py37,py38,py39,coverage-report [testenv] deps = .[test] commands = coverage run -m pytest -vv tests -n auto [testenv:coverage-report] deps = coverage skip_install = true commands = coverage combine coverage report [testenv:lint] deps = flake8 skip_install = true commands = flake8 crosscompute [gh-actions] python = 3.6: py36 3.7: py37 3.8: py38 3.9: py39
check-manifest: Check your
MANIFEST.infor errors and omissions.
- hypothesis: Find edge cases through semi-automated testing.
- pytest-check-links: Check URLs in your Markdown files and Jupyter notebooks.
- pytest-console-scripts: Test console scripts more efficiently.
- pytest-xtest: Run tests in parallel if your machine has multiple cores.
- requests-cache: Cache outgoing HTTP requests.
Here are other packages that we have been using already:
- pytest-cov: Include line-by-line coverage analysis in your test results.
- pytest-mock: Mock functions in your tests.
# Clone repository git clone https://github.com/crosscompute/crosscompute # Install with dependencies for tests cd crosscompute pip install -e .[test] # Run tests pytest \ --cov=crosscompute \ --cov-config=tox.ini \ --cov-report term-missing:skip-covered \ tests # Build package for PyPI pip install build python -m build --sdist --wheel
- After building your source distribution, inspect its contents in the
distfolder to make sure the package contains all the files it needs.
- Change to a different folder that does not contain your package, start Python and try importing your package to check for import errors.
All of your tests have passed. You are now ready to publish your package! You will need to create accounts on both TestPyPI and PyPI.
- Publish on TestPyPI to make sure that everything works properly.
pip install twine --upgrade python -m twine upload --repository testpypi dist/*
- Publish on PyPI.
pip install twine --upgrade python -m twine upload dist/*
If you publish packages frequently, you might want to define the
.pypi.rc configuration file in your home folder. You can specify your PyPI API token in this file.
# .pypi.rc [testpypi] username = __token__ password = Your TestPyPI API token [pypi] username = __token__ password = Your PyPI API token
For more information on Python packaging, please see the official Python Packaging User Guide.