Setupy

gitignore.io for setup.py

Setupy is a tool for generating setup.py files made of composable layers of features and settings.

Use

setup.py files can be generated from setupy.dev in one of two ways

• The web UI (with or without Javascript enabled)
• The API

Some examples of API usage on the web

# Get a base setup.py file that can be extended
curl https://setupy.dev/get

# Include the help text to make it easier to manually edit the output
curl https://setupy.dev/get?include_help=true

# Include a feature and setting 
curl https://setupy.dev/get?features=merge&settings=add_upload_command

# Get a list of all the available features
curl https://setupy.dev/features/list

# Get a list of all available settings
curl https://setupy.dev/settings/list

And on the command line (if setupy is installed)

python -m setupy \
    -s base add_upload_command add_long_description \
    --include-setting "$(cat setupy.yaml)"

python -m setupy help

Note: Order is important when passing in settings

What Setupy generates

The setup.py file for Setupy is generated by Setupy so it serves as a good example to see what can be generated. Here's an earlier (smaller) version:

# 1
import os
import sys
from os import path
from shutil import rmtree

from setuptools import Command, find_packages, setup


# 2
here = path.abspath(path.dirname(__file__))

with open(path.join(here, 'README.md'), encoding='utf-8') as f:
    long_description = f.read()


VERSION = open("VERSION.txt").read()


def merge(*dicts):
    r = {}
    for d in dicts:
        r.update(d)
    return r


# 3
base = {
    "name": "mypackage",
    "version": "0.1.0",
    "packages": find_packages(exclude=['contrib', 'docs', 'test'])
}

add_long_description = {
    "long_description": long_description,
    "long_description_content_type": "text/markdown"
}

setupy = {
    "name": "setupy",
    "version": VERSION,
    "install_requires": ["isort>=4.3", "pyyaml>=3.13", "flask>=1.0.2"],
    "extras_require": {
        "dev": ["pytest", "pytest-cov"],
        "neovim": ["neovim"]
    }
}


# 4
setup(**merge(base, add_upload_command, add_long_description, setupy))

There are 4 primary sections in files generated by Setupy

1. Imports
2. Features
3. Settings
4. Merging all setting layers together and calling setup

Settings

Lets start at the beginning to see the motivation for each section. Our end goal is to call setup from setuptools (which is the end goal of every setup.py file. The setup command takes a number of key word arguments. In order to make things more modular, we can create a number of dictionaries and merge them together, then unpack them with ** and execute setup on the result.

This is Setupy's core model. Settings are used to generate these dictionaries which are ultimately merged and passed to setup. Settings like base:

base = {
    "name": "mypackage",
    "version": "0.1.0",
    "packages": find_packages(exclude=['contrib', 'docs', 'test'])
}

Are provided by Setupy to be used across setup files while settings like setupy in the above example

setupy = {
    "name": "setupy",
    "version": VERSION,
    "install_requires": ["isort>=4.3", "pyyaml>=3.13", "flask>=1.0.2"],
    "extras_require": {
        "dev": ["pytest", "pytest-cov"],
        "neovim": ["neovim"]
    }
}

Override previous settings during the merge. In the final call to merge

merge(base, add_upload_command, add_long_description, setupy)

Earlier dictionaries in the argument list have their values overriden by later dictionaries. So in this case, the default name passed in by base gets overriden by the setup dictionary. You can download a file from Setupy and define a new dictionary with any of the provided features or settings and then augment them by editing the resulting setup file. You can also use the command line to provide custom settings or features as Setupy does in order to generate its own setup.py file:

python -m setupy \
    -s base add_upload_command add_long_description \
    --include-setting "$(cat setupy.yaml)"

Settings will override each other (be passed to merge) in the order they are passed in to the API, whether that's on the web or through the command line. If you want to know how merge works, there's no secret. It's an included feature!

def merge(*dicts):
    r = {}
    for d in dicts:
        r.update(d)
    return r

Features

In order to support special functionality for a setup.py file (for example getting the version from a VERSION.txt file or uploading through twine) we need to be able to add python code to our setup.py files. Features add this functionality. For example a standard feature in Setupy allows extracting the version from VERSION.txt

VERSION = open("VERSION.txt").read()

Later settings can then use the VERSION variable. In order to keep from having to manually track which features are required to activate which settings, settings can declare their feature dependencies:

# setupy/settings/long_description.yaml, one of the standard settings 
---
name: add_long_description
dependencies:
features:
    - long_description # Depends on the long_description feature
properties:
    long_description: long_description
    long_description_content_type: "\"text/markdown\""

Any features that a setting depends on will automatically be pulled in when the setting is added. Features may also depend on other features.

Note: There isn't a lot of validation logic built around this dependency graph behavior just yet. Tread carefully when building long chains of dependencies.

Imports

Features and settings might need to import a python module in order to work. Those dependencies are also declared in the yaml files. Take this example from the upload feature:

# setupy/features/upload.yaml
---
name: upload
dependencies:
    features:
        - version_txt
    imports:
        - import os
        - from shutil import rmtree 
        - import sys
        - from setuptools import Command

Four imports are needed to make this feature work. They will all be pulled in when the file is built. Lots of features and settings may end up depending on the same imports. In order to prevent a messy collection of imports sitting at the top of the file, all imports are run through isort before the final setup.py is written.

Why Setupy

Whenever I start a new python project there's a couple of things I do, I update pipenv and start a new shell, I git init, add a .gitignore file from gitignore.io and create a few directories for my project files (the main module, a __main__, some __init__.py files and tests/ mostly.

Eventually I get to the point that I want to create a setup.py file and maybe publish my module to PyPi. In order to do that I inevitably have to look up a setup.py file to copy, figure out which options I want and spend a couple minutes filling everything out.

I wanted a solution like gitignore.io, something with an easy to use API that would interact well with curl and would be modular enough to work with lots of different projects. My hope is that the number of features and settings in this repository continues to grow and Setupy can become a really useful tool for generating standardized and customizable setup.py files.

Self hosting

The standard repository likely won't work for everybody. You or your organization might also want to use Setupy on projects where you want to retain the rights to any features or settings you build without making them open source. Setupy can be self hosted and pointed to a custom set of settings and features by pulling and running the docker container:

docker pull setupy:latest    
docker run setupy:latest \
    -e SETUPY_SETTINGS /path/to/setupy/settings
    -e SETUPY_FEATURES /path/to/setupy/features

Defining your own features

Features reside in the directory pointed to by the environment variable SETUPY_FEATURES. There are two types of files in this directory

• .yaml files which define the metadata for a feature
• .py files which define the code for a feature

Lets define a new feature called long_description. First we create the metadata file long_description.yaml

---
name: long_description
dependencies:
    imports:
        - from os import path

This feature has only one dependant import. The name is important here, it must be the same as the file name. Next we create the python file that contains the code that will be pulled into our setup.py file:

here = path.abspath(path.dirname(__file__))

with open(path.join(here, 'README.md'), encoding='utf-8') as f:
    long_description = f.read() 

And that's it. We can now generate setup.py files using this custom feature

The full schema for a feature .yaml file looks like:

---
name (required): string 
dependencies (optional):
    imports (optional):
        - list (optional)
        - of 
        - strings
    features (optional):
        - list (optional)
        - of
        - strings

Each entry in the list dependencies.features should be the name of an existing feature. All features must have a unique name.

Defining your own settings

Settings reside in the directory pointed to by the environment variable SETUPY_SETTINGS. This file contains .yaml files that look like:

---
name: add_long_description
dependencies:
    features:
        - long_description
properties:
    long_description: long_description
    long_description_content_type: "\"text/markdown\""

The dependencies object has the same schema as the dependencies object from features. Note that settings may not depend on settings. This is an intentional design choice to:

1. Keep chains of dependencies as short and clear as possible
2. Prevent complex and possibly unpredictable ordering behavior when constructing the final ordered list of dictionaries to merge

You may notice the \" (escaped quotes) characters in the properties object. Because properties may reference variables AND literal strings, there must be a way to differentiate them when the properties object is translated to a python dictionary to be included in the setup.py file. Literal strings must be enclosed in quotes and escaped quotes (as in "\"text/markdown\""). Variables should be enclosed in neither (as in long_description).

The name property in this file should match the filename (minus the .yaml extension) and will be used as the variable to assign the resulting dictionary to when seutp.py is created. Thus all settings must have a unique name.