Creating custom buildpacks for Cloud Foundry

Page last updated:

You can create custom buildpacks for Cloud Foundry.

For more information about how buildpacks work, see How Buildpacks Work.

Packaging custom buildpacks

Cloud Foundry buildpacks can work with limited or no internet connectivity. The Buildpack Packager gives the same flexibility to custom buildpacks, enabling them to work in partially or completely disconnected environments. For more information, see the Buildpack Packager repository on GitHub.

Using the Buildpack Packager

To use the Buildpack Packager:

1. Download the Buildpack Manager from the Buildpack Packager repository on GitHub.

2. Create a manifest.yml file in your buildpack.

3. Run the packager in cached mode:

   buildpack-packager build -cached -any-stack
   

The packager adds everything in your buildpack directory into a ZIP file, and excludes anything marked for exclusion in your manifest.

In cached mode, the packager downloads, and adds dependencies as described in the manifest.

For more information, see Buildpack Packager repository on GitHub.

Using and sharing the packaged buildpack

After you have packaged the buildpack using Buildpack Packager, you can use the ZIP file locally, or share it with others by uploading it to any network location that is accessible to the CLI. You can can then specify the buildpack with the -b option when you push apps. For more information, see Deploying Apps with a Custom Buildpack.

Offline buildpack packages might contain proprietary dependencies that require distribution licensing or export control measures. For more information about offline buildpacks, see About Offline Buildpacks section of the Packaging dependencies for offline Buildpacks topic.

Specifying a default version

As of Buildpack Packager v2.3.0, you can specify the default version for a dependency by adding a default_versions object to the manifest.yml file. The default_versions object has two properties, name and version.

For example:

default_versions:
- name: go
  version: 1.6.3
- name: other-dependency
  version: 1.1.1

To specify a default version:

1. Add the default_version object to your manifest, following the guidance in Rules for Specifying a Default Version. For a complete example, see manifest.yml in the Cloud Foundry Go(Lang) Buildpack repository in GitHub.

2. Run the default_version_for script from the compile-extensions repository, passing the path of your manifest.yml and the dependency name as arguments. Run:

   ./compile-extensions/bin/default_version_for manifest.yml DEPENDENCY-NAME
   

   Where DEPENDENCY-NAME is the name property from the default_versions object in your manifest.yml file.

For more information, see Buildpack Packager v2.3.0 in the Buildpack Packager repository on GitHub.

Rules for specifying a default version

The Buildpack Packager script validates this object according to the following rules:

• You can create at most one entry under default_versions for a single dependency. The following example causes Buildpack Packager to fail with an error because the manifest file specifies two default versions for the same go dependency.

  default_versions:
  - name: go
    version: 1.6.3
  - name: go
    version: 1.7.5
  

• If you specify a default_version for a dependency, you must also list that dependency and version under the dependencies section of the manifest. The following example causes Buildpack Packager to fail with an error because the manifest specifies version: 1.9.2 for the go dependency, but lists version: 1.7.5 under dependencies.

  default_versions:
  - name: go
    version: 1.9.2
  
  dependencies:
  - name: go
    version: 1.7.5
    uri: https://storage.googleapis.com/golang/go1.7.5.linux-amd64.tar.gz
    md5: c8cb76e2308c792e2705c2eb1b55de95
    cf_stacks:
    - cflinuxfs3
  

To avoid security exposure, verify that you migrate your apps and custom buildpacks to use the cflinuxfs4 stack based on Ubuntu 22.04 LTS (Jammy Jellyfish). The cflinuxfs3 stack is based on Ubuntu 18.04 (Bionic Beaver), which reaches end of standard support in April 2023.

Core buildpack communication contract

Learn about the communication contract followed by the Cloud Foundry core buildpacks. This contract enables buildpacks to interact with one another, so that you can use multiple buildpacks with your apps.

You must ensure your custom buildpacks follow the contract.

This section uses the following placeholders:

• IDX is the zero-padded index matching the position of the buildpack in the priority list.
• MD5 is the MD5 checksum of the buildpack’s URL.

For all buildpacks that supply dependencies through /bin/supply:

• The buildpack must create /tmp/deps/IDX/config.yml to provide a name to subsequent buildpacks. This file might also contain miscellaneous configuration for subsequent buildpacks.

• The config.yml file must be formatted as:

  name: BUILDPACK
  config: YAML-OBJECT
  

  Where:

  • BUILDPACK is the name of the buildpack providing dependencies.
  • YAML-OBJECT is the YAML object that contains buildpack-specific configuration.

• The following directories can be created inside of /tmp/deps/IDX/ to provide dependencies to subsequent buildpacks:

  • /bin: Contains binaries intended for $PATH during staging and launch.
  • /lib: Contains libraries intended for $LD_LIBRARY_PATH during staging and launch.
  • /include: Contains header files intended for compilation during staging.
  • /pkgconfig: Contains pkgconfig files intended for compilation during staging.
  • /env: Contains environment variables intended for staging, loaded as FILENAME=FILECONTENTS.
  • /profile.d: Contains scripts intended for /app/.profile.d, sourced before launch.

• The buildpack might make use of previous non-final buildpacks by scanning /tmp/deps/ for index-named directories containing config.yml.

For the last buildpack:

• To make use of dependencies provided by the previously applied buildpacks, the last buildpack must scan /tmp/deps/ for index-named directories containing config.yml.

• To make use of dependencies provided by previous buildpacks, the last buildpack:

  • Can use /bin during staging, or make it available in $PATH during launch.
  • Can use /lib during staging, or make it available in $LD_LIBRARY_PATH during launch.
  • Can use /include, /pkgconfig, or /env during staging.
  • Can copy files from /profile.d to /tmp/app/.profile.d during staging.
  • Can use the supplied config object in config.yml during the staging process.

Deploying apps with a custom buildpack

Once a custom buildpack has been created and pushed to a public Git repository, the Git URL can be passed through the cf CLI when pushing an app.

For example, you can use a buildpack that has been pushed to GitHub by running:

cf push YOUR-APP -b git://github.com/REPOSITORY/BUILDPACK.git

Where:

• YOUR-APP is the name of your app.
• REPOSITORY is the name of your public Git repository.
• BUILDPACK is the name of your custom buildpack.

Alternatively, you can use a private Git repository, with HTTPS, and username and password authentication:

cf push YOUR-APP -b https://USERNAME:[email protected]/REPOSITORY/BUILDPACK.git

Where:

• YOUR-APP is the name of your app.
• USERNAME is your Git username.
• PASSWORD is the name of your Git password.
• REPOSITORY is the name of your public Git repository.
• BUILDPACK is the name of your custom buildpack.

By default, Cloud Foundry uses the default branch of the buildpack’s Git repository. You can specify a different branch using the Git URL:

cf push YOUR-APP -b https://github.com/REPOSITORY/BUILDPACK.git#BRANCH

Where:

• YOUR-APP is the name of your app.
• REPOSITORY is the name of your public Git repository.
• BUILDPACK is the name of your custom buildpack.
• BRANCH is the branch you want to use.

Additionally, you can use tags in a Git repository:

cf push YOUR-APP -b https://github.com/REPOSITORY/BUILDPACK#TAG

Where:

• YOUR-APP is the name of your app.
• REPOSITORY is the name of your public Git repository.
• BUILDPACK is the name of your custom buildpack.
• TAG is the Git repository tag you want to use.

The app is then deployed to Cloud Foundry, and the buildpack is cloned from the repository and applied to the app.

If a buildpack is specified using cf push -b, the detect step is skipped. As a result, no buildpack detect scripts are run.

Disabling Custom Buildpacks

A common development practice for custom buildpacks is to fork existing buildpacks and sync subsequent patches from upstream. To merge upstream patches to your custom buildpack, see Syncing a fork in the GitHub documentation.