Deploying with Application Manifests

Page last updated:

Application manifests tell cf push what to do with applications. This includes everything from how many instances to create and how much memory to allocate to what services applications should use.

A manifest can help you automate deployment, especially of multiple applications at once.

How cf push Finds the Manifest

By default, the cf push command deploys an application using a manifest.yml file in the current working directory.

$ cf push
Using manifest file /path_to_working_directory/manifest.yml

If your manifest is located elsewhere, use the -f option to provide the path to the filename.

$ cf push -f ./some_directory/some_other_directory/alternate_manifest.yml
Using manifest file /path_to_working_directory/some_directory/some_other_directory/alternate_manifest.yml

If you provide a path with no filename, the filename must be manifest.yml.

$ cf push -f ./some_directory/some_other_directory/
Using manifest file /path_to_working_directory/some_directory/some_other_directory/manifest.yml

Example Manifest

You can deploy applications without ever using a manifest. The benefits manifests may provide include consistency and reproducibility. When you want applications to be portable between different clouds, manifests may prove especially useful.

Manifests are written in YAML. The manifest below illustrates some YAML conventions, as follows:

  • The manifest may begin with three dashes.
  • The applications block begins with a heading followed by a colon.
  • The application name is preceded by a single dash and one space.
  • Subsequent lines in the block are indented two spaces to align with name.
---
applications:
- name: nifty-gui
  memory: 512M
  host: nifty

A minimal manifest requires only an application name. To create a valid minimal manifest, remove the memory and host properties from this example.

Always Provide an Application Name to cf push

cf push requires an application name, which you provide either in a manifest or at the command line.

As described in How cf push Finds the Manifest above, the command cf push locates the manifest.yml in the current working directory by default, or in the path provided by the -f option.

If you do not use a manifest, the minimal push command looks like this:

$ cf push my-app

Note: When you provide an application name at the command line, cf push uses that application name whether or not there is a different application name in the manifest. If the manifest describes multiple applications, you can push a single application by providing its name at the command line; the cf CLI does not push the others. Use these behaviors for testing.

How cf push Finds the Application

By default, cf push recursively pushes the contents of the current working directory. Alternatively, you can provide a path using either a manifest or a command line option.

  • If the path is to a directory, cf push recursively pushes the contents of that directory instead of the current working directory.
  • If the path is to a file, cf push pushes only that file.

Note: If you want to push more than a single file, but not the entire contents of a directory, consider using a .cfignore file to tell cf push what to exclude.

Precedence Between Manifests, Command Line Options, and Most Recent Values

When you push an application for the first time, Cloud Foundry applies default values to any attributes that you do not set in a manifest or cf push command line options.

  • For example, cf push my-app with no manifest might deploy one instance of the app with one gigabyte of memory. In this case the default values for instances and memory are “1” and “1G”, respectively.

Between one push and another, attribute values can change in other ways.

  • For example, the cf scale command changes the number of instances.

The attribute values on the server at any one time represent the cumulative result of all settings applied up to that point: defaults, attributes in the manifest, cf push command line options, and commands like cf scale. There is no special name for this resulting set of values on the server. You can think of them as the most recent values.

cf push follows rules of precedence when setting attribute values:

  • Manifests override most recent values, including defaults.
  • Command line options override manifests.

In general, you can think of manifests as just another input to cf push, to be combined with command line options and most recent values.

Optional Attributes

This section explains how to describe optional application attributes in manifests. Each of these attributes can also be specified by a command line option. Command line options override the manifest.

The buildpack attribute

If your application requires a custom buildpack, you can use the buildpack attribute to specify it in one of three ways:

  • By name: MY-BUILDPACK.
  • By GitHub URL: https://github.com/cloudfoundry/java-buildpack.git.
  • By GitHub URL with a branch or tag: https://github.com/cloudfoundry/java-buildpack.git#v3.3.0 for the v3.3.0 tag.
---
  ...
  buildpack: buildpack_URL

Note: The cf buildpacks command lists the buildpacks that you can refer to by name in a manifest or a command line option.

The command line option that overrides this attribute is -b.

The command attribute

Some languages and frameworks require that you provide a custom command to start an application. Refer to the buildpack documentation to determine if you need to provide a custom start command.

You can provide the custom start command in your application manifest or on the command line. See Starting, Restarting, and Restaging Applications for more information on how Cloud Foundry determines its default start command.

To specify the custom start command in your application manifest, add it in the command: START-COMMAND format as the following example shows:

---
  ...
  command: bundle exec rake VERBOSE=true

The start command you specify becomes the default for your application. To return to using the original default start command set by your buildpack, you must explicitly set the attribute to null as follows:

---
  ...
  command: null

On the command line, use the -c option to specify the custom start command as the following example shows:

$ cf push my-app -c "bundle exec rake VERBOSE=true"

Note: The -c option with a value of ‘null’ forces cf push to use the buildpack start command. See Forcing cf push to use the Buildpack Start Command for more information.

If you override the start command for a Buildpack application, Linux uses bash -c YOUR-COMMAND to invoke your application. If you override the start command for a Docker application, Linux uses sh -c YOUR-COMMAND to invoke your application. Because of this, if you override a start command, you should prefix exec to the final command in your custom composite start command.

An app needs to catch termination signals and clean itself up appropriately. Because of the way that shells manage process trees, the use of custom composite shell commands, particularly those that create child processes using &, &&, ||, etc., can prevent your app from receiving signals that are sent to the top level bash process.

To resolve this issue, you can use exec to replace the bash process with your own process. For example:

  • bin/rake cf:on_first_instance db:migrate && bin/rails server -p $PORT -e $RAILS_ENV The process tree is bash -> ruby, so on graceful shutdown only the bash process receives the TERM signal, not the ruby process.

  • bin/rake cf:on_first_instance db:migrate && exec bin/rails server -p $PORT -e $RAILS_ENV Because of the exec prefix included on the final command, the ruby process invoked by rails takes over the bash process managing the execution of the composite command. The process tree is only ruby, so the ruby web server receives the TERM signal and can shutdown gracefully for 10 seconds.

In more complex situations, like making a custom buildpack, you may want to use bash trap, wait, and backgrounded processes to manage your process tree and shut down apps gracefully. In most situations, however, a well-placed exec should be sufficient.

The disk quota attribute

Use the disk_quota attribute to allocate the disk space for your app instance. This attribute requires a unit of measurement: M, MB, G, or GB, in upper case or lower case.

---
  ...
  disk_quota: 1024M

The command line option that overrides this attribute is -k.

The domain attribute

Every cf push deploys applications to one particular Cloud Foundry instance. Every Cloud Foundry instance may have a shared domain set by an admin. Unless you specify a domain, Cloud Foundry incorporates that shared domain in the route to your application.

You can use the domain attribute when you want your application to be served from a domain other than the default shared domain.

---
  ...
  domain: unique-example.com

The command line option that overrides this attribute is -d.

The domains attribute

Use the domains attribute to provide multiple domains. If you define both domain and domains attributes, Cloud Foundry creates routes for domains defined in both of these fields.

---
  ...
  domains:
  - domain-example1.com
  - domain-example2.org

The command line option that overrides this attribute is -d.

The stack attribute

Use the stack attribute to specify which stack to deploy your application to.

To see a list of available stacks, run cf stacks from the cf cli.

---
  ...
  stack: cflinuxfs2

The command line option that overrides this attribute is -s.

The instances attribute

Use the instances attribute to specify the number of app instances that you want to start upon push:

---
  ...
  instances: 2

We recommend that you run at least two instances of any apps for which fault tolerance matters.

The command line option that overrides this attribute is -i.

The memory attribute

Use the memory attribute to specify the memory limit for all instances of an app. This attribute requires a unit of measurement: M, MB, G, or GB, in upper case or lower case. For example:

---
  ...
  memory: 1024M

The default memory limit is 1G. You might want to specify a smaller limit to conserve quota space if you know that your app instances do not require 1G of memory.

The command line option that overrides this attribute is -m.

The health-check-type attribute

Use the health-check-type attribute to set the health_check_type flag to either port or none. If you do not provide a health-check-type attribute, it defaults to port.

---
  ...
  health-check-type: none

The command line option that overrides this attribute is -u.

The host attribute

Use the host attribute to provide a hostname, or subdomain, in the form of a string. This segment of a route helps to ensure that the route is unique. If you do not provide a hostname, the URL for the app takes the form of APP-NAME.DOMAIN.

---
  ...
  host: my-app

The command line option that overrides this attribute is -n.

The hosts attribute

Use the hosts attribute to provide multiple hostnames, or subdomains. Each hostname generates a unique route for the app. hosts can be used in conjunction with host. If you define both attributes, Cloud Foundry creates routes for hostnames defined in both host and hosts.

---
  ...
  hosts:
  - app_host1
  - app_host2

The command line option that overrides this attribute is -n.

The no-hostname attribute

By default, if you do not provide a hostname, the URL for the app takes the form of APP-NAME.DOMAIN. If you want to override this and map the root domain to this app then you can set no-hostname as true.

---
  ...
  no-hostname: true

The command line option that corresponds to this attribute is --no-hostname.

The routes attribute

Use the routes attribute to provide multiple HTTP and TCP routes. Each route for this app is created if it does not already exist.

This attribute is a combination of push options that include --hostname, -d, and --route-path.

---
  ...
  routes:
  - route: example.com
  - route: www.example.com/foo
  - route: tcp-example.com:1234

Manifest Attributes

The routes attribute cannot be used in conjunction with the following attributes: host, hosts, domain, domains, and no-hostname. An error will result.

Push Flag Options

This attribute has unique interactions with different command line options.

Push Flag Option Resulting Behaviour
--no-route All declared routes are ignored.
-d Overrides DOMAIN part of all declared HTTP and TCP routes.
--hostname, -n Sets or overrides HOSTNAME in all HTTP routes.
It has no impact on TCP routes.
--route-path Sets or overrides the PATH in all HTTP routes.
It has no impact on TCP routes.
--random-route Sets or overrides the HOSTNAME in all HTTP routes.
Sets or overrides the PORT in all TCP routes.
The PORT and HOSTNAME will be randomly generated.

The random-route attribute

Use the random-route attribute to create a URL that includes the app name and random words. Use this attribute to avoid URL collision when pushing the same app to multiple spaces, or to avoid managing app URLs.

The command line option that corresponds to this attribute is --random-route.

---
  ...
  random-route: true

The path attribute

You can use the path attribute to tell Cloud Foundry the directory location where it can find your application.

The directory specified as the path, either as an attribute or as a parameter on the command line, becomes the location where the buildpack Detect script executes.

The command line option that overrides this attribute is -p.

---
  ...
  path: /path/to/application/bits

For more information, see the How cf push Finds the Application topic.

The timeout attribute

The timeout attribute defines the number of seconds that Cloud Foundry allocates for starting your application.

For example:

---
  ...
  timeout: 80

You can increase the timeout length for very large apps that require more time to start. The default timeout is 60 seconds, with an upper bound of 180 seconds.

Note: Administrators can set the upper bound of the maximum_health_check_timeout property to any value. Any changes to Cloud Controller properties in the deployment manifest require running bosh deploy.

The command line option that overrides the timeout attribute is -t.

The no-route attribute

By default, cf push assigns a route to every app. But, some apps process data while running in the background and should not be assigned routes.

You can use the no-route attribute with a value of true to prevent a route from being created for your app.

---
  ...
  no-route: true

The command line option that corresponds to this attribute is --no-route.

In the newer Diego architecture, no-route skips creating and binding a route for the app, but does not specify which type of health check to perform. If your app does not listen on a port because it is a worker or a scheduler app, then it does not satisfy the port-based health check and Cloud Foundry marks it as crashed. To prevent this, disable the port-based health check with cf set-health-check APP_NAME none.

In the older Droplet Execution Agent (DEA) architecture, cf set-health-check APP_NAME none is unnecessary because no-route causes the DEAs to skip the port health-check on app startup.

To remove a route from an existing app, perform the following steps:

  1. Remove the route using the cf unmap-route command.
  2. Push the app again with the no-route: true attribute in the manifest or the --no-route command line option.

For more information, see Describing Multiple Applications with One Manifest below.

Environment Variables

The env block consists of a heading, then one or more environment variable/value pairs.

For example:

---
  ...
  env:
    RAILS_ENV: production
    RACK_ENV: production

cf push deploys the application to a container on the server. The variables belong to the container environment.

While the application is running, you can modify environment variables.

  • View all variables: cf env my-app
  • Set an individual variable: cf set-env my-app my-variable_name my-variable_value
  • Unset an individual variable: cf unset-env my-app my-variable_name my-variable_value

Environment variables interact with manifests in the following ways:

  • When you deploy an application for the first time, Cloud Foundry reads the variables described in the environment block of the manifest and adds them to the environment of the container where the application is staged, and the environment of the container where the application is deployed.

  • When you stop and then restart an application, its environment variables persist.

Services

Applications can bind to services such as databases, messaging, and key-value stores.

Applications are deployed into App Spaces. An application can only bind to services instances that exist in the target App Space before the application is deployed.

The services block consists of a heading, then one or more service instance names.

Whoever creates the service chooses the service instance names. These names can convey logical information, as in backend_queue, describe the nature of the service, as in mysql_5.x, or do neither, as in the example below.

---
  ...
  services:
   - instance_ABC
   - instance_XYZ

Binding to a service instance is a special case of setting an environment variable, namely VCAP_SERVICES. See the Bind a Service section of the Delivering Service Credentials to an Application topic.

Describing Multiple Applications with One Manifest

You can deploy multiple applications with one cf push command by describing them in a single manifest. In doing so, you need to pay extra attention to directory structure and path lines in the manifest.

Suppose you want to deploy two applications called respectively spark and flame, and you want Cloud Foundry to create and start spark before flame. You accomplish this by listing spark first in the manifest.

In this situation there are two sets of bits that you want to push. Let’s say that they are spark.rb in the spark directory and flame.rb in the flame directory. One level up, the fireplace directory contains the spark and the flame directories along with the manifest.yml file. Your plan is to run the cf CLI from the fireplace directory, where you know it can find the manifest.

Now that you have changed the directory structure and manifest location, cf push can no longer find your applications by its default behavior of looking in the current working directory. How can you ensure that cf push finds the bits you want to push?

The answer is to add a path line to each application description to lead cf push to the correct bits. Assume that cf push is run from the fireplace directory.

For spark:

---
  ...
  path: ./spark/

For flame:

---
  ...
  path: ./flame/

The manifest now consists of two applications blocks.

---
# this manifest deploys two applications
# apps are in flame and spark directories
# flame and spark are in fireplace
# cf push should be run from fireplace
applications:
- name: spark
  memory: 1G
  instances: 2
  host: flint-99
  domain: shared-domain.example.com
  path: ./spark/
  services:
  - mysql-flint-99
- name: flame
  memory: 1G
  instances: 2
  host: burnin-77
  domain: shared-domain.example.com
  path: ./flame/
  services:
  - redis-burnin-77

Follow these general rules when using a multiple-application manifest:

  • Name and completely describe your applications in the manifest.

  • Use a no-route line in the description of any application that provides background services to another application.

  • Do not provide an application name with cf push.

  • Do not use any command line options with cf push.

There are only two narrow exceptions:

  • If your manifest is not named manifest.yml or not in the current working directory, use the -f command line option.

  • If you want to push a single application rather than all of the applications described in the manifest, provide the desired application name by running cf push my-app.

Minimizing Duplication

In manifests where multiple applications share settings or services, you begin to see content duplicated. While the manifests still work, duplication increases the risk of typographical errors which cause deployment to fail.

The cure for this problem is to “promote” the duplicate content—that is, to move it up above the applications block, where it need appear only once. The promoted content applies to all applications described in the manifest. Note that content in the applications block overrides content above the applications block, if the two conflict.

The manifest becomes shorter, more readable, and more maintainable.

Notice how much content in the manifest below has been promoted in this way.

---
  ...
# all applications use these settings and services
domain: shared-domain.example.com
memory: 1G
instances: 1
services:
- clockwork-mysql
applications:
- name: springtock
  host: tock09876
  path: ./spring-music/build/libs/spring-music.war
- name: springtick
  host: tick09875
  path: ./spring-music/build/libs/spring-music.war

In the next section we carry this principle further by distributing content across multiple manifests.

Multiple Manifests with Inheritance

A single manifest can describe multiple applications. Another powerful technique is to create multiple manifests with inheritance. Here, manifests have parent-child relationships such that children inherit descriptions from a parent. Children can use inherited descriptions as-is, extend them, or override them.

Content in the child manifest overrides content in the parent manifest, if the two conflict.

This technique helps in these and other scenarios:

  • An application has a set of different deployment modes, such as debug, local, and public. Each deployment mode is described in child manifests that extend the settings in a base parent manifest.

  • An application is packaged with a basic configuration described by a parent manifest. Users can extend the basic configuration by creating child manifests that add new properties or override those in the parent manifest.

The benefits of multiple manifests with inheritance are similar to those of minimizing duplicated content within single manifests. With inheritance, though, we “promote” content by placing it in the parent manifest.

Every child manifest must contain an “inherit” line that points to the parent manifest. Place the inherit line immediately after the three dashes at the top of the child manifest. For example, every child of a parent manifest called base-manifest.yml begins like this:

---
inherit: base-manifest.yml
...

You do not need to add anything to the parent manifest.

In the simple example below, a parent manifest gives each application minimal resources, while a production child manifest scales them up.

simple-base-manifest.yml

---
path: .
domain: shared-domain.example.com
memory: 256M
instances: 1
services:
- singular-backend

# app-specific configuration
applications:
 - name: springtock
   host: 765shower
   path: ./april/build/libs/april-weather.war
 - name: wintertick
   host: 321flurry
   path: ./december/target/december-weather.war

simple-prod-manifest.yml

---
inherit: simple-base-manifest.yml
applications:
 - name:springstorm
   memory: 512M
   instances: 1
   host: 765deluge
   path: ./april/build/libs/april-weather.war
 - name: winterblast
   memory: 1G
   instances: 2
   host: 321blizzard
   path: ./december/target/december-weather.war

Note: Inheritance can add an additional level of complexity to manifest creation and maintenance. Comments that precisely explain how the child manifest extends or overrides the descriptions in the parent manifest can alleviate this complexity.

View the source for this page in GitHub