How Apps Are Staged

This topic describes how Diego stages buildpack apps and Docker images in Cloud Foundry.

Overview

Cloud Foundry uses Diego to manage app containers. It is a self-healing system that attempts to keep the correct number of instances running in Diego Cells to avoid network failures and crashes. For more information about Diego, see Diego Components and Architecture.

This topic references tasks and long-running processes (LRPs). For more information about these, see Tasks and Long-Running Processes.

To better understand the flow and caching of source bits during staging, see Cloud Controller Blobstore.

How Diego Stages Buildpack Apps

This section describes how Diego stages buildpack apps.

The following diagram illustrates the steps and components involved in the process of staging a buildpack app.

The staging process for buildpack apps includes a developer and the following components: CF Command Line, Cloud Controller (CCNG), CCNG Blobstore, CCDB, Diego Cell (Staging), and Diego Cell (Running). Step 1 is CF Push from Developer to CF Command Line. Step 2 is Create App from CF Command Line to Cloud Controller (CCNG). Step 3 is Stores App Metadata from CCNG to the CCDB. Step 4 is Upload App Files from the CF Command Line to CCNG. Step 5 is Store App Files from CCNG to CCNG Blobstore. Step 6 is App Start from CF Command Line to CCNG. Step 7 is Stage App from CCNG to Diego Cell (Staging). Step 8 is Stream Staging Output from Diego Cell (Staging) to the Developer. Step 9 is Store App Droplet from Diego Cell (Staging) to CCNG Blobstore. Step 10 is Report Staging Complete from Diego Cell (Staging) to CCNG. Step 11 is Start Staged App from CCNG to Diego Cell (Running). Step 12 is Report App Status from Diego Cell (Running) to CCNG.

For a description of each step in the process of staging a buildpack app, see the following:

  1. A developer runs cf push.

  2. The Cloud Foundry Command Line Interface (cf CLI) tells the Cloud Controller to create a record for the app. See the Cloud Controller topic for more information about the Cloud Controller.

  3. The Cloud Controller stores the app metadata. App metadata can include the app name, number of instances, buildpack, and other information about the app.

  4. This step includes the following:

    1. The cf CLI requests a resource match from the Cloud Controller.
    2. The cf CLI uploads the app source files, omitting any app files that already exist in the resource cache.
    3. The Cloud Controller combines the uploaded app files with files from the resource cache to create the app package.

  5. The Cloud Controller stores the app package in the blobstore. For more information, see the Blobstore section in the Cloud Foundry Components topic.

  6. The cf CLI issues a request to start the app.

  7. This step includes the following:

    1. The Cloud Controller issues a staging request to Diego.
    2. Diego schedules a Diego Cell to run the staging task.
    3. The task downloads buildpacks and the app buildpack cache, if present.
    4. The task uses the buildpack to compile and stage the app.

  8. The Diego Cell streams the output of the staging process. A developer may need to view the output to troubleshoot staging problems.

  9. This step includes the following:

    1. The task creates a tarball, or droplet, with the compiled and staged app.
    2. The Diego Cell stores the droplet in the blobstore.
    3. The task uploads the buildpack cache to the blobstore for use the next time the app is staged.

  10. The Diego Bulletin Board System (BBS) reports to the Cloud Controller that staging is complete. If staging does not complete within 15 minutes, it fails.

  11. Diego schedules the app as a LRP on one or more Diego Cells.

  12. The Diego Cells report the status of the app to the Cloud Controller.

How Diego Stages Docker Images

This section describes how Diego stages Docker images.

The following diagram illustrates the steps and components involved in the process of staging a Docker image.

The staging process for Docker images includes a developer and the following components: CF Command Line, Cloud Controller (CCNG), CCDB, Diego Cell (Staging), and Diego Cell (Running). Step 1 is CF Push from Developer to CF Command Line. Step 2 is Create Record from CF Command Line to CCNG. Step 3 is Stage Image from CCNG to CCDB. Step 4 is Stream Staging Output from Diego Cell (Staging) to Developer. Step 5 is Fetch Metadata from Diego Cell (Staging) to CCNG. Step 6 is Store Metadata from CCNG to CCDB. Step 7 is Submit LRP an Run Start Command from CCNG to Diego Cell (Running)

For a description of each step in the process of staging a Docker image, see the following:

  1. A developer runs cf push and includes the name of a Docker image in an accessible Docker Registry.

  2. The cf CLI tells the Cloud Controller to create a record for the Docker image.

  3. This step includes the following:

    1. The Cloud Controller issues a staging request to Diego.
    2. Diego schedules a Diego Cell to run the task.

  4. The Diego Cell streams the output of the staging process. A developer may need to view the output to troubleshoot staging problems.

  5. The task fetches the metadata associated with the Docker image and returns a portion of it to the Cloud Controller.

  6. The Cloud Controller stores the metadata in the Cloud Controller database (CCDB).

  7. This step includes the following:

    1. The Cloud Controller uses the Docker image metadata to construct a LRP that runs the start command specified in the Dockerfile.
    2. The Cloud Controller submits the LRP to Diego.
    3. Diego schedules the LRP on one or more Diego Cells.
    4. The Cloud Controller instructs Diego and the Gorouter to route traffic to the Docker image.

    Note: The Cloud Controller takes into account any user-specified overrides specified in the Dockerfile, such as environment variables.

Create a pull request or raise an issue on the source for this page in GitHub