Cloud Controller Blobstore Configuration

Page last updated:

This topic describes the options for configuring a blobstore for the Cloud Controller.

Cloud Controller has four types of objects that need to be stored in a blobstore: buildpacks, droplets, packages, and resource_pool. By default, the blobstore configuration uses the Fog Ruby gem, but can also use the WebDAV protocol.

This document describes five common blobstore configurations:

Fog with AWS Credentials

To use Fog blobstores with AWS credentials, perform the steps below.

  1. Insert the following configuration into your manifest under properties.cc:

    cc:
      buildpacks:
        blobstore_type: fog
        buildpack_directory_key: YOUR-AWS-BUILDPACK-BUCKET
        fog_connection: &fog_connection
          aws_access_key_id: AWS_ACCESS_KEY
          aws_secret_access_key: AWS_SECRET_ACCESS_KEY
          provider: AWS
          region: us-east-1
      droplets:
        blobstore_type: fog
        droplet_directory_key: YOUR-AWS-DROPLET-BUCKET
        fog_connection: *fog_connection
      packages:
        blobstore_type: fog
        app_package_directory_key: YOUR-AWS-PACKAGE-BUCKET
        fog_connection: *fog_connection
      resource_pool:
        blobstore_type: fog
        resource_directory_key: YOUR-AWS-RESOURCE-BUCKET
        fog_connection: *fog_connection
    
  2. Replace AWS_ACCESS_KEY and AWS_SECRET_ACCESS_KEY with your AWS credentials.

  3. Replace YOUR-AWS-BUILDPACK-BUCKET, YOUR-AWS-DROPLET-BUCKET, YOUR-AWS-PACKAGE-BUCKET, and YOUR-AWS-RESOURCE-BUCKET with the names of your AWS buckets. Do not use periods (.) in your AWS bucket names. In the AWS console, you must assign your credentials an IAM policy that allows all S3 actions on all of these buckets.

  4. You can provide further configuration through the fog_connection hash, which is passed through to the Fog gem.

Fog with AWS Server-Side Encryption

AWS S3 offers Server-Side Encryption at rest. For more information, see Protecting Data Using Server-Side Encryption.

AWS SSE-S3 blobstore encryption

  1. Insert the following configuration into your manifest under properties.cc:

      cc:
        buildpacks:
          blobstore_type: fog
          buildpack_directory_key: YOUR-AWS-BUILDPACK-BUCKET
          fog_connection: &fog_connection
            aws_access_key_id: AWS_ACCESS_KEY
            aws_secret_access_key: AWS_SECRET_ACCESS_KEY
            provider: AWS
            region: us-east-1
          fog_aws_storage_options: &fog_aws_storage_options
            encryption: 'AES256'
        droplets:
          blobstore_type: fog
          droplet_directory_key: YOUR-AWS-DROPLET-BUCKET
          fog_connection: *fog_connection
          fog_aws_storage_options: *fog_aws_storage_options
        packages:
          blobstore_type: fog
          app_package_directory_key: YOUR-AWS-PACKAGE-BUCKET
          fog_connection: *fog_connection
          fog_aws_storage_options: *fog_aws_storage_options
        resource_pool:
          blobstore_type: fog
          resource_directory_key: YOUR-AWS-RESOURCE-BUCKET
          fog_connection: *fog_connection
          fog_aws_storage_options: *fog_aws_storage_options
    
  2. Replace AWS_ACCESS_KEY and AWS_SECRET_ACCESS_KEY with your AWS credentials.

  3. Replace YOUR-AWS-BUILDPACK-BUCKET, YOUR-AWS-DROPLET-BUCKET, YOUR-AWS-PACKAGE-BUCKET, and YOUR-AWS-RESOURCE-BUCKET with the names of your AWS buckets. Do not use periods (.) in your AWS bucket names. In the AWS console, you must assign your credentials an IAM policy that allows all S3 actions on all of these buckets.

  4. You can provide further configuration through the fog_connection hash, which is passed through to the Fog gem.

  5. fog_aws_storage_options takes a hash with the key encryption. Operators can set its value to a type of encryption algorithm. In the configuration information above, encryption is set to AES256 to enable AWS SSE-S3 encryption.

  6. You can provide further configuration through the fog_aws_storage_options hash, which is passed through to the Fog gem.

AWS SSE-KMS blobstore encryption

  1. Obtain your KMS Key ID. For information about managing KMS keys, see the AWS Key Management Service Getting Started guide.

  2. Insert the following configuration into your manifest under properties.cc:

      cc:
        buildpacks:
          blobstore_type: fog
          buildpack_directory_key: YOUR-AWS-BUILDPACK-BUCKET
          fog_connection: &fog_connection
            aws_access_key_id: AWS_ACCESS_KEY
            aws_secret_access_key: AWS_SECRET_ACCESS_KEY
            provider: AWS
            region: us-east-1
          fog_aws_storage_options: &fog_aws_storage_options
            encryption: 'aws:kms'
            x-amz-server-side-encryption-aws-kms-key-id: "YOUR-AWS-KMS-KEY-ID"
        droplets:
          blobstore_type: fog
          droplet_directory_key: YOUR-AWS-DROPLET-BUCKET
          fog_connection: *fog_connection
          fog_aws_storage_options: *fog_aws_storage_options
        packages:
          blobstore_type: fog
          app_package_directory_key: YOUR-AWS-PACKAGE-BUCKET
          fog_connection: *fog_connection
          fog_aws_storage_options: *fog_aws_storage_options
        resource_pool:
          blobstore_type: fog
          resource_directory_key: YOUR-AWS-RESOURCE-BUCKET
          fog_connection: *fog_connection
          fog_aws_storage_options: *fog_aws_storage_options
    
  3. Replace AWS_ACCESS_KEY and AWS_SECRET_ACCESS_KEY with your AWS credentials.

  4. Replace YOUR-AWS-BUILDPACK-BUCKET, YOUR-AWS-DROPLET-BUCKET, YOUR-AWS-PACKAGE-BUCKET, and YOUR-AWS-RESOURCE-BUCKET with the names of your AWS buckets. Do not use periods (.) in your AWS bucket names. In the AWS console, you must assign your credentials an IAM policy that allows all S3 actions on all of these buckets.

  5. You can provide further configuration through the fog_connection hash, which is passed through to the Fog gem.

  6. Replace YOUR-AWS-KMS-KEY-ID with your KMS Key ID.

  7. fog_aws_storage_options takes a hash with the key encryption. Operators can set its value to a type of encryption algorithm. In the configuration information above, encryption is set to aws:kms to enable AWS SSE-KMS encryption.

  8. You can provide further configuration through the fog_aws_storage_options hash, which is passed through to the Fog gem.

Fog with AWS IAM Instance Profiles

To configure Fog blobstores to use AWS IAM Instance Profiles, perform the steps below.

  1. Insert the following configuration into your manifest under properties.cc:

    cc:
      buildpacks:
        blobstore_type: fog
        buildpack_directory_key: YOUR-AWS-BUILDPACK-BUCKET
        fog_connection: &fog_connection
          provider: AWS
          region: us-east-1
          use_iam_profile: true
      droplets:
        blobstore_type: fog
        droplet_directory_key: YOUR-AWS-DROPLET-BUCKET
        fog_connection: *fog_connection
      packages:
        blobstore_type: fog
        app_package_directory_key: YOUR-AWS-PACKAGE-BUCKET
        fog_connection: *fog_connection
      resource_pool:
        blobstore_type: fog
        resource_directory_key: YOUR-AWS-RESOURCE-BUCKET
        fog_connection: *fog_connection
    
  2. Replace YOUR-AWS-BUILDPACK-BUCKET, YOUR-AWS-DROPLET-BUCKET, YOUR-AWS-PACKAGE-BUCKET, and YOUR-AWS-RESOURCE-BUCKET with the names of your AWS buckets. Do not use periods (.) in your AWS bucket names.

  3. To provide further configuration, create a fog_connection hash to pass through to the Fog gem.

  4. Complete the steps documented in the AWS CPI and Director configured with an S3 blobstore instructions section of the BOSH documentation.

  5. In the AWS console, configure an additional cloud-controller IAM role that gives access to both the BOSH S3 buckets and the Cloud Controller S3 buckets specified in your manifest:

    {
      "Version": "2012-10-17",
      "Statement": [{
        "Effect": "Allow",
        "Action": [ "s3:*" ],
        "Resource": [
          "arn:aws:s3:::BOSH_BUCKET_NAME",
          "arn:aws:s3:::BOSH_BUCKET_NAME/*",
          "arn:aws:s3:::YOUR-AWS-BUILDPACK-BUCKET",
          "arn:aws:s3:::YOUR-AWS-BUILDPACK-BUCKET/*",
          "arn:aws:s3:::YOUR-AWS-DROPLET-BUCKET",
          "arn:aws:s3:::YOUR-AWS-DROPLET-BUCKET/*",
          "arn:aws:s3:::YOUR-AWS-PACKAGE-BUCKET",
          "arn:aws:s3:::YOUR-AWS-PACKAGE-BUCKET/*",
          "arn:aws:s3:::YOUR-AWS-RESOURCE-BUCKET",
          "arn:aws:s3:::YOUR-AWS-RESOURCE-BUCKET/*",
        ]
      }]
    }
    

    If you are using the AWS console, an IAM Role is automatically assigned to an IAM Instance Profile with the same name, cloud-controller. If you are not using the AWS console, you must create an IAM Instance Profile with a single assigned IAM Role.

  6. In your Cloud Foundry deployment manifest, configure a resource pool to use the cloud-controller IAM Instance Profile:

    resource_pools:
    - name: cloud-controller-resource-pool
      network: default
      stemcell: { ... }
      cloud_properties:
        instance_type: c3.large
        iam_instance_profile: cloud-controller
    
  7. Assign all Cloud Controller jobs, identified by api_z*, and Cloud Controller Worker jobs, identified by api_worker_z*, to the cloud-controller-resource-pool.

Fog with Google Cloud Storage

To configure your blobstores to use Google Cloud Storage Interoperability credentials, perform the steps below.

  1. Insert the following configuration into your manifest under properties.cc:

    cc:
      buildpacks:
        blobstore_type: fog
        buildpack_directory_key: YOUR-GCS-BUILDPACK-BUCKET
        fog_connection: &fog_connection
          provider: Google
          google_storage_access_key_id: GCS_ACCESS_KEY
          google_storage_secret_access_key: GCS_SECRET_ACCESS_KEY
      droplets:
        blobstore_type: fog
        droplet_directory_key: YOUR-GCS-DROPLET-BUCKET
        fog_connection: *fog_connection
      packages:
        blobstore_type: fog
        app_package_directory_key: YOUR-GCS-PACKAGE-BUCKET
        fog_connection: *fog_connection
      resource_pool:
        blobstore_type: fog
        resource_directory_key: YOUR-GCS-RESOURCE-BUCKET
    
  2. Create an Interoperability key using the GCP instructions.

  3. Replace GCS_ACCESS_KEY and GCS_SECRET_ACCESS_KEY with your Cloud Storage credentials.

  4. Replace YOUR-GCS-BUILDPACK-BUCKET, YOUR-GCS-DROPLET-BUCKET, YOUR-GCS-PACKAGE-BUCKET, and YOUR-GCS-RESOURCE-BUCKET with the names of your Cloud Storage buckets. Do not use periods (.) in your bucket names.

  5. You can provide further configuration through the fog_connection hash, which is passed through to the Fog gem.

Fog with Azure Storage

To configure your blobstores to use Azure Storage credentials, perform the steps below.

  1. Insert the following configuration into your manifest under properties.cc:

    cc:
      buildpacks:
        blobstore_type: fog
        buildpack_directory_key: YOUR-AZURE-BUILDPACK-CONTAINER
        fog_connection: &fog_connection
          provider: AzureRM
          environment: AzureCloud
          azure_storage_account_name: YOUR-AZURE-STORAGE-ACCOUNT-NAME
          azure_storage_access_key: YOUR-AZURE-STORAGE-ACCESS-KEY
      droplets:
        blobstore_type: fog
        droplet_directory_key: YOUR-AZURE-DROPLET-CONTAINER
        fog_connection: *fog_connection
      packages:
        blobstore_type: fog
        app_package_directory_key: YOUR-AZURE-PACKAGE-CONTAINER
        fog_connection: *fog_connection
      resource_pool:
        blobstore_type: fog
        resource_directory_key: YOUR-AZURE-RESOURCE-CONTAINER
        fog_connection: *fog_connection
    
  2. Replace YOUR-AZURE-STORAGE-ACCOUNT-NAME and YOUR-AZURE-STORAGE-ACCESS-KEY with your Azure Storage credentials.

  3. Replace YOUR-AZURE-BUILDPACK-CONTAINER, YOUR-AZURE-DROPLET-CONTAINER, YOUR-AZURE-PACKAGE-CONTAINER, and YOUR-AZURE-RESOURCE-CONTAINER with the names of your Cloud Storage containers.

  4. You can provide further configuration through the fog_connection hash, which is passed through to the Fog gem.

WebDAV

To configure your blobstores to use the WebDAV protocol, perform the steps below.

  1. Ensure your deployment manifest has a single instance of the blobstore job. For a working example, see the example bosh-lite manifest.

  2. Insert the following configuration into your manifest under properties.blobstore and properties.cc:

    blobstore:
      admin_users:
      - password: WEBDAV_BASIC_AUTH_PASSWORD
        username: WEBDAV_BASIC_AUTH_USER
      port: 8080
      secure_link:
        secret: WEBDAV_SECRET
      tls:
        cert: WEBDAV_CERT
        port: 4443
        private_key: WEBDAV_PRIVATE_KEY
        ca_cert: WEBDAV_CA_CERT_BUNDLE
    cc:
      buildpacks:
        blobstore_type: webdav
        buildpack_directory_key: cc-buildpacks
        webdav_config: &webdav_config
          public_endpoint: WEBDAV_PUBLIC_ENDPOINT
          private_endpoint: WEBDAV_PRIVATE_ENDPOINT
          username: WEBDAV_BASIC_AUTH_USER
          password: WEBDAV_BASIC_AUTH_PASSWORD
          ca_cert: WEBDAV_CA_CERT_BUNDLE
      droplets:
        blobstore_type: webdav
        droplet_directory_key: cc-droplets
        webdav_config: *webdav_config
      packages:
        blobstore_type: webdav
        app_package_directory_key: cc-packages
        webdav_config: *webdav_config
      resource_pool:
        blobstore_type: webdav
        resource_directory_key: cc-resources
        webdav_config: *webdav_config
    
  3. Configure your WebDAV blobstores by performing the following steps:

    • Replace WEBDAV_BASIC_AUTH_USER and WEBDAV_BASIC_AUTH_PASSWORD with Basic AUTH credentials that Cloud Controller can use to talk your WebDAV installation.
    • Replace WEBDAV_SECRET with a secret phrase used to sign URLs.
    • Replace WEBDAV_CERT, WEBDAV_PRIVATE_KEY, and WEBDAV_CA_CERT_BUNDLE with proper TLS configuration that are used for the internal blobstore.
    • Replace WEBDAV_PUBLIC_ENDPOINT with the public URL that resolves to your WebDAV installation. For example, https://blobstore.SYSTEM-DOMAIN.example.com.
    • Replace WEBDAV_PRIVATE_ENDPOINT with a routable URL on your internal network. If not set, this defaults to https://blobstore.service.cf.internal:4443.
    • Replace WEBDAV_BASIC_AUTH_USER and WEBDAV_BASIC_AUTH_PASSWORD with Basic AUTH credentials that Cloud Controller can use to communicate with your WebDAV installation.
  4. Create a webdav_config hash to provide further configuration.

View the source for this page in GitHub