Customizing the Cloud Foundry Deployment Manifest for Azure

This topic describes how to customize the Cloud Foundry deployment manifest for Azure.

To create a Cloud Foundry manifest, you must perform the following steps:

  1. Prepare workspace environment, download cf-release and install spiff.
  2. Use the BOSH CLI to retrieve your BOSH Director UUID, which you use to customize your manifest stub.
  3. Create a manifest stub in YAML format.
  4. Use spiff to merge the manifest stub with Azure IaaS configuration and other configuration files in the cf-release repository to generate your deployment manifest.

Step 1: Preparation

  1. Prepare a work environment

    $ mkdir workspace
    $ cd workspace
    $ export WORK_DIR=$(pwd)
    
  2. Clone the cf-release GitHub repository. Use git clone to clone the latest Cloud Foundry configuration files onto your machine.

    $ git clone https://github.com/cloudfoundry/cf-release.git
    
  3. From the cf-release directory, run the update script to fetch all the submodules.

    $ ${WORK_DIR}/cf-release/scripts/update
    
  4. Install spiff, a command line tool for generating deployment manifests.

Step 2: Retrieve Your BOSH Director UUID

To perform these procedures, you must have installed the BOSH CLI.

  1. Use the bosh target command with the address of your BOSH Director to connect to the BOSH Director. Log in with the default user name and password, admin and admin, or use the username and password that you set when you installed BOSH.
    $ bosh target https://bosh.my-domain.example.com
    Target set to `bosh'
    Your username: admin
    Enter password: *****
    Logged in as 'admin'
    
  2. Use the bosh status --uuid command to view information about your BOSH deployment. Record the UUID of the BOSH Director. You use the UUID when customizing the Cloud Foundry deployment manifest stub.
    $ bosh status --uuid
    abcdef12-3456-7890-abcd-ef1234567890
    

Step 3: Create Your Manifest Stub

To create your manifest stub, download the example manifest stub for Azure from GitHub, then customize it for your deployment.

Download the Example Manifest Stub

Download the example manifest stub for Azure from GitHub.

Customize the Manifest

Follow the instructions in this table to edit the manifest stub for your deployment.

Deployment Manifest Stub Contents Editing Instructions

director_uuid: DIRECTOR-UUID
    
Replace DIRECTOR-UUID with the BOSH Director UUID. Run the BOSH CLI command bosh status --uuid to view the BOSH Director UUID.

meta:
  environment: ENVIRONMENT
    
Replace ENVIRONMENT with a name of your choice that describes your environment, for example azure-prod.

  reserved_static_ips:
  - CLOUD-FOUNDRY-PUBLIC-IP
    
Replace CLOUD-FOUNDRY-PUBLIC-IP with an existing public IP address for your Azure network. This is assigned to the ha_proxy job to receive incoming traffic.

networks:
  - name: reserved
    type: vip

  - name: cf1
    type: manual
    subnets:
    - range: 10.0.16.0/20
      gateway: 10.0.16.1
      dns:
      - 168.63.129.16
      reserved:
      - 10.0.16.2 - 10.0.16.3
      static:
      - 10.0.16.4 - 10.0.16.100
      cloud_properties:
        virtual_network_name: VNET-NAME
        subnet_name: SUBNET-NAME-FOR-CLOUD-FOUNDRY
        security_group: NSG-NAME-FOR-CLOUD-FOUNDRY
    
Update the values for range, gateway, reserved, and static to reflect the available networks and IP addresses in your Azure network. Replace VNET-NAME with the virtual network name, SUBNET-NAME-FOR-CLOUD-FOUNDRY with the subnet name, and NSG-NAME-FOR-CLOUD-FOUNDRY with the security group name of your Azure network.

properties:
  system_domain: SYSTEM-DOMAIN
  system_domain_organization: SYSTEM-DOMAIN-ORGANIZATION
    
Replace SYSTEM-DOMAIN with the full domain you want associated with applications pushed to your Cloud Foundry installation. You must have already acquired these domains and configured their DNS records so that these domains resolve to your load balancer. For testing, you can use CLOUD-FOUNDRY-PUBLIC-IP.xip.io if you do not have a domain available. xip.io is not stable, and can sometimes fail to resolve to related IP addresses.

Replace SYSTEM-DOMAIN-ORGANIZATION with your organization name. This organization will be created and configured to own the SYSTEM-DOMAIN.

  app_ssh:
    host_key_fingerprint: HOST-KEY-FINGERPRINT
    oauth_client_id: ssh-proxy
    
Replace HOST-KEY-FINGERPRINT with the host key fingerprint. Run the following commands to generate the fingerprint:
$ ssh-keygen -N "" -f ssh-proxy-host-key.pem > /dev/null
$ ssh-keygen -lf ssh-proxy-host-key.pem.pub | cut -d ' ' -f2

  ssl:
    skip_cert_verify: true
    
Set skip_cert_verify to true to skip SSL certificate verification if you do not use a real domain and certificate. If you want to use SSL certificates to secure traffic into your deployment, see the Securing Traffic into Cloud Foundry topic.

  cc:
    staging_upload_user: STAGING-UPLOAD-USER
    staging_upload_password: STAGING-UPLOAD-PASSWORD
    bulk_api_password: BULK-API-PASSWORD
    db_encryption_key: DB-ENCRYPTION-KEY
    uaa_skip_ssl_validation: true
    tls_port: CC-MUTUAL-TLS-PORT
    mutual_tls:
      ca_cert: CC-MUTUAL-TLS-CA-CERT
      public_cert: CC-MUTUAL-TLS-PUBLIC-CERT
      private_key: CC-MUTUAL-TLS-PRIVATE-KEY
    
The Cloud Controller API endpoint requires basic authentication. Replace STAGING-UPLOAD-USER and STAGING-UPLOAD-PASSWORD with a username and password of your choosing.

Replace BULK-API-PASSWORD with a password of your choosing. The password cannot contain characters that are not allowed for basic authentication unless you encode the characters. Health Manager uses this password to access the Cloud Controller bulk API.

Replace DB-ENCRYPTION-KEY with a secure key that you generate to encrypt sensitive values in the Cloud Controller database. You can use any random string. For example, run the following command from a command line to generate a 32-character random string: LC_ALL=C tr -dc 'A-Za-z0-9' < /dev/urandom | head -c 32 ; echo

Replace CC-MUTUAL-TLS-PORT with a tls port, for example: 9023

To generate the certificates and keys for the mutual_tls section, you need the generate-cf-diego-certs script from the cf-release repository. Run the generate-cf-diego-certs script to generate the certificates and keys for Cloud Controller.
For example, run the following in a terminal window:
$ ./scripts/generate-cf-diego-certs
This script outputs a directory named cf-diego-certs that contains a set of files with the certificates and keys you need.
In the stub, replace… with the contents of this file…
CC-MUTUAL-TLS-CA-CERT cf-diego-ca.crt
CC-MUTUAL-TLS-PUBLIC-CERT cloud_controller.crt
CC-MUTUAL-TLS-PRIVATE-KEY cloud_controller.key

  blobstore:
    admin_users:
      - username: BLOBSTORE-USERNAME
        password: BLOBSTORE-PASSWORD
    secure_link:
        secret: BLOBSTORE-SECRET
    tls:
        port: 443
        cert: BLOBSTORE-TLS-CERT
        private_key: BLOBSTORE-PRIVATE-KEY
        ca_cert: BLOBSTORE-CA-CERT
    
Replace BLOBSTORE-USERNAME and BLOBSTORE-PASSWORD with a username and password of your choosing.

Replace BLOBSTORE-SECRET with a secure secret of your choosing.

Replace BLOBSTORE-TLS-CERT, BLOBSTORE-PRIVATE-KEY, and BLOBSTORE-CA-CERT with the blobstore TLS certificate, private key, and CA certificate. You can use the scripts/generate-blobstore-certs script in the cf-release repository to generate self-signed certificates.

  consul:
    encrypt_keys:
      - CONSUL-ENCRYPT-KEY
    ca_cert: CONSUL-CA-CERT
    server_cert: CONSUL-SERVER-CERT
    server_key: CONSUL-SERVER-KEY
    agent_cert: CONSUL-AGENT-CERT
    agent_key: CONSUL-AGENT-KEY
    
See the Security Configuration for Consul topic. You can use the scripts/generate-consul-certs script in the cf-release repository to generate self-signed certificates.

  etcd:
    require_ssl: true
    ca_cert: ETCD-CA-CERT
    client_cert: ETCD-CLIENT-CERT
    client_key: ETCD-CLIENT-KEY
    peer_ca_cert: ETCD-PEER-CA-CERT
    peer_cert: ETCD-PEER-CERT
    peer_key: ETCD-PEER-KEY
    server_cert: ETCD-SERVER-CERT
    server_key: ETCD-SERVER-KEY
    
Generate SSL certs for etcd and replace these values. You can use the scripts/generate-etcd-certs script in the cf-release repository to generate self signed certs.

  loggregator:
    tls:
      ca_cert: LOGGREGATOR-CA-CERT
      doppler:
        cert: DOPPLER-CERT
        key: DOPPLER-KEY
      metron:
        cert: METRON-CERT
        key: METRON-KEY
      trafficcontroller:
        cert: TRAFFICCONTROLLER-CERT
        key: TRAFFICCONTROLLER-KEY
      cc_trafficcontroller:
        cert: CCTRAFFICCONTROLLER-CERT
        key: CCTRAFFICCONTROLLER-KEY
      syslogdrainbinder:
        cert: SYSLOGDRAINBINDER-CERT
        key: SYSLOGDRAINBINDER-KEY
      statsd_injector:
        cert: STATSDINJECTOR-CERT
        key: STATSDINJECTOR-KEY
    
To generate the certificates and keys for the Loggregator components, you need the following: Generate certs
For example, run the following in a terminal window:
$  ./scripts/generate-loggregator-certs cf-diego-certs/cf-diego-ca.crt cf-diego-certs/cf-diego-ca.key
$  ./scripts/generate-statsd-injector-certs loggregator-certs/loggregator-ca.crt loggregator-certs/loggregator-ca.key
This script outputs directories named loggregator-certs and statsd-injector-certs that contain a set of files with the certificates and keys you need for Loggregator.
In the stub, replace… with the contents of this file…
LOGGREGATOR-CA-CERT loggregator-ca.crt
DOPPLER-CERT doppler.crt
DOPPLER-KEY doppler.key
TRAFFICCONTROLLER-CERT trafficcontroller.crt
TRAFFICCONTROLLER-KEY trafficontroller.key
METRON-CERT metron.crt
METRON-KEY metron.key
CCTRAFFICCONTROLLER-CERT cc_trafficcontroller.crt
CCTRAFFICCONTROLLER-KEY cc_trafficcontroller.key
SYSLOGDRAINBINDER-CERT syslogdrainbinder.crt
SYSLOGDRAINBINDER-KEY syslogdrainbinder.key
STATSDINJECTOR-CERT statsdinjector.crt
STATSDINJECTOR-KEY statsdinjector.key

  loggregator_endpoint:
    shared_secret: LOGGREGATOR-ENDPOINT-SHARED-SECRET
    
Replace LOGGREGATOR-ENDPOINT-SHARED-SECRET with a secure string that you generate.

  nats:
    user: NATS-USER
    password: NATS-PASSWORD
    
Replace NATS-USER and NATS-PASSWORD with a username and secure password of your choosing. Cloud Foundry components use these credentials to communicate with each other over the NATS message bus.

  router:
    status:
      user: ROUTER-USER
      password: ROUTER-PASSWORD
    
Replace ROUTER-USER and ROUTER-PASSWORD with a username and secure password of your choosing.

  uaa:
    admin:
      client_secret: ADMIN-SECRET
    cc:
      client_secret: CC-CLIENT-SECRET
    clients:
      cc_service_key_client:
        secret: CC-SERVICE-KEY-CLIENT-SECRET
      cc_routing:
        secret: CC-ROUTING-SECRET
      cloud_controller_username_lookup:
        secret: CLOUD-CONTROLLER-USERNAME-LOOKUP-SECRET
      doppler:
        secret: DOPPLER-SECRET
      gorouter:
        secret: GOROUTER-SECRET
      tcp_emitter:
        secret: TCP-EMITTER-SECRET
      tcp_router:
        secret: TCP-ROUTER-SECRET
      login:
        secret: LOGIN-CLIENT-SECRET
      notifications:
        secret: NOTIFICATIONS-CLIENT-SECRET
      cc-service-dashboards:
        secret: CC-SERVICE-DASHBOARDS-SECRET
    
Replace the values for all secret keys with secure secrets that you generate.

You can configure container-to-container networking in this section. If you want to deploy container-to-container networking, see the Enable on an IaaS section of the Administering Container-to-Container Networking topic, beginning with step 4.

    jwt:
      verification_key: JWT-VERIFICATION-KEY
      signing_key: JWT-SIGNING-KEY
    
Generate a PEM-encoded RSA key pair. Replace JWT-SIGNING-KEY with the private key, and JWT-VERIFICATION-KEY with the corresponding public key. Run openssl genrsa -out jwt-key.pem 2048 && openssl rsa -pubout -in jwt-key.pem -out jwt-key.pem.pub to generate a key pair. This command creates jwt-key.pem.pub, which contains your public key, and jwt-key.pem, which contains your private key.
Copy in the full keys, including the BEGIN and END delimiter lines.

    scim:
      users:
      - name: admin
        password: ADMIN-PASSWORD
        groups:
        - scim.write
        - scim.read
        - openid
        - cloud_controller.admin
        - doppler.firehose
    
Generate a secure password and replace ADMIN-PASSWORD with that value to set the password for the Admin user of your Cloud Foundry installation.

    ca_cert: UAA-CA-CERT
    sslCertificate: UAA-SERVER-CERT
    sslPrivateKey: UAA-SERVER-KEY
    
Replace UAA-CA-CERT, UAA-SERVER-CERT, and UAA-SERVER-KEY with UAA certificates. You can use the scripts/generate-uaa-certs script in the cf-release repository to generate self-signed certificates.

  ccdb:
    roles:
    - name: ccadmin
      password: CCDB-PASSWORD
  uaadb:
    roles:
      - name: uaaadmin
        password: UAADB-PASSWORD
  databases:
    roles:
    - name: ccadmin
      password: CCDB-PASSWORD
    - name: uaaadmin
      password: UAADB-PASSWORD
    - name: diego
      password: DIEGODB-PASSWORD
    
Replace CCDB-PASSWORD, UAADB-PASSWORD, and DIEGODB-PASSWORD with secure passwords of your choosing.

  login:
    saml:
      serviceProviderKey: SAML-KEY
      serviceProviderKeyPassword: ''
      serviceProviderCertificate: SAML-CERT
    
Generate a PEM-encoded RSA key pair. Run openssl req -x509 -nodes -newkey rsa:2048 -days 365 -keyout key.pem -out cert.pem to generate a key pair. This command creates cert.pem, which contains your public key, and key.pem, which contains your private key. Replace SERVICE-PROVIDER-PRIVATE-KEY with the full private key, including the BEGIN and END delimiter lines, under serviceProviderKey.
For RSA keys, you only need to configure the private key.

jobs:
  - name: ha_proxy_z1
    networks:
      - name: cf1
        default:
        - dns
        - gateway
    properties:
      ha_proxy:
        ssl_pem: |
          -----BEGIN RSA PRIVATE KEY-----
          RSA-PRIVATE-KEY
          -----END RSA PRIVATE KEY-----
          -----BEGIN CERTIFICATE-----
          SSL-CERTIFICATE-SIGNED-BY-PRIVATE-KEY
          -----END CERTIFICATE-----
    
Replace RSA-PRIVATE-KEY and SSL-CERTIFICATE-SIGNED-BY-PRIVATE-KEY with the PEM-encoded private key and certificate associated with the system domain and apps domains that you configured to terminate at the floating IP address associated with the ha_proxy job. For self-signed certificates, you can run openssl genrsa -out haproxy_ssl.key 2048 to generate a key, and run openssl req -new -x509 -days 365 -key haproxy_ssl.key -out haproxy_ssl.cert to generate the certificate.

Note: You can configure blacklists of IP address ranges to prevent future apps deployed to your Cloud Foundry installation from attempting to drain syslogs to internal Cloud Foundry components. See the Log Drain Blacklist Configuration topic for more information.

Step 4: Generate Your Manifest

  1. Download Azure infrastructure stub cf-infrastructure-azure.yml for Cloud Foundry from this link. You can also set IaaS configuration by changing this file, for example changing instance_type.
  2. Run the following command from the cf-release directory to create a deployment manifest named cf-deployment.yml:

    $ spiff merge \
      "${WORK_DIR}/cf-release/templates/generic-manifest-mask.yml" \
      "${WORK_DIR}/cf-release/templates/cf.yml" \
      "${WORK_DIR}/cf-infrastructure-azure.yml" \
      "${WORK_DIR}/cf-stub.yml" \
      > "${WORK_DIR}/cf-deployment.yml"
    

  3. Use the bosh deployment command to set your deployment to the generated manifest:

    $ bosh deployment ${WORK_DIR}/cf-deployment.yml
    

You are now ready to deploy Cloud Foundry. See the Deploying Cloud Foundry topic for instructions.

Create a pull request or raise an issue on the source for this page in GitHub