Administering CF Networking

This topic describes how to enable and use the CF Networking feature. For an overview of how CF Networking works, see the Understanding CF Networking topic.

Enable CF Networking

This section presents two procedures for enabling CF Networking. See the table below and choose the procedure that corresponds with your use case.

If your deployment runs on … Then follow …
An IaaS, such as AWS or vSphere Enable on an IaaS
BOSH-Lite, a Vagrant box for testing or development Enable on BOSH-Lite

Enable on an IaaS

If your CF deployment runs on an IaaS, follow these steps to enable CF Networking.

  1. Ensure that you have a database ready. To complete this procedure, you need a v5.7 or later MySQL database, or a PostgreSQL database. You can use a database within your Cloud Controller database instance or create a different instance, such as Amazon RDS.
  2. Target your BOSH Director using the BOSH CLI:
    $ bosh target BOSH-DIRECTOR-IP 
  3. If you have not already, upload a stemcell that uses Linux kernel 4.4, such as stemcell 3263.2 or later:
    $ bosh upload stemcell URL-OF-STEMCELL 
  4. Open the CF properties stub that you created when deploying Cloud Foundry and do the following:
    1. Under properties > uaa > scim > users > name: admin > groups add a new group named network.admin.
      scim:
        users:
        - name: admin
          password: 
          groups:
            - scim.write
            - scim.read
            ...
            - routing.router_groups.write
            - network.admin 
    2. Under properties > uaa > clients > cf, to the line beginning scope: add network.admin.
    3. clients:
        cf:
          scope: cloud_controller.read, [...] routing.router_groups.read,network.admin
    4. Under clients, add a network-policy client as follows and replace REPLACE_WITH_UAA_CLIENT_SECRET with a secure password of your choosing.
      clients:
        cf:
          scope: cloud_controller.read, [...] routing.router_groups.read,network.admin
        network-policy:
          authorities: uaa.resource,cloud_controller.admin_read_only
          authorized-grant-types: client_credentials,refresh_token
          secret: REPLACE_WITH_UAA_CLIENT_SECRET
  5. Create a CF Networking stub file stubs/cf-networking/stub.yml and paste in the template below:
    ---
    cf_networking_overrides:
      releases:
      - name: cf-networking
        version: latest
      driver_templates:
      - name: garden-cni
        release: cf-networking
      - name: cni-flannel
        release: cf-networking
      - name: netmon
        release: cf-networking
      - name: vxlan-policy-agent
        release: cf-networking
      properties:
        cf_networking:
          vxlan_policy_agent:
            policy_server_url: https://policy-server.service.cf.internal:4003
            ca_cert: |
              -----BEGIN CERTIFICATE-----
              REPLACE_WITH_CA_CERT
              -----END CERTIFICATE-----
            client_cert: |
              -----BEGIN CERTIFICATE-----
              REPLACE_WITH_CLIENT_CERT
              -----END CERTIFICATE-----
            client_key: |
              -----BEGIN RSA PRIVATE KEY-----
              REPLACE_WITH_CLIENT_KEY
              -----END RSA PRIVATE KEY-----
          garden_external_networker:
            cni_plugin_dir: /var/vcap/packages/flannel/bin
            cni_config_dir: /var/vcap/jobs/cni-flannel/config/cni
          plugin:
            etcd_endpoints:
              - (( config_from_cf.etcd.advertise_urls_dns_suffix ))
            etcd_client_cert: (( config_from_cf.etcd.client_cert ))
            etcd_client_key: (( config_from_cf.etcd.client_key ))
            etcd_ca_cert: (( config_from_cf.etcd.ca_cert ))
          policy_server:
            database:
              type: REPLACE_WITH_DB_TYPE
              username: REPLACE_WITH_USERNAME
              password: REPLACE_WITH_PASSWORD
              host: REPLACE_WITH_DB_HOSTNAME
              port: REPLACE_WITH_DB_PORT
              name: REPLACE_WITH_DB_NAME
            skip_ssl_validation: true
            uaa_client_secret: REPLACE_WITH_UAA_CLIENT_SECRET
            uaa_url: (( "https://uaa." config_from_cf.system_domain ))
            ca_cert: |
              -----BEGIN CERTIFICATE-----
              REPLACE_WITH_CA_CERT
              -----END CERTIFICATE-----
            server_cert: |
              -----BEGIN CERTIFICATE-----
              REPLACE_WITH_SERVER_CERT
              -----END CERTIFICATE-----
            server_key: |
              -----BEGIN RSA PRIVATE KEY-----
              REPLACE_WITH_SERVER_KEY
              -----END RSA PRIVATE KEY-----
      garden_properties:
        network_plugin: /var/vcap/packages/runc-cni/bin/garden-external-networker
        network_plugin_extra_args:
        - --configFile=/var/vcap/jobs/garden-cni/config/adapter.json
      jobs:
      - name: policy-server
        instances: 1
        persistent_disk: 256
        templates:
        - name: policy-server
          release: cf-networking
        - name: route_registrar
          release: cf
        - name: consul_agent
          release: cf
        - name: metron_agent
          release: cf
        resource_pool: database_z1
        networks:
          - name: diego1
        properties:
          nats:
            machines: (( config_from_cf.nats.machines ))
            user: (( config_from_cf.nats.user ))
            password: (( config_from_cf.nats.password ))
            port: (( config_from_cf.nats.port ))
          metron_agent:
            zone: z1
          route_registrar:
            routes:
            - name: policy-server
              port: 4002
              registration_interval: 20s
              uris:
              - (( "api." config_from_cf.system_domain "/networking" ))
          consul:
            agent:
              services:
                policy-server:
                  name: policy-server
                  check:
                    interval: 5s
                    script: /bin/true
    
    config_from_cf: (( merge ))
    
  6. Edit the stub file using the table below as a guide:

    Note: For a test environment, you can use a script such as the one in the CF Networking Release repository to generate the certificates and keys required for the stub. For a production deployment, use certificates signed by a Certificate Authority (CA).

    CF Networking Stub Contents Editing Instructions
    
    properties:
      cf_networking:
        vxlan_policy_agent:
          policy_server_url: [...]
          ca_cert: |
            -----BEGIN CERTIFICATE-----
            REPLACE_WITH_CA_CERT
            -----END CERTIFICATE-----
          client_cert: |
            -----BEGIN CERTIFICATE-----
            REPLACE_WITH_CLIENT_CERT
            -----END CERTIFICATE-----
          client_key: |
            -----BEGIN RSA PRIVATE KEY-----
            REPLACE_WITH_CLIENT_KEY
            -----END RSA PRIVATE KEY-----
    Paste in certificates and keys for the policy agent. The policy agent communicates with the policy server through TLS. See the Architecture section for more information.
    
    policy_server:
      uaa_client_secret: REPLACE_WITH_UAA_CLIENT_SECRET
        
    Paste in the same value you used in the step above.
    
    database:
      type: REPLACE_WITH_DB_TYPE 
      username: REPLACE_WITH_USERNAME
      password: REPLACE_WITH_PASSWORD
      host: REPLACE_WITH_DB_HOSTNAME
      port: REPLACE_WITH_DB_PORT 
      name: REPLACE_WITH_DB_NAME 
       
    Supply the details for the database from step 1.
    The database type must be postgres or mysql.
    Choose a username and password.
    For host, enter the IP address of the database instance.
    Supply a port; for MySQL a typical port is 3360.
    Supply the name of the database.
    
      ca_cert: |
        -----BEGIN CERTIFICATE-----
        REPLACE_WITH_CA_CERT
        -----END CERTIFICATE-----
      server_cert: |
        -----BEGIN CERTIFICATE-----
        REPLACE_WITH_SERVER_CERT
        -----END CERTIFICATE-----
      server_key: |
        -----BEGIN RSA PRIVATE KEY-----
        REPLACE_WITH_SERVER_KEY
        -----END RSA PRIVATE KEY-----
    garden_external_networker:
      cni_plugin_dir: /var/vcap/packages/flannel/bin
      cni_config_dir: /var/vcap/jobs/cni-flannel/config/cni
        
    Paste in the certificates and keys for the policy server. The policy server communicates with the policy agent through TLS. See the Architecture section for more information.
  7. Create a file that contains the following bash script. Name the file generate_diego.sh.
    set -e -x -u
    
    environment_path=STUBS-DIRECTORY
    output_path=MANIFEST-DIRECTORY
    diego_release_path=LOCAL-DIEGO-REPO
    
    pushd cf-release
      ./scripts/generate_deployment_manifest aws \
        ${environment_path}/stubs/director-uuid.yml \
        ${diego_release_path}/examples/aws/stubs/cf/diego.yml \
        ${environment_path}/stubs/cf/properties.yml \
        ${environment_path}/stubs/cf/instance-count-overrides.yml \
        ${environment_path}/stubs/cf/stub.yml \
        > ${output_path}/cf.yml
    popd
    
    pushd diego-release
      ./scripts/generate-deployment-manifest \
        -g \
        -c ${output_path}/cf.yml \
        -i ${environment_path}/stubs/diego/iaas-settings.yml \
        -p ${environment_path}/stubs/diego/property-overrides.yml \
        -n ${environment_path}/stubs/diego/instance-count-overrides.yml \
        -N ${environment_path}/stubs/cf-networking/stub.yml \
        -v ${environment_path}/stubs/diego/release-versions.yml \
        > ${output_path}/diego.yml
    popd
    
    Replace the variables as follows:
    • STUBS-DIRECTORY: The directory containing your stubs for CF, Diego, and CF Networking.
    • MANIFEST-DIRECTORY: The directory where you want the manifest created.
    • LOCAL-DIEGO-REPO: The directory of the local copy of the diego-release repository.
  8. Enter the following commands to make the script executable and run the script.
    $ chmod u+x generate_diego.sh
    $ ./generate_diego.sh
    
  9. Enter the following command to target your BOSH director:
    $ bosh target BOSH-DIRECTOR-IP
    
    For example,
    $ bosh target 192.0.2.1
  10. Enter the following command to set your CF deployment to the manifest you generated.
    $ bosh deployment ${output_path}/cf.yml
    
  11. Enter the following command to deploy CF.
    $ bosh deploy
  12. Enter the following command to set your Diego deployment to the manifest you generated.
    $ bosh deployment ${output_path}/diego.yml
    
  13. Enter the following command to deploy Diego.
    $ bosh deploy
  14. (Optional) Try out the Cats and Dogs example in the CF Networking Release repository. In this tutorial, you deploy two apps and create a CF Networking policy that allows them to communicate directly with each other.

Enable on BOSH-Lite

If your CF deployment runs on BOSH-Lite, follow these steps to enable CF Networking.

  1. Ensure your BOSH-Lite version is 9000.131.0 or later. If you need to upgrade, follow the instructions for Upgrading the BOSH-Lite VM.
  2. Navigate to your bosh-lite directory, for example,
    $ cd ~/workspace/bosh-lite 
  3. To enable bridge-netfilter on the VM running BOSH-Lite, run the following command:
    $ vagrant ssh -c 'sudo modprobe br_netfilter'

    CF Networking on BOSH-Lite requires this Linux kernel feature to enforce network policy.
  4. Upload the latest BOSH-Lite stemcell:
    $ bosh upload stemcell https://bosh.io/d/stemcells/bosh-warden-boshlite-ubuntu-trusty-go_agent
  5. To clone the required CF release repos to your workspace, enter the following commands:
    $ git clone https://github.com/cloudfoundry/diego-release
    $ git clone https://github.com/cloudfoundry/cf-release
    $ git clone https://github.com/cloudfoundry-incubator/cf-networking-release
  6. To enable CF Networking on BOSH-Lite, change directories into cf-networking-release and run the deploy script:
    $ cd ~/workspace/cf-networking-release
    $ ./scripts/deploy-to-bosh-lite
  7. (Optional) Try out the Cats and Dogs example in the CF Networking Release repository. In this tutorial, you deploy two apps and create a CF Networking policy that allows them to communicate directly with each other.

Create Policies for CF Networking

This section describes how to create and modify CF Networking policies using a plugin for the CF CLI.

To use the plugin, you must have the network.admin UAA scope. This scope gives you the right to create a policy between any two apps in your CF deployment. Depending on the security structure of your organization, you can either assign this scope to developers so that they can create their own policies or you can have your developers send you requests. For more information, see Creating and Managing Users with the UAA CLI (UAAC).

Install the Plugin

Follow these steps to download and install the Network Policy plugin for the CF CLI:

  1. Download the network-policy-plugin for your operating system from the CF Networking Release repository.

  2. To change the permissions of the plugin file and complete the installation, enter the following commands:

    $ chmod +x ~/Downloads/network-policy-plugin
    $ cf install-plugin ~/Downloads/network-policy-plugin
    

Create a Policy

To create a policy that allows direct network traffic from one app to another, enter the following command:

$ cf allow-access SOURCE-APP DESTINATION-APP --protocol PROTOCOL --port PORT

Replace the placeholders in the above command as follows:

  • SOURCE-APP is the name of the app that will be sending traffic.
  • DESTINATION-APP is the name of the app that will be receiving traffic.
  • PROTOCOL is one of the following: tcp or udp.
  • PORT is the port at which to connect to the destination app. The allowed range is 1 to 65535.

The following example command allows access from the frontend app to the backend app over TCP at port 8080:

$ cf allow-access frontend backend --protocol tcp --port 8080
Allowing traffic from frontend to backend as admin...
OK 

List Policies

You can list all the policies in your deployment or just the policies for which a single app is either the source or the destination:

  • To list the all the policies in your deployment, enter the following command:

    $ cf list-access
    
  • To list the policies for an app, enter the following command:

    $ cf list-access --app MY-APP
    

    The following example command lists policies for the app frontend:

    $ cf list-access --app frontend
    Listing policies as admin...
    OK
    
    Source    Destination    Protocol    Port
    frontend  backend        tcp         8080
    

Delete a Policy

To delete a policy that allows direct network traffic from one app to another, enter the following command:

$ cf remove-access SOURCE-APP DESTINATION-APP --protocol PROTOCOL --port PORT

For example,

$ cf remove-access frontend backend --protocol tcp --port 8080
Denying traffic from frontend to backend as admin...
OK 

View the source for this page in GitHub