All cloud providers offer automated means of scaling the number of instances of a specific appliance type up and down. Because AppGate ZTNA uses virtual appliances, you can auto-scale the number of Gateways within a Site based on demand. Auto-scaling can be triggered by metrics such as memory and CPU usage, which the cloud provider normally monitors.
User Data (AWS), Custom Data (Azure), or Startup Scripts (Google Cloud) create an appliance configuration for newly started instances based on a template, which also seeds the instance. For the Controller to remove unwanted Gateways from a Site—not just add them—you need two process integrations:
Upscale: Create a new appliance entry and join the AppGate ZTNA Site by seeding the appliance when the auto-scaling group spawns a new instance.
Downscale: Delete the appliance entry when an instance is removed from the auto-scaling group.
AWS
AWS provides auto-scaling groups to automatically increase or decrease the number of running instances based on metrics such as CPU or memory load. See:
GCP
GCP provides instance groups to automatically increase or decrease the number of running instances based on metrics such as CPU or memory load. See:
Azure
Azure provides virtual machine scale sets to automatically increase or decrease the number of running instances based on metrics such as CPU or memory load. See:
Prerequisites
To fully automate auto-scaling, a startup script is passed to the AppGate ZTNA instance using the cloud provider's API. This script creates an appliance configuration for the newly started instance based on a template, which also seeds the instance. Because you can only pass a single base64 blob (the script), it must contain all the necessary parameters to create a Gateway and the seed configuration. In AWS and Google Cloud, this concept is referred to as user data. Microsoft Azure has a similar feature called custom data.
Before you begin, configure the following in AppGate ZTNA. You will also need the Controller's hostname or IP address and admin port 8443. If you have a shared hostname configured with multiple Controllers using DNS round robin, use the shared hostname.
Configure the auto-scale Site
Create an auto-scale Site, as auto-scaling is configured on a per-Site basis. Assign a tag such as siteX-for-auto to the Site to restrict admin access to this auto-scale Site later. To use the script, you will need the Site ID. The Site ID appears in the URL when you navigate to the Site in the admin UI. For example, given the Site URL https://mycontroller.com:8443/ui/sites/edit/750f210a-1c42-4d27-b568-4a8767ef2790, the Site ID is 750f210a-1c42-4d27-b568-4a8767ef2790.
Configure the auto-scale appliance
Create a new Gateway appliance assigned to the auto-scale Site. This appliance becomes a base template for the auto-scaling process, which varies only parameters such as hostname. Configure it with all the usual settings, but use a unique placeholder hostname such as appliance-name.example.com. Use a placeholder hostname even if you use IP addresses instead of hostnames. Assign a tag such as siteX-auto to the appliance to restrict admin access to only the templated and auto-scaled appliances later.
NOTE
Do not activate the appliance. This configuration is used as a template; the new instances created from the template are the ones that get activated.
Configure the auto-scale admin user
Create an admin user called autoscaling. You can use the local user database or an external identity provider. Set a strong, preferably randomly generated password. Exempt the user from multi-factor authentication (MFA) on the MFA for Admin page. To use the script, you will need username and password.
Configure the auto-scale admin role
The auto-scaling script requires specific privileges to view the template appliance for a given Site, create a new appliance, and export its seed configuration. Create an admin role called siteX-autoscaling and add the following privileges:
View all appliances tagged with
siteX-autoand Sites tagged withsiteX-for-autoCreate all appliances with tag
siteX-autoExport all appliances with tag
siteX-autoDelete all appliances with tag
siteX-auto(for downscaling)Edit appliances with tag
siteX-autoscaling-instance(If using deactivation during upscaling or downscaling)Assign the Gateway function on all appliances.
Configure the auto-scale policy
Create a new policy for the autoscaling user and assign it the siteX-autoscaling role. Add the following two criteria for the policy assignment:
Identity provider is the identity provider used for for the
autoscalinguserUsername is
autoscaling
Get the CA certificate
If you use the admin interface (port 8443) with a public certificate, you do not need to retrieve the Controller's CA certificate.
Implementing auto-scaling
Using the cloud template and startup script
User Data (AWS), Custom Data (Azure), or Startup Scripts (Google Cloud) create an appliance configuration for newly started instances based on a template, which also seeds the instance. AppGate ZTNA uses the following mechanisms on each cloud platform:
AWS: Instance user data. See AWS::AutoScaling::LaunchConfiguration.
GCP: Virtual machine startup script. See About startup scripts.
Azure: CustomData. See What are Virtual Machine Scale Sets?
NOTE
Startup scripts are available to the VM only during first boot and initial setup (provisioning). Provisioning is the process where VM create parameters—for example, hostname, username, password, certificates, custom data, and keys—are made available to the VM.
See the examples below to help you create the required scripts for your environment.
Upscaling
If you configure your instance group correctly with your startup script, the script runs when the new appliance starts. The instance finds the appliance template for its Site as defined on the Controller and creates a new appliance configuration based on it. It then exports the configuration, seeds itself, and performs the upscaling. The template is selected from the Gateways in the given Site. Inactive appliances tagged with template are given priority as the appliance template.
Downscaling
When the instance shuts down, it either deactivates or deletes the entry from the Controller.
Sharing the client hostname
By default, the script creates an appliance configuration using the appliance hostname. To use the Client Hostname/IP from the template Gateway configuration instead, pass --share-client-hostname. This is useful when you configure your auto-scaling group to use a load balancer.
Hostname conflicts
If a Gateway with the same hostname already exists in the Site, AppGate ZTNA deactivates it and reuses the entry for the new instance. If the Controller has already marked the Gateway as activated, the autoscale script exits and takes no action.
The script
The Gateway Autoscale Script is hosted on all appliances and run from a cloud instance's startup script. Use the following syntax:
appgate-autoscale.py COMMAND [OPTION1] [OPTION2] [OPTION...] CONTROLLER
CONTROLLER is the primary Controller's admin interface hostname or IP address.
NOTE
The default port is 444, even though the Controller uses 8443 for admin/API access.
The following table describes available commands:
Command | Default | Usage | Comment |
|---|---|---|---|
| Adds a new Gateway. | ||
| Deletes or deactivates an autoscaled Gateway. |
The following table describes available options for the syntax:
Option | Default | Usage | Comment |
|---|---|---|---|
| Prints help and exits. | ||
| 444 | The AppGate Controller admin port is 8443, so this options is required. | |
| admin | Username of the admin user. | |
| local | Identity provider for the admin user. | |
--cacert | Path to the CA certificate file (CER, DER, or PEM format). Not required if you have added a trusted third-party certificate for the admin/API access. | ||
| Auto-discovered | Controller API version | |
| Path to the executable that returns the admin password. | ||
| Use the Client Hostname/IP from the admin UI template. If not set, the appliance hostname is used. | ||
| Upscale | Site identifier (for example, | |
| Upscale | Uses the public IP address as the hostname for the new appliance. | |
| Disabled | Upscale | Enables customizations on the created appliance. |
| Downscale | Deactivates the Gateway (still visible in admin UI) rather than deleting the record. |
The following table describes advanced options for the syntax:
Advanced Option | Default | Usage | Comment |
|---|---|---|---|
| Auto-generated | Device ID for the new instance. | |
|
| Upscale | Seed file path. This normally does not need changing. |
| Read from instance | Cloud platform instance ID assigned to the new autoscale instance. | |
| Read from instance | Upscale | Appliance hostname of the new appliance. |
|
| Upscale | AppGate ZTNA tag used to find the appliance configuration template. You can also set this with the |
| Skips server certificate verification. The |
Upgrading autoscaled Gateways
Do not upgrade active autoscaled Gateways. When a new AppGate ZTNA version is available, a new image will be available for your cloud platform. Point the auto-scaling group to the new image, then terminate the active Gateway instances running the old version one by one. New Gateway instances running the new version replace them automatically.
If you use sdpctl to upgrade other appliances, use the exclude flag to skip the auto-scaling Sites. For example:
sdpctl.exe appliance upgrade prepare --image=<upgrade-image-zip-or-url> --exclude site=string
AWS
On AWS, create a new launch configuration pointing to the new AMI and update the auto-scaling group to point to the new launch configuration.
Advanced setups
You must create the startup script for your site manually. The following examples can serve as templates for your own setup.
Setup using a valid Controller certificate and an external credentials manager
This example uses a placeholder credentials manager similar to HashiCorp Vault. This concept also works with other providers such as AWS Security Token Service (STS). The Controller uses the admin interface on port 8443 with a valid certificate.
bash
#!/bin/sh
# Create a script that outputs the password for the auto-scaling admin user.
# The password is fetched dynamically every time it is needed to not expose credentials unnecessarily.
cat >/tmp/password-executable <<EOL
#!/usr/bin/python3
import JSON
import requests
r = requests.get('http://my-credential-manager.example.com/my-autoscaling-secret', params={'token': 'ASDL...ASZSD'})
password = r.JSON()['data']['password']
print(JSON.dumps({"password": password}))
EOL
chmod +x /tmp/password-executable
# Seed this appliance.
# We can skip the --cacert parameter since we are using a valid public certificate on the admin interface.
python3 /usr/share/admin-scripts/appgate-autoscale.py upscale controller.example.com --port 8443 --username siteAWS-autoscaling --password-path /tmp/password-executable --site af4fedf3-5cc2-43fb-aff7-61b67369a505 --file /home/cz/seed.JSON
# Setup appliance to delete itself on shutdown
cat >/var/cache/cz-scripts/shutdown-script<<EOL
#!/bin/sh
/usr/share/admin-scripts/appgate-autoscale.py downscale controller.example.com --port 8443 --username siteAWS-autoscaling --password-path /tmp/password-executable
EOLSetup with a self-signed Controller certificate using admin interface (port 8443)
This example uses a standard setup with the Controller using the admin interface on port 8443 and a self-signed certificate. The password is hardcoded in the startup script; do not use this approach in production.
bash
#!/bin/sh
# Add your Controller's CA certificate.
# This is not necessary if you are using a valid public certificate on the admin interface.
cat >/tmp/cacert.pem <<EOF
-----BEGIN CERTIFICATE-----
MIIFPDCCAySgAwIBAgIJAKJah5YABq/ZMA0GCSqGSIb3DQEBDQUAMBkxFzAVBgNV
BAMTDkFwcEdhdGUgWERQIENBMB4XDTE3MDIyNDE1MTExOVoXDTI3MDIyMjE1MTEx
[...]
zVAmFQL0rHwjonLsbLGBG54idCNTctN6HisBSdRmd7UlVdrABrbiBj42vQ7H8mwU
5qvvc4QH0X+dwU+QWVW0w+MDQNc+9xkTGTKaEiAiQ8WqigT88xUgTVEbSv25f9fm
-----END CERTIFICATE-----
EOF
# Create a script that outputs the password for the auto-scaling admin user
# In production, this should be getting the password from the appropriate credentials manager, for example STS on AWS.
cat >/tmp/password-executable <<EOL
#!/bin/sh
echo '{"password": "password_for_the_autoscaling_admin_user"}'
EOL
chmod +x /tmp/password-executable
# Seed this appliance
# Use the appropriate port, for example if you are using the admin interface on port 10001 pass --port 10001
# Skip the --cacert parameter if using a valid public certificate on the admin interface
python3 /usr/share/admin-scripts/appgate-autoscale.py upscale controller.example.com --port 8443 --cacert /tmp/cacert.pem --username siteAWS-autoscaling --password-path /tmp/password-executable --site af4fedf3-5cc2-43fb-aff7-61b67369a505 --file /home/cz/seed.JSON
# Setup appliance to delete itself on shutdown
cat >/var/cache/cz-scripts/shutdown-script<<EOL
#!/bin/sh
/usr/share/admin-scripts/appgate-autoscale.py downscale controller.example.com --port 8443 --cacert /tmp/cacert.pem --username siteAWS-autoscaling --password-path /tmp/password-executable
EOL