LogServer migrations

Migrating LogServer to another appliance

If you have your LogServer co-hosted with a Controller, it is a good idea to move it to a separate appliance. You can migrate your LogServer away from your Controller taking the existing audit logs across. Appliances will normally queue logs when they can't talk to the LogServer; but they only do this when there is a LogServer configured. So it is important to minimize the time window while there are no LogServers configured.

You will need a browser access the admin UI and terminal access to both appliances:

1. In the browser, open the admin UI and login as administrator

2. Go to Appliances and create a new appliance (B) which will become the new LogServer

3. Go to Appliances > Appliance (A) (the appliance that is currently acting as LogServer), uncheck the LogServer function, and Save.

4. In a terminal, SSH to Appliance (A) were the LogServer is currently running: ssh cz@applianceA.address

   1. Switch user to root: sudo su  

   2. Go to the OpenSearch directory: cd /mnt/data/elasticsearch5

   3. Compress the whole OpenSearch data directory: tar zcvf es_data.tar.gz data/

   4. Copy the compressed data file to the new appliance: scp es_data.tar.gz cz@applianceB.address:

5. In a terminal, SSH to the new spare Appliance (B): ssh cz@applianceB.address

   1. Switch user to root: sudo su  

   2. Copy the compressed file to its proper directory: mv es_data.tar.gz /mnt/data/elasticsearch5

   3. Go to the OpenSearch directory and uncompress the data file: cd /mnt/data/elasticsearch5; tar zxvf es_data.tar.gz

   4. Set the right permissions to the files: chown -R czd:czd *

6. In the admin UI, go to Appliances > Appliance (B), select the LogServer function, and Save.

Migrating from using LogServer to using LogForwarder

If you have been using the LogServer and want to migrate to an enterprise log management system, the LogForwarder is designed for this purpose.

LogForwarders can be configured for HA operation using two or more appliances. They can be deployed to export the logs by Site to different destinations. Multiple export protocols can be specified at the same time including one for the ELK stack. This means that if there is an ongoing requirement to retain the ELK stack in an enterprise environment, then one can be deployed outside of the AppGate ZTNA Collective. Logs can be forwarded there while also exporting the log data into an enterprise-class logging system.  

It is possible to migrate from the LogServer to the LogForwarder without losing audit logs you have already accumulated on you LogServer. To do this:

1. Disable the LogServer function and enable the LogForwarder function on the LogServer. This will become a LogForwarder.

2. SSH to the LogForwarder and run the following command to generate OpenSearch configuration: sudo cz-config set -j runOpensearch true

3. Wait for OpenSearch to become green. This can take a long time depending on the size of the database. To see this, run:

   es-tools -c cluster-health

   {
     "cluster_name": "appgate",
     "status": "green",
     "timed_out": false,
     "number_of_nodes": 1,
     "number_of_data_nodes": 1,
     "active_primary_shards": 6,
     "active_shards": 6,
     "relocating_shards": 0,
     "initializing_shards": 0,
     "unassigned_shards": 0,
     "delayed_unassigned_shards": 0,
     "number_of_pending_tasks": 0,
     "number_of_in_flight_fetch": 0,
     "task_max_waiting_in_queue_millis": 0,
     "active_shards_percent_as_number": 100
   }

4. Next, es-export is used to export all the log indexes from OpenSearch and feed them to cz-logd like any other appliance. Run es-export script with the custom forward parameter:

   es-export --forward_logd true

   Current index:  logstash-2021-02-23
Done

5. Finally, to stop OpenSearch and remove the temporary OpenSearch configuration on the new LogForwarder, enter: sudo cz-config set -j runOpensearch false

6. Optionally, remove the OpenSearch database by entering: sudo rm -rf /mnt/data/elasticsearch5/data/