Versions Compared

Old Version 8

changes.mady.by.user Sarabjeet Kaur

Saved on

compared with

New Version Current

changes.mady.by.user Sarabjeet Kaur

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Applies to: (tick) Kyvos Enterprise  (tick) Kyvos Cloud (SaaS on AWS) (tick) Kyvos AWS Marketplace

...

  1. Create a new node for Kyvos Manager, and ensure the following:

    1. This node should have the same set of permissions in terms of roles, tags (UsedBy / CreatedBy, CLUSTER_ID, ROLE : KM, LAYER : KM_Service), network access rules and permissions (VirtaulNetwork, Subnet, Security Group, Resource Group), credentials, size and instance type, disk organization (mount point, disks, directories where Kyvos Manager and Kyvos installed) as that of the original Kyvos Manager node which doesn’t exist anymore.

    2. For access purposes, you need to either add the same security group or the security group added must have the same set of access rules and permissions.

    3. If Secrets Manager/Key Vault is in use, then ensure that the roles assigned to the new Kyvos Manager node have access to the Secrets Manager/Key Vault.

    4. Ensure that roles assigned to the new Kyvos Manager node have access to the S3 bucket/ABFS account.

  2. If the Kyvos Manager node is created by attaching a disk image of any old Kyvos Manager node, then ensure the below in mentioned sequence:

    1. Agent service is stopped on that node.

    2. Agent cron entry deleted from crontab.

    3. Kyvos Manager Agent and Kyvos folders were deleted from it.

  3. The OS commands must be present in the path of a non-interactive login session for the user account used to log in to the nodes.

  4. To restore Kyvos Manager on the new node, download a script file named disaster-recovery-kyvosmanager.sh from the DFS at path <engine_work>/setup/scripts/ and execute that script. This will restore the Kyvos Manager server and the Kyvos Manager service will start automatically.

Panel
panelIconIdatlassian-note
panelIcon:note:
bgColor#DEEBFF

Note

  • Keep the following things handy during disaster recovery, depending on what is affected in your cluster.

    • New certificates are applicable if existing settings (domain/subdomain) are changed after recovery.

    • Production license as per new BI nodes in case any BI server impacted

  • You must use the disaster recovery capability in any of the following scenarios: 

    • If Kyvos Manager, BI Server, or Query Engine nodes are affected. 

    • If only the Kyvos Manager nodes are affected. 

    • If Kyvos Manager and all Kyvos nodes (BI Servers, Query Engines, Web Portal, and Postgres Server) are affected. 

  • If only the BI Server or Query Engine nodes are affected, then add a node for that service, and the cluster can be restored. You will not need to use disaster recovery capability for this case.

  • If you enable TLS for Kyvos Manager and Kyvos application, the TLS option is not applicable during the Disaster Recovery restoration. After successful restoration, the TLS-related certificates are restored, and you can continue with the TLS option. 

Disaster recovery through the guided flow on Kyvos Manager

...

  1. Click the Uninstall button corresponding to Step 12: Uninstall Zookeeper in the Restore Cluster area.

Panel
panelIconIdatlassian-note
panelIcon:note:
bgColor#DEEBFF

Note

The option to Uninstall Zookeeper is displayed only when multiple BI Servers are installed on the cluster.

  1. On the displayed confirmation dialog box, provide your Kyvos Manager password, and click the Uninstall button.

    Image Modified

    A new browser tab is opened, showing

...

  1. uninstall Zookeeper operation details and status. You may switch back to the Disaster Recovery browser tab

...

  1. once the operation is completed

...

  1. .

...

...

  1. Click the Delete button corresponding to Step

...

  1. 3: Delete Offline Nodes.

    image-20240927-083543.pngImage Added

  2. From the Delete Offline Nodes dialog box, select the nodes you want to delete and provide your Kyvos Manager Password.
    Note that you will see only the Offline nodes in this list.

...

  2. Click the Delete button.

...

Panel
panelIconIdatlassian-note
panelIcon:note:
bgColor#DEEBFF

Note

Once deleted, nodes cannot be retrieved

...

.

...


You may switch back to the Disaster Recovery browser tab.
Once the operation is completed, you will see the status shown in the following figure. At this point, you will be able to perform the next step for adding new nodes.

...

image-20240927-083901.pngImage Added

  1. Click the Add button corresponding to Step

...

  1. 4: Add Nodes.

...

  1. On the Add Nodes

...

  1. dialog box,

...

  1. click the

...

  1. Next button.

...

  1. Ensure that you can add as many new nodes

...

  1. as you need.

    image-20240926-133702.pngImage Added

  2. Once done, provide your Kyvos Manager Password, and click the Add button.

...

  1. A new browser tab is opened, showing add node operation details and status. You may switch back to the Disaster Recovery browser tab.
    Once the operation is completed, you will see the status shown in the following figure. At this point, you will be able to perform the next step for installing Zookeeper.

...

  1. image-20240927-084533.pngImage Added

  2. Click the Install button corresponding to Step

...

  1. 5: Install Zookeeper.

    image-20240927-084814.pngImage Added

  2. Provide your Kyvos Manager Password on the confirmation boxand click the Install button.

...

  1. A new browser tab is opened, showing

...

  1. install Zookeeper operation details and status. You may switch back to the Disaster Recovery browser tab

...

  1. once the operation is completed

...

  1. .

...

  2. Click the Switch button corresponding to Step

...

  1. 6: Switch Repository. You will be redirected to the Switch Repository page.
    Refer to the Manage Kyvos Repository section to learn more.

...

  1. image-20240927-085241.pngImage Added
Panel
panelIconIdatlassian-info
panelIcon:info:
bgColor#FFFAE6

Important

  • After performing Disaster Recovery, if you want to enable TLS, you need to restart Kyvos Manager. To do this, go to Terminal, and run the stop-km.sh and then start-km.sh).

  • When Kyvos Manager HA is enabled and Managed zookeeper is used then after completing the Disaster Recovery activity, stop and start Kyvos Manager from terminal (not from Kyvos Manager UI) irrespective of whether TLS is enabled or not. Prior to the Kyvos 2024.1 release, Kyvos Manager restart is required only when TLS is enabled.

  • After completing disaster recovery, ensure that the following snapshots are pushed from Kyvos Manager. To do this, navigate to Utilities > Update Snapshot Bundles.

    image-20240124-144518.png

...

  1. Log in to Kyvos Manager.

  2. After login, the same old cluster should be visible by default.

  3. After the addition of the new Kyvos Manager node, the license error on the Kyvos Manager cluster Dashboard goes away.

  4. Log in to the Kyvos Web portal.

  5. Run the Sanity suite.

Snapshot Bundles:

Anchor
snapshots
snapshots
Snapshot Bundles:

Name and location to find snapshot bundles in DFS

...