HA Utilities

Setting up multiple nodes behind shared Corda Firewall require preparation of various keystores and config files, which can be time consuming and error prone. The HA Utilities aims to provide tools to streamline the node provision and deployment process.

The tool is distributed as part of Corda Enterprise in the form of runnable .jar. file - corda-tools-ha-utilities-4.11.jar.

To run simply pass in the file or URL as the first parameter:

java -jar corda-tools-ha-utilities-4.11.jar <sub-command> <command line options>

Use the --help flag for a full list of command line options.

Sub-commands

• node-registration: Corda registration tool for registering 1 or more node with the Corda Network, using provided node configuration.
• import-ssl-key: Key copying tool for creating bridge SSL keystore or add new node SSL identity to existing bridge SSL keystore.
• generate-internal-artemis-ssl-keystores: Generate self-signed root and SSL certificates for internal communication between the services and external Artemis broker.
• generate-internal-tunnel-ssl-keystores: Generate self-signed root and SSL certificates for internal communication between Bridge and Float.
• configure-artemis: Generates required configuration files for the external Artemis broker.
• install-shell-extensions: Install alias and autocompletion for bash and zsh. See cli-application-shell-extensions for more info.
• notary-registration: Corda registration tool for registering a single HA notary service identity, using provided worker node configuration.

Node Registration Tool

The registration tool can be used to register multiple Corda nodes with the network operator, it is useful when managing multiple identities and setting up multiple Corda nodes sharing Corda firewall infrastructures. For convenience the tool is also downloading network parameters. Additionally, the tool can use the crypto services configured in the bridge (if any) to generate SSL keys and import them into the bridge.

The tool does not include any third party supplied client side jar files needed when connecting to an HSM. These jar files are supplied by the HSM vendor. The tool does however assume that it can load these jar files from the drivers sub directory of the configured base-directory option. Before running the tool you need to make sure the required HSM client side jar files are in the drivers directory. This is only necessary when connecting to an HSM.

Command-line options

ha-utilities node-registration [-hrRvV] [--logging-level=<loggingLevel>] [-b=FOLDER] [-d=<days>] [-g=FILE] -p=PASSWORD -t=FILE -f=FILE... [-f=FILE...]...

• -v, --verbose, --log-to-console: If set, prints logging to the console as well as to a file.
• --logging-level=<loggingLevel>: Enable logging at this level and higher. Possible values: ERROR, WARN, INFO, DEBUG, TRACE. Default: INFO
• -b, --base-directory=FOLDER: The node working directory where all the files are kept.
• -f, --config-files=FILE...: The path to the config file
• -t, --network-root-truststore=FILE: Network root trust store obtained from network operator.
• -p, --network-root-truststore-password=PASSWORD: Network root trust store password obtained from network operator.
• -g, --bridge-config-file: The path to the bridge configuration file.
• -d, --tls-cert-validity=<days>: Validity for TLS certificate in days, for example -d 825. If not specified, equals to the validity of the issuer Node CA certificate or 10 years, whichever is shorter.
• -r, --renew-tls-cert: Renew TLS certificate for already registered node without changing keys in HSM. Node registration is not performed with this option.
• -R, --renew-tls-cert-with-keys: Renew TLS certificate for already registered node with re-generating TLS keys in HSM. Node registration is not performed with this option.
• -h, --help: Show this help message and exit.
• -V, --version: Print version information and exit.
• -C, --node-identity-cert-not-ca: A Boolean value, true or false. Creates a node legal identity certificate which is not a certificate authority (CA).

Keys and certificates

The following table shows keys and certificates generated by the tool. For more details on a certificate hierarchy see permissioning.

[1] Certificate validity cannot exceed the validity of Doorman CA certificate.

[2] For Node TLS certificate it’s recommended to set 27 months validity by specifying --tls-cert-validity 825.

[3] HSM for TLS keys is only supported with standalone bridge component. TLS keys for ‘all-in-one’ node can be only stored in file.

Registering nodes with bridge

Before running the tool you need to copy the configuration files from Corda nodes and bridge running on different servers into the same location, for example, as follows:

.
├── corda-tools-ha-utilities-4.11.jar
├── drivers
    ├── <HSM driver>
├── entity_A
    ├── node.conf
    ├── HSM.conf
├── entity_B
    ├── node.conf
    ├── HSM.conf
├── bridge
    ├── firewall.conf
    ├── HSM.conf

Given the configuration above, the tool can be run with the following command:

java -jar corda-tools-ha-utilities-4.11.jar node-registration --base-directory=./output --config-files=./entity_A/node.conf --config-files=./entity_B/node.conf --bridge-config-file=./bridge/firewall.conf --network-root-truststore=network-root-truststore.jks --network-root-truststore-password=trustpass --tls-cert-validity 825

After successful execution this will produce output/ folder containing the following files:

.
├── output
    ├── entity_A
        ├── certificates
            ├── nodekeystore.jks
            ├── sslkeystore.jks
            └── truststore.jks
    ├── entity_B
        ├── certificates
            ├── nodekeystore.jks
            ├── sslkeystore.jks
            └── truststore.jks
    ├── network-parameters
    └── bridge.jks

Generated files need to be copied from the output dir to the actual node and bridge locations, as follows:

Renewing TLS certificates

The tool allows to re-issue TLS certificate, which validity is shorter than the validity of corresponding Node CA certificate, by specifying -r or -R option. In this case node re-registration will not occur, so Node CA and Node Identity keys and certificates remain intact.

Pre-requisites for TLS certificate renewal are:

• All nodes are already registered with the tool.
• Configuration files which were used for initial registration are in place.
• nodekeystore.jks and sslkeystore.jks files produced by the tool during initial registration remain at the same output location.

Given the configuration above, TLS certificates can be renewed with the following command:

java -jar corda-tools-ha-utilities-4.11.jar node-registration --base-directory=./output --config-files=./entity_A/node.conf --config-files=./entity_B/node.conf --bridge-config-file=./bridge/firewall.conf --network-root-truststore=network-root-truststore.jks --network-root-truststore-password=trustpass --tls-cert-validity 825 --renew-tls-cert

This will update certificates in sslkeystore.jks and bridge.jks at the output location.

SSL key copier

When using shared external bridge, the bridge will need to have access to nodes’ SSL key in order to establish connections to counterparties on behalf of the nodes. The SSL key copier sub command can be used to provision the SSL keystore and add additional keys when adding more nodes to the shared infrastructure.

Command-line options

ha-utilities import-ssl-key [-hvV] [--logging-level=<loggingLevel>] [-b=FOLDER] [-k=FILES] -p=PASSWORDS --node-keystore-passwords=PASSWORDS... [--node-keystore-passwords=PASSWORDS...]... --node-keystores=FILES... [--node-keystores=FILES...]...

• -v, --verbose, --log-to-console: If set, prints logging to the console as well as to a file.
• --logging-level=<loggingLevel>: Enable logging at this level and higher. Possible values: ERROR, WARN, INFO, DEBUG, TRACE. Default: INFO
• --node-keystores=FILES...: The path to the node SSL keystore(s)
• --node-keystore-passwords=PASSWORDS...: The password(s) of the node SSL keystore(s)
• -b, --base-directory=FOLDER: The working directory where all the files are kept.
• -k, --bridge-keystore=FILES: The path to the bridge SSL keystore.
• -p, --bridge-keystore-password=PASSWORDS: The password of the bridge SSL keystore.
• -h, --help: Show this help message and exit.
• -V, --version :Print version information and exit.

Self signed internal Artemis SSL keystore

TLS is used to ensure communications between HA components and standalone Artemis are secured. This tool can be used to generate the required keystores if TLS cert signing infrastructure is not available within your organisation. Please note that for Artemis to work correctly, the password for the store and the password for the private key will need to be set to the same value. This tool can generate the private key used by the Bridge or the Node in an HSM. This will happen if the --bridge-hsm-name and --bridge-hsm-config-file options are specified. Otherwise the file-based keystore is used. Regardless where the private keys are stored the public certificates are stored in the file based keystores.

The tool does not include any third-party supplied client side JAR files needed when connecting to an HSM. These JAR files are supplied by the HSM vendor. The tool does however assume that it can load these JAR files from the drivers sub-directory of the configured base directory option. Before running the tool you need to make sure the required HSM client side JAR files are in the drivers directory. This is only necessary when connecting to an HSM.

The following files are produced when you run the tool:

• artemisbridge.jks: used by the standalone bridge. The path to this file should be specified in the artemisSSLConfiguration section in firewall.conf.
• artemisnode.jks: optionally used by the node. When used, the path to this file should be specified in the messagingServerSslConfiguration section in node.conf.
• artemis.jks: used by the standalone P2P Artemis broker.
• artemis-truststore.jks: must be placed together with any of the above keystore files.

Command-line options

ha-utilities generate-internal-artemis-ssl-keystores [-hvV] [--logging-level=<loggingLevel>] [-b=<baseDirectory>] [-c=<country>] [-d=<days>] [-l=<locality>] [-o=<organization>] [-p=<keyStorePassword>] [-t=<trustStorePassword>]

• -v, --verbose, --log-to-console: If set, prints logging to the console as well as to a file.
• --logging-level=<loggingLevel>: Enable logging at this level and higher. Possible values: ERROR, WARN, INFO, DEBUG, TRACE. Default: INFO
• -p, --keystore-password=<keyStorePassword>: Password ^{[4] for all generated keystores. Default: changeit}
• -t, --truststore-password=<trustStorePassword>: Password ^{[4] for the trust store. Default: changeit}
• -o, --organization=<organization>: X500Name’s organization attribute. Default: Corda
• -l, --locality=<locality>: X500Name’s locality attribute. Default: London
• -c, --country=<country>: X500Name’s country attribute. Default: GB
• -d, --tls-cert-validity=<days>: Validity for TLS certificate in days. Default: 3650 days (10 years).
• -b, --base-directory=<baseDirectory>: The working directory where all the files are kept.
• -m, --bridge-hsm-name: The HSM name used by the bridge.
• -f, --bridge-hsm-config-file: The path to the bridge HSM config file. Only required if the HSM name has been specified.
• -s, --node-hsm-name: The HSM name used by the node.
• -i, --node-hsm-config-file: The path to the node HSM config file. Only required if the HSM name has been specified.
• -h, --help: Show this help message and exit.
• -V, --version: Print version information and exit.

The name of HSM provider to be used, e.g. UTIMACO, GEMALTO_LUNA, etc.

Self signed internal Tunnel SSL keystore

TLS is used for communications between Bridge and Float components. This tool can be used to generate the required keystores if TLS cert signing infrastructure is not available within your organisation. This tool can also create the private keys used by the Bridge and Float for the SSL communication in an HSM. This will happen if the HSM name and HSM config file option for the Bridge or Float is specified, otherwise the file based keystore is used. Regardless where the private keys are stored the public certificates are stored in the file based keystores.

The tool does not include any third party supplied client side jar files needed when connecting to an HSM. These jar files are supplied by the HSM vendor. The tool does however assume that it can load these jar files from the drivers sub directory of the configured base-directory option. Before running the tool you need to make sure the required HSM client side jar files are in the drivers directory. This is only necessary when connecting to an HSM.

Command-line options

ha-utilities generate-internal-tunnel-ssl-keystores [-hvV] [--logging-level=<loggingLevel>] [-b=<baseDirectory>] [-c=<country>] [-d=<days>] [-l=<locality>] [-o=<organization>] [-p=<keyStorePassword>] [-e=<entryPassword>] [-t=<trustStorePassword>]

• -v, --verbose, --log-to-console: If set, prints logging to the console as well as to a file.
• --logging-level=<loggingLevel>: Enable logging at this level and higher. Possible values: ERROR, WARN, INFO, DEBUG, TRACE. Default: INFO
• -p, --keystore-password=<keyStorePassword>: Password for all generated keystores. Default: changeit
• -e, --entry-password=<entryPassword>: Password for all the keystores private keys. Default: changeit
• -t, --truststore-password=<trustStorePassword>: Password for the trust store. Default: changeit
• -o, --organization=<organization>: X500Name’s organization attribute. Default: Corda
• -l, --locality=<locality>: X500Name’s locality attribute. Default: London
• -c, --country=<country>: X500Name’s country attribute. Default: GB
• -d, --tls-cert-validity=<days>: Validity for TLS certificate in days. Default: 3650 days (10 years).
• -b, --base-directory=<baseDirectory>: The working directory where all the files are kept.
• -m, --float-hsm-name: The HSM name for the Float.
• -f, --float-hsm-config-file: The path to the Float HSM config file. Only required if the HSM name has been specified.
• -s, --bridge-hsm-name: The HSM name for the Bridge.
• -i, --bridge-hsm-config-file: The path to the Bridge HSM config file. Only required if the HSM name has been specified.
• -h, --help: Show this help message and exit.
• -V, --version: Print version information and exit.

The name of HSM provider to be used, e.g. UTIMACO, GEMALTO_LUNA, etc.

Artemis configuration

Configuring an external Artemis broker to be used by Corda nodes and firewall components can be a little daunting. This tool will generate the necessary configuration files and install (if needed) a new Artemis instance. Please note that the generated configuration files will be copied to a destination supplied as an argument to the command, replacing existing ones. The keystore and truststore information will be downloaded by the clients when Artemis is configured in HA mode. Therefore, the tool will configure broker.xml with paths relative to the Artemis instance (for acceptors) and client’s working directory (for connectors). For example, using the option –keystore ./artemiscerts/keystore.jks will require the keystore to be installed in $ARTEMIS_DIR/etc/artemiscerts and $NODE_OR_BRIDGE_DIR/artemiscerts.

Command-line options

ha-utilities configure-artemis [-hvV] [--logging-level=<loggingLevel>] [--install] [--distribution=<dist>] --path=<workingDir>  --user=<userX500Name> --acceptor-address=<acceptorHostAndPort> --keystore=<keyStore> --keystore-password=<keyStorePass> --truststore=<trustStore> --truststore-password=<trustStorePass> [--ha=<mode>] [--connectors=<connectors>[;<connectors>...]]

• -v, --verbose, --log-to-console: If set, prints logging to the console as well as to a file.
• --logging-level=<loggingLevel>: Enable logging at this level and higher. Possible values: ERROR, WARN, INFO, DEBUG, TRACE. Default: INFO
• --install: Install an Artemis instance at the specified path.
• --distribution: The path to the Artemis distribution used to install an instance.
• --path: The path where the generated configuration files will be installed. Used as the new instance location when using the –install option.
• --user: The X500 name of connecting users (clients). CN must be “artemis”. Other fields must match those passed in to generate-internal-artemis-ssl-keystores. This parameter is whitespace-sensitive. Example value: “CN=artemis, O=Corda, L=London, C=GB”
• --acceptor-address: The broker instance acceptor network address for incoming connections.
• --keystore: The SSL keystore path.
• --keystore-password: The SSL keystore password ^[4].
• --truststore: The SSL truststore path.
• --truststore-password: The SSL truststore password ^[4].
• --ha: The broker’s working mode. If specified, the broker will be configured to work in HA mode. Valid values: NON_HA, MASTER, SLAVE
• --connectors: A list of network hosts and ports representing the Artemis connectors used for the Artemis HA cluster. The first entry in the list will be used by the instance configured with this tool. The connector entries are separated by commas (e.g. localhost:10000,localhost:10001,localhost:10002)
• --cluster-user: The username of the Artemis cluster. Default: corda-cluster-user
• --cluster-password: Artemis cluster password. Default: changeit
• -h, --help: Show this help message and exit.
• -V, --version: Print version information and exit.

[4] The ampersand (&) character is not allowed in the password for Artemis SSL keystore or truststore.

Notary registration

The HA Utilities tool is used to register the identity of the HA notary service, and generates the key and certificate of the notary service that is shared by all workers of the notary cluster. It is required to be run before registering any notary worker nodes.

The tool does not include any third party supplied client side jar files needed when connecting to an HSM. These jar files are supplied by the HSM vendor. The tool does however assume that it can load these jar files from the drivers sub directory of the configured base-directory option. Before running the tool you need to make sure the required HSM client side jar files are in the drivers directory. This is only necessary when connecting to an HSM.

Command-line options

ha-utilities notary-registration [-hrRvV] [--logging-level=<loggingLevel>] [-b=FOLDER] [-f=FILE] -t=FILE -p=PASSWORD [-k=FILE]...

• -v, --verbose, --log-to-console: If set, prints logging to the console as well as to a file.
• --logging-level=<loggingLevel>: Enable logging at this level and higher. Possible values: ERROR, WARN, INFO, DEBUG, TRACE. Default: INFO
• -b, --base-directory=FOLDER: The working directory where all the files are kept.
• -f, --config-file=FILE...: The path to the config file. Default: node.conf
• -t, --network-root-truststore=FILE: Network root trust store obtained from network operator.
• -p, --network-root-truststore-password=PASSWORD: Network root trust store password obtained from network operator.
• -k, --output-keystore=FILE: If set, stores the generated notary service certificate (and key if not using a HSM) in the configured key store.
• -h, --help: Show this help message and exit.
• -V, --version: Print version information and exit.
• -r, --renew: Send a CSR to a different endpoint for certificate renewal. For more information about this feature, contact your R3 account manager.

Output

After successful registration, a keystore file is created by the tool. The location of this file can be explicitly set using the optional –output-keystore commandline argument. If this argument has not been passed then the file location will default to certificates/nodekeystore.jks.

This key store contains the service identity certificate (and key if not using a HSM) that all notary workers of this notary cluster share. Due to HSM restrictions around storing certificate chains, the key store will still be generated when using a HSM however it will only contain the notary service certificate chain. See HSM Support for more information.

Node Certificate Rotation Tool

This tool enables the reissuing of node legal identity keys and certificates, allowing for re-registration of a node (including a notary node) with a new certificate in the Network Map in Corda Enterprise Network Manager. You must not change the node’s myLegalName during certificate rotation.

For more information about this feature, contact your R3 account manager.