Configuring the MySQL notary backend [Deprecated]

The MySQL notary service is tested against Percona XtraDB Cluster 5.7. Percona’s documentation page explains the installation in detail.

When setting up on Red Hat Enterprise Linux and CentOS make sure SELinux is disabled (we found issues even with the permissive mode). Otherwise you might get state transfer errors when starting up the second node, such as: [Warning] WSREP: 0.0 (pxc-cluster-node-1): State transfer to 1.0 (pxc-cluster-node-2) failed: -2 (No such file or directory)

Note also that each Percona XtraDB Cluster node requires multiple ports to be opened, the defaults are: 3306, 4444, 4567 and 4568.

If Percona is used as the database, each worker node requires a MySQL JDBC driver to be placed in the drivers directory to be able to communicate with the Percona XtraDB Cluster. The official driver can be obtained from Maven or the MySQL Connector/J download page.

The Percona nodes communicate with each other via group communication (GComm). The Percona replicas should only be reachable from each other and from the worker nodes.

R3 recommends running the worker nodes and the Percona service in a joined private subnet, opening up the P2P ports of the workers for external traffic.

In this section we’re setting up a three-node Percona cluster. A three-node cluster can tolerate one crash fault. In production, you probably want to run five nodes, to be able to tolerate up to two faults.

Host names and IP addresses used in the example are listed in the table below.

HostIP
percona-110.1.0.1
percona-210.1.0.2
percona-310.1.0.3

Percona provides repositories for the YUM and APT package managers. Alternatively you can install from source. For simplicity, we are going to install Percona using the default data directory /var/lib/mysql.

Run the commands below on all nodes of your Percona cluster to configure the Percona repositories and install the service.

wget https://repo.percona.com/apt/percona-release_0.1-4.$(lsb_release -sc)_all.deb
sudo dpkg -i percona-release_0.1-4.$(lsb_release -sc)_all.deb
sudo apt-get update
sudo apt-get install percona-xtradb-cluster-57

The service will start up automatically after the installation, you can confirm that the service is running with service mysql status, start the service with sudo service mysql start and stop with sudo service mysql stop.

Some distributions allow root access to the database through a Unix domain socket, others require you to find the temporary password in the log file and change it upon first login.

sudo service mysql stop

Variables you need to change from the defaults are listed in the table below.

Variable NameExampleDescription
wsrep_cluster_addressgcomm://10.1.0.1,10.1.0.2,10.1.0.3The addresses of all the cluster nodes (host and port)
wsrep_node_address10.1.0.1The address of the Percona node
wsrep_cluster_namenotary-cluster-1The name of the Percona cluster
wsrep_sst_authusername:passwordThe credentials for SST
wsrep_provider_options“gcache.size=8G”Replication options

Configure all replicas via /etc/mysql/percona-xtradb-cluster.conf.d/wsrep.cnf as shown in the template below.

[mysqld]
# Path to Galera library
wsrep_provider=/usr/lib/galera3/libgalera_smm.so
wsrep_provider_options="gcache.size=8G"
# TODO set options related to the timeouts for WAN:
# evs.keepalive_period=PT3s
# evs.inactive_check_period=PT10S
# evs.suspect_timeout=PT30S
# evs.install_timeout=PT1M
# evs.send_window=1024
# evs.user_send_window=512

# Cluster connection URL contains IPs of nodes
#If no IP is found, this implies that a new cluster needs to be created,
#in order to do that you need to bootstrap this node
wsrep_cluster_address="gcomm://{{ your_cluster_IPs }}"

# In order for Galera to work correctly binlog format should be ROW
binlog_format=ROW

# MyISAM storage engine has only experimental support
default_storage_engine=InnoDB

# Slave thread to use
wsrep_slave_threads= 8

wsrep_log_conflicts

# This changes how InnoDB autoincrement locks are managed and is a requirement for Galera
innodb_autoinc_lock_mode=2

# Node IP address
wsrep_node_address={{ node_address }}

# Cluster name
wsrep_cluster_name={{ cluster_name }}

#If wsrep_node_name is not specified,  then system hostname will be used
#wsrep_node_name=

#pxc_strict_mode allowed values: DISABLED,PERMISSIVE,ENFORCING,MASTER
pxc_strict_mode=ENFORCING

# SST method
wsrep_sst_method=xtrabackup-v2

#Authentication for SST method
wsrep_sst_auth={{ sst_user }}:{{ sst_pass }}

The file /etc/mysql/percona-xtradb-cluster.conf.d/mysqld.cnf contains additional settings like the data directory. We’re assuming you keep the default /var/lib/mysql.

If you’re changing the location of the database data directory, you might need to configure your security module accordingly.

sudo /etc/init.d/mysql bootstrap-pxc

Watch the logs using tail -f /var/log/mysqld.log. Look for a log entry like WSREP: Setting wsrep_ready to true.

You can now connect to the database using a standard tool (e.g. the mysql command line tool).

CREATE USER corda IDENTIFIED BY '{{ password }}';

We need to create three tables in our database:

  • notary_committed_states
  • notary_request_log
  • notary_committed_transactions

We can do this using the following commands:

CREATE DATABASE corda;

CREATE TABLE IF NOT EXISTS corda.notary_committed_states(
  issue_transaction_id BINARY(32) NOT NULL,
  issue_transaction_output_id INT UNSIGNED NOT NULL,
  consuming_transaction_id BINARY(32) NOT NULL,
  CONSTRAINT id PRIMARY KEY (issue_transaction_id, issue_transaction_output_id)
);

GRANT SELECT, INSERT ON corda.notary_committed_states TO 'corda';

CREATE TABLE IF NOT EXISTS corda.notary_request_log(
  consuming_transaction_id BINARY(32) NOT NULL,
  requesting_party_name TEXT NOT NULL,
  request_signature BLOB NOT NULL,
  request_date TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
  request_id INT UNSIGNED NOT NULL AUTO_INCREMENT,
  worker_node_x500_name TEXT,
  CONSTRAINT rid PRIMARY KEY (request_id)
);

GRANT INSERT ON corda.notary_request_log TO 'corda';

CREATE TABLE IF NOT EXISTS corda.notary_committed_transactions(
  transaction_id BINARY(32) NOT NULL,
  CONSTRAINT tid PRIMARY KEY (transaction_id)
);

GRANT SELECT, INSERT ON corda.notary_committed_transactions TO 'corda';
CREATE USER ‘{{ sst_user }}’@localhost IDENTIFIED BY ‘{{ sst_pass }}‘;
GRANT RELOAD, LOCK TABLES, PROCESS, REPLICATION CLIENT ON *.* TO ‘{{ sst_user }}’@localhost;
FLUSH PRIVILEGES;

Once you have updated the wsrep.cnf on all nodes, start MySQL on all the remaining nodes of your cluster. Run this command on all nodes of your cluster, except the first one.

service mysql start

Watch the logs using tail -f /var/log/mysqld.log. Make sure you can start the MySQL client on the command line and access the corda database on all nodes.

mysql
mysql> use corda;
# The output should be `Database changed`.

In the next section, we’re Configuring the notary worker nodes.

Was this page helpful?

Thanks for your feedback!

Chat with us

Chat with us on our #docs channel on slack. You can also join a lot of other slack channels there and have access to 1-on-1 communication with members of the R3 team and the online community.

Propose documentation improvements directly

Help us to improve the docs by contributing directly. It's simple - just fork this repository and raise a PR of your own - R3's Technical Writers will review it and apply the relevant suggestions.

We're sorry this page wasn't helpful. Let us know how we can make it better!

Chat with us

Chat with us on our #docs channel on slack. You can also join a lot of other slack channels there and have access to 1-on-1 communication with members of the R3 team and the online community.

Create an issue

Create a new GitHub issue in this repository - submit technical feedback, draw attention to a potential documentation bug, or share ideas for improvement and general feedback.

Propose documentation improvements directly

Help us to improve the docs by contributing directly. It's simple - just fork this repository and raise a PR of your own - R3's Technical Writers will review it and apply the relevant suggestions.