Great Tandem: VoltDB & Delphix

Great Tandem: VoltDB & Delphix

VoltDB is gaining its market share as a great real time analytics database. The buzz word is "Smart Data Fast". Gartner Magic Quadrant for Operational Databases makes it clear.

What if we merge VoltDB real time capabilities with power of Data as a Service? Below I'm testing if it is possible to make VoltDB's persistent store reside within Delphix vFiles.

Moving Parts

• Delphix Engine: I am using Delphix Express 4.2.3.1

 

• Test Host: Linux CentOS 6.6

• VoltDB Trial 5.6 Enterprise Edition (https://voltdb.com/products/editions)

Test Steps

• Prepare volt user

[root@volt1 home]# groupadd volt

[root@volt1 home]# useradd -m -g volt volt

[root@volt1 home]# passwd volt

• Add entry with volt to /etc/sudoers

Defaults:volt !requiretty

volt ALL=NOPASSWD: /bin/mkdir, /bin/rmdir, /bin/ps, /bin/mount, /bin/umount, /sbin/mount, /sbin/umount

• Unpack VoltDB binaries

[volt@volt1 ~]$ tar -xvzf LINUX-voltdb-ent-5.6.tar.gz

• Turn off Transparent Huge Pages

[root@volt1 ~]# echo never >/sys/kernel/mm/transparent_hugepage/enabled

[root@volt1 ~]# echo never >/sys/kernel/mm/transparent_hugepage/defrag

• Create vFiles for persistence space

[volt@volt1 ~]$ mkdir voltdbroot2

[volt@volt1 ~]$ cd voltdbroot2/

[volt@volt1 voltdbroot2]$ pwd

/home/volt/voltdbroot2

 

• Prepare config file voltdbconfig_2.xml

<?xml version="1.0"?>

<deployment>

  <cluster hostcount="1"

           sitesperhost="1"

           kfactor="0"

  />

  <paths>

     <voltdbroot path="/home/volt/voltdbroot2" />

  </paths>

</deployment>

• Create VoltDB instance using provisioned Delphix storage

[volt@volt1 ~]$ voltdb create --deployment=voltdbconfig_2.xml
Initializing VoltDB...

 _    __      ____  ____  ____
| |  / /___  / / /_/ __ \/ __ )
| | / / __ \/ / __/ / / / __  |
| |/ / /_/ / / /_/ /_/ / /_/ /
|___/\____/_/\__/_____/_____/

--------------------------------

Build: 5.6 voltdb-5.6-0-g3743684-local Enterprise Edition
Connecting to VoltDB cluster as the leader...
Host id of this node is: 0
Starting VoltDB with trial license. License expires on Oct 26, 2015.
Initializing the database and command logs. This may take a moment...
WARN: This is not a highly available cluster. K-Safety is set to 0.
Server completed initialization.

• Check Delphix storage contents

[volt@volt1 voltdbroot2]$ ls

catalog-report.html  command_log  command_log_snapshot  config_log  dr_overflow  export_overflow  snapshots

[volt@volt1 voltdbroot2]$ pwd

/home/volt/voltdbroot2

•  Create snapshot of "voltdbroot2" object in Delphix. We have ingested and fixed current database into Delphix.

• Let us play a little bit with VoltDB instance

[volt@volt1 voltdbroot2]$ sqlcmd

SQL Command :: localhost:21212

1> CREATE TABLE towns (

2>     town VARCHAR(64),

3>     county VARCHAR(64),

4>     state VARCHAR(2)

5>  );

Command succeeded.

6> insert into towns values ('Billerica','Middlesex','MA');

(Returned 1 rows in 0.05s)

7> insert into towns values ('Buffalo','Erie','NY');

(Returned 1 rows in 0.01s)

8> insert into towns values ('Bay View','Erie','OH');

(Returned 1 rows in 0.00s)

9> select count(*) as total from towns;

TOTAL

------

     3

(Returned 1 rows in 0.02s)

10> select town, state from towns ORDER BY town;

TOWN       STATE

---------- ------

Bay View   OH

Billerica  MA

Buffalo    NY

(Returned 3 rows in 0.01s)

11>

• Now again Delphix snapshot
• Suddenly small disaster happens

[volt@volt1 ~]$ sqlcmd
SQL Command :: localhost:21212
1> select * from towns;
TOWN       COUNTY     STATE
---------- ---------- ------
Billerica  Middlesex  MA
Buffalo    Erie       NY
Bay View   Erie       OH

(Returned 3 rows in 0.02s)
2> delete from towns;
(Returned 3 rows in 0.01s)
3> select * from towns;
TOWN  COUNTY  STATE
----- ------- ------

(Returned 0 rows in 0.00s)
4> exit

• Fortunately we have Delphix snapshot done before our disaster. What we can do?

1. Shutdown VoltDB
2. Rewind to Delphix snapshot done before our unwanted operation
3. Restore VoltDB

[volt@volt1 ~]$ voltadmin shutdown
Status: -1
Information: Connection broken

Click "Rewind" button.

[volt@volt1 ~]$ voltdb recover --deployment=voltdbconfig_2.xml
Initializing VoltDB...

 _    __      ____  ____  ____
| |  / /___  / / /_/ __ \/ __ )
| | / / __ \/ / __/ / / / __  |
| |/ / /_/ / / /_/ /_/ / /_/ /
|___/\____/_/\__/_____/_____/

--------------------------------

Build: 5.6 voltdb-5.6-0-g3743684-local Enterprise Edition
Connecting to VoltDB cluster as the leader...
Host id of this node is: 0
Starting VoltDB with trial license. License expires on Oct 26, 2015.
Initializing the database and command logs. This may take a moment...
WARN: This is not a highly available cluster. K-Safety is set to 0.
Restoring from path: /home/volt/voltdbroot2/command_log_snapshot with nonce: 1442441721521
Finished restore of /home/volt/voltdbroot2/command_log_snapshot with nonce: 1442441721521 in 0.13 seconds
Server completed initialization.
 

• Now we can again select our table. Just after few seconds. In case of big databases this will be no more than 15 minutes.

[volt@volt1 ~]$ sqlcmd
SQL Command :: localhost:21212
1> select * from towns;
TOWN       COUNTY     STATE
---------- ---------- ------
Billerica  Middlesex  MA
Buffalo    Erie       NY
Bay View   Erie       OH

Conclusion

VoltDB and Delphix are great tandem. If we store our persistence data in Delphix vFiles we can have:

• Live backup
• Easy way to multiply environments. We can provision new instances of VoltDB instantly and without administrative overhead. It is easy to imagine that we can store in Delphix not only persistence space but also VoltDB binaries.
• Easy way to recover from human errors just by rewinding Delphix vFiles
• Extreme agility for test and development purposes