Great Tandem: VoltDB & Delphix
VoltDB is gaining its market share as a great real time analytics database. The buzz word is "Smart Data Fast".
Gartner Magic Quadrant for Operational Databases makes it clear.
What if we merge VoltDB real time capabilities with power of Data as a Service? Below I'm testing if it is possible to make VoltDB's persistent store reside within Delphix vFiles.
Moving Parts
- Delphix Engine: I am using Delphix Express 4.2.3.1
- Test Host: Linux CentOS 6.6
Test Steps
[root@volt1 home]# groupadd volt
[root@volt1 home]# useradd -m -g volt volt
[root@volt1 home]# passwd volt
- Add entry with volt to /etc/sudoers
Defaults:volt !requiretty
volt ALL=NOPASSWD: /bin/mkdir, /bin/rmdir, /bin/ps,
/bin/mount, /bin/umount, /sbin/mount, /sbin/umount
[volt@volt1 ~]$ tar -xvzf LINUX-voltdb-ent-5.6.tar.gz
- Turn off Transparent Huge Pages
[root@volt1 ~]# echo never
>/sys/kernel/mm/transparent_hugepage/enabled
[root@volt1 ~]# echo never
>/sys/kernel/mm/transparent_hugepage/defrag
- Create vFiles for persistence space
[volt@volt1 ~]$ mkdir voltdbroot2
[volt@volt1 ~]$ cd voltdbroot2/
[volt@volt1 voltdbroot2]$ pwd
/home/volt/voltdbroot2
- Prepare config file voltdbconfig_2.xml
<?xml version="1.0"?>
<deployment>
<cluster
hostcount="1"
sitesperhost="1"
kfactor="0"
/>
<paths>
<voltdbroot
path="/home/volt/voltdbroot2" />
</paths>
</deployment>
- Create VoltDB instance using provisioned Delphix storage
[volt@volt1 ~]$ voltdb create --deployment=voltdbconfig_2.xml
Initializing VoltDB...
_ __ ____ ____ ____
| | / /___ / / /_/ __ \/ __ )
| | / / __ \/ / __/ / / / __ |
| |/ / /_/ / / /_/ /_/ / /_/ /
|___/\____/_/\__/_____/_____/
--------------------------------
Build: 5.6 voltdb-5.6-0-g3743684-local Enterprise Edition
Connecting to VoltDB cluster as the leader...
Host id of this node is: 0
Starting VoltDB with trial license. License expires on Oct 26, 2015.
Initializing the database and command logs. This may take a moment...
WARN: This is not a highly available cluster. K-Safety is set to 0.
Server completed initialization.
- Check Delphix storage contents
[volt@volt1 voltdbroot2]$ ls
catalog-report.html
command_log
command_log_snapshot
config_log dr_overflow export_overflow snapshots
[volt@volt1 voltdbroot2]$ pwd
/home/volt/voltdbroot2
- Create snapshot of "voltdbroot2" object in Delphix. We have ingested and fixed current database into Delphix.
- Let us play a little bit with VoltDB instance
[volt@volt1 voltdbroot2]$ sqlcmd
SQL Command :: localhost:21212
1> CREATE TABLE towns (
2> town
VARCHAR(64),
3> county
VARCHAR(64),
4> state
VARCHAR(2)
5> );
Command succeeded.
6> insert into towns values
('Billerica','Middlesex','MA');
(Returned 1 rows in 0.05s)
7> insert into towns values
('Buffalo','Erie','NY');
(Returned 1 rows in 0.01s)
8> insert into towns values ('Bay
View','Erie','OH');
(Returned 1 rows in 0.00s)
9> select count(*) as total from towns;
TOTAL
------
3
(Returned 1 rows in 0.02s)
10> select town, state from towns ORDER BY town;
TOWN STATE
---------- ------
Bay View OH
Billerica MA
Buffalo NY
(Returned 3 rows in 0.01s)
11>
- Now again Delphix snapshot
- Suddenly small disaster happens
[volt@volt1 ~]$ sqlcmd
SQL Command :: localhost:21212
1> select * from towns;
TOWN COUNTY STATE
---------- ---------- ------
Billerica Middlesex MA
Buffalo Erie NY
Bay View Erie OH
(Returned 3 rows in 0.02s)
2> delete from towns;
(Returned 3 rows in 0.01s)
3> select * from towns;
TOWN COUNTY STATE
----- ------- ------
(Returned 0 rows in 0.00s)
4> exit
- Fortunately we have Delphix snapshot done before our disaster. What we can do?
- Shutdown VoltDB
- Rewind to Delphix snapshot done before our unwanted operation
- Restore VoltDB
[volt@volt1 ~]$ voltadmin shutdown
Status: -1
Information: Connection broken
Click "Rewind" button.
[volt@volt1 ~]$ voltdb recover --deployment=voltdbconfig_2.xml
Initializing VoltDB...
_ __ ____ ____ ____
| | / /___ / / /_/ __ \/ __ )
| | / / __ \/ / __/ / / / __ |
| |/ / /_/ / / /_/ /_/ / /_/ /
|___/\____/_/\__/_____/_____/
--------------------------------
Build: 5.6 voltdb-5.6-0-g3743684-local Enterprise Edition
Connecting to VoltDB cluster as the leader...
Host id of this node is: 0
Starting VoltDB with trial license. License expires on Oct 26, 2015.
Initializing the database and command logs. This may take a moment...
WARN: This is not a highly available cluster. K-Safety is set to 0.
Restoring from path: /home/volt/voltdbroot2/command_log_snapshot with nonce: 1442441721521
Finished restore of /home/volt/voltdbroot2/command_log_snapshot with nonce: 1442441721521 in 0.13 seconds
Server completed initialization.
- Now we can again select our table. Just after few seconds. In case of big databases this will be no more than 15 minutes.
[volt@volt1 ~]$ sqlcmd
SQL Command :: localhost:21212
1> select * from towns;
TOWN COUNTY STATE
---------- ---------- ------
Billerica Middlesex MA
Buffalo Erie NY
Bay View Erie OH
Conclusion
VoltDB and Delphix are great tandem. If we store our persistence data in Delphix vFiles we can have:
- Live backup
- Easy way to multiply environments. We can provision new instances of VoltDB instantly and without administrative overhead. It is easy to imagine that we can store in Delphix not only persistence space but also VoltDB binaries.
- Easy way to recover from human errors just by rewinding Delphix vFiles
- Extreme agility for test and development purposes
