Wednesday 18 November 2015

Delphix Developer Edition, Bonita and Docker

Delphix Developer Edition, Bonita and Docker


What if you want to spin up quickly new Bonita BPM application? Delphix DE and Docker are great choice to do this!

Delphix Developer Edition is a part of Lanshark demo environment or can be requested seperately.

More information here:

Delphix Express GUI

Delphix Express vs Delphix Enterprise
Landshark environment together with Delphix Express engine contains two VMs. Linuxsource VM and Linuxtarget VM have preinstalled Postgresql, Oracle XE and more...


"Bonita BPM is an open-source business process management and workflow suite created in 2001. It was started in France National Institute for Research in Computer Science, and then had incubated
years inside of the French computer science company Groupe Bull. Since 2009, the development of Bonita is supported by a company dedicated to this activity: Bonitasoft. ..."

Bonita BPM

Show Time

We are using:

  • Delphix Express
  • Landshark source VM (linuxsource)
  • Landshark target VM (linuxtarget)
  • Official Docker Bonita image (  


  1. Register Postgresql Data Source (dSource) - 1 time action
  2. Provision Postgresql cloned database
  3. Spin up Bonita docker image

Let's do it!

1. Data Source

Menu Environments
Open "Environments"
Choose "Databases"
Scroll down to postgresql part.
Click "Add dSource"

After clicking "Add dSource"
Enter user "postgres" password "delphix". Next.
Change name to "usda".

Click on Delphix logo. Left upper corner.

dSource "usda"

2. VDB (Virtual DB)

Click on "usda".

"Provision" in right lower corner.

Click on "Provision".

Provisioning. First Tab.
Choose LINUXTARGET. Enter port number 5433. Click advanced.

Enter new rule in pg_hba tab.
Enter rule "host   all   all   trust" in pg_hba tab. Next. Next. Next. Finish.

VDB Ready!

3. Docker part.

  • Login to linuxsource host. User root password delphix.
  • Enter following commands staying in /root
chkconfig docker on
service docker start

export PGUSER=postgres
export PGPASSWORD=delphix
export PGHOST=`hostname --ip-address`
export PGDATABASE=usda
export PGPORT=5433
psql -U $PGUSER  -h $PGHOST -d $PGDATABASE --command "DROP DATABASE bonita;"
psql -U $PGUSER  -h $PGHOST -d $PGDATABASE --command "DROP USER bonita;"
psql -U $PGUSER  -h $PGHOST -d $PGDATABASE --command "DROP DATABASE businessdb;"
psql -U $PGUSER  -h $PGHOST -d $PGDATABASE --command "DROP USER businessuser;"
psql -U $PGUSER  -h $PGHOST -d $PGDATABASE --command "CREATE DATABASE bonita OWNER bonita;"

rm -rf bonita
mkdir -p bonita

docker run -e DB_VENDOR=postgres -e DB_HOST=$PGHOST -e DB_NAME=bonita -e DB_USER=bonita -e DB_PASS=bpm -e DB_ADMIN_PASS=delphix -e DB_PORT=5433 -e TENANT_LOGIN=install -e TENANT_PASSWORD=install --name bonita_custom -v /root/bonita:/opt/bonita -d -p 8080:8080 bonita
  •  Open url "http://linuxtarget:8080/bonita" in your browser.

Bonita BPM using VDB ready to go!


  • You can use Delphix VDBs together with Docker containerized application without much effort. It's very easy.
  • You can deliver data to containerized applications in agile way. No expensive data duplication. No storage waste. Fast!


Thursday 5 November 2015

MemSQL & Delphix - Win Win Option

MemSQL & Delphix Express - Win Win Option


  • Delphix Express

Delphix: "Delphix Positioned as a Leader in the Gartner Magic Quadrant for Structured Data Archiving and Application Retirement" 



About Delphix:
  • MemSQL 4.1.9 Community Edition

According to Wikipedia (

"MemSQL is a distributed, in-memory database that is part of the NewSQL movement.[2] It is an ACID-compliant RDBMS that most notably converts SQL into C++ through code generation.[3] It is being developed by MemSQL Inc., that was founded in 2011 and is a graduate of the Y Combinator startup program. MemSQL Inc. has raised more than $40 million to date from a variety of investors including First Round Capital, IA Ventures, NEA, and several prominent angels including Paul Buchheit, Max Levchin, Aaron Levie, and Ashton Kutcher.[4] MemSQL Inc. launched its database to the public on June 18, 2012."

MemSQL & Gartner:
"Gartner Positions MemSQL in the Visionaries Quadrant of the Magic Quadrant for Operational Database Management Systems
MemSQL scores high for its completeness of vision, technology innovation and ability to execute against its R&D model"

Source: Home Page


I'm trying to test 2 things
  1. Is it possible to use Delphix vFiles functionality to handle persistence in MemSQL?
  2. What is its the impact on MemSQL performance?

Test Environment

  • Delphix Express VM

Delphix Express in VMWare Player 7

Delphix Express Console

  • Centos VM for MemSQL

MemSQL Linux VM

MemSQL Linux VM Box Parameters

Test Steps Breakdown

  1. Install MemSQL
  2. Create a table and fill with some data
  3. Test performance (using MemSQL Ops)
  4. Shutdown MemSQL
  5. Replace Data directory with resource provisioned with Delphix
  6. Copy existing data to new provisioned resource
  7. Startup MemSQL
  8. Check if everything works
  9. Check performance again (using MemSQL Ops)
  10. Conclusions

Test Steps

1. Install MemSQL

[memsql@volt1 ~]$ tar -xvzf memsql-ops-4.1.9.tar.gz
[memsql@volt1 ~]$ cd memsql-ops-4.1.9
[memsql@volt1 memsql-ops-4.1.9]$ ls  memsql-ops  README
[memsql@volt1 memsql-ops-4.1.9]$ sudo

MemSQL Ops

2. New Table and Some Data

[memsql@volt1 memsql]$ mysql -u root -h volt1 -P 3306 --prompt="memsql> "
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 5
Server version: 5.5.8 MemSQL source distribution (compatible; MySQL Enterprise & MySQL Commercial)

Copyright (c) 2000, 2013, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

memsql> -- create a new user
memsql> grant all on *.* to 'myuser'@'%' identified by 'mypassword';
Query OK, 0 rows affected (0.38 sec)

memsql> -- show all memsql leaf nodes
memsql> show leaves;
| Host            | Port | Availability_Group | Pair_Host | Pair_Port | State  | Opened_Connections | Average_Roundtrip_Latency_ms |
| | 3307 |                  1 | NULL      |      NULL | online |                  1 |                        0.255 |
1 row in set (0.00 sec)
memsql> -- create a new database
memsql> create database mymemsqldb;
Query OK, 1 row affected (3.25 sec)

memsql> show databases;
| Database           |
| information_schema |
| memsql             |
| mymemsqldb         |
| sharding           |
4 rows in set (0.01 sec)

memsql> use mymemsqldb;
Database changed
memsql> -- create a new table
memsql> create table mytable (id INT PRIMARY KEY, name varchar(10));
Query OK, 0 rows affected (10.03 sec)

memsql> show tables;
| Tables_in_mymemsqldb |
| mytable              |
1 row in set (0.00 sec)

memsql> describe mytable;
| Field | Type        | Null | Key  | Default | Extra |
| id    | int(11)     | NO   | PRI  | NULL    |       |
| name  | varchar(10) | YES  |      | NULL    |       |
2 rows in set (0.01 sec)

memsql> -- perform data query operations
memsql> insert into mytable values (1, 'hello');
Query OK, 1 row affected (1.66 sec)

memsql> select * from mytable;
| id | name  |
|  1 | hello |
1 row in set (3.10 sec)

3. Test performance (using MemSQL Ops)


Reference Performance Score


4. Shutdown MemSQL

 [memsql@volt1 memsql]$ memsql-ops memsql-stop
 Index  ID       Agent Id  Process State  Cluster State  Role    Host             Port
 1      DB6DBFC  Ad92cbe   RUNNING        CONNECTED      MASTER  3306
 2      F0031A1  Ad92cbe   RUNNING        CONNECTED      LEAF  3307
Select a MemSQL node: 2
2015-11-05 19:24:33: J60c2bf [INFO] Stopping MemSQL node F0031A1E49417690DC51BD4F718863DCD901DE6B on Agent Ad92cbe2bc08845b8b5958b69673e26ec
2015-11-05 19:24:38: J60c2bf [INFO] Successfully stopped MemSQL node F0031A1E49417690DC51BD4F718863DCD901DE6B
[memsql@volt1 memsql]$ memsql-ops memsql-stop
 Index  ID       Agent Id  Process State  Cluster State  Role    Host             Port
 1      DB6DBFC  Ad92cbe   RUNNING        CONNECTED      MASTER  3306
 2      F0031A1  Ad92cbe   NOT RUNNING    OFFLINE        LEAF  3307
Select a MemSQL node: 1
2015-11-05 19:24:44: J476b0d [INFO] Stopping MemSQL node DB6DBFCC07F087C826D73597A7F2488A5E7F51DD on Agent Ad92cbe2bc08845b8b5958b69673e26ec
2015-11-05 19:24:47: J476b0d [INFO] Successfully stopped MemSQL node DB6DBFCC07F087C826D73597A7F2488A5E7F51DD

 5. Replace Data directory with resource provisioned with Delphix

[memsql@volt1 leaf-3307-1]$ pwd
We have to find data folder and replace it with Delphix provisioned folder

[memsql@volt1 data]$ pwd

Data folder renamed.
Data folder of the MemSQL leaf renamed and new empty folder created.

Environment Registered in Delphix
We have to register host with MemSQL in Delphix.

Provisioning Delphix Storage to MemSQL Host
Next we have to provision Delphix storage to volt1 host and mount it in new empty data folder.

Provisioning to Path /var/lib/memsql/leaf-3307-1/data

Storage Provisioned to /var/lib/memsql/leaf-3307-1/data
We can check this using mount command.

Storage mounted from Delphix platform

6. Copy existing data to new provisioned resource

Now we are copying contents of old __data__ folder to provisioned one.

Copying Data to Delphix Storage
Now snapshot of data in Delphix.

Snapshot done

Now time for starting MemSQL nodes.

7. Startup MemSQL

Nodes started.

MemSQL Ops shows running nodes
Success! Nodes are up and running. Now SQL check.

8. Check if everything works

Database is running fine. Select statement returned correct response.

9. Check performance again (using MemSQL Ops)

Performance after switching to Delphix storage

 10. Conclusions

  • It is possible to use Delphix storage as persistence store for MemSQL nodes.
  • Performance is not affected. 
  • Technically using Delphix provisioned storage has to be orchestrated using some automation tool. We have to replace storage for every leaf node.


Possibility of symbiosis between MemSQL and Delphix is really interesting.
  • We get live backup of MemSQL.
  • We get amazing refresh/rewind/clone features of Delphix.
  • We optimize storage required for persistence purposes.
  • What more?