Add multi host support

toolset/deployment/vagrant-multi/.gitignore

@@ -0,0 +1 @@
+.vagrant/

toolset/deployment/vagrant-multi/README.md

@@ -0,0 +1,143 @@
+# Vagrant Automated Deployment
+
+Currently, our [Vagrant](https://www.vagrantup.com) setup can automatically
+setup a development environment running in a single virtual machine. 
+This virtual machine can run inside either [VirtualBox](https://www.virtualbox.org/) or [Amazon EC2](http://aws.amazon.com/ec2/). Feel free to submit a PR 
+to help us support more providers, such as [rackspace](http://www.rackspace.com/)
+or [VMware](http://www.vmware.com/). 
+
+This single VM runs the `app server`, `client server`, and `database server`, 
+and allows you to rapidly test if your framework/framework changes integrate
+properly with TFB. While you *can* run a full benchmark using 
+this virtual machine, your results would be quite useless. 
+
+## Prerequisites
+
+* **A recent version of Vagrant**, like 1.6.3 (NOTE: `apt-get` is 
+too old, download the newest `deb` directly). See 
+[here](https://www.vagrantup.com/downloads.html) for downloads.
+
+* **A CPU that can virtualize a 64-bit virtual OS**, because TFB
+downloads a number of static binaries that require 64-bit. See
+the FAQs section below for more on this
+
+* **VirtualBox**, if you plan to run the Virtual Machine locally. If 
+you plan to solely use Amazon-based Virtual Machines you can skip this. 
+
+## Using Vagrant to Run VirtualBox-powered Virtual Machine
+
+In short, you need to clone the project and then run `vagrant up` 
+followed by `vagrant ssh`. Your cloned working directory will be 
+exposed inside the VM at `~/FrameworkBenchmarks`, so you can 
+continue to edit files using your favorite IDE and the changes will
+show up inside the virtual machine. 
+
+Details: 
+
+```bash
+# Go into the right directory
+$ cd FrameworkBenchmarks/toolset/deployment/vagrant
+# Setup the VM with all initial software (takes 15-20 minutes)
+$ vagrant up
+# SSH into the machine (you get a nice welcome message)
+$ vagrant ssh
+Welcome to the FrameworkBenchmarks project!
+   
+To get started, perhaps try this:
+   $ cd FrameworkBenchmarks
+   $ toolset/run-tests.py --install server --test go
+   $ cat results/ec2/latest/logs/go/out.txt
+
+You can get lots of help:
+   $ toolset/run-tests.py --help
+
+This Vagrant environment is already setup and ready to go, so you
+can ignore any flags about users, hosts, or identity files
+```
+
+## Using Vagrant to Run Amazon-powered Virtual Machine
+
+The high level steps are similar, run `vagrant up --provider=aws` followed 
+by `vagrant ssh`. The main differences are 1) you need to provide a 
+number of environment variables to let vagrant log into your AWS account, 
+and 2) your git working copy will not be synced with the remote virtual 
+machine (by default). 
+
+#### Required Environment Variables for Using Amazon Web Services
+
+These are all required: 
+
+    TFB_AWS_ACCESS_KEY : Your Amazon Web Services Access Key
+    TFB_AWS_SECRET_KEY : Your Amazon Web Services Secret Access Key
+    TFB_AWS_KEY_NAME   : The name of the keypair you are using
+    TFB_AWS_KEY_PATH   : Path to the *.pem file for the keypair you are using
+
+You can declare these in your `~/.bash_profile`, using lines like:
+
+    export TFB_AWS_ACCESS_KEY="<your access key>"
+
+Or you can specify them on the command line as so: 
+
+    $ TFB_AWS_ACCESS_KEY="<your key>" TFB_AWS_SECRET_KEY="<your secret>" TFB_AWS_KEY_NAME="<your keypair name>" TFB_AWS_KEY_PATH="/home/you/.ssh/your_key.pem" vagrant up --provider aws
+
+#### Directoy Syncing and Amazon Web Services
+
+By default, our Vagrant setup will not syncronize your local git working copy 
+with the remote filesystem on Amazon. Instead, the `~/FrameworkBenchmarks` 
+folder will be a fresh clone of github.com/TechEmpower/FrameworkBenchmarks. 
+
+This is due to the project currently being multiple GB in size (see #1050). 
+Synchronizing the local filesystem with the remote filesystem would require an
+initial upload to the remote system, and most people do not want to wait for 
+a multi-GB upload. 
+
+If you are interested in having this synchronization, you can use set an 
+environment variable as so: 
+
+    $ TFB_FORCE_SYNC=true vagrant up --provider aws
+
+This will tell vagrant to keep your git working copy in sync with the remote
+filesystem. While the initial upload is quite arduous, any later changes are
+synchronized quite quickly (the underlying system is rsync). This would 
+allow you to continue editing files on your local machine and then using the 
+remote virtual machine to test your edits. As long as you don't run `vagrant 
+destroy`, any later synchronization shouldn't take more than a few seconds. 
+
+
+### FAQs
+
+**I'm using a 32-bit computer, can I run your Virtual Machine?**: 
+
+If 1) your CPU provides the [vmx or smv](http://en.wikipedia.org/wiki/X86_virtualization) features, and 2) they are enabled
+in your BIOS and 3) they are enabled in Virtualbox, then yes. 
+The easiest way to check this all is to attempt to launch the VM 
+without hiding the window:
+
+    $ TFB_SHOW_VM=true vagrant up
+
+If you see a boot sequence and a login, you're good to go. If you 
+only see a black screen, examine if you meet all three requirements. 
+If you do not, then you can either 1) run on Amazon or 2) try your
+luck with a 32-bit virtual machine. A number of the downloaded 
+framework binaries are 64-bit only, so those will be unable to run. 
+
+To force FrameworkBenchmarks to use a 32-bit VM, do this: 
+     
+    $ TFB_ARCH=32 vagrant up
+
+See [here](http://askubuntu.com/questions/41550) for some helpful information.  
+
+**How much does running on Amazon cost?**: 
+
+Roughly `$1/day` with default settings. 
+
+Running `vagrant up --provider aws` takes about 20 minutes to complete. Running 
+`toolset/run-tests.py --install server --mode verify --test <foo>` takes between
+5 and 60 minutes, depending on `<foo>`. [Prices as of writing](http://aws.amazon.com/ec2/previous-generation/) show the default `m1.small` instance type is 
+`$0.044` per Hour, or roughly `$1/day`
+
+Running `toolset/run-tests.py --mode benchmark --test <foo>` takes between 15-60
+minutes per test in `<foo>`, and running the full benchmark would likely take 
+between 1 and 5 days of continuous execution. Note that this single VM runs the 
+framework, the load generation, and the database, and is therefore not a stable 
+benchmarking setup. 

toolset/deployment/vagrant-multi/Vagrantfile

@@ -0,0 +1,72 @@
+# -*- mode: ruby -*-
+# vi: set ft=ruby :
+
+Vagrant.configure("2") do |config|
+  
+  # Put the keys inside each box
+  Dir['keys/*'].each do |fname|
+    config.vm.provision :file do |file|
+      file.source = fname
+      file.destination = "~/.ssh/" + File.basename(fname)
+    end
+  end
+
+  # Build the DB and client servers before the 
+  # app server
+  config.vm.define "db" do |db|
+    db.vm.hostname = "TFB-dbserver"
+    db.vm.box = "ubuntu/trusty64"
+    db.vm.network "private_network", ip: "172.16.16.18"
+    db.vm.synced_folder "../../..", "/FrameworkBenchmarks"
+
+    # Database-specific setup
+    db.vm.provision "shell" do |sh|
+      sh.inline = "cat ~/.ssh/database.pub >> ~/.ssh/authorized_keys"
+      sh.privileged = false
+    end
+
+    db.vm.provision "shell" do |sh|
+      sh.path = "bootstrap.sh"
+      sh.privileged = false
+      sh.args = "database"
+    end
+  end
+
+  # Build the DB and client servers before the 
+  # app server
+  config.vm.define "load" do |load|
+    load.vm.hostname = "TFB-loadserver"
+    load.vm.box = "ubuntu/trusty64"
+    load.vm.network "private_network", ip: "172.16.16.17"
+    load.vm.synced_folder "../../..", "/FrameworkBenchmarks"
+
+    load.vm.provision "shell" do |sh|
+      sh.inline = "cat ~/.ssh/client.pub >> ~/.ssh/authorized_keys"
+      sh.privileged = false
+    end
+
+    load.vm.provision "shell" do |sh|
+      sh.path = "bootstrap.sh"
+      sh.privileged = false
+      sh.args = "client"
+    end
+  end
+
+  # Define the app server as the primary VM
+  config.vm.define "app", primary: true do |app|
+    app.vm.hostname = "TFB-appserver"
+    app.vm.box = "ubuntu/trusty64"
+    app.vm.network "private_network", ip: "172.16.16.16"
+    app.vm.synced_folder "../../..", "/FrameworkBenchmarks"
+    app.vm.network :forwarded_port, guest: 8080, host: 28080
+
+    app.vm.provision "shell" do |sh|
+      sh.path = "bootstrap.sh"
+      sh.privileged = false
+      sh.args = "server"
+    end
+  end
+
+
+  
+end

toolset/deployment/vagrant-multi/bootstrap.sh

@@ -0,0 +1,58 @@
+#!/usr/bin/env bash
+#
+# Prepares a virtual machine for running TFB
+#
+# Intentionally uses ~, $HOME, and $USER so that the 
+# same script can work for VirtualBox (username vagrant)
+# and Amazon (username ubuntu)
+
+# Setup some nice TFB defaults
+echo "export TFB_SERVER_HOST=172.16.16.16" >> ~/.bash_profile
+echo "export TFB_CLIENT_HOST=172.16.16.17" >> ~/.bash_profile
+echo "export TFB_DATABASE_HOST=172.16.16.18" >> ~/.bash_profile
+echo "export TFB_CLIENT_USER=$USER" >> ~/.bash_profile
+echo "export TFB_DATABASE_USER=$USER" >> ~/.bash_profile
+echo "export TFB_CLIENT_IDENTITY_FILE=$HOME/.ssh/client" >> ~/.bash_profile
+echo "export TFB_DATABASE_IDENTITY_FILE=$HOME/.ssh/database" >> ~/.bash_profile
+echo "export FWROOT=$HOME/FrameworkBenchmarks" >> ~/.bash_profile 
+if [ -e "~/FrameworkBenchmarks/benchmark.cfg" ]; then
+  echo "You have a benchmark.cfg file that will interfere with Vagrant, moving to benchmark.cfg.bak"
+  mv ~/FrameworkBenchmarks/benchmark.cfg ~/FrameworkBenchmarks/benchmark.cfg.bak
+fi
+source ~/.bash_profile
+
+# Enable SSH to localhost
+ssh-keygen -t rsa -N '' -f ~/.ssh/id_rsa
+cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
+chmod 600 ~/.ssh/authorized_keys
+
+# Workaround mitchellh/vagrant#289
+echo "grub-pc grub-pc/install_devices multiselect     /dev/sda" | sudo debconf-set-selections
+
+# Install prerequisite tools
+sudo apt-get install -y git
+sudo apt-get install -y python-pip
+
+# Initial project setup
+if [ -d "/FrameworkBenchmarks" ]; then
+  ln -s /FrameworkBenchmarks $FWROOT
+  echo "Removing any install or results files so they do not interfere"
+  rm -rf $FWROOT/installs $FWROOT/results
+else
+  # If there is no synced folder, clone the project
+  git clone https://github.com/TechEmpower/FrameworkBenchmarks.git $FWROOT
+fi
+sudo pip install -r $FWROOT/config/python_requirements.txt
+
+# Setup 
+cd $FWROOT
+toolset/run-tests.py --verbose --install $1 --install-only --test ''
+
+# Setup a nice welcome message for our guest
+sudo rm -f /etc/update-motd.d/51-cloudguest
+sudo rm -f /etc/update-motd.d/98-cloudguest
+sudo cp /vagrant/custom_motd.sh /etc/update-motd.d/55-tfbwelcome
+
+touch ~/.firstboot
+
+

toolset/deployment/vagrant-multi/custom_motd.sh

@@ -0,0 +1,15 @@
+#!/bin/sh
+
+echo "$(tput setaf 4)"
+echo "Welcome to the FrameworkBenchmarks project!"
+echo ""
+echo "  To get started, perhaps try this:"
+echo "    \$ cd FrameworkBenchmarks"
+echo "    \$ toolset/run-tests.py --install server --test go"
+echo "    \$ cat results/ec2/latest/logs/go/out.txt"
+echo ""
+echo "  You can get lots of help:"
+echo "    \$ toolset/run-tests.py --help"
+echo ""
+echo "  This Vagrant environment is already setup and ready to go, so you"
+printf "  can safely ignore any flags about users, hosts, or identity files $(tput sgr 0)"

toolset/deployment/vagrant-multi/keys/client

@@ -0,0 +1,27 @@
+-----BEGIN RSA PRIVATE KEY-----
+MIIEpAIBAAKCAQEAqbwcjGjUhqqlTKt8CPiS7IyZr3AHV/ZY/lU5hqixDtX0rNUU
+LY0gSNNgDOLsk4refgZbqxUlUykcjhYv4Lbmkav7ru2cqkQcyGcXMvCluh4rZSCe
+1omCsu8RHCTVY0bR/A2/tZifmIEPMbqVxJYyYxUOeyTJfvO/SRGJDuoKNAEtpuYV
+JtvwOm6G6qHEAscrHD9aLrOVfkhUfhZ5Ikf5pehnxKI5J+i2uMGOsT4XS5BCVcMG
+ZDpy0EjNQCQN738fjbCuSWT6N+bURAKZpAayLOaJJzA0ckJwuBLN6+URnYeyT+JL
+/hAXSWC6YSdfGf7ZRzKlQ+P1keG+mnbU2rR8ZQIDAQABAoIBAB3AEgTuTun3uRdy
+K0BrSV1a24kriq35SOnzSCz5HmeqSsOMaYf+Z308R2aa07qcqZjPRnv7Ldqgpxek
+fenWRR2dVnT3wvD5FU5u+1r7YSD5LsP/7rdNzPHOA2K6dgh7nkyeoE6VHMwfOumN
+ebjGVriVNT+SWBr/YDTPdQV/MiChFyR2HgfT9wkDS4rtML1WpaQeNVHSyHNhjCdQ
+CHV8yLJGrPopXs2+DlD/QY3y9i6S6nFFLQCDgZYMW21LgtWIHdQ53xYnfR3qXrB6
+zqq5Qys0y283acRG/jwNlyu4o2K/Z95fqH9+WIn99j89UH89WUy8Ps4EM1yiik1g
+o61e+WECgYEA1jp1qJmwIz0jbV04Sf+IYspsGfdTBYxcQ3KcKfkUMOZ8t8pO7O96
+vY8JozZIyzWrh5yFZGqNBPKMAeBvFbaU3py+vBEHF7fpNsvo7TKXqx539wbThUPO
+uKrtw6+KQOa5vJfi8ASCvbzqD+hDQBseOEqiAGKJSZQy0WHMuImBMzkCgYEAytSw
+BHoMvK0wnJE045HpeM7l30skhHx/NaREUK/k0g/dc74C9Q6fwlDXCorhl+m+G3aX
+djZflOvYh60yajyjTyE5MBA9L2C8GldfjzbYYE02av3pQ+GGoN9wm2Z1FpKBU51m
+AwFme3AxkuObRhhl/7kGrmmzQCFMcijXup1ndo0CgYEAj+teOBJn3oSoVESYGD5v
+Zc8PqLIbOFL/eOB6QUhwZBm0F9T2CP8pukdp9hf1927q+YQRqVBbxMMdg/U/K5sp
+kBRFnxrhZPHM319F8HRYqEEj3vTbDWBsdGuAqEI54zgd97EiaJTfMJDFvkIPS/Xr
+621YQGHyUVgae8f0PNVD0yECgYAampMLw0ZwaPb7UkBxqO3Knrif9ScxCXNWAdnE
+C44jz1jomX+hzO0/e4L5KwYz+sCosYFkS/V1fwnDof/+1z1EX8M0fqzHzhReEB/E
+lRkEaJw+pFc4RBh4eVY3y9SHxWeZJAgBVOmwPyMBdUEkjIa0Z/xsXAja3IM+ZgUF
+wSeU8QKBgQDRv/nbrPc06JFYhGI8U+fE6Y2Qy1k5XQzZ+lAi7zHVmlS3qHpaltgM
+eU6JI34RByAofTppC4AmFI5y8Y/XYiD+yweDQzeI6jgnVyzBM/lNMdPkCtg/MoWt
+lqiAWjrXBHfTUNqtXyf8eQG8YB5Llv0Di3K8JbIS6LA2VCIje+aI0g==
+-----END RSA PRIVATE KEY-----

toolset/deployment/vagrant-multi/keys/client.pub

@@ -0,0 +1 @@
+ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpvByMaNSGqqVMq3wI+JLsjJmvcAdX9lj+VTmGqLEO1fSs1RQtjSBI02AM4uyTit5+BlurFSVTKRyOFi/gtuaRq/uu7ZyqRBzIZxcy8KW6HitlIJ7WiYKy7xEcJNVjRtH8Db+1mJ+YgQ8xupXEljJjFQ57JMl+879JEYkO6go0AS2m5hUm2/A6bobqocQCxyscP1ous5V+SFR+FnkiR/ml6GfEojkn6La4wY6xPhdLkEJVwwZkOnLQSM1AJA3vfx+NsK5JZPo35tREApmkBrIs5oknMDRyQnC4Es3r5RGdh7JP4kv+EBdJYLphJ18Z/tlHMqVD4/WR4b6adtTatHxl [email protected]

toolset/deployment/vagrant-multi/keys/database

@@ -0,0 +1,27 @@
+-----BEGIN RSA PRIVATE KEY-----
+MIIEpAIBAAKCAQEAsA0K9e7N+A1CvdxVUfM1GWR9rM/Kyo703MRw9TZHdI15QgUU
+VwIGLoNyaGeQJMfbDY6k3QzC1y2DmnVkRgy55Q8LLXN7bfHcgme8Yie1fsK9YNrN
+nzfEbzwsO+8t9tR7IJufWQ36TDl0pskBOuArABFIh22ZMD4iKc1SWPqlZYHWdA4u
+LWlQ5ZmqOhUCCdtYyQoKq/IF2G0ahiYhpEmdf+zmTvHc7BZxz0wdHCPbAc3JMMBQ
+XcjgHEPb2qVqMzx5V0X5WH7w42Q0rYkr69efJJmjj86kL5ZIMrHhNPUjGdRyw6IO
+4WK8osBoOZnCfqr668F8Nso7heOM/f8EeCkJLwIDAQABAoIBAFDurgtBLQ7LidMe
+Z1q1I8P4no3DOMbF0WtsI0GXImJCZ2wiSad2FsFrTD0S3wEJn3osCT5Vj3y8P7PS
+I7bQQuU6dTj/zc5NUcHhTokRGC6d99F+xOhdCsDYOL2Uxtyh8bib0jVuZ5KDF01V
+OmuGgj4Vs/IOQE4dQjdVXHXrQ/3VXx0xsKgVmepNUBeTd1Vvxu6LPxPLcjj3ViC/
+fADuv23sL6kqSCSICDcCYxM2jmEMN3XTHVqBubm/V7x9QaS13EyoehzqzM0R4z7f
+hFtsgrtZJ+NautzgvVq4wcBFATzQXVl08vVN/gJmf4gq92GGGzo+HU3q0C7A+bkC
+0jdJ3YECgYEA3GVcGpmzbXEpVGgkaTcx3DwSGoiJ7tbuTS84F4I5p9uOILX2JDyu
+tIicfVb0YFtnZ7wJ7ggZZUdz9gWV4TIpmZsjDDre/7TJq53z2dQpvA8DLfw1G+KN
+WjNH/o5f00hKgLgNKaApkSiENvvmXlKqLSUOigk84uOXLKXp6MdwD8UCgYEAzH3D
+OY+UJ7RGgl8HckXBCYxn+9ajuOOBHvqR5BAGcJJ2LSYErugYpGeh8W46v+vq/7Ar
+jIAcV6hqDmG62ibOErIr13QMn0V0dz6Zgf20bbCcAYCLRcFP8kA2HfxP9ntUs48q
+M+J0sG+6efzDB54W2xi8Qz1jwrtwmpVKcUFGMGMCgYEAgnwcQL95JLWXw5p43DZK
+ddq22TC3gUmNAvOO4RQ/sGrjxegVV7T5tncfN3IQ8y+KlDcXJe9zoavhMupxp0d8
+q63W3TnJp1jRAaNdqTzcjLPSrRoiE2VZTOsJl+ORRlUJjxXo7RkfIwr65FbVOEj2
+6g2DgyQfATG/4jZEu4XAigUCgYEAnb82ZqT36VkMO24LVH+y1ibV4FRHoE0AQmGR
+QZmLHjQdrd5yJmss90ZqitA5yeu4MN45+fyp4IUuq0GA3uCof4uqfl4rjoIJwiMs
+UDbihOuErTP4PCtP/NTD6oXFOaMSSCHcsIG0+pZ2B7yQxsORZhrCwCwPVR07VERu
+VMfB2XkCgYA1Fp/pThnPQAOjYUftj9LSTj6Zm+IYZKyv3oXiO6IGjc4MWvpRmyj+
+G/gXbCZhL40z0hGIHMdyeu3ddQ+pBSncI0ycccbRGvPbITw9EXs+uzFflaYR/ocf
+v0hOmP/FvhuKH6shk8QjMPYENt5M5dRDZMujqtIgD+p40z98YqIxow==
+-----END RSA PRIVATE KEY-----

toolset/deployment/vagrant-multi/keys/database.pub

@@ -0,0 +1 @@
+ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCwDQr17s34DUK93FVR8zUZZH2sz8rKjvTcxHD1Nkd0jXlCBRRXAgYug3JoZ5Akx9sNjqTdDMLXLYOadWRGDLnlDwstc3tt8dyCZ7xiJ7V+wr1g2s2fN8RvPCw77y321Hsgm59ZDfpMOXSmyQE64CsAEUiHbZkwPiIpzVJY+qVlgdZ0Di4taVDlmao6FQIJ21jJCgqr8gXYbRqGJiGkSZ1/7OZO8dzsFnHPTB0cI9sBzckwwFBdyOAcQ9vapWozPHlXRflYfvDjZDStiSvr158kmaOPzqQvlkgyseE09SMZ1HLDog7hYryiwGg5mcJ+qvrrwXw2yjuF44z9/wR4KQkv [email protected]