Updated how we invoke the start scripts

Before we were calling Popen which spawns a new process
and continues in a non-blocking way. This can (and did)
lead to some longer-running setup scripts to not be ready
when we tried to verify. Additionally, it was hard to
determine the error in the case where the setup.sh file
failed to run successfully.

Therefore, I can changed the call back to a blocking one
(check_call) and added back the retcode and logging in
the case of errors. I tested this and if you return a
non-zero result, you get some nice logging, and if you
return nothing it continues as expected.

One caveat to this is that the startup scripts needed a
way to run a given script in a non-blocking way. For
example, when frameworkX starts the server, it just sits
and waits which would never return. Therefore, any/all
tests that perform this way should invoke their server
start code in the background, which will not block.

Example:
java myserver.jar &

That will start a process in the background as the current
user and move on.

frameworks/C++/ULib/setup.sh

@@ -11,4 +11,4 @@ echo $ORM_OPTION
 sed -i 's|PREFORK_CHILD .*|PREFORK_CHILD '"${MAX_THREADS}"'|g' $ULIB_ROOT/benchmark.cfg
 
 # 2. Start ULib Server (userver_tcp)
-$ULIB_ROOT/bin/userver_tcp -c $ULIB_ROOT/benchmark.cfg
+$ULIB_ROOT/bin/userver_tcp -c $ULIB_ROOT/benchmark.cfg &

frameworks/C++/cpoll_cppsp/setup.sh

@@ -5,4 +5,4 @@ sed -i 's|#define BENCHMARK_DB_HOST ".*"|#define BENCHMARK_DB_HOST "'"$DBHOST"'"
 make clean
 make -C $CPPSP_HOME
 cd $CPPSP_HOME
-./run_application $TROOT/www -g g++-4.8 -m /forcedynamic.cppsm
+./run_application $TROOT/www -g g++-4.8 -m /forcedynamic.cppsm &

frameworks/C++/treefrog/setup.sh

@@ -17,4 +17,4 @@ make -j8
 rm -f log/*.log
 
 # 4. Start TreeFrog
-treefrog -d $TROOT
+treefrog -d $TROOT &

frameworks/C++/wt/setup.sh

@@ -7,4 +7,4 @@ sed -i 's|INSERT_DB_HOST_HERE|'"${DBHOST}"'|g' benchmark.cpp
 
 g++-4.8 -O3 -DNDEBUG -std=c++0x -L${BOOST_LIB} -I${BOOST_INC} -L${WT_LIB} -I${WT_INC} -o benchmark.wt benchmark.cpp -lwt -lwthttp -lwtdbo -lwtdbomysql -lboost_thread
 
-./benchmark.wt -c wt_config.xml -t ${MAX_THREADS} --docroot . --http-address 0.0.0.0 --http-port 8080 --accesslog=- --no-compression
+./benchmark.wt -c wt_config.xml -t ${MAX_THREADS} --docroot . --http-address 0.0.0.0 --http-port 8080 --accesslog=- --no-compression &

frameworks/C/duda/setup.sh

@@ -1,3 +1,3 @@
 #!/bin/bash
 
-dudac -w $TROOT/webservice -p 2001
+dudac -w $TROOT/webservice -p 2001 &

frameworks/C/lwan/setup.sh

@@ -1,4 +1,4 @@
 #!/bin/bash
 
 cd $LWAN_ROOT/techempower
-$LWAN_BUILD/techempower/techempower
+$LWAN_BUILD/techempower/techempower &

frameworks/C/onion/setup.sh

@@ -3,4 +3,5 @@
 rm -f *.o
 cp -R $IROOT/onion onion/onion
 rm CMakeCache.txt
-make && ./hello
+make
+./hello &

frameworks/Clojure/http-kit/setup.sh

@@ -8,4 +8,4 @@ rm -rf target
 $LEIN_HOME/bin/lein uberjar
 # -server is much faster
 # 'lein run' passes '-client -XX:+TieredCompilation -XX:TieredStopAtLevel=1' which make it starts fast, but runs slow
-$JAVA_HOME/bin/java -server -jar target/http-kit-standalone.jar --db-host ${DBHOST}
+$JAVA_HOME/bin/java -server -jar target/http-kit-standalone.jar --db-host ${DBHOST} &

toolset/benchmark/framework_test.py

@@ -197,8 +197,23 @@ class FrameworkTest:
     # This requires superuser privs, so `sudo` is necessary.
     #   -u [username] The username
     #   -E Preserves the current environment variables
-    subprocess.Popen('sudo -u %s -E ./%s.sh' % (self.benchmarker.runner_user, self.setup_file), cwd=self.directory, shell=True, stderr=err, stdout=out)
-
+    # Note: check_call is a blocking call, so any startup scripts
+    # run by the framework that need to continue (read: server has
+    # started and needs to remain that way), then they should be
+    # executed in the background.
+    try:
+      retcode = subprocess.check_call('sudo -u %s -E ./%s.sh' % 
+        (self.benchmarker.runner_user, self.setup_file), 
+        cwd=self.directory, shell=True, stderr=err, stdout=out)
+      if retcode == None:
+        retcode = 0
+    except Exception:
+      retcode = 1
+      st = traceback.format_exc()
+      st = '\n'.join((4 * ' ') + x for x in st.splitlines())
+      st = "Start exception:\n%s" % st
+      logging.info(st)
+      err.write(st + '\n')
     os.chdir(previousDir)
 
     # Stop the progress printer
@@ -206,7 +221,7 @@ class FrameworkTest:
 
     logging.info("Called setup.py start")
 
-    return 0
+    return retcode
   ############################################################
   # End start
   ############################################################