Merge branch 'master' of https://github.com/LadyMozzarella/FrameworkBenchmarks into get-rack-rails-sinatra-running

frameworks/C/lwan/README.md

@@ -1,11 +1,82 @@
 # Lwan
 
-This is the configuration files to benchmark the [Lwan](http://lwan.ws)
+This is the configuration files to benchmark the [Lwan](http://lwan.ws/?src=twfb)
 web server/framework.
 
+## FAQ
+
+### Where's the source code for the benchmark?
+
+Source code for the benchmark itself is currently living in [in Lwan's
+repository](https://github.com/lpereira/lwan/tree/master/techempower) for
+practical reasons, but will move to [this repository right after Round 10 is
+complete](https://github.com/TechEmpower/FrameworkBenchmarks/issues/1372).
+
+In any case, the script to install the framework in the test machines [pulls
+a specific snapshot of the Lwan source
+code](https://github.com/lpereira/lwan/tree/49607addb31879e2aa2b701317773674662315aa),
+including the benchmark code; and, due to the nature of Git, it's not
+possible to alter that code without changing which commit hash to checkout
+in this repository.
+
+### The JSON test RPS value seems ridiculous; is Lwan cheating?
+
+Come on. There's no optimization without cheating. There's a somewhat
+[detailed blog post about
+this](http://tia.mat.br/blog/html/2014/10/06/life_of_a_http_request.html?src=twfb).
+However, that level of cheating is considered by Lwan's author to be fair
+game, since it's not related to the benchmark code itself.
+
+In fact, the serialization library has been picked solely because it was
+available in the [CCAN](http://ccodearchive.net/).  It's written by [Joseph
+A.  Adams](http://ccodearchive.net/info/json.html), and has been slightly
+tweaked to fix some compiler warnings and a few other minor changes.  No
+major optimization or trickery has been made there: it's essentially the
+same code.  Props to him for a clean code that performs well.
+
+It's all open source anyway, feel free to go ahead and inspect the code.
+
+### The database results aren't that good. Why?
+
+The database tests are included for completeness sake. The database
+abstraction layer (which works with SQLite and MySQL) was written
+specifically for this benchmark, and is completely synchronous: while
+one query is being performed, all other clients wait.
+
+So the uninspiring results are expected. Database connectivity is
+important and will eventually be revisited.
+
+### I've seen the preliminary results and Lwan didn't fare well in the plaintext benchmark. Is there a reason why?
+
+These benchmarks are performed using pipelined requests. For practical
+reasons (almost no client supports them, at least by default), this was not
+implemented in Lwan: the server was dropping the request buffer and waiting
+for the next request to come in, while `wrk` was waiting for responses to
+come out, eventually timeouting and sending more pipelined requests, only to
+repeat, until the test actually ended.
+
+Pipelining has been implemented in a hurry to get included in Round 10.
+
+### The results are pretty good, can I use this in my next web app?
+
+Writing web apps in C? Only if you're Crazy. (The uppercase C is not a typo.)
+
+Seriously, Lwan is just a toy.  It's an experiment.  Most things you'll need
+from real frameworks won't be available.  You'll have a hard time trying to do
+the simplest things.  There are far more important things to look for while
+picking a framework, and performance, while important, is not the most
+important factor.  However, if you're feeling adventurous, know that Lwan is
+free software, and is very open to pull requests.
+
+### I have other questions, is there a FAQ somewhere else?
+
+Sure there is. [See Lwan's web page](http://lwan.ws/?src=twfb) or contact the author
+directly.
+
 ## Requirements
 
-GCC, SQLite 3.7, Linux 3.0, optionally jemalloc or tcmalloc.
+GCC, SQLite 3.7, MySQL/MariaDB client libraries, Lua 5.1/LuaJit 2.0, Linux 3.0,
+optionally jemalloc or tcmalloc.
 
 ## Tests available
 
@@ -29,7 +100,6 @@ URL: /fortunes
 
 URL: /plaintext
 
-
 ## Contact
 
-Leandro Pereira <[email protected]>
+[Leandro Pereira](http://tia.mat.br/?src=twfb) <[[email protected]](mailto:[email protected])>

frameworks/Java/servlet3-cass/bash_profile.sh

@@ -1,2 +1,3 @@
-export JAVA_HOME=/usr/lib/jvm/java-1.7.0-openjdk-amd64
+export JAVA_HOME=/opt/java8
+export JAVA_EXE=$JAVA_HOME/bin/java
 export RESIN_HOME=${IROOT}/resin-4.0.41

frameworks/Java/servlet3-cass/install.sh

@@ -1,3 +1,3 @@
 #!/bin/bash
 
-fw_depends java resin maven
+fw_depends java8 resin maven

frameworks/Java/servlet3-cass/pom.xml

@@ -10,7 +10,7 @@
         <sourceEncoding>UTF-8</sourceEncoding>
         <java.version>1.7</java.version>
 
-        <slf4j.version>1.7.7</slf4j.version>
+        <slf4j.version>1.7.10</slf4j.version>
         <logback.version>1.1.2</logback.version>
     </properties>
 
@@ -22,7 +22,7 @@
             <plugin>
                 <groupId>org.apache.maven.plugins</groupId>
                 <artifactId>maven-compiler-plugin</artifactId>
-                <version>3.1</version>
+                <version>3.2</version>
                 <configuration>
                     <source>${java.version}</source>
                     <target>${java.version}</target>
@@ -33,7 +33,7 @@
             <plugin>
                 <groupId>org.apache.maven.plugins</groupId>
                 <artifactId>maven-war-plugin</artifactId>
-                <version>2.4</version>
+                <version>2.6</version>
                 <configuration>
                     <failOnMissingWebXml>false</failOnMissingWebXml>
                 </configuration>
@@ -62,13 +62,13 @@
         <dependency>
             <groupId>com.fasterxml.jackson.core</groupId>
             <artifactId>jackson-databind</artifactId>
-            <version>2.4.1</version>
+            <version>2.5.1</version>
         </dependency>
 
         <dependency>
             <groupId>com.datastax.cassandra</groupId>
             <artifactId>cassandra-driver-core</artifactId>
-            <version>2.0.3</version>
+            <version>2.1.4</version>
             <scope>compile</scope>
         </dependency>
 

frameworks/Java/servlet3-cass/source_code

@@ -4,8 +4,6 @@
 ./servlet3-cass/src/main/java/fi/markoa
 ./servlet3-cass/src/main/java/fi/markoa/tfb
 ./servlet3-cass/src/main/java/fi/markoa/tfb/servlet3
-./servlet3-cass/src/main/java/fi/markoa/tfb/servlet3/AsyncErrorServlet1.java
-./servlet3-cass/src/main/java/fi/markoa/tfb/servlet3/AsyncErrorServlet2.java
 ./servlet3-cass/src/main/java/fi/markoa/tfb/servlet3/DatabaseBaseServlet.java
 ./servlet3-cass/src/main/java/fi/markoa/tfb/servlet3/DatabaseQueriesServlet.java
 ./servlet3-cass/src/main/java/fi/markoa/tfb/servlet3/DatabaseQueryServlet.java

frameworks/Perl/mojolicious/benchmark_config

@@ -11,7 +11,7 @@
       "plaintext_url": "/plaintext",
       "port": 8080,
       "approach": "Realistic",
-      "classification": "Full",
+      "classification": "Fullstack",
       "database": "MongoDB",
       "framework": "mojolicious",
       "language": "Perl",

toolset/benchmark/benchmarker.py

@@ -6,6 +6,7 @@ from benchmark.test_types import *
 from utils import header
 from utils import gather_tests
 from utils import gather_frameworks
+from utils import verify_database_connections
 
 import os
 import json
@@ -539,6 +540,15 @@ class Benchmarker:
           """)
           time.sleep(10)
 
+          st = verify_database_connections([
+            ("mysql", self.database_host, 3306),
+            ("mongodb", self.database_host, 27017),
+            ("redis", self.database_host, 6379),
+            ("postgresql", self.database_host, 5432),
+            ("cassandra", self.database_host, 9160)
+          ])
+          print "database connection test results:\n" + "\n".join(st[1])
+
         if self.__is_port_bound(test.port):
           # This can happen sometimes - let's try again
           self.__stop_test(out, err)

toolset/benchmark/utils.py

@@ -2,6 +2,7 @@ import ConfigParser
 import os
 import glob
 import json
+import socket
 
 from ast import literal_eval
 
@@ -138,3 +139,33 @@ def header(message, top='-', bottom='-'):
       else:
         result += "\n%s" % bottomheader
     return result + '\n'
+
+def check_services(services):
+
+  def check_service(address, port):
+    try:
+      s = socket.socket()
+      s.settimeout(20)
+      s.connect((address, port))
+      return (True, "")
+    except Exception as ex:
+      return (False, ex)
+    finally:
+      s.close
+
+  res = []
+  for s in services:
+    r = check_service(s[1], s[2])
+    res.append((s[0], r[0], str(r[1])))
+  return res
+
+def verify_database_connections(services):
+  allGo = True
+  messages = []
+  for r in check_services(services):
+    if r[1]:
+      messages.append(r[0] + ": is GO!")
+    else:
+      messages.append(r[0] + ": is _NO_ GO!: ERROR: " + r[2])
+      allGo = False
+  return (allGo, messages)