version 1.1

README.md

@@ -1,33 +1,195 @@
 
 Go-Guerrilla SMTPd
+====================
+
 An minimalist SMTP server written in Go, made for receiving large volumes of mail.
 
 ![Go Guerrilla](https://raw.github.com/flashmob/go-guerrilla/master/GoGuerrilla.png)
 
-What is Go Guerrilla SMTPd?
-It's a small SMTP server written in Go, optimized for receiving email.
+### What is Go Guerrilla SMTPd?
+
+It's a small SMTP server written in Go, for the purpose of receiving large volume of email.
 Written for GuerrillaMail.com which processes tens of thousands of emails
 every hour.
 
-Version: 1.0
-Author: Flashmob, GuerrillaMail.com
-Contact: [email protected]
-License: MIT
-Repository: https://github.com/flashmob/Go-Guerrilla-SMTPd
-Site: http://www.guerrillamail.com/
+The purpose of this daemon is to grab the email, save it to the database
+and disconnect as quickly as possible.
+
+A typical user of this software would probably want to customize the saveMail function for
+their own systems.
+
+This server does not attempt to filter HTML, check for spam or do any sender 
+verification. These steps should be performed by other programs.
+The server does NOT send any email including bounces. This should
+be performed by a separate program.
+
+
+### So what's the story?
+
+Originally, Guerrilla Mail was running the Exim mail server, and the emails
+were fetched using POP.
+
+This proved to be inefficient and unreliable, so the system was replaced in 
+favour of piping the emails directly in to a PHP script.
+
+Soon, the piping solution  became a problem too; it required a new process to 
+be started for each arriving email, and it also required a new database 
+connection every time. 
+
+So, how was the bottleneck eliminated? Conveniently, PHP has a socket 
+library which means we can use PHP to quickly prototype a simple SMTP server.
+If the server runs as a daemon, then the system doesn't need to launch a new 
+process for each incoming email. It also doesn't need to run and insane amount 
+of checks for each connection (eg, NS Lookups, white-lists, black-lists, SPF
+domain keys, Spam Assassin, etc).
+
+We only need to open a single database connection and a single process can be 
+re-used indefinitely. The PHP server was able to multiplex simultaneous 
+connections without the need for forking/threading thanks to socket_select()
+
+Therefore, we could receive, process and store email all in the one process.
+The performance improvement has been dramatic. 
+
+However, a few months later, the volume of email increased again and
+this time our server's CPU was under pressure. You see, the problem was that
+the SMTP server was checking the sockets in a loop all the time. This is fine
+if you have a few sockets, but horrible if you have 1000! A common way to solve
+this is to have a connection per thread - and a thread can sleep while it is
+waiting for a socket, not polling all the time.
+
+Threads are great if you have many CPU cores, but our server only had two.
+Besides, our process was I/O bound - there's a better way than to block.
+So how to solve this one? 
+
+Instead of polling/blocking in an infinite loop to see if there are any sockets to be 
+operated, it was more efficent to be notified when the sockets are ready.
+This Wikipedia entry explains it best, 
+http://en.wikipedia.org/wiki/Asynchronous_I/O see "Select(/poll) loops"
+
+Luck had it that an extension is available for PHP! One night later, and version 2 was made
+to use libevent http://pecl.php.net/package/libevent
+
+The improvement was superb. The load average has decreased
+substantially, freeing the CPU for other tasks. It was even surprising to see that
+PHP could handle so many emails.
+
+Fast forward to 2012, to where we are now. Golang 1.0 was released, so it was
+decided to give it a go.
+
+As a language, Go is brilliant for writing server back-ends, with support for concurrency. 
+There's a nice tutorial for getting started on the Golang.org website. 
+There were a lot of great discoveries along the way, particulary the 'channels' 
+in Go can be simple, yet very powerful and the defer statement is quite convenient. 
+It looks like there's a wealth of packages available for
+almost everything, including MySQL.
+
+
+
+HOW TO USE / Installation:
+===========================
+
+To build, you will need to install the following Go libs:
+
+$ go get github.com/ziutek/mymysql/thrsafe
+$ go get github.com/ziutek/mymysql/autorc
+$ go get github.com/ziutek/mymysql/godrv
+$ go get github.com/sloonz/go-iconv
+$ go get github.com/garyburd/redigo/redis
+
+Rename goguerrilla.conf.sample to goguerrilla.conf
+
+Setup the following database:
+
+
+
+Configuration
+============================================
+The configuration is in strict JSON format. Here is an anotated configuration
+
+	{
+	    "GM_ALLOWED_HOSTS":"example.com,sample.com,foo.com,bar.com", // which domains accept mail
+	    "GM_MAIL_TABLE":"new_mail", // name of new email table
+	    "GM_PRIMARY_MAIL_HOST":"mail.example.com", // given in the SMTP greeting
+	    "GSMTP_HOST_NAME":"mail.example.com", // given in the SMTP greeting
+	    "GSMTP_LOG_FILE":"/dev/stdout", // not used yet
+	    "GSMTP_MAX_SIZE":"131072", // max size of DATA command
+	    "GSMTP_PRV_KEY":"/etc/ssl/private/example.com.key", // private key for TLS
+	    "GSMTP_PUB_KEY":"/etc/ssl/certs/example.com.crt", // public key for TLS
+	    "GSMTP_TIMEOUT":"100", // tcp connection timeout
+	    "GSMTP_VERBOSE":"N", // set to Y for debugging
+	    "GSTMP_LISTEN_INTERFACE":"5.9.7.183:25",
+	    "MYSQL_DB":"gmail_mail", // database name
+	    "MYSQL_HOST":"127.0.0.1:3306", // database connect
+	    "MYSQL_PASS":"$ecure1t", // database connection pass
+	    "MYSQL_USER":"gmail_mail", // database username
+	    "GM_CONN_BACKLOG":"100", // connection backlog queue
+	    "GM_MAX_CLIENTS":"500", // max clients that can be handled
+	    "SGID":"508", // group id of the user from /etc/passwd
+		"GUID":"504" // uid from /etc/passwd
+	}
+
+Using Nginx as a proxy
+=========================================================
+Nginx can be used to proxy SMTP traffic for GoGuerrilla SMTPd
+
+Why proxy SMTP?
+
+ - Terminate TLS connections: Golang is not there yet when it comes to TLS.
+At present, only a partial implementation of TLS is provided (as of Nov 2012). 
+OpenSSL on the other hand, used in Nginx, has a complete implementation of
+ SSL v2/v3 and TLS protocols
+
+- Could be used for load balancing and authentication in the future.
+
+The following Nginx proxy configuration:
+
+
+	mail {
+	
+	        auth_http 127.0.0.1:8025/; # This is the URL to GoGuerrilla's http service which tells Nginx where to proxy the traffic to 
+									
+	        server {
+	                listen  5.9.7.183:25;
+	                protocol smtp;
+	                server_name  ak47.guerrillamail.com;
+	
+	                smtp_auth none;
+	                timeout 30000;
+					smtp_capabilities "SIZE 15728640"
+					
+					# ssl default off. Leave off if starttls is on
+	                #ssl                  on;
+	                ssl_certificate      /etc/ssl/certs/ssl-cert-snakeoil.pem;
+	                ssl_certificate_key  /etc/ssl/private/ssl-cert-snakeoil.key;
+	                ssl_session_timeout  5m;
+	                ssl_protocols  SSLv2 SSLv3 TLSv1;
+	                ssl_ciphers  HIGH:!aNULL:!MD5;
+	                ssl_prefer_server_ciphers   on;
+					# TLS off unless client issues STARTTLS command
+	                starttls on;
+	                proxy on;
+	
+	        }
+	
+	}
+	
+Assuming that Guerrilla SMTPd has the following configuration settings:
+	"GSMTP_MAX_SIZE"		  "15728640",
+	"NGINX_AUTH_ENABLED":     "Y",
+	"NGINX_AUTH":             "127.0.0.1:8025", 
+
+
+Starting / Command Line usage
+==========================================================
 
-Copyright (c) 2012 Flashmob, GuerrillaMail.com
+All command line arguments are optional
 
-Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated
-documentation files (the "Software"), to deal in the Software without restriction, including without limitation the
-rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to
-permit persons to whom the Software is furnished to do so, subject to the following conditions:
+  -config="goguerrilla.conf": Path to the configuration file
+  -if="": Interface and port to listen on, eg. 127.0.0.1:2525
+  -v="n": Verbose, [y | n]
 
-The above copyright notice and this permission notice shall be included in all copies or substantial portions of the
-Software.
+Starting from the command line (example)
 
-THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE
-WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
-COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR
-OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
+/usr/bin/nohup /home/mike/goguerrilla -config=/home/mike/goguerrilla.conf 2>&1 &
 
+This will place goguerrilla in the background and continue running

goguerrilla.conf.sample

@@ -2,15 +2,14 @@
     "GM_ALLOWED_HOSTS":"example.com,sample.com,foo.com,bar.com",
     "GM_MAIL_TABLE":"new_mail",
     "GM_PRIMARY_MAIL_HOST":"mail.example.com",
-    "GSMTP_GID":"",
     "GSMTP_HOST_NAME":"mail.example.com",
-    "GSMTP_LOG_FILE":"/dev/stdout",
-    "GSMTP_MAX_SIZE":"131072",
+    "GSMTP_LOG_FILE":"",
+    "GSMTP_MAX_SIZE":"15728640",
     "GSMTP_PRV_KEY":"/etc/ssl/private/example.com.key",
     "GSMTP_PUB_KEY":"/etc/ssl/certs/example.com.crt",
     "GSMTP_TIMEOUT":"100",
     "GSMTP_VERBOSE":"N",
-    "GSTMP_LISTEN_INTERFACE":"5.9.7.183:25",
+    "GSTMP_LISTEN_INTERFACE":"33.1.2.133:25",
     "MAX_SMTP_CLIENTS":"10000",
     "MYSQL_DB":"gmail_mail",
     "MYSQL_HOST":"127.0.0.1:3306",

goguerrilla.go

@@ -1,6 +1,8 @@
-/*
+/** 
 Go-Guerrilla SMTPd
-An minimalist SMTP server written in Go, made for receiving large volumes of mail.
+A minimalist SMTP server written in Go, made for receiving large volumes of mail.
+Works either as a stand-alone or in conjunction with Nginx SMTP proxy.
+TO DO: add http server for nginx
 
 Copyright (c) 2012 Flashmob, GuerrillaMail.com
 
@@ -22,7 +24,12 @@ It's a small SMTP server written in Go, optimized for receiving email.
 Written for GuerrillaMail.com which processes tens of thousands of emails
 every hour.
 
-Version: 1.0
+Benchmarking:
+http://www.jrh.org/smtp/index.html
+Test 500 clients:
+$ time smtp-source -c -l 5000 -t [email protected] -s 500 -m 5000 5.9.7.183
+
+Version: 1.1
 Author: Flashmob, GuerrillaMail.com
 Contact: [email protected]
 License: MIT
@@ -31,15 +38,20 @@ Site: http://www.guerrillamail.com/
 
 See README for more details
 
-*/
-
-/*
-Install mysql drivers
+To build
+Install the following
 $ go get github.com/ziutek/mymysql/thrsafe
 $ go get github.com/ziutek/mymysql/autorc
 $ go get github.com/ziutek/mymysql/godrv
 $ go get github.com/sloonz/go-iconv
+
+TODO: after failing tls, 
+
+patch:
+rebuild all: go build -a -v new.go
+
 */
+
 package main
 
 import (
@@ -49,7 +61,6 @@ import (
 	"crypto/md5"
 	"crypto/rand"
 	"crypto/tls"
-	// "database/sql"
 	"encoding/base64"
 	"encoding/hex"
 	"encoding/json"
@@ -65,42 +76,15 @@ import (
 	"io/ioutil"
 	"log"
 	"net"
-	//"os"
+	"net/http"
+	"os"
 	"regexp"
 	"runtime"
 	"strconv"
 	"strings"
-	"syscall"
 	"time"
 )
 
-// defaults. Overwrite any of these in the configure() function which loads them from a json file
-var gConfig = map[string]string{
-	"MAX_SMTP_CLIENTS":       "10000",
-	"GSMTP_MAX_SIZE":         "131072",
-	"GSMTP_HOST_NAME":        "server.example.com", // This should also be set to reflect your RDNS
-	"GSMTP_VERBOSE":          "Y",
-	"GSMTP_TIMEOUT":          "100", // how many seconds before timeout.
-	"MYSQL_HOST":             "127.0.0.1:3306",
-	"MYSQL_USER":             "gmail_mail",
-	"MYSQL_PASS":             "ok",
-	"MYSQL_DB":               "gmail_mail",
-	"GM_MAIL_TABLE":          "new_mail",
-	"GSMTP_USER":             "nobody",
-	"GSTMP_LISTEN_INTERFACE": "1.0.0.0:25",
-	"GSMTP_LOG_FILE":         "gosmtp.log",
-	"GSMTP_GID":              "",
-	"GSMTP_UID":              "",
-	"GSMTP_PUB_KEY":          "/etc/ssl/certs/ssl-cert-snakeoil.pem",
-	"GSMTP_PRV_KEY":          "/etc/ssl/private/ssl-cert-snakeoil.key",
-	"GM_ALLOWED_HOSTS":       "guerrillamail.de,guerrillamailblock.com",
-	"GM_PRIMARY_MAIL_HOST":   "guerrillamail.com",
-	"GM_CONN_BACKLOG":        "100",
-	"GM_MAX_CLIENTS":         "500",
-	"SGID":                   "1008", // group id
-	"SUID":                   "1008", // user id, from /etc/passwd
-}
-
 type Client struct {
 	state       int
 	helo        string
@@ -114,7 +98,7 @@ type Client struct {
 	hash        string
 	time        int64
 	tls_on      bool
-	socket      net.Conn
+	conn        net.Conn
 	bufin       *bufio.Reader
 	bufout      *bufio.Writer
 	kill_time   int64
@@ -123,22 +107,59 @@ type Client struct {
 	savedNotify chan int
 }
 
+var TLSconfig *tls.Config
+var max_size int // max email DATA size
+var timeout time.Duration
+var allowedHosts = make(map[string]bool, 15)
+var sem chan int // currently active clients
+
+var SaveMailChan chan *Client // workers for saving mail
+// defaults. Overwrite any of these in the configure() function which loads them from a json file
+var gConfig = map[string]string{
+	"GSMTP_MAX_SIZE":         "131072",
+	"GSMTP_HOST_NAME":        "server.example.com", // This should also be set to reflect your RDNS
+	"GSMTP_VERBOSE":          "Y",
+	"GSMTP_LOG_FILE":         "",    // Eg. /var/log/goguerrilla.log or leave blank if no logging
+	"GSMTP_TIMEOUT":          "100", // how many seconds before timeout.
+	"MYSQL_HOST":             "127.0.0.1:3306",
+	"MYSQL_USER":             "gmail_mail",
+	"MYSQL_PASS":             "ok",
+	"MYSQL_DB":               "gmail_mail",
+	"GM_MAIL_TABLE":          "new_mail",
+	"GSTMP_LISTEN_INTERFACE": "0.0.0.0:25",
+	"GSMTP_PUB_KEY":          "/etc/ssl/certs/ssl-cert-snakeoil.pem",
+	"GSMTP_PRV_KEY":          "/etc/ssl/private/ssl-cert-snakeoil.key",
+	"GM_ALLOWED_HOSTS":       "guerrillamail.de,guerrillamailblock.com",
+	"GM_PRIMARY_MAIL_HOST":   "guerrillamail.com",
+	"GM_MAX_CLIENTS":         "500",
+	"NGINX_AUTH_ENABLED":     "N",              // Y or N
+	"NGINX_AUTH":             "127.0.0.1:8025", // If using Nginx proxy, ip and port to serve Auth requsts
+	"SGID":                   "1008",           // group id
+	"SUID":                   "1008",           // user id, from /etc/passwd
+}
+
 type redisClient struct {
 	count int
 	conn  redis.Conn
 	time  int
 }
 
-var TLSconfig *tls.Config
-var clientChan chan *Client // connection backlog
+func logln(level int, s string) {
 
-var sem chan int              // currently active clients
-var SaveMailChan chan *Client // workers for saving mail
-// hosts allowed in the 'to' address'
-var allowedHosts = make(map[string]bool, 15)
+	if gConfig["GSMTP_VERBOSE"] == "Y" {
+		fmt.Println(s)
+	}
+	if level == 2 {
+		log.Fatalf(s)
+	}
+	if len(gConfig["GSMTP_LOG_FILE"]) > 0 {
+		log.Println(s)
+	}
+}
 
 func configure() {
 	var configFile, verbose, iface string
+	log.SetOutput(os.Stdout)
 	// parse command line arguments
 	flag.StringVar(&configFile, "config", "goguerrilla.conf", "Path to the configuration file")
 	flag.StringVar(&verbose, "v", "n", "Verbose, [y | n] ")
@@ -147,16 +168,13 @@ func configure() {
 	// load in the config.
 	b, err := ioutil.ReadFile(configFile)
 	if err != nil {
-		fmt.Println("Could not read config file")
-		panic(err)
+		log.Fatalln("Could not read config file")
 	}
 	var myConfig map[string]string
 	err = json.Unmarshal(b, &myConfig)
 	if err != nil {
-		fmt.Println("Could not parse config file")
-		panic(err)
+		log.Fatalln("Could not parse config file")
 	}
-
 	for k, v := range myConfig {
 		gConfig[k] = v
 	}
@@ -172,161 +190,88 @@ func configure() {
 	}
 	var n int
 	var n_err error
-	if n, n_err = strconv.Atoi(gConfig["GM_CONN_BACKLOG"]); n_err != nil {
-		n = 50
-	}
-	// connection backlog list
-	clientChan = make(chan *Client, n)
+	// sem is an active clients channel used for counting clients
 	if n, n_err = strconv.Atoi(gConfig["GM_MAX_CLIENTS"]); n_err != nil {
 		n = 50
 	}
 	// currently active client list
 	sem = make(chan int, n)
 	// database writing workers
-	SaveMailChan = make(chan *Client, 4)
-
-	return
-}
-
-func logln(level int, s string) {
-	if level == 2 {
-		log.Fatalf(s)
+	SaveMailChan = make(chan *Client, 5)
+	// timeout for reads
+	if n, n_err = strconv.Atoi(gConfig["GSMTP_TIMEOUT"]); n_err != nil {
+		timeout = time.Duration(10)
+	} else {
+		timeout = time.Duration(n)
 	}
-	if gConfig["GSMTP_VERBOSE"] == "Y" {
-		fmt.Println(s)
+	// max email size
+	if max_size, n_err = strconv.Atoi(gConfig["GSMTP_MAX_SIZE"]); n_err != nil {
+		max_size = 131072
 	}
+	// custom log file
+	if len(gConfig["GSMTP_LOG_FILE"]) > 0 {
+		logfile, err := os.OpenFile(gConfig["GSMTP_LOG_FILE"], os.O_WRONLY|os.O_APPEND|os.O_CREATE|os.O_SYNC, 0600)
+		if err != nil {
+			log.Fatal("Unable to open log file ["+gConfig["GSMTP_LOG_FILE"]+"]: ", err)
+		}
+		log.SetOutput(logfile)
+	}
+
+	return
 }
 
 func main() {
 	configure()
-	logln(1, "Loading priv:"+gConfig["GSMTP_PRV_KEY"]+" and pub:"+gConfig["GSMTP_PRV_KEY"])
 	cert, err := tls.LoadX509KeyPair(gConfig["GSMTP_PUB_KEY"], gConfig["GSMTP_PRV_KEY"])
 	if err != nil {
 		logln(2, fmt.Sprintf("There was a problem with loading the certificate: %s", err))
 	}
 	TLSconfig = &tls.Config{Certificates: []tls.Certificate{cert}, ClientAuth: tls.VerifyClientCertIfGiven, ServerName: gConfig["GSMTP_HOST_NAME"]}
 	TLSconfig.Rand = rand.Reader
+	// start some savemail workers
+	for i := 0; i < 3; i++ {
+		go saveMail()
+	}
+	if gConfig["NGINX_AUTH_ENABLED"] == "Y" {
+		go nginxHTTPAuth()
+	}
+	// Start listening for SMTP connections
 	listener, err := net.Listen("tcp", gConfig["GSTMP_LISTEN_INTERFACE"])
 	if err != nil {
-		logln(2, fmt.Sprintf("Cannot listen on port, %s", err))
-	}
-	gid, _ := strconv.ParseInt(gConfig["SGID"], 10, 32)
-	uid, _ := strconv.ParseInt(gConfig["SUID"], 10, 32)
-	syscall.Setgid(int(gid))
-	syscall.Setuid(int(uid))
-	logln(1, fmt.Sprintf("server listening on "+gConfig["GSTMP_LISTEN_INTERFACE"]))
-	go Serve(clientChan) // Start our SMTP client worker pool
-	go saveMail()        // start our email saving worker pool
-	clientId := int64(1)
+		logln(2, fmt.Sprintf("Cannot listen on port, %v", err))
+	} else {
+		logln(1, fmt.Sprintf("Listening on tcp %s", gConfig["GSTMP_LISTEN_INTERFACE"]))
+	}
+	var clientId int64
+	clientId = 1
 	for {
 		conn, err := listener.Accept()
 		if err != nil {
 			logln(1, fmt.Sprintf("Accept error: %s", err))
-			break
+			continue
 		}
-		logln(1, fmt.Sprintf("server: accepted from %s", conn.RemoteAddr()))
-		// place a new client on the channel
-		clientChan <- &Client{
-			socket:      conn,
+		logln(1, fmt.Sprintf(" There are now "+strconv.Itoa(runtime.NumGoroutine())+" serving goroutines"))
+		sem <- 1 // Wait for active queue to drain.
+		go handleClient(&Client{
+			conn:        conn,
 			address:     conn.RemoteAddr().String(),
 			time:        time.Now().Unix(),
 			bufin:       bufio.NewReader(conn),
 			bufout:      bufio.NewWriter(conn),
 			clientId:    clientId,
 			savedNotify: make(chan int),
-		}
+		})
 		clientId++
 	}
 }
 
-func Serve(clientChan chan *Client) {
-	for {
-		// get new clients off the queue and pass them to the handler
-		c := <-clientChan
-		sem <- 1           // Wait for active queue to drain.
-		go handleClient(c) // Don't wait for handle to finish.
-		logln(1, fmt.Sprintf("There are now "+strconv.Itoa(runtime.NumGoroutine())+" goroutines"))
-	}
-}
-
-func closeClient(client *Client) {
-	client.socket.Close()
-	<-sem // Done; enable next client to run.
-}
-
-func readSmtp(client *Client) (input string, err error) {
-	var reply string
-	// Command state terminator by default
-	suffix := "\r\n"
-	if client.state == 2 {
-		// DATA state
-		suffix = "\r\n.\r\n"
-	}
-	for err == nil {
-		client.socket.SetDeadline(time.Now().Add(100 * time.Second))
-		reply, err = client.bufin.ReadString('\n')
-		if reply != "" {
-			input = input + reply
-			if client.state == 2 {
-				// Extract the subject while we are at it.
-				scanSubject(client, reply)
-			}
-		}
-		if err != nil {
-			break
-		}
-		if strings.HasSuffix(input, suffix) {
-			break
-		}
-	}
-	return input, err
-}
-
-// Scan the data part for a Subject line. Can be a multi-line
-func scanSubject(client *Client, reply string) {
-	if client.subject == "" && (len(reply) > 8) {
-		test := strings.ToUpper(reply[0:9])
-		if i := strings.Index(test, "SUBJECT: "); i == 0 {
-			// first line with \r\n
-			client.subject = reply[9:]
-		}
-	} else if strings.HasSuffix(client.subject, "\r\n") {
-		// chop off the \r\n
-		client.subject = client.subject[0 : len(client.subject)-2]
-		if (strings.HasPrefix(reply, " ")) || (strings.HasPrefix(reply, "\t")) {
-			// subject is multi-line
-			client.subject = client.subject + reply[1:]
-		}
-	}
-}
-
-func responseWrite(client *Client) (err error) {
-	var size int
-	client.socket.SetDeadline(time.Now().Add(100 * time.Second))
-	size, err = client.bufout.WriteString(client.response)
-	client.bufout.Flush()
-	client.response = client.response[size:]
-	return err
-}
-
-func responseAdd(client *Client, line string) {
-	client.response = line + "\r\n"
-}
-func responseClear(client *Client) {
-	client.response = ""
-}
-
-func killClient(client *Client) {
-	client.kill_time = time.Now().Unix()
-}
-
 func handleClient(client *Client) {
-	var input_hist string
 	defer closeClient(client)
+	//	defer closeClient(client)
 	greeting := "220 " + gConfig["GSMTP_HOST_NAME"] +
 		" SMTP Guerrilla-SMTPd #" + strconv.FormatInt(client.clientId, 10) + " (" + strconv.Itoa(len(sem)) + ") " + time.Now().Format(time.RFC1123Z)
 	advertiseTls := "250-STARTTLS\r\n"
-	for i := 0; i < 10; i++ {
+	for i := 0; i < 100; i++ {
 		switch client.state {
 		case 0:
 			responseAdd(client, greeting)
@@ -334,6 +279,7 @@ func handleClient(client *Client) {
 		case 1:
 			input, err := readSmtp(client)
 			if err != nil {
+				logln(1, fmt.Sprintf("Read error: %v", err))
 				if err == io.EOF {
 					// client closed the connection already
 					return
@@ -345,7 +291,6 @@ func handleClient(client *Client) {
 				break
 			}
 			input = strings.Trim(input, " \n\r")
-			input_hist = input_hist + input + "\n"
 			cmd := strings.ToUpper(input)
 			switch {
 			case strings.Index(cmd, "HELO") == 0:
@@ -357,15 +302,19 @@ func handleClient(client *Client) {
 				if len(input) > 5 {
 					client.helo = input[5:]
 				}
-				if client.tls_on {
-					advertiseTls = ""
-				}
 				responseAdd(client, "250-"+gConfig["GSMTP_HOST_NAME"]+" Hello "+client.helo+"["+client.address+"]"+"\r\n"+"250-SIZE "+gConfig["GSMTP_MAX_SIZE"]+"\r\n"+advertiseTls+"250 HELP")
 			case strings.Index(cmd, "MAIL FROM:") == 0:
 				if len(input) > 10 {
 					client.mail_from = input[10:]
 				}
 				responseAdd(client, "250 Ok")
+			case strings.Index(cmd, "XCLIENT") == 0:
+				// Nginx sends this
+				// XCLIENT ADDR=212.96.64.216 NAME=[UNAVAILABLE]
+				client.address = input[13:]
+				client.address = client.address[0:strings.Index(client.address, " ")]
+				fmt.Println("client address:[" + client.address + "]")
+				responseAdd(client, "250 OK")
 			case strings.Index(cmd, "RCPT TO:") == 0:
 				if len(input) > 8 {
 					client.rcpt_to = input[8:]
@@ -388,10 +337,10 @@ func handleClient(client *Client) {
 				responseAdd(client, "221 Bye")
 				killClient(client)
 			default:
-				responseAdd(client, fmt.Sprintf("500 unrecognized command %v", err))
+				responseAdd(client, fmt.Sprintf("500 unrecognized command"))
 				client.errors++
 				if client.errors > 3 {
-					responseAdd(client, fmt.Sprintf("500 Too many unrecognized commands %v", err))
+					responseAdd(client, fmt.Sprintf("500 Too many unrecognized commands"))
 					killClient(client)
 				}
 			}
@@ -404,23 +353,31 @@ func handleClient(client *Client) {
 				SaveMailChan <- client
 				// wait for the save to complete
 				status := <-client.savedNotify
+
 				if status == 1 {
 					responseAdd(client, "250 OK : queued as "+client.hash)
 				} else {
 					responseAdd(client, "554 Error: transaction failed, blame it on the weather")
 				}
+			} else {
+				logln(1, fmt.Sprintf("DATA read error: %v", err))
 			}
 			client.state = 1
 		case 3:
 			// upgrade to TLS
 			var tlsConn *tls.Conn
-			tlsConn = tls.Server(client.socket, TLSconfig)
-			tlsConn.Handshake() // not necessary to call here, but might as well
-			client.socket = net.Conn(tlsConn)
-			client.bufin = bufio.NewReader(client.socket)
-			client.bufout = bufio.NewWriter(client.socket)
+			tlsConn = tls.Server(client.conn, TLSconfig)
+			err := tlsConn.Handshake() // not necessary to call here, but might as well
+			if err == nil {
+				client.conn = net.Conn(tlsConn)
+				client.bufin = bufio.NewReader(client.conn)
+				client.bufout = bufio.NewWriter(client.conn)
+				client.tls_on = true
+			} else {
+				logln(1, fmt.Sprintf("Could not TLS handshake:%v", err))
+			}
+			advertiseTls = ""
 			client.state = 1
-			client.tls_on = true
 		}
 		// Send a response back to the client
 		err := responseWrite(client)
@@ -438,6 +395,77 @@ func handleClient(client *Client) {
 			return
 		}
 	}
+
+}
+
+func responseAdd(client *Client, line string) {
+	client.response = line + "\r\n"
+}
+func closeClient(client *Client) {
+	client.conn.Close()
+	<-sem // Done; enable next client to run.
+}
+func killClient(client *Client) {
+	client.kill_time = time.Now().Unix()
+}
+
+func readSmtp(client *Client) (input string, err error) {
+	var reply string
+	// Command state terminator by default
+	suffix := "\r\n"
+	if client.state == 2 {
+		// DATA state
+		suffix = "\r\n.\r\n"
+	}
+	for err == nil {
+		client.conn.SetDeadline(time.Now().Add(timeout * time.Second))
+		reply, err = client.bufin.ReadString('\n')
+		if reply != "" {
+			input = input + reply
+			if len(input) > max_size {
+				err = errors.New("Maximum DATA size exceeded (" + strconv.Itoa(max_size) + ")")
+				return input, err
+			}
+			if client.state == 2 {
+				// Extract the subject while we are at it.
+				scanSubject(client, reply)
+			}
+		}
+		if err != nil {
+			break
+		}
+		if strings.HasSuffix(input, suffix) {
+			break
+		}
+	}
+	return input, err
+}
+
+// Scan the data part for a Subject line. Can be a multi-line
+func scanSubject(client *Client, reply string) {
+	if client.subject == "" && (len(reply) > 8) {
+		test := strings.ToUpper(reply[0:9])
+		if i := strings.Index(test, "SUBJECT: "); i == 0 {
+			// first line with \r\n
+			client.subject = reply[9:]
+		}
+	} else if strings.HasSuffix(client.subject, "\r\n") {
+		// chop off the \r\n
+		client.subject = client.subject[0 : len(client.subject)-2]
+		if (strings.HasPrefix(reply, " ")) || (strings.HasPrefix(reply, "\t")) {
+			// subject is multi-line
+			client.subject = client.subject + reply[1:]
+		}
+	}
+}
+
+func responseWrite(client *Client) (err error) {
+	var size int
+	client.conn.SetDeadline(time.Now().Add(timeout * time.Second))
+	size, err = client.bufout.WriteString(client.response)
+	client.bufout.Flush()
+	client.response = client.response[size:]
+	return err
 }
 
 func saveMail() {
@@ -461,7 +489,6 @@ func saveMail() {
 	if sql_err != nil {
 		logln(2, fmt.Sprintf("Sql statement incorrect: %s", sql_err))
 	}
-	//defer db.Close()
 
 	//  receives values from the channel repeatedly until it is closed.
 	for {
@@ -494,9 +521,8 @@ func saveMail() {
 				client.data = ""
 				body = "redis"
 			}
-			//fmt.Println(do_reply, do_err)
 		} else {
-			fmt.Println("redis err", redis_err)
+			logln(1, fmt.Sprintf("redis: %v", redis_err))
 		}
 		// bind data to cursor
 		ins.Bind(
@@ -517,7 +543,7 @@ func saveMail() {
 			logln(1, "Email saved "+client.hash+" len:"+strconv.Itoa(length))
 			_, _, err = incr.Exec()
 			if err != nil {
-				fmt.Println(err)
+				logln(1, fmt.Sprintf("Failed to incr count:", err))
 			}
 			client.savedNotify <- 1
 		}
@@ -539,15 +565,6 @@ func (c *redisClient) redisConnection() (err error) {
 	return nil
 }
 
-func mysqlTest() {
-	//var mysqlCon *sql.DB
-	//mysqlCon, err := sql.Open("mymysql", gConfig["MYSQL_DB"]+"/"+gConfig["MYSQL_USER"]+"/"+gConfig["MYSQL_PASS"])
-	//if err != nil {
-	//	log.Fatalf("Cannot open Mysql connection: %s", err)
-	//}
-	// defer mysqlCon.Close()
-}
-
 func validateEmailData(client *Client) (user string, host string, addr_err error) {
 	if user, host, addr_err = extractEmail(client.mail_from); addr_err != nil {
 		return user, host, addr_err
@@ -570,9 +587,9 @@ func extractEmail(str string) (name string, host string, err error) {
 		host = validHost(matched[2])
 		name = matched[1]
 	} else {
-		if res := strings.Split(name, "@"); len(res) > 1 {
-			name = matched[0]
-			host = validHost(matched[1])
+		if res := strings.Split(str, "@"); len(res) > 1 {
+			name = res[0]
+			host = validHost(res[1])
 		}
 	}
 	if host == "" || name == "" {
@@ -689,3 +706,27 @@ func md5hex(str string) string {
 	sum := h.Sum([]byte{})
 	return hex.EncodeToString(sum)
 }
+
+// If running Nginx as a proxy, give Nginx the IP address and port for the SMTP server
+// Primary use of Nginx is to terminate TLS so that Go doesn't need to deal with it.
+// This could perform auth and load balancing too
+// See http://wiki.nginx.org/MailCoreModule
+func nginxHTTPAuth() {
+	parts := strings.Split(gConfig["GSTMP_LISTEN_INTERFACE"], ":")
+	gConfig["HTTP_AUTH_HOST"] = parts[0]
+	gConfig["HTTP_AUTH_PORT"] = parts[1]
+	fmt.Println(parts)
+	http.HandleFunc("/", nginxHTTPAuthHandler)
+	err := http.ListenAndServe(gConfig["NGINX_AUTH"], nil)
+	if err != nil {
+		log.Fatal("ListenAndServe: ", err)
+	}
+
+}
+
+func nginxHTTPAuthHandler(w http.ResponseWriter, r *http.Request) {
+	w.Header().Add("Auth-Status", "OK")
+	w.Header().Add("Auth-Server", gConfig["HTTP_AUTH_HOST"])
+	w.Header().Add("Auth-Port", gConfig["HTTP_AUTH_PORT"])
+	fmt.Fprint(w, "")
+}