HTTP server hangs while accepting packets

Go To StackoverFlow.com

5

I have written a simple http server to handle POST requests:

class MyHandler( BaseHTTPServer.BaseHTTPRequestHandler):
    def do_POST( self ):
        ctype, pdict = cgi.parse_header(self.headers.getheader('content-type'))
        postvars = {}
        try:
          if ctype == 'application/x-www-form-urlencoded':
              length = int(self.headers.getheader('content-length'))
              postvars = cgi.parse_qs(self.rfile.read(length), keep_blank_values=1)

          self.send_response( 200 )
          self.send_header( "Content-type", "text")
          self.send_header( "Content-length", str(len(body)) )
          self.end_headers()
          self.wfile.write(body)
        except:
          print "Error"


def httpd(handler_class=MyHandler, server_address = ('2.3.4.5', 80)):
    try:
        print "Server started"
        srvr = BaseHTTPServer.HTTPServer(server_address, handler_class)
        srvr.serve_forever() # serve_forever
    except KeyboardInterrupt:
        server.socket.close()


if __name__ == "__main__":
    httpd( )

The server runs fine but sometimes it just hangs. When I press CTRL+C it gives the following error and then continues receiving data:

Exception happened during processing of request from ('1.1.1.2', 50928)
Traceback (most recent call last):
  File "/usr/lib/python2.6/SocketServer.py", line 281, in _handle_request_noblock
    self.process_request(request, client_address)
  File "/usr/lib/python2.6/SocketServer.py", line 307, in process_request
    self.finish_request(request, client_address)
  File "/usr/lib/python2.6/SocketServer.py", line 320, in finish_request
    self.RequestHandlerClass(request, client_address, self)
  File "/usr/lib/python2.6/SocketServer.py", line 615, in __init__
    self.handle()
  File "/usr/lib/python2.6/BaseHTTPServer.py", line 329, in handle
    self.handle_one_request()
  File "/usr/lib/python2.6/BaseHTTPServer.py", line 312, in handle_one_request
    self.raw_requestline = self.rfile.readline()
  File "/usr/lib/python2.6/socket.py", line 406, in readline
    data = self._sock.recv(self._rbufsize)
KeyboardInterrupt

Can someone tell me how to correct this? I can't make sense of the errors.

2012-04-04 01:35
by Bruce
Is that really the code you're running? That dangling except: looks like a syntax error - A B 2012-04-04 01:40
I missed the statement while copying. Sorry about that - Bruce 2012-04-04 01:48
Are you sure the client does not cause the problem (e.g. by connecting and sending nothing or an unlimited amount of data) - katzenversteher 2012-04-04 07:00
@affenlehrer: The client is sending very small amount of data...how can I close the connection on server side if the client does not send does not send data for say 2 minutes - Bruce 2012-04-04 17:00
You could e.g. use this: http://docs.python.org/library/socket.html#socket.socket.settimeout However I don't know how the HTTPServer/SocketServer implementation behaves if you use that - katzenversteher 2012-04-04 18:15
The traceback says that the socket did wait for data and no exception socket.timeout has been raised until you raised KeyboardInterrupt. This is OK - hynekcer 2012-04-21 13:28


11

I completely rewrote my solution in order to fix two disadvantages:

  • It can treat timeout even if client has opened only a connection but did not start communicate.
  • If the client opens a connection, a new server process is forked. A slow client can not block other.

It is based on your code as most as possible. It is now a complete independent demo. If you open a browser on http://localhost:8000/ and write 'simulate error' to form and press submit, it simulates runtime error.

import BaseHTTPServer
import SocketServer
import cgi


class MyHandler(BaseHTTPServer.BaseHTTPRequestHandler):
    def do_POST(self):
        ctype, pdict = cgi.parse_header(self.headers.getheader('content-type'))
        postvars = {}
        try:
            if ctype == 'application/x-www-form-urlencoded':
                length = int(self.headers.getheader('content-length'))
                postvars = cgi.parse_qs(self.rfile.read(length),
                        keep_blank_values=1)
                assert postvars.get('foo', '') != ['simulate error']
            body = 'Something'
            self.send_response(200)
            self.send_header("Content-type", "text")
            self.send_header("Content-length", str(len(body)))
            self.end_headers()
            self.wfile.write(body)
        except:
            self.send_error(500)
            raise

    def do_GET(self):
        # demo for testing POST by web browser - without valid html
        body = 'Something\n<form method="post" action="http://%s:%d/">\n' \
                '<input name="foo" type="text"><input type="submit"></form>\n'\
                % self.server.server_address
        self.send_response(200)
        self.send_header("Content-type", "text/html")
        self.send_header("Content-length", str(len(body)))
        self.end_headers()
        self.wfile.write(body)


class ForkingHTTPServer(SocketServer.ForkingMixIn, BaseHTTPServer.HTTPServer):
    def finish_request(self, request, client_address):
        request.settimeout(30)
        # "super" can not be used because BaseServer is not created from object
        BaseHTTPServer.HTTPServer.finish_request(self, request, client_address)


def httpd(handler_class=MyHandler, server_address=('localhost', 8000)):
    try:
        print "Server started"
        srvr = ForkingHTTPServer(server_address, handler_class)
        srvr.serve_forever()  # serve_forever
    except KeyboardInterrupt:
        srvr.socket.close()

if __name__ == "__main__":
    httpd()

Forking can be disabled for e.g. debugging purposes by removing class SocketServer.ForkingMixIn from code.

2012-04-21 13:18
by hynekcer
Awesome answer. I was looking was that exactly - Bruce 2012-04-21 19:12
How can I close the client connection? I tried self.rfile._sock.close() but it doesn't help...The timeout is somehow not working. I want to forcibly close the connection when exception happens in self.rfile.read()...I am open to any suggestions.. - Bruce 2012-04-23 23:28
I wrote a complete solution instead of advices - hynekcer 2012-04-24 20:16
request.settimeout(30q) should be request.settimeout(30)bain 2017-06-16 15:27