Python's http.server module builds on socketserver. Let's see if we can similarly make http.server compatible with upstart-socket-bridge.
About http.server
The http.server module is intended to be a simple way to create webservers. Most of the module is devoted to classes that handle incoming and outgoing data. Only one class, HTTPServer, handles the networking stuff.Example 1
Here is the first example http.server.SimpleHTTPRequestHandler in the Python documentation:import http.server
import socketserver
PORT = 8000
Handler = http.server.SimpleHTTPRequestHandler
httpd = socketserver.TCPServer(("", PORT), Handler)
print("serving at port", PORT)
httpd.serve_forever()
When you run this code, and point a web browser to port 8000, the code serves the current working directory, with links to files and subdirectories.
Convert Example 1 to work with upstart-socket-bridge
The example server is only seven lines.Line 3, the PORT, is no longer needed. Upstart will define the port.
Line 5, the socketserver.TCPServer line, will be the biggest change. We need to define a new class based on TCPServer, and override two methods. The is exactly what we did in the previous post.
Line 6, the print statement, can be deleted. When the job is started by Upstart, there is no display - nowhere to print to. An important side effect of starting the job using Upstart is that the Present Working Directory is / (root), unless you specify otherwise in the /etc/init config file that starts the job.
Because of the change to Line 5, the final result is 6 more lines...not bad, and now it works with upstart-socket-bridge.
import http.server
import socketserver
class MyServer(socketserver.TCPServer):
def server_bind(self):
""" Replace the socket FD with the Upstart-provided FD"""
fd = int(os.environ["UPSTART_FDS"])
self.socket = socket.fromfd(fd, socket.AF_INET, socket.SOCK_STREAM)
def server_activate(self):
pass # Upstart takes care of listen()
Handler = http.server.SimpleHTTPRequestHandler
server = MyServer(None, Handler)
server.serve_forever()
As always, the script is triggered by an appropriate Upstart job.
Test the script, in this case, by pointing a web browser at the port specified in the Upstart job.
Example 2
This example is also from the Python http.server documentation:
def run(server_class=HTTPServer, handler_class=BaseHTTPRequestHandler):
server_address = ('', 8000)
httpd = server_class(server_address, handler_class)
httpd.serve_forever()
It's not a standalone example, but instead an abstract example of how to use the module in a larger application.
Let's reformat it into a working example:
import http.server
server_address = ('', 8000)
handler_class=http.server.BaseHTTPRequestHandler
httpd = http.server.HTTPServer(server_address, handler_class)
httpd.serve_forever()
Now the server runs...sort of. It gives us a 501 error message. Let's add a class to the handler that reads the path and gives a real, valid response. (Source)
import http.server
class MyHandler(http.server.BaseHTTPRequestHandler):
def do_HEAD(self):
self.send_response(200)
self.send_header("Content-type", "text/html")
self.end_headers()
def do_GET(self):
self.send_response(200)
self.send_header("Content-type", "text/html")
self.end_headers()
print(self.wfile)
content = ["<html><head><title>The Title</title></head>",
"<body>This is a test.<br />",
"You accessed path: ", self.path, "<br />",
"</body></html>"]
self.wfile.write("".join(content).encode("UTF-8"))
server_address = ('', 8000)
handler_class=MyHandler
httpd = http.server.HTTPServer(server_address, handler_class)
httpd.serve_forever()
Okay, now this is a good working example of a working http.server. It really shows how http.server hides all the complexity of networking on one simple .server line and focuses all your effort on content in the handler. It's clear how you would parse the URL input using self.path. It's clear how you can create and send content using self.wfile.
Convert Example #2 to work with upstart-socket-bridge
There are two classes at work: http.server.BaseHTTPRequestHandler handles content doesn't care about the networking. http.server.HTTPServer handles netwokring and doesn't care about content.Good news: HTTPServer is based on socketserver.TCPServer, which we already know how to patch!
The changes I made:
1) I want it to launch at each connection, exchange data once, and then terminate. Each connection will launch a separate instance. We no longer want the service to serve_forever().
httpd.serve_forever() # Old httpd.handle_request() # New
2) Let's make it compatible with all three inits: sysvinit (deamon), Upstart, and systemd.
import os
if __name__ == "__main__":
if "UPSTART_FDS" in os.environ.keys() \ # Upstart
or "LISTEN_FDS" in os.environ.keys(): # systemd
httpd = MyServer(None, MyHandler) # Need a custom Server class
httpd.handle_request() # Run once
else: # sysvinit
server_address = ('', 8000)
httpd = http.server.HTTPServer(server_address, MyHandler)
httpd.serve_forever() # Run forever
3) Add a custom handler that overrides server_bind() and server_activate() in both the socketserver and http.server.HTTPServer modules. This is the secret sauce that makes Upstart compatibility work:
import http.server, socketserver, socket, os
class MyServer(http.server.HTTPServer):
def server_bind(self):
# Get the File Descriptor from an Upstart-created environment variable
if "UPSTART_FDS" in os.environ:
fd = int(os.environ["UPSTART_FDS"]) # Upstart
else:
fd = int(os.environ["LISTEN_FDS"]) # Systemd
self.socket = socket.fromfd(fd, socket.AF_INET, socket.SOCK_STREAM)
# Only http.server.CGIHTTPRequestHandler uses these
host, port = self.socket.getsockname()[:2]
self.server_name = socket.getfqdn(host)
self.server_port = port
4) Add the Upstart job to monitor the port:
# /etc/init/socket-test.py description "upstart-socket-bridge test" start on socket PROTO=inet PORT=8000 ADDR=127.0.0.1 setuid your_username # Does not need to run as root exec /usr/bin/python3 /tmp/socket-server.py
And the final product looks like:
#!/usr/bin/python3
import http.server, socketserver, socket, os
class MyServer(http.server.HTTPServer):
"""
This class overrides two methods in the socketserver module:
socketserver __init__ uses both server_bind() and server_activate()
This class overrides one method in the http.server.HTTPServer class:
HTTPServer uses both socketserver __init__ and it's own custom
server_bind
These overrides makes it compatible with Upstart and systemd socket-bridges
Warning: It won't bind() or listen() to a socket anymore
"""
def server_bind(self):
"""
Get the File Descriptor from an Upstart-created environment variable
instead of binding or listening to a socket.
"""
if "UPSTART_FDS" in os.environ:
fd = int(os.environ["UPSTART_FDS"])
else:
fd = int(os.environ["LISTEN_FDS"])
self.socket = socket.fromfd(fd, socket.AF_INET, socket.SOCK_STREAM)
# From http.server:
# http.server.CGIHTTPRequestHandler uses these.
# Other handler classes don't use these.
host, port = self.socket.getsockname()[:2]
self.server_name = socket.getfqdn(host)
self.server_port = port
def server_activate(self):
"""
This socketserver method sends listen(), so it needs to be overridden
"""
pass
class MyHandler(http.server.BaseHTTPRequestHandler):
"""
A very simple custom handler.
It merely reads the URL and responds with the path.
This shows how you read data from a GET, and send a response.
"""
def do_HEAD(self):
self.send_response(200)
self.send_header("Content-type", "text/html")
self.end_headers()
def do_GET(self):
self.send_response(200)
self.send_header("Content-type", "text/html")
self.end_headers()
print(self.wfile)
content = ["The Title ",
"This is a test.
",
"You accessed path: ", self.path, "
",
""]
self.wfile.write("".join(content).encode("UTF-8"))
if __name__ == "__main__":
if "UPSTART_FDS" in os.environ.keys() \ # Upstart
or "LISTEN_FDS" in os.environ.keys(): # systemd
httpd = MyServer(None, MyHandler) # Use fd to get connection
httpd.handle_request() # Handle once, then terminate
else:
server_address = ('', 8000) # sysvinit, classic bind()
httpd = http.server.HTTPServer(server_address, MyHandler)
httpd.serve_forever()
Test the service:
- The Python 3 script and Upstart job both use Port 8000.
- Save the Python 3 script. Make it executable.
- Save the Upstart job. Change the Upstart job to Port 8001. Make sure it points to the Python 3 script.
Webserver daemon using sysvinit - run forever:
- Run the Python 3 script.
- Start a web browser, and point it to http://localhost:8000/test/string
- The browser should show a response
- Kill the Python 3 script
Web service using upstart - run once:
- Don't start the Python 3 script. Upstart will do it for you.
- Start a web browser, and point it to http://localhost:8001/test/string
- The browser should show a response.
No comments:
Post a Comment