{ "metadata": { "name": "", "signature": "sha256:d8ce3daf4b5f7634bd45f3aa3142516ecb1a1342251f6a7022b65ba69cd9f608" }, "nbformat": 3, "nbformat_minor": 0, "worksheets": [ { "cells": [ { "cell_type": "heading", "level": 1, "metadata": {}, "source": [ "Network Scripting" ] }, { "cell_type": "heading", "level": 1, "metadata": {}, "source": [ "Plumbing the Internet" ] }, { "cell_type": "heading", "level": 2, "metadata": {}, "source": [ "The Socket Layer" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "In simple terms, sockets are a programmable interface to connections between pro-\n", "grams, possibly running on different computers of a network. They allow data format-\n", "ted as byte strings to be passed between processes and machines. Sockets also form the\n", "basis and low-level \u201cplumbing\u201d of the Internet itself: all of the familiar higher-level Net\n", "protocols, like FTP, web pages, and email, ultimately occur over sockets. Sockets are\n", "also sometimes called communications endpoints because they are the portals through\n", "which programs send and receive bytes during a conversation.\n", "\n", "\n", "socket\u7684\u82f1\u6587\u539f\u4e49\u662f\u201c\u5b54\u201d\u6216\u201c\u63d2\u5ea7\u201d\uff0c\u901a\u5e38\u4e5f\u79f0\u4f5c\"\u5957\u63a5\u5b57\"\uff0c\u7528\u4e8e\u63cf\u8ff0IP\u5730\u5740\u548c\u7aef\u53e3\uff0c\u662f\u4e00\u4e2a\u901a\u4fe1\u94fe\u7684\u53e5\u67c4\u3002\u5728Internet\u4e0a\u7684\u4e3b\u673a\u4e00\u822c\u8fd0\u884c\u4e86\u591a\u4e2a\u670d\u52a1\u8f6f\u4ef6\uff0c\u540c\u65f6\u63d0\u4f9b\u51e0\u79cd\u670d\u52a1\u3002\u6bcf\u79cd\u670d\u52a1\u90fd\u6253\u5f00\u4e00\u4e2aSocket\uff0c\u5e76\u7ed1\u5b9a\u5230\u4e00\u4e2a\u7aef\u53e3\u4e0a\uff0c\u4e0d\u540c\u7684\u7aef\u53e3\u5bf9\u5e94\u4e8e\u4e0d\u540c\u7684\u670d\u52a1\u3002Socket\u6b63\u5982\u5176\u82f1\u6587\u539f\u610f\u90a3\u6837\uff0c\u8c61\u4e00\u4e2a\u591a\u5b54\u63d2\u5ea7\u3002\u4e00\u53f0\u4e3b\u673a\u72b9\u5982\u5e03\u6ee1\u5404\u79cd\u63d2\u5ea7\u7684\u623f\u95f4\uff0c\u6bcf\u4e2a\u63d2\u5ea7\u6709\u4e00\u4e2a\u7f16\u53f7\uff0c\u6709\u7684\u63d2\u5ea7\u63d0\u4f9b220\u4f0f\u4ea4\u6d41\u7535\uff0c \u6709\u7684\u63d0\u4f9b110\u4f0f\u4ea4\u6d41\u7535\uff0c\u6709\u7684\u5219\u63d0\u4f9b\u6709\u7ebf\u7535\u89c6\u8282\u76ee\u3002 \u5ba2\u6237\u8f6f\u4ef6\u5c06\u63d2\u5934\u63d2\u5230\u4e0d\u540c\u7f16\u53f7\u7684\u63d2\u5ea7\uff0c\u5c31\u53ef\u4ee5\u5f97\u5230\u4e0d\u540c\u7684\u670d\u52a1\u3002" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Although often used for network conversations, sockets may also be used as a com-\n", "munication mechanism between programs running on the same computer, taking the\n", "form of a general Inter-Process Communication (IPC) mechanism. We saw this socket\n", "usage mode briefly in Chapter 5. Unlike some IPC devices, sockets are bidirectional\n", "data streams: programs may both send and receive data through them.\n", "\n", "\n", "\u4e5f\u53efIPC\uff0c\u53cc\u5411\u6570\u636e" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "To programmers, sockets take the form of a handful of calls available in a library. These\n", "socket calls know how to send bytes between machines, using lower-level operations\n", "such as the TCP network transmission control protocol. At the bottom, TCP knows\n", "how to transfer bytes, but it doesn\u2019t care what those bytes mean. For the purposes of\n", "this text, we will generally ignore how bytes sent to sockets are physically transferred.\n", "To understand sockets fully, though, we need to know a bit about how computers are\n", "named." ] }, { "cell_type": "heading", "level": 3, "metadata": {}, "source": [ "Machine identifiers" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "- Machine identifiers = Machine names(IP address) + Port numbers(0..1023) \n", "- domain name server = {domain name : IP address}" ] }, { "cell_type": "heading", "level": 3, "metadata": {}, "source": [ "The Protocol Layer" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Python provides support for standard protocols, which auto-\n", "mates most of the socket and message formatting details.\n", "Standard Internet protocols define a structured way to talk over sockets. \n", "They generally standardize both message formats and socket port numbers:\n", "- Message formats provide structure for the bytes exchanged over sockets during\n", "conversations.\n", "- Port numbers are reserved numeric identifiers for the underlying sockets over which\n", "messages are exchanged." ] }, { "cell_type": "heading", "level": 3, "metadata": {}, "source": [ "Port number rules" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "make it easier for programs to locate the standard protocols,\n", "port numbers in the range of 0 to 1023 are reserved and preassigned to the standard\n", "higher-level protocols.\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
\n", "
Table 12-1.\n", " Port Numbers Reserved for Common Protocols
\n", "
\n", "

Protocol

\n", " 		\n", "

Common\n", " Function

\n", " 		\n", "

Port\n", " Number

\n", " 		\n", "

Python\n", " Module

\n", "
\n", "

HTTP

\n", " 		\n", "

Web pages

\n", " 		\n", "

80

\n", " 		\n", "

http.client , http.server

\n", "
\n", "

NNTP

\n", " 		\n", "

Usenet news

\n", " 		\n", "

119

\n", " 		\n", "

nntplib

\n", "
\n", "

FTP data default

\n", " 		\n", "

File transfers

\n", " 		\n", "

20

\n", " 		\n", "

ftplib

\n", "
\n", "

FTP control

\n", " 		\n", "

File transfers

\n", " 		\n", "

21

\n", " 		\n", "

ftplib

\n", "
\n", "

SMTP

\n", " 		\n", "

Sending email

\n", " 		\n", "

25

\n", " 		\n", "

smtplib

\n", "
\n", "

POP3

\n", " 		\n", "

Fetching email

\n", " 		\n", "

110

\n", " 		\n", "

poplib

\n", "
\n", "

IMAP4

\n", " 		\n", "

Fetching email

\n", " 		\n", "

143

\n", " 		\n", "

imaplib

\n", "
\n", "

Finger

\n", " 		\n", "

Informational

\n", " 		\n", "

79

\n", " 		\n", "

n/a

\n", "
\n", "

Telnet

\n", " 		\n", "

Command lines

\n", " 		\n", "

23

\n", " 		\n", "

telnetlib

\n", "
" ] }, { "cell_type": "heading", "level": 3, "metadata": {}, "source": [ "Clients and servers" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "On one side of a conversation, machines that support standard\n", "protocols perpetually run a set of programs that listen for connection requests on the\n", "reserved ports. On the other end of a dialog, other machines contact those programs\n", "to use the services they export.\n", "We usually call the perpetually running listener program a server and the connecting\n", "program a client. Let\u2019s use the familiar web browsing model as an example. As shown\n", "in Table 12-1, the HTTP protocol used by the Web allows clients and servers to talk\n", "over sockets on port 80:" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Server\n", "- A machine that hosts websites usually runs a web server program that constantly\n", "listens for incoming connection requests, on a socket bound to port 80. Often, the\n", "server itself does nothing but watch for requests on its port perpetually; handling\n", "requests is delegated to spawned processes or threads.\n", "\n", "Clients\n", "- Programs that wish to talk to this server specify the server machine\u2019s name and\n", "port 80 to initiate a connection. For web servers, typical clients are web browsers\n", "like Firefox, Internet Explorer, or Chrome, but any script can open a client-side\n", "connection on port 80 to fetch web pages from the server. The server\u2019s machine\n", "name can also be simply \u201clocalhost\u201d if it\u2019s the same as the client\u2019s." ] }, { "cell_type": "heading", "level": 3, "metadata": {}, "source": [ "Protocol structures" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The structure of those message bytes varies from protocol to protocol, is hidden\n", "by the Python library. For example, the FTP protocol prevents deadlock by conversing over two sockets: one\n", "for control messages only and one to transfer file data. An FTP server listens for control\n", "messages (e.g., \u201csend me a file\u201d) on one port, and transfers file data over another. FTP\n", "clients open socket connections to the server machine\u2019s control port, send requests,\n", "and send or receive file data over a socket connected to a data port on the server machine. FTP also defines standard message structures passed between client and server.\n", "The control message used to request a file, for instance, must follow a standard format." ] }, { "cell_type": "heading", "level": 2, "metadata": {}, "source": [ "Python\u2019s Internet Library Modules" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "In fact, each supported protocol is represented in Python\u2019s standard library by either a\n", "module package of the same name as the protocol or by a module file with a name of\n", "the form xxxlib.py" ] }, { "cell_type": "heading", "level": 1, "metadata": {}, "source": [ "Socket Programming" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Although sockets them-\n", "selves transfer only byte strings, we can also transfer Python objects through them by\n", "using Python\u2019s pickle module. Because this module converts Python objects such as\n", "lists, dictionaries, and class instances to and from byte strings, it provides the extra step\n", "needed to ship higher-level objects through sockets when required." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Beyond basic data communication tasks, the socket module also includes a variety of\n", "more advanced tools. For instance, it has calls for the following and more:\n", "- Converting bytes to a standard network ordering ( ntohl , htonl )\n", "- Querying machine name and address ( gethostname , gethostbyname )\n", "- Wrapping socket objects in a file object interface ( sockobj.makefile )\n", "- Making socket calls nonblocking ( sockobj.setblocking )\n", "- Setting socket timeouts ( sockobj.settimeout )" ] }, { "cell_type": "heading", "level": 2, "metadata": {}, "source": [ "Socket Basics" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Server side: open a TCP/IP socket on a port, listen for a message from\n", "a client, and send an echo reply; this is a simple one-shot listen/reply\n", "conversation per client, but it goes into an infinite loop to listen for\n", "more clients as long as this server script runs; the client may run on\n", "a remote machine, or on same computer if it uses 'localhost' for server" ] }, { "cell_type": "code", "collapsed": false, "input": [ "from socket import *\n", "myHost = '' # '' = all available interfaces on host\n", "myPort = 50007 # listen on a non-reserved port number\n", "sockobj = socket(AF_INET, SOCK_STREAM) #make a TCP socket object\n", "sockobj.bind((myHost, myPort)) #bind it to server port number\n", "sockobj.listen(5) #listen, allow 5 pending connects\n", "while True:\n", " connections, address = sockobj.accept() #wait for next client connect\n", " print 'Server connected by', address #connection is a new socket\n", " while True:\n", " data = connections.recv(1024) # read next line on client socket\n", " if not data: break # send a reply line to the client\n", " connections.send(b'Echo =>' + data) # until eof when socket closed\n", " connections.close()" ], "language": "python", "metadata": {}, "outputs": [], "prompt_number": 8 }, { "cell_type": "markdown", "metadata": {}, "source": [ "Client side: use sockets to send data to the server, and print server's\n", "reply to each message line; 'localhost' means that the server is running\n", "on the same machine as the client, which lets us test client and server\n", "on one machine; to test over the Internet, run a server on a remote\n", "machine, and set serverHost or argv[1] to machine's domain name or IP addr;\n", "Python sockets are a portable BSD socket interface, with object methods\n", "for the standard socket calls available in the system's C library;" ] }, { "cell_type": "code", "collapsed": false, "input": [ "import sys\n", "from socket import *\n", "serverHost = 'localhost' # portable socket interface plus constants \n", " # server name, or: 'starship.python.net'\n", "serverPort = 50007 # non-reserved port used by the server\n", "message = [b'Hello network world'] # default text to send to server\n", " # requires bytes: b'' or str,encode()\n", "if len(sys.argv) > 1:\n", " serverHost = sys.argv[1] # server from cmd line arg 1\n", " if len(sys.argv) > 2: # text from cmd line args 2..n\n", " message = (x.encode() for x in sys.argv[2:]) \n", " \n", "sockobj = socket(AF_INET, SOCK_STREAM) # make a TCP/IP socket object\n", "sockobj.connect((serverHost, serverPort)) # connect to server machine + port\n", "\n", "for line in message:\n", " sockobj.send(line) # send line to server over socket\n", " data = sockobj.recv(1024) # receive line from server: up to 1k\n", " print 'Client received:', data # bytes are quoted, was `x`, repr(x)\n", "sockobj.close()" ], "language": "python", "metadata": {}, "outputs": [], "prompt_number": 17 }, { "cell_type": "heading", "level": 3, "metadata": {}, "source": [ "Server socket calls" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Uses the Python socket module to create a TCP socket object. The names\n", "AF_INET and SOCK_STREAM are preassigned variables defined by and imported from\n", "the socket module; using them in combination means \u201ccreate a TCP/IP socket,\u201d\n", "the standard communication device for the Internet. More specifically, AF_INET\n", "means the IP address protocol, and SOCK_STREAM means the TCP transfer protocol.\n", "The AF_INET / SOCK_STREAM combination is the default because it is so common, but\n", "it\u2019s typical to make this explicit." ] }, { "cell_type": "code", "collapsed": false, "input": [ "sockobj = socket(AF_INET, SOCK_STREAM)" ], "language": "python", "metadata": {}, "outputs": [], "prompt_number": 24 }, { "cell_type": "markdown", "metadata": {}, "source": [ "Associates the socket object with an address\u2014for IP addresses, we pass a server\n", "machine name and port number on that machine. This is where the server identifies\n", "the machine and port associated with the socket. In server programs, the hostname\n", "is typically an empty string (\u201c\u201d), which means the machine that the script runs on\n", "(formally, all available local and remote interfaces on the machine), and the port\n", "is a number outside the range 0 to 1023 (which is reserved for standard protocols,\n", "described earlier).\n", "Note that each unique socket dialog you support must have its own port number;\n", "if you try to open a socket on a port already in use, Python will raise an exception.\n", "Also notice the nested parentheses in this call\u2014for the AF_INET address protocol\n", "socket here, we pass the host/port socket address to bind as a two-item tuple object\n", "(pass a string for AF_UNIX ). Te" ] }, { "cell_type": "code", "collapsed": false, "input": [ "sockobj.bind((myHost, myPort))" ], "language": "python", "metadata": {}, "outputs": [], "prompt_number": 25 }, { "cell_type": "markdown", "metadata": {}, "source": [ "Starts listening for incoming client connections and allows for a backlog of up to\n", "five pending requests. The value passed sets the number of incoming client requests\n", "queued by the operating system before new requests are denied (which happens\n", "only if a server isn\u2019t fast enough to process requests before the queues fill up). A\n", "value of 5 is usually enough for most socket-based programs; the value must be at\n", "least 1." ] }, { "cell_type": "code", "collapsed": false, "input": [ "sockobj.listen(5)" ], "language": "python", "metadata": {}, "outputs": [], "prompt_number": 31 }, { "cell_type": "markdown", "metadata": {}, "source": [ "At this point, the server is ready to accept connection requests from client programs\n", "running on remote machines (or the same machine) and falls into an infinite loop\u2014\n", "while True (or the equivalent while 1 for older Pythons and ex-C programmers)\u2014\n", "waiting for them to arrive:" ] }, { "cell_type": "code", "collapsed": false, "input": [ "connection, address = sockobj.accept()" ], "language": "python", "metadata": {}, "outputs": [] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Waits for the next client connection request to occur; when it does, the accept call\n", "returns a brand-new socket object over which data can be transferred from and to\n", "the connected client. Connections are accepted on sockobj , but communication\n", "with a client happens on connection , the new socket. This call actually returns a\n", "two-item tuple\u2014 address is the connecting client\u2019s Internet address. We can call\n", "accept more than one time, to service multiple client connections; that\u2019s why each\n", "call returns a new, distinct socket for talking to a particular client." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Once we have a client connection, we fall into another loop to receive data from the\n", "client in blocks of up to 1,024 bytes at a time, and echo each block back to the client:" ] }, { "cell_type": "code", "collapsed": false, "input": [ "data = connection.recv(1024)" ], "language": "python", "metadata": {}, "outputs": [], "prompt_number": 33 }, { "cell_type": "markdown", "metadata": {}, "source": [ "Reads at most 1,024 more bytes of the next message sent from a client (i.e., coming\n", "across the network or IPC connection), and returns it to the script as a byte string.\n", "We get back an empty byte string when the client has finished\u2014end-of-file is triggered when the client closes its end of the socket." ] }, { "cell_type": "code", "collapsed": false, "input": [ "connection.send(b'Echo=>' + data)" ], "language": "python", "metadata": {}, "outputs": [], "prompt_number": 35 }, { "cell_type": "markdown", "metadata": {}, "source": [ "Sends the latest byte string data block back to the client program, prepending the\n", "string 'Echo=>' to it first. The client program can then recv what we send here\u2014\n", "the next reply line. Technically this call sends as much data as possible, and returns\n", "the number of bytes actually sent. To be fully robust, some programs may need to\n", "resend unsent portions or use connection.sendall to force all bytes to be sent." ] }, { "cell_type": "heading", "level": 3, "metadata": {}, "source": [ "Transferring byte strings and objects" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Although the socket model is limited to transferring byte strings, you can\n", "send and receive nearly arbitrary Python objects with the standard library pickle object\n", "serialization module. Its dumps and loads calls convert Python objects to and from byte\n", "strings, ready for direct socket transfer:" ] }, { "cell_type": "code", "collapsed": false, "input": [ "import pickle" ], "language": "python", "metadata": {}, "outputs": [], "prompt_number": 36 }, { "cell_type": "code", "collapsed": false, "input": [ "x = pickle.dumps([99,100]) # on sending end... convert to byte strings" ], "language": "python", "metadata": {}, "outputs": [], "prompt_number": 38 }, { "cell_type": "code", "collapsed": false, "input": [ "x # string passed to send, returned by recv" ], "language": "python", "metadata": {}, "outputs": [ { "metadata": {}, "output_type": "pyout", "prompt_number": 41, "text": [ "'(lp0\\nI99\\naI100\\na.'" ] } ], "prompt_number": 41 }, { "cell_type": "code", "collapsed": false, "input": [ "pickle.loads(x) # on receiving end... convert back to object" ], "language": "python", "metadata": {}, "outputs": [ { "metadata": {}, "output_type": "pyout", "prompt_number": 42, "text": [ "[99, 100]" ] } ], "prompt_number": 42 }, { "cell_type": "markdown", "metadata": {}, "source": [ "For simpler types that correspond to those in the C language, the struct module provides the byte-string conversion we need as well:" ] }, { "cell_type": "code", "collapsed": false, "input": [ "import struct" ], "language": "python", "metadata": {}, "outputs": [], "prompt_number": 43 }, { "cell_type": "code", "collapsed": false, "input": [ "x = struct.pack('>ii', 99 ,100) # convert simpler types for transmission" ], "language": "python", "metadata": {}, "outputs": [], "prompt_number": 44 }, { "cell_type": "code", "collapsed": false, "input": [ "x" ], "language": "python", "metadata": {}, "outputs": [ { "metadata": {}, "output_type": "pyout", "prompt_number": 45, "text": [ "'\\x00\\x00\\x00c\\x00\\x00\\x00d'" ] } ], "prompt_number": 45 }, { "cell_type": "code", "collapsed": false, "input": [ "struct.unpack('>ii',x)" ], "language": "python", "metadata": {}, "outputs": [ { "metadata": {}, "output_type": "pyout", "prompt_number": 48, "text": [ "(99, 100)" ] } ], "prompt_number": 48 }, { "cell_type": "heading", "level": 3, "metadata": {}, "source": [ "Client socket calls" ] }, { "cell_type": "code", "collapsed": false, "input": [ "sockobj.connect((serverHost, serverPort))" ], "language": "python", "metadata": {}, "outputs": [], "prompt_number": 50 }, { "cell_type": "markdown", "metadata": {}, "source": [ "Opens a connection to the machine and port on which the server program is lis-\n", "tening for client connections. This is where the client specifies the string name of\n", "the service to be contacted. In the client, we can either specify the name of the\n", "remote machine as a domain name (e.g., starship.python.net) or numeric IP ad-\n", "dress. We can also give the server name as localhost (or the equivalent IP address\n", "127.0.0.1 ) to specify that the server program is running on the same machine as\n", "the client; that comes in handy for debugging servers without having to connect\n", "to the Net. And again, the client\u2019s port number must match the server\u2019s exactly.\n", "Note the nested parentheses again\u2014just as in server bind calls, we really pass the\n", "server\u2019s host/port address to connect in a tuple object." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Once the client establishes a connection to the server, it falls into a loop, sending a\n", "message one line at a time and printing whatever the server sends back after each line\n", "is sent:" ] }, { "cell_type": "code", "collapsed": false, "input": [ "sockobj.send(line)" ], "language": "python", "metadata": {}, "outputs": [] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Transfers the next byte-string message line to the server over the socket. Notice\n", "that the default list of lines contains bytes strings ( b'...' ). Just as on the server,\n", "data passed through the socket must be a byte string, though it can be the result\n", "of a manual str.encode encoding call or an object conversion with pickle or\n", "struct if desired. When lines to be sent are given as command-line arguments\n", "instead, they must be converted from str to bytes ; the client arranges this by en-\n", "coding in a generator expression (a call map(str.encode, sys.argv[2:]) would have\n", "the same effect)." ] }, { "cell_type": "heading", "level": 3, "metadata": {}, "source": [ "Running Socket Programs Locally" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The server keeps\n", "running and responds to requests made each time you run the client script in the other\n", "window." ] }, { "cell_type": "heading", "level": 3, "metadata": {}, "source": [ "Running Socket Programs Remotely" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "First, upload the server\u2019s source file to a remote machine where\n", "you have an account and a Python.\n", "The & syntax in Unix/Linux shells can be used to run the server\n", "script in the background.\n", "Now that the server is listening for connections on the Net, run the client on your local\n", "computer multiple times again. This time, the client runs on a different machine than\n", "the server, so we pass in the server\u2019s domain or IP name as a client command-line\n", "argument. The server still uses a machine name of \"\" because it always listens on what-\n", "ever machine it runs on." ] }, { "cell_type": "code", "collapsed": false, "input": [ "!ping learning-python.com" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "stream", "stream": "stdout", "text": [ "PING learning-python.com (97.74.215.115) 56(84) bytes of data.\r\n" ] }, { "output_type": "stream", "stream": "stdout", "text": [ "64 bytes from p3nlh266.shr.prod.phx3.secureserver.net (97.74.215.115): icmp_seq=1 ttl=38 time=210 ms\r\n" ] }, { "output_type": "stream", "stream": "stdout", "text": [ "64 bytes from p3nlh266.shr.prod.phx3.secureserver.net (97.74.215.115): icmp_seq=2 ttl=38 time=212 ms\r\n" ] }, { "output_type": "stream", "stream": "stdout", "text": [ "64 bytes from p3nlh266.shr.prod.phx3.secureserver.net (97.74.215.115): icmp_seq=3 ttl=38 time=214 ms\r\n" ] }, { "output_type": "stream", "stream": "stdout", "text": [ "64 bytes from p3nlh266.shr.prod.phx3.secureserver.net (97.74.215.115): icmp_seq=4 ttl=38 time=209 ms\r\n" ] }, { "output_type": "stream", "stream": "stdout", "text": [ "64 bytes from p3nlh266.shr.prod.phx3.secureserver.net (97.74.215.115): icmp_seq=5 ttl=38 time=217 ms\r\n" ] }, { "output_type": "stream", "stream": "stdout", "text": [ "^C\r\n", "--- learning-python.com ping statistics ---\r\n", "6 packets transmitted, 5 received, 16% packet loss, time 5004ms\r\n", "rtt min/avg/max/mdev = 209.249/212.888/217.931/3.055 ms\r\n", "\r\n" ] } ], "prompt_number": 51 }, { "cell_type": "heading", "level": 3, "metadata": {}, "source": [ "Socket pragmatics" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "1. you can run the client and server like this on any two Internet-aware machines where Python is installed. All you need then is a computer that allows sockets, and most do.\n", "2. the socket module generally raises exceptions if you ask for something invalid.\n", "3. be sure to kill the server process before restarting it again, or else the port number will still be in use, and you\u2019ll get another exception." ] }, { "cell_type": "heading", "level": 2, "metadata": {}, "source": [ "Spawning Clients in Parallel" ] }, { "cell_type": "code", "collapsed": false, "input": [ "import sys\n", "from PP4E.launchmodes import QuietPortableLauncher" ], "language": "python", "metadata": {}, "outputs": [], "prompt_number": 52 }, { "cell_type": "code", "collapsed": false, "input": [ "numclients = 8\n", "def start(cmdline):\n", " QuietPortableLauncher(cmdline, cmdline)()" ], "language": "python", "metadata": {}, "outputs": [], "prompt_number": 53 }, { "cell_type": "code", "collapsed": false, "input": [ "# start('echo-server.py') # spawn server locally if not yet started\n", "args = ' '.join(sys.argv[1:]) # pass server name if running remotely\n", "for i in range(numclients):\n", " start('echo-client.py %s' % args) # spawn 8? clients to test the server" ], "language": "python", "metadata": {}, "outputs": [], "prompt_number": 54 }, { "cell_type": "heading", "level": 2, "metadata": {}, "source": [ "Talking to Reserved Ports" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "It\u2019s also important to know that this client and server engage in a proprietary sort of\n", "discussion, and so use the port number 50007 outside the range reserved for standard\n", "protocols (0 to 1023). There\u2019s nothing preventing a client from opening a socket on\n", "one of these special ports, however. For instance, the following client-side code con-\n", "nects to programs listening on the standard email, FTP, and HTTP web server ports\n", "on three different server machines:" ] }, { "cell_type": "code", "collapsed": false, "input": [ "from socket import *" ], "language": "python", "metadata": {}, "outputs": [], "prompt_number": 56 }, { "cell_type": "heading", "level": 4, "metadata": {}, "source": [ "talk to POP email server" ] }, { "cell_type": "code", "collapsed": false, "input": [ "sock = socket(AF_INET,SOCK_STREAM)" ], "language": "python", "metadata": {}, "outputs": [], "prompt_number": 57 }, { "cell_type": "code", "collapsed": false, "input": [ "sock.connect(('pop.secureserver.net', 110)) " ], "language": "python", "metadata": {}, "outputs": [], "prompt_number": 58 }, { "cell_type": "code", "collapsed": false, "input": [ "print sock.recv(70)" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "stream", "stream": "stdout", "text": [ "+OK <28789.1401261144@p3plpop05-10.prod.phx3.secureserver.net>\r\n", "\n" ] } ], "prompt_number": 59 }, { "cell_type": "code", "collapsed": false, "input": [ "sock.close()" ], "language": "python", "metadata": {}, "outputs": [], "prompt_number": 60 }, { "cell_type": "heading", "level": 4, "metadata": {}, "source": [ "talk to FTP server" ] }, { "cell_type": "code", "collapsed": false, "input": [ "sock = socket(AF_INET,SOCK_STREAM)" ], "language": "python", "metadata": {}, "outputs": [], "prompt_number": 63 }, { "cell_type": "code", "collapsed": false, "input": [ "sock.connect(('learning-python.com', 21))" ], "language": "python", "metadata": {}, "outputs": [], "prompt_number": 64 }, { "cell_type": "code", "collapsed": false, "input": [ "print sock.recv(70)" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "stream", "stream": "stdout", "text": [ "220---------- Welcome to Pure-FTPd [privsep] [TLS] ----------\r\n", "220-You\n" ] } ], "prompt_number": 65 }, { "cell_type": "code", "collapsed": false, "input": [ "sock.close()" ], "language": "python", "metadata": {}, "outputs": [], "prompt_number": 60 }, { "cell_type": "heading", "level": 4, "metadata": {}, "source": [ "talk to Python's HTTP server" ] }, { "cell_type": "code", "collapsed": false, "input": [ "sock = socket(AF_INET,SOCK_STREAM)" ], "language": "python", "metadata": {}, "outputs": [], "prompt_number": 78 }, { "cell_type": "code", "collapsed": false, "input": [ "sock.connect(('www.python.net', 80))" ], "language": "python", "metadata": {}, "outputs": [], "prompt_number": 79 }, { "cell_type": "code", "collapsed": false, "input": [ "sock.send(b'GET /\\r\\n') # fetch root page reply" ], "language": "python", "metadata": {}, "outputs": [ { "metadata": {}, "output_type": "pyout", "prompt_number": 80, "text": [ "7" ] } ], "prompt_number": 80 }, { "cell_type": "code", "collapsed": false, "input": [ "sock.recv(70)" ], "language": "python", "metadata": {}, "outputs": [ { "metadata": {}, "output_type": "pyout", "prompt_number": 81, "text": [ "'\\r\\n\u001b[0m in \u001b[0;36m\u001b[1;34m()\u001b[0m\n\u001b[1;32m----> 1\u001b[1;33m \u001b[0msock\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mbind\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;34m''\u001b[0m\u001b[1;33m,\u001b[0m\u001b[1;36m80\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m", "\u001b[1;32m/usr/lib/python2.7/socket.pyc\u001b[0m in \u001b[0;36mmeth\u001b[1;34m(name, self, *args)\u001b[0m\n\u001b[0;32m 222\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m 223\u001b[0m \u001b[1;32mdef\u001b[0m \u001b[0mmeth\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mname\u001b[0m\u001b[1;33m,\u001b[0m\u001b[0mself\u001b[0m\u001b[1;33m,\u001b[0m\u001b[1;33m*\u001b[0m\u001b[0margs\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m--> 224\u001b[1;33m \u001b[1;32mreturn\u001b[0m \u001b[0mgetattr\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0m_sock\u001b[0m\u001b[1;33m,\u001b[0m\u001b[0mname\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m*\u001b[0m\u001b[0margs\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m 225\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m 226\u001b[0m \u001b[1;32mfor\u001b[0m \u001b[0m_m\u001b[0m \u001b[1;32min\u001b[0m \u001b[0m_socketmethods\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n", "\u001b[1;31merror\u001b[0m: [Errno 13] Permission denied" ] } ], "prompt_number": 88 }, { "cell_type": "heading", "level": 1, "metadata": {}, "source": [ "Handling Multiple Clients" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "In real-world client/server programs, it\u2019s far more typical to code a server so as to avoid\n", "blocking new requests while handling a current client\u2019s request. Perhaps the easiest\n", "way to do so is to service each client\u2019s request in parallel\u2014in a new process, in a new\n", "thread, or by manually switching (multiplexing) between clients in an event loop. This\n", "isn\u2019t a socket issue per se, and we already learned how to start processes and threads\n", "in Chapter 5. But since these schemes are so typical of socket server programming, let\u2019s\n", "explore all three ways to handle client requests in parallel here." ] }, { "cell_type": "heading", "level": 2, "metadata": {}, "source": [ "Forking Servers" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Server side: open a socket on a port, listen for a message from a client,\n", "and send an echo reply; forks a process to handle each client connection;\n", "child processes share parent's socket descriptors; fork is less portable\n", "than threads--not yet on Windows, unless Cygwin or similar installed" ] }, { "cell_type": "code", "collapsed": false, "input": [ "import os, time, sys\n", "from socket import *\n", "myHost = '' # '' = all available interfaces on host\n", "myPort = 50007 # listen on a non-reserved port number\n", "\n", "sockobj = socket(AF_INET, SOCK_STREAM) #make a TCP socket object\n", "sockobj.bind((myHost, myPort)) #bind it to server port number\n", "sockobj.listen(5) #listen, allow 5 pending connects\n", "\n", "def now():\n", " return time.ctime(time.time())\n", "\n", "activeChildren = []\n", "def reapChildren(): # reap any dead child processes\n", " while activeChildren: # else may fill up system table\n", " pid, stat = os.waitpid(0, os.WNOHANG) # don't hang if no child exited\n", " if not pid: break\n", " activeChildren.remove(pid)\n", "\n", "# child process: reply, exit simulate a blocking activity\n", "# read, write a client socket till eof when socket closed \n", "def handleClient(connection):\n", " time.sleep(5)\n", " while True:\n", " data = connections.recv(1024) # read next line on client socket\n", " if not data: break # send a reply line to the client\n", " reply = 'Echo=>%s at %s' % (data,now())\n", " connections.send(reply.encode()) # until eof when socket closed\n", " connections.close()\n", " os._exit(0)\n", "\n", "def dispatcher(): # listen until process killed \n", " while True: # wait for next connection,\n", " connection, address = sockobj.accept() # pass to process for service\n", " print 'Server connected by', address, 'at', now()\n", " reapChildren() # clean up exited children now\n", " childPid = os.fork() # copy this process\n", " if childPid == 0: # if in child process: handle\n", " handleClient(connection)\n", " else: # else: go accept next connect\n", " activeChildren.append(childPid) # add to active child pid list" ], "language": "python", "metadata": {}, "outputs": [], "prompt_number": 92 }, { "cell_type": "code", "collapsed": false, "input": [ "dispatcher()" ], "language": "python", "metadata": {}, "outputs": [], "prompt_number": 96 }, { "cell_type": "heading", "level": 3, "metadata": {}, "source": [ "Running the forking server" ] }, { "cell_type": "code", "collapsed": false, "input": [ "!netstat -pant | grep 50007 #show 50007 port\n", "!kill -9 pid # kill python server" ], "language": "python", "metadata": {}, "outputs": [] }, { "cell_type": "markdown", "metadata": {}, "source": [ "the test proceeds as follows:\n", "1. The server starts running remotely.\n", "2. All three clients are started and connect to the server a few seconds apart.\n", "3. On the server, the client requests trigger three forked child processes, which all\n", "immediately go to sleep for five seconds (to simulate being busy doing something\n", "useful).\n", "4. Each client waits until the server replies, which happens five seconds after their\n", "initial requests." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "In a more realistic application, that delay could be fatal if many clients were trying to\n", "connect at once\u2014the server would be stuck in the action we\u2019re simulating with\n", "time.sleep , and not get back to the main loop to accept new client requests. With\n", "process forks per request, clients can be serviced in parallel." ] }, { "cell_type": "heading", "level": 3, "metadata": {}, "source": [ "Killing dead-but-listed child processes zombies" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "ps -af full process listing command shows that all the dead child pro-\n", "cesses stay in the system process table (show as )" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "When the reapChildren command is reactivated, dead child zombie entries are cleaned\n", "up each time the server gets a new client connection request, by calling the Python\n", "os.waitpid function. A few zombies may accumulate if the server is heavily loaded, but\n", "they will remain only until the next client connection is received (you get only as many\n", "zombies as processes served in parallel since the last accept )" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "In fact, if you type fast enough, you can actually see a child process morph from a real\n", "running program into a zombie. Here, for example, a child spawned to handle a new\n", "request changes to on exit. Its connection cleans up lingering zombies, and\n", "its own process entry will be removed completely when the next request is received." ] }, { "cell_type": "heading", "level": 3, "metadata": {}, "source": [ "Preventing zombies with signal handlers on Linux" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "On some systems, it\u2019s also possible to clean up zombie child processes by resetting the\n", "signal handler for the SIGCHLD signal delivered to a parent process by the operating\n", "system when a child process stops or exits. If a Python script assigns the SIG_IGN (ignore)\n", "action as the SIGCHLD signal handler, zombies will be removed automatically and im-\n", "mediately by the operating system as child processes exit; the parent need not issue\n", "wait calls to clean up after them. Because of that, this scheme is a simpler alternative\n", "to manually reaping zombies on platforms where it is supported." ] }, { "cell_type": "code", "collapsed": false, "input": [ "# Demo Python's signal module; pass signal number as a command-line arg, and use\n", "# a \"kill -N pid\" shell command to send this process a signal; on my Linux machine,\n", "# SIGUSR1=10, SIGUSR2=12, SIGCHLD=17, and SIGCHLD handler stays in effect even if\n", "# not restored: all other handlers are restored by Python after caught, but SIGCHLD\n", "# behavior is left to the platform's implementation; signal works on Windows too,\n", "# but defines only a few signal types; signals are not very portable in general" ], "language": "python", "metadata": {}, "outputs": [], "prompt_number": 1 }, { "cell_type": "code", "collapsed": false, "input": [ "import sys, signal, time\n", "def now():\n", " return time.asctime()\n", "def onSignal(signum, stackframe): # Python signal handler\n", " print 'Got signal', signum, 'at', now()\n", " # most handlers stay in effect but sigchld handler is not\n", " if signum == signal.SIGCHLD: #signal.signal(signal.SIGCHLD, onSignal)\n", " print 'sigchld caught'\n", "\n", "signum = int(sys.argv[1])\n", "signal.signal(signum, onSignal) # install signal handler\n", "while True: signal.pause() # sleep waiting for signals" ], "language": "python", "metadata": {}, "outputs": [], "prompt_number": 4 }, { "cell_type": "markdown", "metadata": {}, "source": [ "To run this script, simply put it in the background and send it signals by typing the\n", "kill -signal-number process-id shell command line; this is the shell\u2019s equivalent of\n", "Python\u2019s os.kill function available on Unix-like platforms only. Process IDs are listed\n", "in the PID column of ps command results. Here is this script in action catching signal\n", "numbers 10 (reserved for general use) and 9 (the unavoidable terminate signal)." ] }, { "cell_type": "code", "collapsed": false, "input": [ "import os, time, sys\n", "from socket import *\n", "myHost = '' # '' = all available interfaces on host\n", "myPort = 50007 # listen on a non-reserved port number\n", "\n", "sockobj = socket(AF_INET, SOCK_STREAM) #make a TCP socket object\n", "sockobj.bind((myHost, myPort)) #bind it to server port number\n", "sockobj.listen(5) #listen, allow 5 pending connects\n", "signal.signal(signal.SIGCHLD, signal.SIG_IGN) #avoid child zombie processes\n", "\n", "def now():\n", " return time.ctime(time.time())\n", "\n", "# child process: reply, exit simulate a blocking activity\n", "# read, write a client socket till eof when socket closed \n", "def handleClient(connection):\n", " time.sleep(5)\n", " while True:\n", " data = connections.recv(1024) # read next line on client socket\n", " if not data: break # send a reply line to the client\n", " reply = 'Echo=>%s at %s' % (data,now())\n", " connections.send(reply.encode()) # until eof when socket closed\n", " connections.close()\n", " os._exit(0)\n", "\n", "def dispatcher(): # listen until process killed \n", " while True: # wait for next connection,\n", " connection, address = sockobj.accept() # pass to process for service\n", " print 'Server connected by', address, 'at', now()\n", " childPid = os.fork() # copy this process\n", " if childPid == 0: # if in child process: handle\n", " handleClient(connection)" ], "language": "python", "metadata": {}, "outputs": [], "prompt_number": 5 }, { "cell_type": "markdown", "metadata": {}, "source": [ "1. Much simpler; we don\u2019t need to manually track or reap child processes.\n", "2. More accurate; it leaves no zombies temporarily between client requests." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "this technique is not universally supported across\n", "all flavors of Unix. If you care about portability, manually reaping children as we did\n", "in Example 12-4 may still be desirable." ] }, { "cell_type": "heading", "level": 3, "metadata": {}, "source": [ "Why multiprocessing doesn\u2019t help with socket server portability" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Though it's crash on Win,open sockets are not correctly\n", "pickled when passed as arguments into the new process, it's ok on linux." ] }, { "cell_type": "heading", "level": 2, "metadata": {}, "source": [ "Threading Servers" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Because threads all run in the same process and memory space, they automatically share\n", "sockets passed between them, similar in spirit to the way that child processes inherit\n", "socket descriptors. Unlike processes, though, threads are usually less expensive to start,\n", "and work on both Unix-like machines and Windows under standard Python today.\n", "Furthermore, many (though not all) see threads as simpler to program\u2014child threads\n", "die silently on exit, without leaving behind zombies to haunt the server." ] }, { "cell_type": "code", "collapsed": false, "input": [ "# Server side: open a socket on a port, listen for a message from a client,\n", "# and send an echo reply; echoes lines until eof when client closes socket;\n", "# spawns a thread to handle each client connection; threads share global\n", "# memory space with main thread; this is more portable than fork: threads\n", "# work on standard Windows systems, but process forks do not" ], "language": "python", "metadata": {}, "outputs": [] }, { "cell_type": "code", "collapsed": false, "input": [ "import time, _thread as thread # or use threading.Thread().start()\n", "from socket import * # get socket constructor and constants\n", "myHost = '' # server machine, '' means local host\n", "myPort = 50007 # listen on a non-reserved port number\n", "\n", "sockobj = socket(AF_INET, SOCK_STREAM) # make a TCP socket object\n", "sockobj.bind((myHost, myPort)) # bind it to server port number\n", "sockobj.listen(5) # allow up to 5 pending connects\n", "\n", "def now():\n", " return time.ctime(time.time()) # current time on the server\n", "\n", "def handleClient(connection): # in spawned thread: reply\n", " time.sleep(5) # simulate a blocking activity\n", " while True: # read, write a client socket\n", " data = connection.recv(1024)\n", " if not data: break\n", " reply = 'Echo=>%s at %s' % (data, now())\n", " connection.send(reply.encode())\n", " connection.close()\n", "\n", "def dispatcher(): # listen until process killed\n", " while True: # wait for next connection,\n", " connection, address = sockobj.accept() # pass to thread for service\n", " print 'Server connected by', address, 'at', now()\n", " thread.start_new_thread(handleClient, (connection,))\n", "\n", "dispatcher()" ], "language": "python", "metadata": {}, "outputs": [] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Remember that a thread silently exits when the function it is running returns; unlike\n", "the process fork version, we don\u2019t call anything like os . _exit in the client handler func-\n", "tion (and we shouldn\u2019t\u2014it may kill all threads in the process, including the main loop\n", "watching for new connections!). Because of this, the thread version is not only more\n", "portable, but also simpler." ] }, { "cell_type": "heading", "level": 2, "metadata": {}, "source": [ "Standard Library Server Classes" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "socketserver module defines classes that implement all flavors of forking and threading\n", "servers that you are likely to be interested in." ] }, { "cell_type": "code", "collapsed": false, "input": [ "\"\"\"\n", "Server side: open a socket on a port, listen for a message from a client, and \n", "send an echo reply; this version uses the standard library module socketserver to\n", "do its work; socketserver provides TCPServer, ThreadingTCPServer, ForkingTCPServer,\n", "UDP variants of these, and more, and routes each client connect request to a new \n", "instance of a passed-in request handler object's handle method; socketserver also\n", "supports Unix domain sockets, but only on Unixen; see the Python library manual.\n", "\"\"\"\n", "\n", "import SocketServer as socketserver, time # get socket server, handler objects\n", "myHost = '' # server machine, '' means local host\n", "myPort = 50007 # listen on a non-reserved port number\n", "def now():\n", " return time.ctime(time.time())\n", "\n", "class MyClientHandler(socketserver.BaseRequestHandler):\n", " def handle(self): # on each client connect\n", " print(self.client_address, now()) # show this client's address\n", " time.sleep(5) # simulate a blocking activity\n", " while True: # self.request is client socket\n", " data = self.request.recv(1024) # read, write a client socket\n", " if not data: break\n", " reply = 'Echo=>%s at %s' % (data, now())\n", " self.request.send(reply.encode())\n", " self.request.close()\n", "\n", "# make a threaded server, listen/handle clients forever\n", "myaddr = (myHost, myPort)\n", "server = socketserver.ThreadingTCPServer(myaddr, MyClientHandler)\n", "server.serve_forever()\n" ], "language": "python", "metadata": {}, "outputs": [] }, { "cell_type": "heading", "level": 2, "metadata": {}, "source": [ "Multiplexing Servers with select" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Technically, though, threads and processes don\u2019t really run in parallel, unless you\u2019re\n", "lucky enough to have a machine with many CPUs. Instead, your operating system\n", "performs a juggling act\u2014it divides the computer\u2019s processing power among all active\n", "tasks. It runs part of one, then part of another, and so on. All the tasks appear to run\n", "in parallel, but only because the operating system switches focus between tasks so fast\n", "that you don\u2019t usually notice. This process of switching between tasks is sometimes\n", "called time-slicing when done by an operating system; it is more generally known as\n", "multiplexing." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "In select-asynchronous servers, a single main\n", "loop run in a single process and thread decides which clients should get a bit of attention\n", "each time through. Client requests and the main dispatcher loop are each given a small\n", "slice of the server\u2019s attention if they are ready to converse." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "That is, when the sources passed to select are sockets, we can be sure that socket calls like accept , recv , and send will not\n", "block (pause) the server when applied to objects returned by select . Because of that,\n", "a single-loop server that uses select need not get stuck communicating with one client\n", "or waiting for new ones while other clients are starved for the server\u2019s attention." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Because this type of server does not need to start threads or processes, it can be efficient\n", "when transactions with clients are relatively short-lived. However, it also requires that\n", "these transactions be quick; if they are not, it still runs the risk of becoming bogged\n", "down waiting for a dialog with a particular client to end, unless augmented with threads\n", "or forks for long-running transactions." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Confusingly, select-based servers are often called asynchronous, to describe their multiplexing of short-lived transactions. \n", "Really, though, the classic forking and threading servers we met earlier are asynchronous, too,\n", "as they do not wait for completion of a given client\u2019s request. \n", "There is a clearer distinction between serial and parallel servers\n", "- {\u201csynchronous\u201d: \u201cserial, process one transaction at a time\u201d, \u201casynchronous\u201d : \u201cparallel\u201d }\n", "- forking, threading, and select loops are three alternative ways to implement parallel, asynchronous servers." ] }, { "cell_type": "heading", "level": 3, "metadata": {}, "source": [ "A select-based echo server" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "can handle multiple clients without ever starting new\n", "processes or threads" ] }, { "cell_type": "code", "collapsed": false, "input": [ "# P822\n", "\"\"\"\n", "Server: handle multiple clients in parallel with select. use the select\n", "module to manually multiplex among a set of sockets: main sockets which\n", "accept new client connections, and input sockets connected to accepted\n", "clients; select can take an optional 4th arg--0 to poll, n.m to wait n.m\n", "seconds, or omitted to wait till any socket is ready for processing.\n", "\"\"\"\n", "\n", "import sys\n", "import time\n", "from select import select\n", "from socket import socket, AF_INET, SOCK_STREAM\n", "\n", "\n", "def now():\n", " return time.ctime(time.time())\n", "\n", "myHost = '' # server machine, '' means local host\n", "myPort = 50007 # listen on a non-reserved port number\n", "if len(sys.argv) == 3: # allow host/port as cmdline args too\n", " myHost, myPort = sys.argv[1:]\n", "numPortSocks = 2 # number of ports for client connects\n", "\n", "# make main sockets for accepting new client requests\n", "mainsocks, readsocks, writesocks = [], [], []\n", "for i in range(numPortSocks):\n", " portsock = socket(AF_INET, SOCK_STREAM) # make a TCP/IP socket object\n", " portsock.bind((myHost, myPort)) # bind it to server port number\n", " portsock.listen(5) # listen, allow 5 pending connects\n", " mainsocks.append(portsock) # add to main list to identify\n", " readsocks.append(portsock) # add to select inputs list\n", " myPort += 1 # bind on consecutive ports\n", "\n", "# event loop: listen and multiplex until server process killed\n", "print('select-server loop starting')\n", "while True:\n", " # print(readsocks)\n", " readables, writeables, exceptions = select(readsocks, writesocks, [])\n", " for sockobj in readables:\n", " if sockobj in mainsocks: # for ready input sockets\n", " # port socket: accept new client\n", " newsock, address = sockobj.accept() # accept should not block\n", " print('Connect:', address, id(newsock)) # newsock is a new socket\n", " readsocks.append(newsock) # add to select list, wait\n", " else:\n", " # client socket: read next line\n", " data = sockobj.recv(1024) # recv should not block\n", " print('\\tgot', data, 'on', id(sockobj))\n", " if not data: # if closed by the clients\n", " sockobj.close() # close here and remv from\n", " readsocks.remove(sockobj) # del list else reselected\n", " else:\n", " # this may block: should really select for writes too\n", " reply = 'Echo=>%s at %s' % (data, now())\n", " sockobj.send(reply.encode())\n" ], "language": "python", "metadata": {}, "outputs": [] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Formally, select is called with three lists of selectable objects (input sources, out-\n", "put sources, and exceptional condition sources), plus an optional timeout. The\n", "timeout argument may be a real wait expiration value in seconds (use floating-point\n", "numbers to express fractions of a second), a zero value to mean simply poll and\n", "return immediately, or omitted to mean wait until at least one object is ready (as\n", "done in our server script). The call returns a triple of ready objects\u2014subsets of the\n", "first three arguments\u2014any or all of which may be empty if the timeout expired\n", "before sources became ready." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "If you\u2019re interested in using select , you will probably also be interested in checking\n", "out the asyncore.py module in the standard Python library. It implements a class-\n", "based callback model, where input and output callbacks are dispatched to class\n", "methods by a precoded select event loop. As such, it allows servers to be con-\n", "structed without threads or forks, and it is a select -based alternative to the sock\n", "etserver module\u2019s threading and forking module we met in the prior sections. As\n", "for this type of server in general, asyncore is best when transactions are short\u2014\n", "what it describes as \u201cI/O bound\u201d instead of \u201cCPU bound\u201d programs, the latter of\n", "which still require threads or forks. See the Python library manual for details and\n", "a usage example." ] }, { "cell_type": "heading", "level": 3, "metadata": {}, "source": [ "Twisted" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "For other server options, see also the open source Twisted system (http://twistedmatrix.com). Twisted is an asynchronous networking framework written in Python\n", "that supports TCP, UDP, multicast, SSL/TLS, serial communication, and more. It\n", "supports both clients and servers and includes implementations of a number of\n", "commonly used network services such as a web server, an IRC chat server, a mail\n", "server, a relational database interface, and an object broker.\n", "Although Twisted supports processes and threads for longer-running actions, it\n", "also uses an asynchronous, event-driven model to handle clients, which is similar\n", "to the event loop of GUI libraries like tkinter. It abstracts an event loop, which\n", "multiplexes among open socket connections, automates many of the details in-\n", "herent in an asynchronous server, and provides an event-driven framework for\n", "scripts to use to accomplish application tasks. Twisted\u2019s internal event engine is\n", "similar in spirit to our select -based server and the asyncore module, but it is re-\n", "garded as much more advanced. Twisted is a third-party system, not a standard\n", "library tool; see its website and documentation for more details." ] }, { "cell_type": "heading", "level": 2, "metadata": {}, "source": [ "Summary: Choosing a Server Scheme" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "1. select\n", " - perform very well when client transactions are relatively short and are not CPU-bound.\n", " - split up the processing of a client\u2019s request in such a\n", " way that it can be multiplexed with other requests and not block the server\u2019s main loop\n", " - select also seems more complex than spawning either processes or threads,\n", " because we need to manually transfer control among all tasks (for instance, compare\n", " the threaded and select versions of our echo server, even without write selects).\n", "\n", "2. threads or forks\n", " - Threads and forks are especially useful if clients require\n", " long-running processing above and beyond the socket calls used to pass data.\n", "\n", "3. The asyncore standard library module\n", "4. Twisted" ] }, { "cell_type": "heading", "level": 1, "metadata": {}, "source": [ "Making Sockets Look Like Files and Streams" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "allow a script to use standard stream tools such as the print and input built-in\n", "functions and sys module file calls (e.g., sys.stdout.write ), and connect them to sock-\n", "ets only when needed." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The socket object makefile method comes in handy anytime you wish to process a\n", "socket with normal file object methods or need to pass a socket to an existing interface\n", "or program that expects a file.\n", "\n", "The makefile method also allows us to treat normally binary socket data as text instead\n", "of byte strings, and has additional arguments such as encoding that let us specify non-\n", "default Unicode encodings for the transferred text\n", "\n", "Although text can always\n", "be encoded and decoded with manual calls after binary mode socket transfers, make\n", "file shifts the burden of text encodings from your code to the file wrapper object." ] }, { "cell_type": "heading", "level": 3, "metadata": {}, "source": [ "A Stream Redirection Utility" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "even when line buffering is requested, socket wrapper file writes (and\n", "by association, prints) are buffered until the program exits, manual flushes are reques-\n", "ted, or the buffer becomes full." ] }, { "cell_type": "code", "collapsed": false, "input": [ "# socket-unbuff-server.py\n", "from __future__ import print_function\n", "from socket import * # read three messages over a raw socket\n", "sock = socket()\n", "sock.bind(('', 60000))\n", "sock.listen(5)\n", "print('accepting...')\n", "conn, id = sock.accept() # blocks till client connect\n", "\n", "for i in range(3):\n", " print('receiving...')\n", " msg = conn.recv(1024) # blocks till data received\n", " print(msg) # gets all print lines at once unless flushed" ], "language": "python", "metadata": {}, "outputs": [] }, { "cell_type": "code", "collapsed": false, "input": [ "# socket-unbuff-client.py\n", "# send three msgs over wrapped and raw socket\n", "from __future__ import print_function\n", "import time\n", "from socket import *\n", "sock = socket() # default=AF_INET, SOCK_STREAM (tcp/ip)\n", "sock.connect(('localhost', 60000))\n", "# default=full buff, 0=error, 1 not linebuff!\n", "file = sock.makefile('w', buffering=1)\n", "\n", "print('sending data1')\n", "file.write('spam\\n')\n", "time.sleep(5) # must follow with flush() to truly send now\n", "# file.flush() # uncomment flush lines to see the difference\n", "\n", "print('sending data2')\n", "# adding more file prints does not flush buffer either\n", "print('eggs', file)\n", "time.sleep(5)\n", "# file.flush() # output appears at server recv only upon\n", "# flush or exit\n", "\n", "print('sending data3')\n", "sock.send(b'ham\\n') # low-level byte string interface sends immediately\n", "time.sleep(5) # received first if don't flush other two!" ], "language": "python", "metadata": {}, "outputs": [] }, { "cell_type": "heading", "level": 3, "metadata": {}, "source": [ "Buffering in other contexts: Command pipes revisited" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Buffered streams and deadlock are general issues that go beyond\n", "socket wrapper files." ] }, { "cell_type": "code", "collapsed": false, "input": [ "# pipe-unbuff-writer.py\n", "# output line buffered (unbuffered) if stdout is a terminal, buffered by default for\n", "# other devices: use -u or sys.stdout.flush() to avoid delayed output on pipe/socket\n", "import time, sys\n", "for i in range(5):\n", " print(time.asctime()) # print transfers per stream buffering\n", " sys.stdout.write('spam\\n') # ditto for direct stream file access\n", " time.sleep(2) # unles sys.stdout reset to other file" ], "language": "python", "metadata": {}, "outputs": [] }, { "cell_type": "code", "collapsed": false, "input": [ "# no output for 10 seconds unless Python -u flag used or sys.stdout.flush()\n", "# but writer's output appears here every 2 seconds when either option is used\n", "from __future__ import print_function\n", "import os\n", "for line in os.popen('python -u pipe-unbuff-writer.py'): # iterator reads lines\n", " print(line, end='') # blocks without -u!" ], "language": "python", "metadata": {}, "outputs": [] }, { "cell_type": "heading", "level": 3, "metadata": {}, "source": [ "Sockets versus command pipes" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "why use sockets in this redirection role at all? Programs require a direct spawning relationship, command pipes do not support longerlived or remotely running servers the way that sockets do.\n", "\n", "- With sockets, we can start client and server independently, and the server may continue\n", "running perpetually to serve multiple clients (albeit with some changes to our utility\n", "module\u2019s listener initialization code). Moreover, passing in remote machine names to\n", "our socket redirection tools would allow a client to connect to a server running on a\n", "completely different machine.\n", "- named pipes (fifos) accessed\n", "with the open call support stronger independence of client and server, too, but unlike\n", "sockets, they are usually limited to the local machine, and are not supported on all\n", "platforms." ] }, { "cell_type": "heading", "level": 1, "metadata": {}, "source": [ "A Simple Python File Server" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "implements both the server-side and the client-side\n", "logic needed to ship a requested file from server to client machines over a raw socket.\n", "\n", "implement client and server-side logic to transfer an arbitrary file from\n", "server to client over a socket; uses a simple control-info protocol rather\n", "than separate sockets for control and data (as in ftp), dispatches each\n", "client request to a handler thread, and loops to transfer the entire file\n", "by blocks; see ftplib examples for a higher-level transport scheme" ] }, { "cell_type": "code", "collapsed": false, "input": [ "\"\"\"\n", "#############################################################################\n", "implement client and server-side logic to transfer an arbitrary file from\n", "server to client over a socket; uses a simple control-info protocol rather\n", "than separate sockets for control and data (as in ftp), dispatches each\n", "client request to a handler thread, and loops to transfer the entire file\n", "by blocks; see ftplib examples for a higher-level transport scheme;\n", "#############################################################################\n", "\"\"\"\n", "\n", "import sys, os, time, thread\n", "from socket import *\n", "\n", "blksz = 1024\n", "defaultHost = 'localhost'\n", "defaultPort = 50001\n", "\n", "helptext = \"\"\"\n", "Usage...\n", "server=> getfile.py -mode server [-port nnn] [-host hhh|localhost]\n", "client=> getfile.py [-mode client] -file fff [-port nnn] [-host hhh|localhost]\n", "\"\"\"\n", "\n", "def now():\n", " return time.asctime()\n", "\n", "def parsecommandline():\n", " dict = {} # put in dictionary for easy lookup\n", " args = sys.argv[1:] # skip program name at front of args\n", " while len(args) >= 2: # example: dict['-mode'] = 'server'\n", " dict[args[0]] = args[1]\n", " args = args[2:]\n", " return dict\n", "\n", "def client(host, port, filename):\n", " sock = socket(AF_INET, SOCK_STREAM)\n", " sock.connect((host, port))\n", " sock.send((filename + '\\n').encode()) # send remote name with dir: bytes\n", " dropdir = os.path.split(filename)[1] # filename at end of dir path\n", " file = open(dropdir, 'wb') # create local file in cwd\n", " while True:\n", " data = sock.recv(blksz) # get up to 1K at a time\n", " if not data: break # till closed on server side\n", " file.write(data) # store data in local file\n", " sock.close()\n", " file.close()\n", " print('Client got', filename, 'at', now())\n", "\n", "def serverthread(clientsock):\n", " sockfile = clientsock.makefile('r') # wrap socket in dup file obj\n", " filename = sockfile.readline()[:-1] # get filename up to end-line\n", " try:\n", " file = open(filename, 'rb')\n", " while True:\n", " bytes = file.read(blksz) # read/send 1K at a time\n", " if not bytes: break # until file totally sent\n", " sent = clientsock.send(bytes)\n", " assert sent == len(bytes)\n", " except:\n", " print 'Error downloading file on server:', filename\n", " clientsock.close()\n", "\n", "def server(host, port):\n", " serversock = socket(AF_INET, SOCK_STREAM) # listen on TCP/IP socket\n", " serversock.bind((host, port)) # serve clients in threads\n", " serversock.listen(5)\n", " while True:\n", " clientsock, clientaddr = serversock.accept()\n", " print 'Server connected by', clientaddr, 'at', now()\n", " thread.start_new_thread(serverthread, (clientsock,))\n", "\n", "def main(args):\n", " host = args.get('-host', defaultHost) # use args or defaults\n", " port = int(args.get('-port', defaultPort)) # is a string in argv\n", " if args.get('-mode') == 'server': # None if no -mode: client\n", " if host == 'localhost': host = '' # else fails remotely\n", " server(host, port)\n", " elif args.get('-file'): # client mode needs -file\n", " client(host, port, args['-file'])\n", " else:\n", " print helptext\n", "\n", "if __name__ == '__main__':\n", " args = parsecommandline()\n", " main(args)" ], "language": "python", "metadata": {}, "outputs": [], "prompt_number": 6 }, { "cell_type": "markdown", "metadata": {}, "source": [ "1. The server function farms out each incoming client request to a thread that trans-\n", "fers the requested file\u2019s bytes.\n", "2. The client function sends the server a file\u2019s name and stores all the bytes it gets\n", "back in a local file of the same name.\n", "3. The most novel feature here is the protocol between client and server: the client starts\n", "the conversation by shipping a filename string up to the server, terminated with an end-\n", "of-line character, and including the file\u2019s directory path in the server. At the server, a\n", "spawned thread extracts the requested file\u2019s name by reading the client socket, and\n", "opens and transfers the requested file back to the client, one chunk of bytes at a time." ] }, { "cell_type": "heading", "level": 2, "metadata": {}, "source": [ "Running the File Server and Clients" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "One subtle security point here: the server instance code is happy to send any server-\n", "side file whose pathname is sent from a client, as long as the server is run with a user-\n", "name that has read access to the requested file. If you care about keeping some of your\n", "server-side files private, you should add logic to suppress downloads of restricted files.\n", "I\u2019ll leave this as a suggested exercise here, but we will implement such filename checks\n", "in a different getfile download tool later in this book." ] }, { "cell_type": "heading", "level": 3, "metadata": {}, "source": [ "Adding a User-Interface Frontend" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "For instance, it would be easy to implement a simple tkinter GUI frontend to the client-\n", "side portion of the getfile script we just met. Such a tool, run on the client machine,\n", "may simply pop up a window with Entry widgets for typing the desired filename, server,\n", "and so on. Once download parameters have been input, the user interface could either\n", "import and call the getfile.client function with appropriate option arguments, or\n", "build and run the implied getfile.py command line using tools such as os.system ,\n", "os.popen , subprocess , and so on." ] }, { "cell_type": "heading", "level": 4, "metadata": {}, "source": [ "Using row frames and command lines" ] }, { "cell_type": "code", "collapsed": false, "input": [ "\"\"\"\n", "launch getfile script client from simple tkinter GUI;\n", "could also use os.fork+exec, os.spawnv (see Launcher);\n", "windows: replace 'python' with 'start' if not on path;\n", "\"\"\"\n", "\n", "import os\n", "from tkinter import *\n", "from tkinter.messagebox import showinfo\n", "def onReturnKey():\n", " cmdline = ('python getfile.py -mode client -file %s -port %s -host %s' %\n", " (content['File'].get(),\n", " content['Port'].get(),\n", " content['Server'].get()))\n", " os.system(cmdline)\n", " showinfo('getfilegui-1', 'Download complete')\n", "\n", "box = Tk()\n", "labels = ['Server', 'Port', 'File']\n", "content = {}\n", "for label in labels:\n", " row = Frame(box)\n", " row.pack(fill=X)\n", " Label(row, text=label, width=6).pack(side=LEFT)\n", " entry = Entry(row)\n", " entry.pack(side=RIGHT, expand=YES, fill=X)\n", " content[label] = entry\n", "\n", "box.title('getfilegui-1')\n", "box.bind('', (lambda event: onReturnKey()))\n", "mainloop()" ], "language": "python", "metadata": {}, "outputs": [] }, { "cell_type": "heading", "level": 4, "metadata": {}, "source": [ "Using grids and function calls" ] }, { "cell_type": "code", "collapsed": false, "input": [ "\"\"\"\n", "same, but with grids and import+call, not packs and cmdline;\n", "direct function calls are usually faster than running files;\n", "\"\"\"\n", "\n", "import getfile\n", "from tkinter import *\n", "from tkinter.messagebox import showinfo\n", "\n", "def onSubmit():\n", " getfile.client(content['Server'].get(),\n", " int(content['Port'].get()),\n", " content['File'].get())\n", " showinfo('getfilegui-2', 'Download complete')\n", "\n", "box = Tk()\n", "labels = ['Server', 'Port', 'File']\n", "rownum = 0\n", "content = {}\n", "for label in labels:\n", " Label(box, text=label).grid(column=0, row=rownum)\n", " entry = Entry(box)\n", " entry.grid(column=1, row=rownum, sticky=E+W)\n", " content[label] = entry\n", " rownum += 1\n", "\n", "box.columnconfigure(0, weight=0) # make expandable\n", "box.columnconfigure(1, weight=1)\n", "Button(text='Submit', command=onSubmit).grid(row=rownum, column=0, columnspan=2)\n", "\n", "box.title('getfilegui-2')\n", "box.bind('', (lambda event: onSubmit()))\n", "mainloop()" ], "language": "python", "metadata": {}, "outputs": [] }, { "cell_type": "heading", "level": 4, "metadata": {}, "source": [ "Using a reusable form-layout class" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "If you\u2019re like me, though, writing all the GUI form layout code in those two scripts can\n", "seem a bit tedious, whether you use packing or grids. In fact, it became so tedious to\n", "me that I decided to write a general-purpose form-layout class, shown in Exam-\n", "ple 12-20, which handles most of the GUI layout grunt work." ] }, { "cell_type": "code", "collapsed": false, "input": [ "\"\"\"\n", "##################################################################\n", "a reusable form class, used by getfilegui (and others)\n", "##################################################################\n", "\"\"\"\n", "\n", "from tkinter import *\n", "entrysize = 40\n", "\n", "class Form: # add non-modal form box\n", " def __init__(self, labels, parent=None): # pass field labels list\n", " labelsize = max(len(x) for x in labels) + 2\n", " box = Frame(parent) # box has rows, buttons\n", " box.pack(expand=YES, fill=X) # rows has row frames\n", " rows = Frame(box, bd=2, relief=GROOVE) # go=button or return key\n", " rows.pack(side=TOP, expand=YES, fill=X) # runs onSubmit method\n", " self.content = {}\n", " for label in labels:\n", " row = Frame(rows)\n", " row.pack(fill=X)\n", " Label(row, text=label, width=labelsize).pack(side=LEFT)\n", " entry = Entry(row, width=entrysize)\n", " entry.pack(side=RIGHT, expand=YES, fill=X)\n", " self.content[label] = entry\n", " Button(box, text='Cancel', command=self.onCancel).pack(side=RIGHT)\n", " Button(box, text='Submit', command=self.onSubmit).pack(side=RIGHT)\n", " box.master.bind('', (lambda event: self.onSubmit()))\n", "\n", " def onSubmit(self): # override this\n", " for key in self.content: # user inputs in\n", " print(key, '\\t=>\\t', self.content[key].get()) # self.content[k]\n", "\n", " def onCancel(self): # override if need\n", " Tk().quit() # default is exit\n", "\n", "class DynamicForm(Form):\n", " def __init__(self, labels=None):\n", " labels = input('Enter field names: ').split()\n", " Form.__init__(self, labels)\n", " def onSubmit(self):\n", " print('Field values...')\n", " Form.onSubmit(self)\n", " self.onCancel()\n", "\n", "if __name__ == '__main__':\n", " import sys\n", " if len(sys.argv) == 1:\n", " Form(['Name', 'Age', 'Job']) # precoded fields, stay after submit\n", " else:\n", " DynamicForm() # input fields, go away after submit\n", " mainloop()" ], "language": "python", "metadata": {}, "outputs": [] }, { "cell_type": "code", "collapsed": false, "input": [ "\"\"\"\n", "launch getfile client with a reusable GUI form class;\n", "os.chdir to target local dir if input (getfile stores in cwd);\n", "to do: use threads, show download status and getfile prints;\n", "\"\"\"\n", "\n", "from form import Form\n", "from tkinter import Tk, mainloop\n", "from tkinter.messagebox import showinfo\n", "import getfile, os\n", "\n", "class GetfileForm(Form):\n", " def __init__(self, oneshot=False):\n", " root = Tk()\n", " root.title('getfilegui')\n", " labels = ['Server Name', 'Port Number', 'File Name', 'Local Dir?']\n", " Form.__init__(self, labels, root)\n", " self.oneshot = oneshot\n", "\n", " def onSubmit(self):\n", " Form.onSubmit(self)\n", " localdir = self.content['Local Dir?'].get()\n", " portnumber = self.content['Port Number'].get()\n", " servername = self.content['Server Name'].get()\n", " filename = self.content['File Name'].get()\n", " if localdir:\n", " os.chdir(localdir)\n", " portnumber = int(portnumber)\n", " getfile.client(servername, portnumber, filename)\n", " showinfo('getfilegui', 'Download complete')\n", " if self.oneshot: Tk().quit() # else stay in last localdir\n", "\n", "if __name__ == '__main__':\n", " GetfileForm()\n", " mainloop()" ], "language": "python", "metadata": {}, "outputs": [] }, { "cell_type": "markdown", "metadata": {}, "source": [ "One caveat worth pointing out here: the GUI is essentially dead while the download is\n", "in progress (even screen redraws aren\u2019t handled\u2014try covering and uncovering the\n", "window and you\u2019ll see what I mean). We could make this better by running the down-\n", "load in a thread, but since we\u2019ll see how to do that in the next chapter when we explore\n", "the FTP protocol, you should consider this problem a preview." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "In\n", "particular, getfile clients can talk only to machines that are running a getfile server.\n", "In the next chapter, we\u2019ll discover another way to download files\u2014FTP\u2014which also\n", "runs on sockets but provides a higher-level interface and is available as a standard\n", "service on many machines on the Net. We don\u2019t generally need to start up a custom\n", "server to transfer files over FTP, the way we do with getfile . In fact, the user-interface\n", "scripts in this chapter could be easily changed to fetch the desired file with Python\u2019s\n", "FTP tools, instead of the getfile module. But instead of spilling all the beans here, I\u2019ll\n", "just say, \u201cRead on.\u201d" ] }, { "cell_type": "heading", "level": 1, "metadata": {}, "source": [ "Using Serial Ports" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "If you\u2019re looking for a lower-level way to communicate with devices in general, though,\n", "you may also be interested in the topic of Python\u2019s serial port interfaces. This isn\u2019t quite\n", "related to Internet scripting, but it\u2019s similar enough in spirit and is discussed often\n", "enough on the Net to merit a few words here.\n", "\n", "In brief, scripts can use serial port interfaces to engage in low-level communication with\n", "things like mice, modems, and a wide variety of serial devices and hardware. Serial port\n", "interfaces are also used to communicate with devices connected over infrared ports\n", "(e.g., hand-held computers and remote modems). Such interfaces let scripts tap into\n", "raw data streams and implement device protocols of their own. Other Python tools\n", "such as the ctypes and struct modules may provide additional tools for creating and\n", "extracting the packed binary data these ports transfer.\n", "\n", "At this writing, there are a variety of ways to send and receive data over serial ports in\n", "Python scripts. Notable among these options is an open source extension package\n", "known as pySerial, which allows Python scripts to control serial ports on both Windows\n", "and Linux, as well as BSD Unix, Jython (for Java), and IronPython (for .Net and Mono).\n", "Unfortunately, there is not enough space to cover this or any other serial port option\n", "in any sort of detail in this text. As always, see your favorite web search engine for up-\n", "to-date details on this front." ] } ], "metadata": {} } ] }