Python and Basic Networking Operations | Developer.com

Every programmer has asked the question “How do I do this (networking operation) programmatically?” at some point. Routine tasks such as website browsing, file transfers and sending or receiving email that human-facing software clients facilitate seem to be enigmatic and complex tasks when they need to be integrated into a software solution. The goal of this Python programming tutorial is to break down some of the more routine networking tasks that a developer may need to undertake using the Python programming language.

This article conspicuously omits any references to MacOS and Python. This is because Apple has decided to remove Python from MacOS.

Interested in learning Python through an online class? We have a great article listing the Top Online Courses to Learn Python.

Sockets in Python

A common definition of a socket, as stated by both IBM, Oracle and the Linux Manual Pages, is “an endpoint of a two-way communications channel.” Sockets are the foundation of any network-enabled application, regardless of how it is developed or what language they are created in.

Without a socket connection, no advanced networking operation of any kind can take place. As such, socket connections are considered to be very “low level.” But even without specialization, socket connections can be very useful in their own right from a diagnostic perspective. Additionally, If a custom or non-standard communications protocol is being implemented, then a socket connection can be used to implement that connectivity.

Sockets in Python are composed of an Address Family and a Type. Depending on the Address Family, an optional protocol number can be specified. An open file stream can also be used to create a socket. As sockets in Python are built on top of the C/C++ library functions that are built-into Linux, the same manual pages for those libraries can be used to get information about how to implement sockets in Python. The information below comes directly from the Linux Manual Page that can be accessed with the command:

$ man 2 socket

Figure 1 – Selected output of the second manual page section for Unix Sockets

This command accesses “Section 2” of the manual page for the topic “socket”. Section 2 of a manual entry for a given topic usually provides information about Linux System Calls. For the purposes of an introductory tutorial, the focus will be on one of the most common Address Families, namely AF_INET for IPv4-based networking and the SOCK_STREAM socket type, which provides for reliable two-way communications. And, while many different Address Families and Types are listed, not all are supported in Linux or Windows. On top of this, a programmer is limited further by the Address Families and Types that are supported within Python. While the Windows Python interpreter supports a subset of these constants, the information contained in Linux manual pages will not be available in Windows.

An alternative way to get this information is to simply type “man 2 socket” into a search engine. All of the Linux manual pages are available online.

Once a socket is instantiated, it must be bound to one or more interfaces and a single port. An interface refers to any available network interface on the device. The examples in this article will bind to all interfaces, but there may be situations in which binding to a single interface may be desirable. To list the interfaces in Linux, use the command:

$ ip a

Below is an example output of this command:

Figure 2 – Example listing of interfaces on a device

127.0.0.1 is the loopback interface. If this interface is used, then the only connections that can be made to a server would be those originating from that server. Those connections do not route to anywhere on any network, even if the device is connected to a network. The other interface shown here, 10.0.2.15, would allow connections from the local network, albeit this is a non-routable IP address. Connections from the broader Internet could be permitted if the router used on the network was configured for port forwarding.

In Windows, the interfaces can be listed using the command:

C…> ipconfig

This command gives output similar to what is shown below:

Figure 3 – Example output of ipconfig command in Windows

Note, the loopback interface of 127.0.0.1 may not be listed in Windows, but it is still present.

Regarding the port, 60000 was chosen because it is a high-numbered port that other applications are unlikely to be using. Only one socket can be bound to a particular port, and if another socket tries to bind to a port that is already bound, then an exception will be raised. And, while there is no restriction on what port to which a socket can be bound, there are certain expectations when it comes to particular ports. For example, port 25 is “expected” to be an SMTP port; ports 80 and 443 are “expected” to be HTTP and HTTPS ports, respectively; port 22 is “expected” to be an SSH port. Developers should always check and verify that the port being chosen for their application is not a popular port used by a major application.

Read: Text Scraping in Python

How to Create A Basic Server using Python

The code below creates a rudimentary server in Python that repeats back what a user typed upon connecting and then disconnects that user. It also provides a means for the remote user to shut the server down. Note that, any attempt to create a socket using an invalid Family Address or Type will result in an Operation not supported error.

# demo-basic-server.py

import datetime
import socket
import sys

def main(argv):
 try:
  # Any use of an unsupported Address Family or Type will give an
  # "Operation not supported" error.
  # This method does not allow for reusing of previously closed 
  # sockets. This means that depending on which state the socket
  # was in when the socket was closed, this program may not be 
  # immediately restartable. This is resolved by setting reusability
  # in the socket options (see the next line)
  s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)

  # This option enables a socket to be reused without having to wait
  # for a timeout to expire.
  s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)

  # Network addresses in Python are represented as "(interface, port)"
  # including the parenthesis.

  # Bind to all interfaces on the device, on port 60000
  # Ports below 1024 must be bound as root. In this example, if it
  # necessary to bind to a specific interface for this device, then 
  # the '' below would be replaced with '127.0.0.1' or whatever other 
  # interface is configured.
  s.bind(('', 60000))

  # If the listen method does not immediately follow the bind method, 
  # then an "Invalid argument" error will occur. Mote that as Python 3.5
  # a parameter to this is optional.
  s.listen()

  # Since the accept method blocks, there is no way to trap a
  # Keyboard interrupt within the while loop.

  try:
   while True:
    # The accept method will block until a connection is made.
    conn, addr = s.accept()
    print ("Got a connection from [" + str(addr) + "] at [" + 
     str(datetime.datetime.now()) + "]")
    # This command presumes UTF-8 encoding for strings. If this is not
    # the default system encoding, then it must be changed or else the
    # command will raise an exception.
    conn.sendall(b"You connected at [" + bytes(str(datetime.datetime.now()), 'utf-8') 
     + b"]\n")

    # Read any input from the connection, so it can be echoed back.
    # The 4096 value is recommended by the official Python documentation.
    # The command blocks until either the data count below is received 
    # or when a user enters a newline character.
    data = conn.recv(4096)

    conn.sendall (b"You sent: [" + data + b"]\n")

    # Do some rudimentary processing. Note the binary data conversion.
    if b"shutdown" == data.strip().lower():
     conn.sendall(b"You told me to shut down!\n")
     raise Exception ("Remote user initiated shutdown.")

    conn.close()
  except KeyboardInterrupt:
   print ("User pressed Ctrl-C, stopping.")
  except Exception as err:
   print ("Execution ended because of [" + str(err) + "]")
  # Closes the socket.
  s.close()
  s = None

 except Exception as err:
  print ("Socket listen failed due to error [" + str(err) + "]")


if __name__ == "__main__":
 main(sys.argv[1:])

Any firewall or antivirus program that is running on the same device as this code must be configured to allow for the Python interpreter to bind to ports and communicate over the network.

The example above makes use of the telnet client to connect to the server. This allows for an end user to directly send commands to the server interactively. However, there are times when a programmatic basic client is necessary.

How to Create a Basic Client in Python

The first thing to consider when writing any client that programmatically communicates with a server is to ensure that the server behavior is matched perfectly. In the case of the basic server above, a message is sent on connection, then an input is expected, and then another message is sent. After all of this, the server disconnects the client.

If this behavior is not matched perfectly, then unpredictable actions will occur. Depending on the server being communicated with, this could cause major problems.

The Python code below compliments the basic server code above:

# demo-basic-client.py

import socket
import sys

def ProcessConnection (inSocket, message):
 # Note how the client program has to match the functionality of the
 # server program.
 # Get any message first.
 inMsg = GetData (inSocket)
 print ("Got Message [" + inMsg + "]")
 # Some sample commands... 
 SendData (inSocket, message)
 inMsg = GetData (inSocket)
 print ("Got Second Message [" + inMsg + "]")

def GetData (inSocket):
 message = ""
 try:
  message = inSocket.recv(4096)
 except socket.error as err:
  print ("Couldn't receive message due to error [" + str(err) + "]")
 
 return message.decode()

def SendData (inSocket, message):
 try:
  inSocket.sendall(bytes(message, "utf-8"))
  print ("Successfully sent message [" + message + "]")
 except socket.error as err:
  print ("Couldn't send message due to error [" + str(err) + "]")

def main(argv):
 try:
  # As it was with the server, any use of an unsupported Address 
  # Family or Type will give an "Operation not supported" error.
  # This method likewise does not allow for reusing of previously 
  # closed sockets. This means that depending on which state the 
  # socket was in when the socket was closed, this program may not be 
  # immediately restartable. This is resolved by setting reusability
  # in the socket options (see the next line)
  s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)

  # This option enables a socket to be reused without having to wait
  # for a timeout to expire.
  s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
  
  # Connect to the server. Both the host and port must match the 
  # what the server is listening on. In this case we connect on the 
  # localhost interface.
  s.connect (("127.0.0.1", 60000))
 
  msgFromPrompt = "Hello!"
  try:
   msgFromPrompt = sys.argv[1]
  except Exception as err:
   print ("No message specified. Defaulting to \"Hello!\"")
   msgFromPrompt = "Hello!"
  ProcessConnection (s, msgFromPrompt)

  # Closes the socket.
  s.close()
  s = None
 except Exception as err:
  print ("Socket action failed due to error [" + str(err) + "]")


if __name__ == "__main__":
 main(sys.argv[1:])

Read: Best Python IDE and Code Editors

Other Socket Programming Considerations

The biggest potential issue with these basic examples is that no encryption or security of any kind is provided by the socket module. Encryption is expected to be provided by the application itself.

Two of the biggest sources of contention in Python socket code are:

• If sockets are not configured to be reusable (the s.setsockopt method) and the program is terminated, there may be a minute or so wait required until the program can be restarted. This is due to Linux waiting for the socket to time out.
• All data sent from a client and sent from a server must be represented as binary and must have consistent encoding.

Windows-Specific Considerations

Windows presents its own considerations to this code. For starters, when first running the basic server code, it is not only possible to encounter the following dialog box, but it is imperative to expect it and to not click it away:

Figure 4 – Windows Firewall Dialog Box

If this dialog box is canceled, then any Python server application will not work properly. Even worse, the only way to get this dialog box back if it is unintentionally canceled or dismissed is to reset all Windows Firewall settings and re-run the Python server script.

Windows Command Prompt boxes also present their own quirks. When running the server code, do not click or drag the cursor in the Command Prompt box that runs the server. This may suppress messages that the server is printing.

Lastly, the keyboard combination Ctrl-C or Ctrl-Break may not be caught by Python when running a server application. The application may need to be recoded to use Windows signals directly, or it can be halted by closing the Windows Command Prompt box. Note that the demonstration of the client passes the proper shutdown command to the server.

Additional Python Socket Resources

The Python Socket Module is fully documented at https://docs.python.org/3/library/socket.html#module-socket. Most implementations involving sockets in Python do not involve working directly with sockets. Instead, modules implement higher-level protocol interfaces which take care of all of the calls to the lower level sockets automatically, and without any direct involvement by the developer. These implementations also automatically take care of encryption/decryption, as well as other security concerns. Going even further, these server implementations handle more complex tasks such as spreading the server load among multiple instances of the server.

Two such examples, SFTP (Secure File Transfer Protocol) and HTTPS (encrypted web client) will be discussed in the next part of this tutorial series on basic Python networking operations.