B. TCP sockets with Tcl

Note

This text is extracted from the book: Tcl 8.5 Networking Programming. (Tcl 8.5 Networking Programming, 2010)

In general, reading or writing from/to a file and sending/receiving data over/from the network are similar concepts, because of the universal abstract concept of channels.

When you open a file in Tcl, you operate on the channel leading to that file-you read data from that channel, and write to that channel.

When you open a TCP connection to the server, you also get a channel that is essentially identical in usage; you can also read from and write to it.

Moreover, for all types of channels, you use the same set of commands such as puts , gets or read. This is the beauty of the Tcl architecture that makes networking so simple.

Of course, some differences must occur, as the underlying devices are significantly different between a file on the disk and the connection over a network.

TCP communication forms the basis of modern networking and the basic Tcl command to use it is socket , and it is built into the Tcl interpreter core.

A socket is an abstract term representing the endpoint of a bidirectional connection across the network. We often use the terms socket and channel interchangeably, although the channel term is more general (not every channel is a socket, but every socket is a channel).

A channel can be an open file, but it can also be a network socket, a pipe, or any other channel type.

Depending on the type of the channel, it can support reading from it, writing to it, or both.

Tcl comes with three default channels- stdin , stdout , and stderr . These channels correspond to the standard input, standard output, and standard error channels of operating systems. Standard input can be used to read information from the user, standard output should be used to write information to user, and standard error is used to write errors.

The execution of socket will result in using channels and the effect of executing this command is usually opening the connection over the TCP protocol (socket supports only TCP) and returning a channel identifier, which may be used for receiving or sending data through that newly created channel, with commands like read or puts.

The command may be used in two flavors in order to create client-side or server-side sockets.

Note

Client sockets serve as the connection opened from a client application to a server of your choice. On the contrary, a server socket does not connect to anything by itself, its primary task is to listen for incoming connections.

Such connections will be accepted automatically, and a new channel will be created for each of them, enabling communication to each of the connecting clients.

Let's explain the details of TCP networking based on simple, yet working example code consisting of two parts: a server and a client.

B.1. Using TCP sockets

B.1.1. Server socket

First look at the server code stored in the server.tcl file:

  socket -server serverProc 9876
  puts "server started and waiting for connections..."
  
  proc serverProc {channelId clientAddress clientPort} {
       puts "connection accepted from $clientAddress:$clientPort"
       puts "server socket details: [fconfigure $channelId -sockname]"
       puts "peer socket details: [fconfigure $channelId -peername]"
       after 5000 set cont 1; vwait cont
       puts $channelId "thank you for connecting to our server!"
       close $channelId
  }

  vwait forever

To create the listening socket, you have to use the socket command in the following format:

socket -server procedureName ?options? port

The last argument is the value of the port on which the server will listen for connections. Before this value, you can use optional parameters. To be more specific, only one option is available: -myaddr address , and it can be used if the computer where you run your server program has more than one IP interface, so you can specify on which IP address the server should accept connections.

The first parameter is the name of the command that will be called once a new connection is established. In our example, this procedure is called serverProc.

The procedure has to accept three arguments:

  • [channelId]

    the identifier of the channel which can be used to send or receive data to/from the client program,

  • [clientAddress]

    the client IP address,

  • [clientPort]

    and the port.

The procedure serverProc first prints out some details about the client-its IP address and port. Next, it prints information about the server and client sockets, illustrating the usage of fconfigure command.

After this, some time-consuming data processing is simulated, by forcing the execution to be suspended for 5 seconds. After that time, it writes a thank you message to the channel, effectively sending it to the client and closing the channel (and thereby, a TCP network connection).

The server socket will listen for the connection only when the Tcl event loop is enabled (otherwise, the the program would end), so the last line enters the event loop with the vwait forever command.

We run the server.tcl:

$ tclsh server.tcl 
server started and waiting for connections...

We may try a connection to this server using the telnet program:

$ telnet localhost 9876

And after 5 seconds the connection is closed:

$ telnet localhost 9876
Trying ::1...
Connected to localhost.
Escape character is '^]'.
thank you for connecting to our server!
Connection closed by foreign host.

And at the server side:

$  tclsh server_socket.tcl 
server started and waiting for connections...
connection accepted from ::1:56327
server socket details: ::1 localhost 9876
peer socket details: ::1 localhost 56327

B.1.2. Client socket

And write a client.tcl:

set serverChannel [socket localhost 9876]

set startTime [clock seconds]

puts "client socket details: [fconfigure $serverChannel -sockname]"
puts "peer socket details: [fconfigure $serverChannel -peername]"
puts [read $serverChannel]
puts "execution has been blocked for [expr [clock seconds] - $startTime] seconds"

close $serverChannel

First let's present the output produced by this client code:

$ tclsh client.tcl 
client socket details: ::1 localhost 51430
peer socket details: ::1 localhost 9876
thank you for connecting to our server!

execution has been blocked for 5 seconds

What happens in the client code is that in the first line we create a TCP connection towards the localhost:9876 socket and save the channel reference in the serverChannel variable.

A TCP socket is uniquely identified by the four values (remote IP, local IP, remote port, local port). You can have multiple connections/sockets, as long as at least one of those differs.

We also store the current time (expressed in seconds) to calculate how long we had to wait for the answer from the server. The code prints out some details about the connection using the command fconfigure which allows us to get/set various options for a channel.

In the case of network channels, the fconfigure command has some usefull configuration options:

  • sockname - returns the details about the closer end of the connection, that is about the socket used by the client: the IP address, the name of the host, and the number of the port. The data is returned as a three-element list. In this example, the client connects to the server from the port 51430. Different connections will be made by different ports randomly chosen by the operating system. But the port number may be forced along with the -myport option for the socket command.

    set serverChannel [ socket -myport 51430 localhost 9876 ]

  • peername - similar to previous option, but the data returned is related to second end of the connection. We can see that we are indeed connected to port 9876, the same one where the server is listening for connections. This option can not be used on the channel identifier returned directly by the socket -server command, as it is not connected to anything and cannot be used to send or receive any data, it will only listen for incoming connections.

  • error - returns the current error status for the connection. If there is no error, an empty string is returned.

Next, the command read read data from the network connection as we would do for any other type of channel. This command reads all the data until the end-of-file marker (in this case, caused by close $channelId being executed on the server's side).

As we know, the server waits for 5 seconds before sending any answer. This effectively causes blocking of the entire client application, because execution hangs at the read command waiting for the data. In many cases such a behavior is unacceptable; therefore an alternative was introduced -nonblocking sockets.

B.1.3. Using nonblocking sockets

The concept of a nonblocking socket is rather simple, instead of executing the command that would wait for (and therefore block), and eventually read the data, you just register a procedure that should be called when there is some data to be read (in other words, when the channel becomes readable).

It is the duty of the underlying nonblocking I/O subsystem to call this procedure. The advantages are obvious: your code does not block, is more responsive (which is crucial in the case of GUI-based applications) and may do some other work in the meantime.

It can also handle multiple connections at the same time. As for the drawbacks, the code may become a bit more complicated, but this is not something you could not handle. :^)

First, let's modify the server code a little:

socket -server serverProc 9876

puts "server started and waiting for connections..."

proc serverProc {channelId clientAddress clientPort} {
     after 5000 set cont 1; vwait cont
     puts -nonewline $channelId "12345"
     flush $channelId
     after 5000 set cont 1; vwait cont
     puts $channelId "6789"
     flush $channelId
     after 5000 set cont 1; vwait cont
     puts $channelId "thank you for connecting to our server!"
     close $channelId
   }
vwait forever

Now it returns 2 lines, wherein the first line is produced in two phases-first it sends 12345, but without the end-of-line character (the -nonewline option for puts ), and after 5 seconds, the rest of line 6789.

Following that it sends the line identical to the earlier one (also without the newline character), and closes the connection. Each time, the flush command is executed to make sure the data is sent, otherwise, the data could be buffered. Effectively, it takes 15 seconds to finish sending the data, and we would like to have client code that will not be blocked for that long.

The following is the client-side code that will not be blocked, due to usage of event programming:

set serverChannel [socket -async localhost 9876]
fconfigure $serverChannel -blocking 0
fileevent $serverChannel readable [list readData $serverChannel]

proc readData {serverChannel} {
     global end
     set startTime [clock seconds]
     set data [read $serverChannel]
     if {[eof $serverChannel]} {
        close $serverChannel
        set end 1
        break
     }
     puts "read: $data"
     puts "execution has been blocked for [expr [clock seconds] - $startTime] seconds"
  }
vwait end

The first line is almost identical to that of the previous example, with the difference that the -async option is used. This option causes the socket command to not wait until the connection is established, and to exit immediately. This may matter in the case of slow, overloaded networks.

The -async option causes connection to happen in the background, and the socket command returns immediately. The socket becomes writable when the connection completes, or fails. You can use fileevent to get a callback when this occurs. If you use the socket before the connection completes, and the socket is in blocking mode, then Tcl automatically blocks and waits for the connection to complete. If the socket is in non-blocking mode, attempts to use the socket return immediately. (Socket Programming)

The next line causes the channel to be switched from the default blocking mode to nonblocking.

In this mode, the command operating on this channel will not block, for example read will return only the data that is available at the moment in the input buffer, without waiting for the end of the file notification.

The nonblocking mode will only make sense when Tcl enters the event loop, which is why the last line calls vwait end , effectively causing the interpreter to wait until the end variable is written.

B.1.4. Handling errors

Network operations are especially vulnerable to various kinds of errors, so if you wish to create reliable application, proper handling of such a situation is indispensable. The basis of this is appropriate reaction to a writable event.

Note

The issue is that the channel is considered writable not only when it is possible to write to it, but also if some error occurred on the underlying device/file.

Once an error occurs, your script will get a bunch of writable events to handle. Therefore, the proper implementation of the handler command must check the error condition, using the command fconfigure $socket -error.

Let's discuss error related issues using the following example:

  if {[catch {set serverChannel [socket -async somehostname.com 9876]} e]} {
      puts $e
      exit 1
  }

  fconfigure $serverChannel -blocking 0

  fileevent $serverChannel writable [list socketWritable $serverChannel]

  fileevent $serverChannel readable [list socketReadable $serverChannel]

  set timer [after 5000 [list timeout $serverChannel]]

  proc timeout {serverChannel} {
       fileevent $serverChannel writable ""
       catch {close $serverChannel}
       puts "custom timeout"
  }

  proc socketWritable {serverChannel} {

       variable timer

       set error [fconfigure $serverChannel -error]

       switch $error {
              "connection timed out" -
              "connection refused"
                {
                   after cancel $timer
                   catch {close $serverChannel}
                }

		""
                 {
                   puts "all OK"
                   after cancel $timer
                 }
                default
                 {
                   puts $error
                 }

              }
  }

 proc socketReadable {serverChannel} {
      set error [fconfigure $serverChannel -error]
      if {$error == ""} {
         catch {gets $serverChannel}
         if {[eof $serverChannel]} {
            puts "the remote peer closed the connection"
            catch {close $serverChannel}
         }
      }
 }

 vwait forever
  

The socket command may throw an error-for example, if the specified target host domain name is invalid. In order to handle it properly, the catch command must be used.

Another important issue is timeouts. The network may be slow, or the target server may be unresponsive. If you do not want to rely on system defaults, there is no other way to specify your user-defined timeout value in Tcl.

However, a little trick with after may be used. In our example, we decided that we would like to terminate it 5 seconds after the initial attempt to connect-the timeout procedure was executed to do so.

First, we need to unregister ourselves from any new writable events and then close the channel. Note that the closing operation may also throw some errors (for example, the channel may already have been closed), so it is good practice to wrap it into catch.

The same applies to commands such as gets or puts. So that we can cancel the timer in case the connection is successful, we store it in the $timer variable.

Once the writable event occurs, the socketWritable procedure is called. First, the current error status of the $serverChannel socket is retrieved by calling fconfigure $serverChannel -error , and then it's stored in the $error variable.

f there is no error, an empty string is returned, and in the consecutive switch block, the all OK message is printed and the timeout timer is cancelled.

If the returned string is not empty, it means that some error occurred, for example:

  • Connection refused - the server refused the connection

  • Connection timed out - the system-defined timeout has occurred

In case of these errors, the timer is cancelled and the channel closed manually. For any other non-empty error messages, the script ( default section) will simply print out the error message, but of course, the appropriate clean-up actions can also be done here.

The other important situation is detecting that the remote peer has closed the connection. This case can only be handled in the procedure called for readable events, socketReadable in this example.

This procedure first checks for errors, just as socketWritable does. Following that, some input operation must be called (in this case, gets), because without it, the eof command would not notify that the EOF state occurred, as mentioned earlier.

Once it is detected, the channel is closed. Note that any concurrent puts $serverChannel operations will throw an error if the channel was closed, so this should also be handled properly with a catch.

The following table sums up the most common issues along with the ways of detecting them:

Table B.1. Alternatives for error detection and handling in socket

IssueDetection
Initial connecting issuessocket may throw errors (for example, when the specified port is already being used by another application)
writable event along with appropriate error information from fconfigure -error
Peer disconnectedDetectable only in the readable event handler using eof
Transmission errorsHandled transparently by the TCP protocol