Network Connection OO Design Pattern

[MyndFyre]

Okay...

I'm working on a set of network connection objects that handle incoming and outgoing data processing in a variety of ways that can be extended.  What I'm primarily concerned with is applying an arbitrary number of filters, such as compression / encryption algorithms, to a data stream.  I don't want this to be required, however, for connections such as a BNLS connection.

This is what I have so far, and I'd like your opinions.

ConnectionBase class
--Handles a single TCP/IP end-to-end connection in either direction.
+ OnDataSending( ) : void
    --Called when data is about to actually be sent -- that is, it was dequeued (if it was enqueued previously) or was immediately sent.  Upon method completion (callbacks called, etc.), this method actually passes the data onto the socket.  This method can be cancelled via a parameter.
+ OnDataSent( ) : void
    --This method is called back when the asynchronous "send" operation has completed.  It contains the data that was sent as a parameter.
+ OnDataReceiving( ) : void
    --This method is called back when the asynchronous "receive" operation has completed with data read from the network.  It contains the data that was received as a parameter.  This method can be cancelled via a parameter.
+ OnDataReceived( ) : void
    --This method is called with the received data buffer as a parameter.  Upon completion of this method, the data is processed.

FilteredConnection class : Derives from ConnectionBase
+ OnDataSending( ) : void
    --Notifies callbacks with the unfiltered data.  Applies filters to the data. Initiates the data transfer with filtered data.
+ OnDataSent( ) : void
    --Method is called with data on the socket callback.  Parameter's data is filtered.
+ OnDataReceiving( ) : void
    --Method is called with data on socket callback.  Parameter's data is filtered.
+ OnDataReceived( ) : void
    -- Data is unfiltered. Callbacks are notified with the unfiltered data.

Am I being redundant?  This design doesn't "feel" good.

[Adron]

Regarding OnDataSending - what if a filter wants to modify the data, but needs more information before it's able to decide what the output will be?

[MyndFyre]

Regarding OnDataSending - what if a filter wants to modify the data, but needs more information before it's able to decide what the output will be?

Like what?

I can think of a few filter scenarios:

1.) Data compression.
2.) Data encryption.
3.) Blocking of certain data from being transferred.

The first two are negotiated at the very beginning of the connection.  Generally, the connection negotiates version and then filter settings, and then applies filters to all communication that follows.

The third scenario is something like iago had in JavaOp -- I can optionally apply it to -- say -- a Battle.net connection, and it checks to see if my CD key or password is being sent, and then if it is, changing the bytes in the buffer to be sent.  If the filter is being applied to the connection, it will already have the data (such as CD key and password) available to it in the connection configuration object.

So I guess my question is -- what do you mean?

[Adron]

Example 1: You have a block mode cipher working with a block size of 8 bytes. You just received 7 bytes, but you need one more before you can get to work.

Example 2: You are filtering b.net packets. You just received four bytes, "ff 0f 21 00", but before you process them you need to look at the rest of the packet.

[MyndFyre]

For example 1: the header of the protocol I am designing specifies the length of data contained in the message.  The OnDataReceiving() method won't be called until all of the data comes in.  So, there is not necessarily a 1-to-1 correspondence between the actual callbacks and the OnDataReceiving() method calls.  But the callbacks are not visible to outside or descendent classes (they are private).

For example 2: well, I really can't think of an instance where I would be filtering incoming packets, necessarily.  But the same instance applies as in example 1.  Battle.net packets wouldn't be processed until the entire payload is received, and that is part of the BNCS message header.

[Arta]

The OnDataReceiving() method won't be called until all of the data comes in. 

That sounds bad. These kinds of classes should be generic, so that they are reusable. To wait for all the data to come in requires that you know when all the data is there, and that is protocol-specific.

This situation sounds like a job for the Chain of Responsibility pattern:

Avoid coupling the sender of a request to its receiver by giving more than one object a chance to handle
the request. Chain the receiving objects and pass the request along the chain until an object handles it.

Use Chain of Responsibility when:

- more than one object may handle a request, and the handler isn't known a priori. The handler
should be ascertained automatically.
- you want to issue a request to one of several objects without specifying the receiver explicitly.
- the set of objects that can handle a request should be specified dynamically.

You could have the handler as the last object in the chain. Objects that are responsible for decompressing or decrypting would have the opportunity to do that before the message reaches the handler. You could also write these classes such that objects in the chain can say "I need more data to be able to process this", in which case, the network IO class can buffer whatever has been received and send the old data along with any new data that arrives, until one of the objects in the chain reports that they received enough data to be able to process something. This pattern has excellent reusability/extensibility, because new functionality can often be created simply by adding an object to the chain.

I'm sure Google will yield more information on the pattern.

[Banana fanna fo fanna]

Uh.

See the Java streams API. It's what you want. In particular, InputStream, OutputStream, and take a look at some FilterInput/OutputStreams.

[MyndFyre]

Uh.

See the Java streams API. It's what you want. In particular, InputStream, OutputStream, and take a look at some FilterInput/OutputStreams.

I'm not using Java. 

Arta: Thanks for the info.  I'll look that up.  I've not heard of the design pattern you're talking about, but it sounds interesting.

[Adron]

the network IO class can buffer whatever has been received and send the old data along with any new data that arrives, until one of the objects in the chain reports that they received enough data to be able to process something.

From this, it sounds like if one object in the chain doesn't have enough data to process, the data should be passed on to other objects to see if it's enough data for them. That's probably not really what you want.

[Arta]

Perhaps not - maybe a "queue of responsibility" would be better

[Adron]

Perhaps not - maybe a "queue of responsibility" would be better

Especially if object 1 decrypts the data and object 2 processes it

[MyndFyre]

So, it seems I should decouple the connection handling code from any type of data processing, and have prioritized handling objects?

In your opinion, should the handling objects specify their own default or suggested priority, or demand a priority, or be user-configurable?

[MyndFyre]

Okay, I've generated a generic schematic of how a chain of responsibility would work in my situation (forgive me for the yucky Word-illustrated pictures):

Incoming:


Outgoing:


Doing so has raised a few questions in my mind...

1.) In my mind, a "Need More Data" request in the incoming chain makes sense, since you can treat it as a pull model.  But in the past, I've spawned new threads to parse data; for example, my incoming request would do something like this:
--Data Received callback function (thread A)
---Store all data in a buffer.
---Start async receive method
---Figure out how long the data was supposed to be.
---If it is long enough, start Thread B to process it.
---If it is not long enough, store the buffer and wait to get the rest of the data.

Now, however, it seems I can't start thread B because Thread A might need to give Thread B more data.  Is this accurate?  Note that in the past, my connection objects were aware of the protocols they implemented.

2.) Also, the "Need more data" idea seems to not make sense in a push-type function such as when the data being sent.  It doesn't really make sense that the RSP service would tell my client's BNCS handler "I need more data to be able to send to the server," because the BNCS client is actually pushing the data to the server.

The idea of passing on to other objects is fine, except that for certain filters such as compression and encryption, I believe that the order in which they are applied is important.  The server and client negotiate these ahead of time, and they are applied in the order in which they are negotiated.

[UserLoser.]

RSP? .. RBP?

[MyndFyre]

RSP? .. RBP?

RSP is an RBP-like protocol that does somewhat more.  It is preliminarily documented here and when I finish the documentation I was going to be asking for comments here.