Drawbacks of handling packets asynchronously?

[iago]

I'm just trying to think if there are any drawbacks to handling battle.net packets asynchronously.  Here is what I mean:

Code Select Expand

Packet 1 - Recieved
Buffer contents:
ff 0a 25 00 69 61 67 6f 00 52 41 54 53 20 30 20         ..%.iago.RATS 0
30 20 30 20 30 20 30 20 30 20 30 20 30 20 30 00         0 0 0 0 0 0 0 0.
69 61 67 6f 00                                          iago.
Length: 37
Packet 1 - Unknown packet recieved
Packet 2 - Recieved
Buffer contents:
ff 0f 3e 00 12 00 00 00 10 00 00 00 8c 00 00 00         ..>.............
00 00 00 00 0d f0 ad ba 0d f0 ad ba 69 61 67 6f         ............iago
00 54 68 69 73 20 69 73 20 61 20 74 65 6d 70 6f         .This is a tempo
72 61 72 79 20 6d 65 73 73 61 67 65 3a 00               rary message:.
Length: 62
Packet 3 - Recieved
Buffer contents:
ff 0f 87 00 12 00 00 00 10 00 00 00 8c 00 00 00         ................
00 00 00 00 0d f0 ad ba 0d f0 ad ba 69 61 67 6f         ............iago
00 59 6f 75 72 20 63 6f 6e 6e 65 63 74 69 6f 6e         .Your connection
20 61 64 64 72 65 73 73 20 69 73 20 32 30 37 2e          address is 207.
31 36 31 2e 31 34 34 2e 31 36 32 3a 34 34 39 32         161.144.162:4492
38 2e 20 50 4c 45 41 53 45 20 49 4e 43 4c 55 44         8. PLEASE INCLUD
45 20 54 48 49 53 20 49 4e 20 41 4e 59 20 42 55         E THIS IN ANY BU
47 20 52 45 50 4f 52 54 2e 20 4d 61 6e 79 20 74         G REPORT. Many t
68 61 6e 6b 73 2e 00                                    hanks..
Length: 135
Packet 2 - Unknown packet recieved
Packet 3 - Unknown packet recieved


For each packet that comes in, I create a process thread and let it go.  Since I'm using Java, synchronization isn't difficult and overhead isn't really a big concern (since Java already has plenty of overhead) so I don't really care about those.  Also, the length is used so each one is always handled entirely, I won't get parts shuffled.

Can anybody suggest any reason not to handle it like this, though?  The only thing I can think of is that chat messages may not display in the correct order ever time; however, if two messages are sent closely enough that this occurs anyway, it's really not that big of a problem.

One thing I like it for is SID_PING, though, because it will instantly handle it, even while if it's processing SID_AUTH_INFO and SID_AUTH_CHECK.

Anybody else have any ideas for potential problems?

[Eli_1]

I'm just trying to think if there are any drawbacks to handling battle.net packets asynchronously.  Here is what I mean:

Code Select Expand

Packet 1 - Recieved
Buffer contents:
ff 0a 25 00 69 61 67 6f 00 52 41 54 53 20 30 20         ..%.iago.RATS 0
30 20 30 20 30 20 30 20 30 20 30 20 30 20 30 00         0 0 0 0 0 0 0 0.
69 61 67 6f 00                                          iago.
Length: 37
Packet 1 - Unknown packet recieved
Packet 2 - Recieved
Buffer contents:
ff 0f 3e 00 12 00 00 00 10 00 00 00 8c 00 00 00         ..>.............
00 00 00 00 0d f0 ad ba 0d f0 ad ba 69 61 67 6f         ............iago
00 54 68 69 73 20 69 73 20 61 20 74 65 6d 70 6f         .This is a tempo
72 61 72 79 20 6d 65 73 73 61 67 65 3a 00               rary message:.
Length: 62
Packet 3 - Recieved
Buffer contents:
ff 0f 87 00 12 00 00 00 10 00 00 00 8c 00 00 00         ................
00 00 00 00 0d f0 ad ba 0d f0 ad ba 69 61 67 6f         ............iago
00 59 6f 75 72 20 63 6f 6e 6e 65 63 74 69 6f 6e         .Your connection
20 61 64 64 72 65 73 73 20 69 73 20 32 30 37 2e          address is 207.
31 36 31 2e 31 34 34 2e 31 36 32 3a 34 34 39 32         161.144.162:4492
38 2e 20 50 4c 45 41 53 45 20 49 4e 43 4c 55 44         8. PLEASE INCLUD
45 20 54 48 49 53 20 49 4e 20 41 4e 59 20 42 55         E THIS IN ANY BU
47 20 52 45 50 4f 52 54 2e 20 4d 61 6e 79 20 74         G REPORT. Many t
68 61 6e 6b 73 2e 00                                    hanks..
Length: 135
Packet 2 - Unknown packet recieved
Packet 3 - Unknown packet recieved


For each packet that comes in, I create a process thread and let it go.  Since I'm using Java, synchronization isn't difficult and overhead isn't really a big concern (since Java already has plenty of overhead) so I don't really care about those.  Also, the length is used so each one is always handled entirely, I won't get parts shuffled.

Can anybody suggest any reason not to handle it like this, though?  The only thing I can think of is that chat messages may not display in the correct order ever time; however, if two messages are sent closely enough that this occurs anyway, it's really not that big of a problem.

One thing I like it for is SID_PING, though, because it will instantly handle it, even while if it's processing SID_AUTH_INFO and SID_AUTH_CHECK.

Anybody else have any ideas for potential problems?

I really like that idea...

[iago]

You didn't have to quote that huge block, but thanks

I modelled it after how Starcraft handles game packets, except that I dispatch a new thread rather than wait for a processing thread to pick it up (or alerting it, anyway).

So, I guess it's not really modelled after Starcraft at all.  Oh well

[Eli_1]

You didn't have to quote that huge block, but thanks

I'v takin up the habit of quoting the authors post because in most cases it ends up getting deleted/modified way to much...

[iago]

You didn't have to quote that huge block, but thanks

I'v takin up the habit of quoting the authors post because in most cases it ends up getting deleted/modified way to much...

That's fine, I would recommend replace the code tags with <CODE BLOCK> or something just to save me scrolling.

Anyway, back to the original point.  

[Eli_1]

before we get back to the origional point...

Code Select Expand


'Using the CreateThread function in Visual Basic
'is very risky! VB5 is 'kinda' stable, but VB6
'applications will probably crash when you
'use the CreateThread function.


those comments were taken from a CreateThread() example in VB6...
what's that all about?

[iago]

before we get back to the origional point...

<comments>

those comments were taken from a CreateThread() example in VB6...
what's that all about?

The problem with using threads is that the same variable can be modified by your program in more than one place.  If your program isn't "thread safe" (ie, the variables aren't protected from this), it'll corrupt and destroy the variables.

Visual Basic probably isn't thread safe.

[MyndFyre]

You didn't have to quote that huge block, but thanks

I modelled it after how Starcraft handles game packets, except that I dispatch a new thread rather than wait for a processing thread to pick it up (or alerting it, anyway).

So, I guess it's not really modelled after Starcraft at all.  Oh well

As a matter of fact, that is almost exactly how I process incoming packets.  Here's how mine works (C# is almost identical to Java, so it's probably similar anyway):

1.) I create a thread-safe ArrayList
2.) I create a Timer that fires every 10 ms.
3.) Incoming packets are stuck into the thread-safe ArrayList as they come in.  If an incoming packet is longer than the BNCS packet header says (the 2nd word), I split at the length of the packet before adding it to the ArrayList.  If the BNCS packet header is invalid (doesn't begin with 0xff), it is ignored but still added to the ArrayList.
4.) The Timer fires its event, which causes the first packet to come out of the ArrayList (arrayList.RemoveAt(0)).  If the packet is shorter than the BNCS headers say it should be, then the ArrayList is looked through for a packet without a valid header (which will occur in one of the first few).  If it finds one, it combines them.

The big caveat here is the .NET event handler model -- it handles threading automatically, and I'm not sure how Java's threading model works.  Each event gets its own thread spawned.

As you pointed out, the order of chat events isn't really crucial anyway, but I've never seen any extremely odd behavior with it.

Cheers.

[iago]

that's actually very dissimilar to mine.  When a packet is recieved, it creates a new instance of Processor, then I can Processor.start() which creates a new instance of the thread and returns immediately(ish).  When the Processor is finished, the thread returns (ie, is over)/

[MyndFyre]

that's actually very dissimilar to mine.  When a packet is recieved, it creates a new instance of Processor, then I can Processor.start() which creates a new instance of the thread and returns immediately(ish).  When the Processor is finished, the thread returns (ie, is over)/

Your Processor class handles the packet parsing, correct?  It is simply another path of execution for your program....  A funciton callback and a function called start() that serve the same purpose are relatively close

[Dark-Feanor]

What I do, is I have a thread that is always checking for incomming data. Once it finds some, it saves it as a big byte array, terminated by a whole lot of nulls. Then I pass it through a function that shaves all of the null bytes off the end of the array and sends it to be parsed.
I dont bother dealing with parsing the packet length.
The only drawback from using this method is that a large byte array can take up a good deal of RAM. I doubt that it will be any problem for the computer that you are using.

[Adron]

What I do, is I have a thread that is always checking for incomming data. Once it finds some, it saves it as a big byte array, terminated by a whole lot of nulls. Then I pass it through a function that shaves all of the null bytes off the end of the array and sends it to be parsed.
I dont bother dealing with parsing the packet length.
The only drawback from using this method is that a large byte array can take up a good deal of RAM. I doubt that it will be any problem for the computer that you are using.

This sounds like it won't work if you happen to get a packet that is split, and where the last bytes of the first part are 00's.

[iago]

What I do, is I have a thread that is always checking for incomming data. Once it finds some, it saves it as a big byte array, terminated by a whole lot of nulls. Then I pass it through a function that shaves all of the null bytes off the end of the array and sends it to be parsed.
I dont bother dealing with parsing the packet length.
The only drawback from using this method is that a large byte array can take up a good deal of RAM. I doubt that it will be any problem for the computer that you are using.

This sounds like it won't work if you happen to get a packet that is split, and where the last bytes of the first part are 00's.

Yes, especially when I first log on I frequently find that the channel lists are split or stacked.

Also, the way I deal with that is using blocking input in my recv thread that doesn't get through until enough bytes are recieved.

[Dark-Feanor]

This sounds like it won't work if you happen to get a packet that is split, and where the last bytes of the first part are 00's.

You could always split the packets by every FF byte and send them to be parsed seperately. Additionally, a packet terminating null byte will never be significant, at least in any battle.net packet that I have seen.

[Kp]

You could always split the packets by every FF byte and send them to be parsed seperately.

IIRC, NullBot did this.  As such, you could confuse it rather badly by sending a 0xff embedded in a talk event, causing it to believe it had encountered the next packet in the sequence.