Valhalla Legends Archive

I am having lots of trouble with this packet because whenever I recieve it I get a big cluster of them all in one packet....I am usally getting about 5-7 packets shuved into one. Does anyone had this problem or know of a solution?

Any Help would be gladly appreciated...

Edit:

You can also reach me on aim, my name is KKBlazekK

Quote from: Kk)Blaze(kK on August 11, 2004, 11:01 PM
I am having lots of trouble with this packet because whenever I recieve it I get a big cluster of them all in one packet....I am usally getting about 5-7 packets shuved into one. Does anyone had this problem or know of a solution?

Any Help would be gladly appreciated...

Edit:

You can also reach me on aim, my name is KKBlazekK

I would suggest separating the individual packets as you receive them.

I have tried that...

Didn't work so well....

I tried something else, parsing the first packet then removing it then doing the next, but that don't work either...

heres the script...



       ParsingData = Data
ParsingPoint:
   ptemp = StrToHex(ParsingData)
   AddChat ptemp, vbRed
   ParsingLength = Mid(ptemp, 8, 2)
   ParsingLength = HexToDec(ParsingLength)
   
   Flags = 0: Ping = 0: Message = vbNullString: Product = vbNullString:
   
   
       Flags = MakeLong(Mid(ParsingData, 9, 4))
       Ping = MakeLong(Mid(ParsingData, 13, 4))
       Username = KillNull(Mid(ParsingData, 29))
       Message = KillNull(Mid(ParsingData, Len(Username) + 30))
       Product = StrReverse(Left(Message, 4))
                   Call OnUser(Username, Flags, Message, Ping, Product)
       ParsingData = Mid(ParsingData, ParsingLength, Len(ParsingData) - ParsingLength)
       
   If Len(ParsingData) > 4 Then
   GoTo ParsingPoint
   Else
   Exit Function
   End If


Quote from: Kk)Blaze(kK on August 11, 2004, 11:01 PM
I am having lots of trouble with this packet because whenever I recieve it I get a big cluster of them all in one packet....

If I understand correctly, it doesn't sound that hard at all. The packets have a header that tells you their length for a reason - use it.

Of course you could do what i do which is:-

(1) Do a recv on your socket using a length of HEADER_SIZE which is 0x04 on a bnchat connection.

(2) Copy the header buffer into a fixed length structure for easy access.

(3) The next recv event use a length of the size field in the header structure you read previously.

This works very well for me on bnchat and also saves you having to read a massive chunk of data then having to split it up with the possibility of incomplete data.

Of course this method assumes 2 things:-

(1) Your underlying socket paradigm allows you to read a data fragment then if there is still data left on the network stack raises additional events until that entire segment is read.

(2) The packet buffer you are using has sufficient space to accomodate a complete event message, which is not a problem with bnchat as the events are very small.

You people took no look at my script did you? My script gets the length from the header. It works untill the thrid packet then it screws up...

Quote from: OnlyMeat on August 12, 2004, 07:46 AM
(3) The next recv event use a length of the size field in the header structure you read previously.

This works very well for me on bnchat and also saves you having to read a massive chunk of data then having to split it up with the possibility of incomplete data.

You still have the possibility of incomplete data. You have to handle that in (3)...

Kind've nasty to look at. Let me clean it up a little:

 ParsingData = Data
ParsingPoint:
 ptemp = StrToHex(ParsingData)
 AddChat ptemp, vbRed
 ParsingLength = Mid(ptemp, 8, 2)
 ParsingLength = HexToDec(ParsingLength)
 
 Flags = 0: Ping = 0: Message = vbNullString: Product = vbNullString:
 
 'assuming you're in the event parsing, here?
 'if not, this could be screwing you up?
 Flags = MakeLong(Mid(ParsingData, 9, 4))
 Ping = MakeLong(Mid(ParsingData, 13, 4))
 Username = KillNull(Mid(ParsingData, 29))
 Message = KillNull(Mid(ParsingData, Len(Username) + 30))
 Product = StrReverse(Left(Message, 4))
 Call OnUser(Username, Flags, Message, Ping, Product)
 ParsingData = Mid(ParsingData, ParsingLength, Len(ParsingData) - ParsingLength)
     
 If Len(ParsingData) > 4 Then
   GoTo ParsingPoint
 Else
   Exit Function
 End IfWhy are you reading the packet header from the hexadecimal string? That's not very easy to work with. Going from one source, to another, and back... that's confusing.

Also, what are you doing in case you don't receive a full complete packet? It looks like you're able to break out if you don't have at least a header, but it doesn't look like you're checking that the whole packet is there.

Try this code:
 ParsingData = Data
ParsingPoint:
 ptemp = StrToHex(ParsingData)
 AddChat ptemp, vbRed
 ParsingLength = Mid(ParsingData, 3, 2)'length byte starts at the third byte
 'ahh, i see now why you were converting to hexadecimal
 'you need to learn how to convert from the raw string directly into the integer, not using middle-man hexadecimal
 If ParsingLength > Len(ParsingData) Then
   'incomplete packet
   Exit Function
 End If
 
 Flags = 0: Ping = 0: Message = vbNullString: Product = vbNullString:
 
 'assuming you're in the event parsing, here?
 'if not, this could be screwing you up?
 Flags = MakeLong(Mid(ParsingData, 9, 4))
 Ping = MakeLong(Mid(ParsingData, 13, 4))
 Username = KillNull(Mid(ParsingData, 29))
 Message = KillNull(Mid(ParsingData, Len(Username) + 30))
 Product = StrReverse(Left(Message, 4))
 Call OnUser(Username, Flags, Message, Ping, Product)
 ParsingData = Mid(ParsingData, ParsingLength, Len(ParsingData) - ParsingLength)
     
 If Len(ParsingData) >= 4 Then
   'need greaterthan or equalto, because null packet is only 4 bytes long
   GoTo ParsingPoint
 Else
   Exit Function
 End If

Quote from: Adron on August 12, 2004, 12:45 PM

Quote from: OnlyMeat on August 12, 2004, 07:46 AM
(3) The next recv event use a length of the size field in the header structure you read previously.

This works very well for me on bnchat and also saves you having to read a massive chunk of data then having to split it up with the possibility of incomplete data.

You still have the possibility of incomplete data. You have to handle that in (3)...

There is no possibility of incomplete data my socket classes just wait on the next recv after (2) and compares the length to the header length to ensure it's complete if not it just waits for the next data fragment until the header size = the total packet size accumulated.

Try it out it;s an impossibility to break my logic

I have used this system for many years without ever running into incomplete data, comparing this to just reading a chunk and parsing it mine is 99% accurate.

Quote from: OnlyMeat on August 12, 2004, 03:17 PM
There is no possibility of incomplete data my socket classes just wait on the next recv after (2) and compares the length to the header length to ensure it's complete if not it just waits for the next data fragment until the header size = the total packet size accumulated.

Then you are handling it in (3) just like I said you had to. If you're going to recommend this solution to others, you better make it clear that they have to handle calling recv multiple times to get all the data.

You're also handling the case of incomplete headers?


It's still not a very efficient solution, not as good as receiving a large buffer and parsing it out yourself.

Quote from: Adron on August 12, 2004, 03:44 PM

Quote from: OnlyMeat on August 12, 2004, 03:17 PM
There is no possibility of incomplete data my socket classes just wait on the next recv after (2) and compares the length to the header length to ensure it's complete if not it just waits for the next data fragment until the header size = the total packet size accumulated.

Then you are handling it in (3) just like I said you had to. If you're going to recommend this solution to others, you better make it clear that they have to handle calling recv multiple times to get all the data.

You're also handling the case of incomplete headers?


It's still not a very efficient solution, not as good as receiving a large buffer and parsing it out yourself.

First response i am just providing a basic framework which does not cover all eventualities otherwise i would be here all day, also im not going to write someones code for them, basic checks and such and standard programming practise, also if he has any specific questions i would be only to pleased to provide more information with a pm or response here.

If you feel the need to elaborate in more detail or provide some source code for him be my guest otherwise then i suggest you allow him to work the finer details for himself.

Second point yes i make all basic checks to do with lengths//structures i read the first 4 bytes for bnchat and all subsequent reads i then first check the header is intact, if thats passes my requirements then i continue and check the data length received against the actually size defined in the header, and if those both succeed then i dispatch the packet

Just as a note the complete message is guarenteed delivery as this is tcp/ip i.e send/acknowlege protocol if the entrire packet sent from bnet is not completley received then it will not be passed on from the network subsystem ^^

Considering the sizes of normal event messages in chat im assuming bnet would send an entire message at once rather splitting it up which would mean guarenteed delivery, or at the very least it would send a complete header, and my logic handles both of these conditions with ease.

Ps. Maybe it would be more helpful if you make a contribution to the original question rather than just critising what i say.

Also forgot last point it;s actually less efficient doing it that way because you have to allocate additional resources to store the buffer and cycle time for parsing etc.

Quote from: OnlyMeat on August 12, 2004, 04:12 PM

Quote from: Adron on August 12, 2004, 03:44 PM

Quote from: OnlyMeat on August 12, 2004, 03:17 PM
There is no possibility of incomplete data my socket classes just wait on the next recv after (2) and compares the length to the header length to ensure it's complete if not it just waits for the next data fragment until the header size = the total packet size accumulated.

Then you are handling it in (3) just like I said you had to. If you're going to recommend this solution to others, you better make it clear that they have to handle calling recv multiple times to get all the data.

You're also handling the case of incomplete headers?


It's still not a very efficient solution, not as good as receiving a large buffer and parsing it out yourself.

First response i am just providing a basic framework which does not cover all eventualities otherwise i would be here all day, also im not going to write someones code for them, basic checks and such and standard programming practise, also if he has any specific questions i would be only to pleased to provide more information with a pm or response here.

If you feel the need to elaborate in more detail or provide some source code for him be my guest otherwise then i suggest you allow him to work the finer details for himself.

Second point yes i make all basic checks to do with lengths//structures i read the first 4 bytes for bnchat and all subsequent reads i then first check the header is intact, if thats passes my requirements then i continue and check the data length received against the actually size defined in the header, and if those both succeed then i dispatch the packet

Just as a note the complete message is guarenteed delivery as this is tcp/ip i.e send/acknowlege protocol if the entrire packet sent from bnet is not completley received then it will not be passed on from the network subsystem ^^

Considering the sizes of normal event messages in chat im assuming bnet would send an entire message at once rather splitting it up which would mean guarenteed delivery, or at the very least it would send a complete header, and my logic handles both of these conditions with ease.

Ps. Maybe it would be more helpful if you make a contribution to the original question rather than just critising what i say.

Also forgot last point it;s actually less efficient doing it that way because you have to allocate additional resources to store the buffer and cycle time for parsing etc.

The time it takes to parse incoming data is negligible compared to a blocking call to recv.

Quote from: Eibro[yL] on August 12, 2004, 04:22 PM
The time it takes to parse incoming data is negligible compared to a blocking call to recv.

If you are using event based async sockets like i am when data becomes available the processing thread is awakened from an efficient wait state.

The logic i am using means the data is already available on the network stack so the blocking is only for a very small inmeasureable time, it simply rewakens my thread for the next data fragment i think maybe you are confusing this situation with non asyc blocking sockets.

The system i use there is no delay no wasted processor cycles doing superflous parsing just efficient kernel based waitstates.

Also we were not talking about *times* we were talking about efficiency in terms of resources etc.

Quote from: OnlyMeat on August 12, 2004, 04:12 PM
First response i am just providing a basic framework which does not cover all eventualities otherwise i would be here all day, also im not going to write someones code for them, basic checks and such and standard programming practise, also if he has any specific questions i would be only to pleased to provide more information with a pm or response here.

If you feel the need to elaborate in more detail or provide some source code for him be my guest otherwise then i suggest you allow him to work the finer details for himself.

If you're going to tell him about your solution, don't make it look easier than it is. If you do, all that is going to happen is he implements the simple solution you described, and it doesn't work. Considering that this whole question was about handling the situation when b.net stuffs together packets and you have to parse out the right pieces, giving him an answer that doesn't correctly split up packets is bad. Giving a wrong answer is worse than giving no answer at all you know...

And no, I don't feel like providing source code, but I do feel like pointing out the things that are important for him to think of when writing his own source code.

Quote from: OnlyMeat on August 12, 2004, 04:12 PM
Just as a note the complete message is guarenteed delivery as this is tcp/ip i.e send/acknowlege protocol if the entrire packet sent from bnet is not completley received then it will not be passed on from the network subsystem ^^

That's not correct. It will be passed on in pieces from the network subsystem. Pieces will not be passed on in the wrong order, and if you read data, you'll eventually get all the pieces, but there's no guaranteed entire packet at once delivery in tcp/ip.

Quote from: OnlyMeat on August 12, 2004, 04:12 PM
Considering the sizes of normal event messages in chat im assuming bnet would send an entire message at once rather splitting it up which would mean guarenteed delivery, or at the very least it would send a complete header, and my logic handles both of these conditions with ease.

As your previous point was incorrect, even though bnet might send an entire message at once, or a complete header, you may not receive a whole header at once from winsock.

Quote from: OnlyMeat on August 12, 2004, 04:12 PM
Ps. Maybe it would be more helpful if you make a contribution to the original question rather than just critising what i say.

I like to let people think for themselves. I point out the problems and what they have to think about, and then I let them solve the problems. If they get stuck, I hint more. I don't like to write people's bots for them.

Quote from: OnlyMeat on August 12, 2004, 04:12 PM
Also forgot last point it;s actually less efficient doing it that way because you have to allocate additional resources to store the buffer and cycle time for parsing etc.

It's actually more efficient doing it that way, because there will be fewer API calls. It's true that you'll need additional resources to store the buffer, but if you don't provide the buffer, winsock will have to buffer the data itself anyway, so it's just moving the same amount of resources to a different place. About the cycle time for parsing, I really don't see what you mean. You're parsing the packet and the length either way.

Quote from: Adron on August 12, 2004, 09:01 PM
If you're going to tell him about your solution, don't make it look easier than it is. If you do, all that is going to happen is he implements the simple solution you described, and it doesn't work. Considering that this whole question was about handling the situation when b.net stuffs together packets and you have to parse out the right pieces, giving him an answer that doesn't correctly split up packets is bad. Giving a wrong answer is worse than giving no answer at all you know...

And who exactly said it was a solution? did you even read my post?, i said a basic framework not a solution i suggest you read the original post before replying next time.

A basic framework in no way contitutes a complete solution and i never stated that.

I imagine if every first reply to a thread on this forum provided complete solutions first time there would'nt be much activity no?.

I was attempting to try and guide that is all plain and simple i never intended to give a complete solution just an idea of what i try and do at a basic level.

Instead of bashing people who are trying to help maybe you should counsider contributing first?.

And once again as i said in my previous posts ( which you must have failed to read by what you have just written ) that if he needs any further assistance i would be pleased to help, so what is exactly wrong with trying to help?, what would you suggest? maybe go away for 3 hours write a complete soution then post//send to him? lol

Quote from: Adron on August 12, 2004, 09:01 PM
That's not correct. It will be passed on in pieces from the network subsystem. Pieces will not be passed on in the wrong order, and if you read data, you'll eventually get all the pieces, but there's no guaranteed entire packet at once delivery in tcp/ip.

Yes once again did you read what i said ?

Quote from: OnlyMeat on August 12, 2004, 04:12 PM
if the entrire packet sent from bnet is not completley received then it will not be passed on from the network subsystem ^^

Which means yes the network layer can transport the data in fragments but if not all the fragments that comprise a transaction are received by the destination tcp/ip network then the subsystem will abort that transaction.

So if all fragments are received then the subsystem will generate the required system events which the application layer can handle, if not then that transaction is aborted!!.

Thats why tcp/ip has guarenteed delivery!, if This did not happen then you could send your data packet and you could not say for certain if the destination received it. That uncertainty does not exist for stream sockets, UDP does have that behaviour though.

Quote from: Adron on August 12, 2004, 09:01 PM
As your previous point was incorrect, even though bnet might send an entire message at once, or a complete header, you may not receive a whole header at once from winsock.

Although true in theory yes a header could come in fragments my own code does handle this condition as i discribed, i simply continuing reading until both my header structure and data body are both complete.

But the likleyhood of receiving say 1/2 bytes and not being able to receive the rest of the 4 byte header in 1 network cycle is extremely low i would say probably in the 0.01% mark if not less.

Quote from: Adron on August 12, 2004, 09:01 PM
I like to let people think for themselves. I point out the problems and what they have to think about, and then I let them solve the problems. If they get stuck, I hint more. I don't like to write people's bots for them.

But you are not trying to point out things for him instead you are trying to pick holes in my comments very pedantically.

Quote from: Adron on August 12, 2004, 09:01 PM
It's actually more efficient doing it that way, because there will be fewer API calls. It's true that you'll need additional resources to store the buffer, but if you don't provide the buffer, winsock will have to buffer the data itself anyway, so it's just moving the same amount of resources to a different place. About the cycle time for parsing, I really don't see what you mean. You're parsing the packet and the length either way.

And you dont call api's when parsing ( and i mean the term api can apply to not only OS based functions but also C runtime functions ) for example strlen() ?.

Your may now say that system API's require more resources etc well one call to recv ( which uses code allready active in memory and an efficient low level API which would simply copy one buffer to another seems very low in cpu cycles because in the end if data is already on the network stack thats all it does copy the specified length of the network buffer to you local buffer which for single event messages is very small.

Also note ws32 dll is loaded into the address space of the application just the same way as the c runtime therefore the performance hit will be just the same if not less.

Compared with having to loop through a multi-packet clump parsing and extracting the relevent data.

I can bet you now my method requires less cpu cycles and leaves the system free to do other things, try it out if you dont believe me.