Valhalla Legends Archive

Ahem.


messageid = 0
messagelen = -1
buffer = new ByteBuffer()
while (socket.connected) {
   buffer.append(socket.readdata())
   if messagelen == -1 {
       if buffer.position >= 4 {
           hdr = buffer.read(4)
           messageid = hdr[1]
           messagelen = bytes2word(hdr[2],hdr[3])
       }
   }
   if messagelen != -1 && buffer.position >= messagelen {
       handle_message(messageid, buffer.read(messagelen))
       messageid = -1
   }
}


That should work, AFAIK.

Quote from: $t0rm on August 13, 2004, 10:49 AM
Ahem.

Code Select Expand


messageid = 0
messagelen = -1
buffer = new ByteBuffer()
while (socket.connected) {
   buffer.append(socket.readdata())
   if messagelen == -1 {
       if buffer.position >= 4 {
           hdr = buffer.read(4)
           messageid = hdr[1]
           messagelen = bytes2word(hdr[2],hdr[3])
       }
   }
   if messagelen != -1 && buffer.position >= messagelen {
       handle_message(messageid, buffer.read(messagelen))
       messageid = -1
   }
}


That should work, AFAIK.

Ye that is the same kinda logic i was talking about, it's quite efficient doing it that way, although you would have to implement that code a in separate thread and use async event based sockets to get the most out of it .

Actually thinking about it that is flawed in quite a few ways, what happends if you read in multiple events do you then have to parse them, if so that is completely Unnecessary.

When you do a read you can just set the number of bytes to read as the message length then you dont read multiple events in one cycle.

Getting the data into your own buffer is better than letting it stagnate in the TCP buffer.

Quote from: $t0rm on August 13, 2004, 05:01 PM
Getting the data into your own buffer is better than letting it stagnate in the TCP buffer.

Eh? stagnate meaning what exactly?, it will just raise additional events within a very small amount of time how is that stagnating?

The kernel is generating these events we are talking about which has high system priority i hardly see how it's stagnating please elborate in detail.

Why let it waste your kernel's memory, when you can have it waste your own memory?

Some kernels leave data directly on the network card. If you leave your data there long enough, it may be overwritten. At that point, youeither have to have the remote host resend the data (which might not even be possible if your kernel has already acknowledged to the remote host that it's received the packets for the data being overwritten), or close the connection since the data is lost and thus the guaranteed reliability of TCP is broken.

Also, even if your kernel does copy the data to its own location (off the network card), it's better to not waste the kernal's memory.

Quote from: OnlyMeat on August 12, 2004, 10:31 PM

Quote from: Adron on August 12, 2004, 09:01 PM
If you're going to tell him about your solution, don't make it look easier than it is. If you do, all that is going to happen is he implements the simple solution you described, and it doesn't work. Considering that this whole question was about handling the situation when b.net stuffs together packets and you have to parse out the right pieces, giving him an answer that doesn't correctly split up packets is bad. Giving a wrong answer is worse than giving no answer at all you know...

And who exactly said it was a solution? did you even read my post?, i said a basic framework not a solution i suggest you read the original post before replying next time.

Indeed, let's read the original post:

Quote from: OnlyMeat on August 12, 2004, 07:46 AM
Of course you could do what i do which is:-

(1) Do a recv on your socket using a length of HEADER_SIZE which is 0x04 on a bnchat connection.

(2) Copy the header buffer into a fixed length structure for easy access.

(3) The next recv event use a length of the size field in the header structure you read previously.

This works very well for me on bnchat and also saves you having to read a massive chunk of data then having to split it up with the possibility of incomplete data.

Of course this method assumes 2 things:-

(1) Your underlying socket paradigm allows you to read a data fragment then if there is still data left on the network stack raises additional events until that entire segment is read.

(2) The packet buffer you are using has sufficient space to accomodate a complete event message, which is not a problem with bnchat as the events are very small.

I can't see any other way to interpret this than that it is a step by step explanation of what he needs to do to have a framework for separating out packets correctly from the data received from b.net. I see nothing indicating that it's just intended as a "basic framework" (which I would still require to be correct instead of faulty) and no disclaimer saying "this is not a solution".

Quote from: OnlyMeat on August 12, 2004, 10:31 PM
A basic framework in no way contitutes a complete solution and i never stated that.

I was attempting to try and guide that is all plain and simple i never intended to give a complete solution just an idea of what i try and do at a basic level.

As I said above, there is no indication in your post that this is just food for thought and not a valid way of doing it.

Quote from: OnlyMeat on August 12, 2004, 10:31 PM
I imagine if every first reply to a thread on this forum provided complete solutions first time there would'nt be much activity no?.

Activity... Yes, I suppose posting non-working solutions on the forum is one way of generating activity.

Quote from: OnlyMeat on August 12, 2004, 10:31 PM
Instead of bashing people who are trying to help maybe you should counsider contributing first?.

I'm contributing by bashing out the people who spread lies and flawed solutions. Weeding out a bit.

Quote from: OnlyMeat on August 12, 2004, 10:31 PM
And once again as i said in my previous posts ( which you must have failed to read by what you have just written ) that if he needs any further assistance i would be pleased to help, so what is exactly wrong with trying to help?, what would you suggest? maybe go away for 3 hours write a complete soution then post//send to him? lol

I'd suggest always pointing out when you post incomplete or flawed solutions, so that it's clear that it's not supposed to work. It's really annoying to get a solution from someone, assume it has been tested and is going to work, go on to implement it, find that it doesn't work, then look at your code over and over for hours, trying to see where the difference is between what that guy said you were supposed to do and what your code does, before you give up and realize that that someone was just messing with you, making you waste your time.

Quote from: OnlyMeat on August 12, 2004, 10:31 PM

Quote from: Adron on August 12, 2004, 09:01 PM
That's not correct. It will be passed on in pieces from the network subsystem. Pieces will not be passed on in the wrong order, and if you read data, you'll eventually get all the pieces, but there's no guaranteed entire packet at once delivery in tcp/ip.

Yes once again did you read what i said ?

Quote from: OnlyMeat on August 12, 2004, 04:12 PM
if the entrire packet sent from bnet is not completley received then it will not be passed on from the network subsystem ^^

Which means yes the network layer can transport the data in fragments but if not all the fragments that comprise a transaction are received by the destination tcp/ip network then the subsystem will abort that transaction.

So if all fragments are received then the subsystem will generate the required system events which the application layer can handle, if not then that transaction is aborted!!.

Thats why tcp/ip has guarenteed delivery!, if This did not happen then you could send your data packet and you could not say for certain if the destination received it. That uncertainty does not exist for stream sockets, UDP does have that behaviour though.

The transaction isn't aborted if all fragments aren't received at a certain time. It's delayed, waiting for the rest of the fragments. After some timeout I'm assuming the connection will be reset.

You said: "if the entrire packet sent from bnet is not completley received then it will not be passed on from the network subsystem". I say: Even if only the first two bytes of the packet sent from bnet have been received, the network subsystem will pass those two bytes on to the application.

Quote from: OnlyMeat on August 12, 2004, 10:31 PM
But the likleyhood of receiving say 1/2 bytes and not being able to receive the rest of the 4 byte header in 1 network cycle is extremely low i would say probably in the 0.01% mark if not less.

I guesstimated that each time you join a channel full of people, there'll be a 5% chance of getting only half a header.

Quote from: OnlyMeat on August 12, 2004, 10:31 PM

Quote from: Adron on August 12, 2004, 09:01 PM
I like to let people think for themselves. I point out the problems and what they have to think about, and then I let them solve the problems. If they get stuck, I hint more. I don't like to write people's bots for them.

But you are not trying to point out things for him instead you are trying to pick holes in my comments very pedantically.

I started by pointing out things for him that you hadn't mentioned in your description of what he was supposed to do - that in addition to what you said, he also needs to handle incomplete packets in some way. From that, I've gone on to correct other incorrectnesses, as well as commenting on your comments.

Quote from: OnlyMeat on August 12, 2004, 10:31 PM

Quote from: Adron on August 12, 2004, 09:01 PM
It's actually more efficient doing it that way, because there will be fewer API calls. It's true that you'll need additional resources to store the buffer, but if you don't provide the buffer, winsock will have to buffer the data itself anyway, so it's just moving the same amount of resources to a different place. About the cycle time for parsing, I really don't see what you mean. You're parsing the packet and the length either way.

And you dont call api's when parsing ( and i mean the term api can apply to not only OS based functions but also C runtime functions ) for example strlen() ?.

Of course I don't call strlen when parsing out a packet from the b.net stream. There's no reason whatsoever to call strlen for that. You have the length in the header!

Quote from: OnlyMeat on August 12, 2004, 10:31 PM
Your may now say that system API's require more resources etc well one call to recv ( which uses code allready active in memory and an efficient low level API which would simply copy one buffer to another seems very low in cpu cycles because in the end if data is already on the network stack thats all it does copy the specified length of the network buffer to you local buffer which for single event messages is very small.

Also note ws32 dll is loaded into the address space of the application just the same way as the c runtime therefore the performance hit will be just the same if not less.

I'd like to point out that it's not very low in cycles at all. recv calls WSARecv, which calls WSPRecv, which calls NtDeviceIoControlFile, which calls into kernel code. This takes a whole lot of cycles. Many many many more than a simple memmove. Therefore the performance hit for recv will be much larger than that for most C runtime library functions.

Quote from: OnlyMeat on August 12, 2004, 10:31 PM
Compared with having to loop through a multi-packet clump parsing and extracting the relevent data.

I can bet you now my method requires less cpu cycles and leaves the system free to do other things, try it out if you dont believe me.

I can bet you now that calling recv with a large buffer and splitting up the pieces yourself requires less cpu cycles and leaves the system free to do other things. If you have a suggestion for how to test it, I'd love to try it out.

Wow that took me like a while to read and I got a headache reading it, but you made some very good points, OnlyMeat is a no good flamer/basher who makes fun of others on the forum.

Well, I ended up using The-Fool gave me after tweaking it a little I made it work for my bot. I have found nothing you people have posted useful (Maybe I shouldn't just skim-read....), and storm, why did you paste the C code to it?

I pasted working psuedocode which 100% totally completely solved this (you'd have to translate it to VB, thats it)

...but of course, no one ever listens to me, anyway. Your loss.

I know too little of C to translate and I am too lazy to see if there is a program to do it. On the finishing note, I am Lazy.


Private Sub Winsock1_DataArrival(ByVal bytesTotal As Long)
     
   Static sBuffer As String
   Dim strTmp As String
   Dim intMessageID As Integer
   Dim intMessageLen As Integer
       
   Winsock1.GetData sTmp, vbString
   sBuffer = sBuffer & sTmp
   
   While LenB(sBuffer) > 0
       If Len(sBuffer) >= 4 Then
           ' Get header information.
           intMessageID = Asc(Mid$(sBuffer, 2, 1))
           intMessageLen = Bytes2Word(Mid$(sBuffer, 3, 2))
           
           ' Check to see if it's a complete packet.
           If intMessageLen <= Len(sBuffer) Then
               HandlePacket intMessageID, Mid$(sBuffer, 5, intMessageLen - 4)
               
               ' Remove handled packet from buffer
               sBuffer = Right$(sBuffer, Len(sBuffer) - intMessageLen)
           Else
               ' Incomplete packet.
               Exit Do
           End If
       Else
           ' Incomplete header.
           Exit Do
       End If
   Wend
   
End Sub


Quote from: Kk)Blaze(kK on August 15, 2004, 12:40 PM
I know too little of C to translate and I am too lazy to see if there is a program to do it. On the finishing note, I am Lazy.

On a side note, your attiture attitude fucking blows.

Quote from: Kk)Blaze(kK on August 15, 2004, 12:40 PM
I know too little of C to translate and I am too lazy to see if there is a program to do it. On the finishing note, I am Lazy.

You apparently don't read too well. $t0rm posted PSEUDOCODE, that is, code that does not belong to any particular language, but is intended to demonstrate the steps necessary to solve the problem. There was no translation to or from C required.

Quote from: Kk)Blaze(kK on August 15, 2004, 12:40 PM
I know too little of C to translate and I am too lazy to see if there is a program to do it. On the finishing note, I am Lazy.

i fucking hate you and i'm never helping you again.

it's easy as hell to read, even for the vb "programmer" that you are. if you'd rather copy someone elses code and then get bitched off the forum cause everyone hates you, that's your business.

good bye.

Quote from: $t0rm on August 15, 2004, 11:26 AM
I pasted working psuedocode which 100% totally completely solved this (you'd have to translate it to VB, thats it)

...but of course, no one ever listens to me, anyway. Your loss.

I was in the middle of something else, otherwise I would've commented that you should loop parsing the buffer in case there's more than one message received in the same readdata.

Whoops, knew I forgot something...

messageid = 0
messagelen = -1
buffer = new ByteBuffer()
while (socket.connected) {
   buffer.append(socket.readdata())
   [b]while[/b] messagelen == -1 {
       if buffer.position >= 4 {
           hdr = buffer.read(4)
           messageid = hdr[1]
           messagelen = bytes2word(hdr[2],hdr[3])
       }
   }
   if messagelen != -1 && buffer.position >= messagelen {
       handle_message(messageid, buffer.read(messagelen))
       messageid = -1
   }
}