Winsock & Merging Data?

[Anubis]

I've been working on a chat program lately (similar to the bnet chatroom) and I'm getting errors because of merging data. The data almost always merges together when logging onto the chat server. Right after logon the server sends the client a list of people in the channel they are joining. So, for example, if there are 3 people in the channel named Bob, Billy, and Jack, the server send the data as "BobBillyJack" when it should be sending three separate messages ("Bob", "Billy", and "Jack"). This is then causing the client to list all the users in the channel on one line in the listbox.

Is there a way to stop the merging? If not, should I just run a loop and split the users into an array and then add them to the list? Preferebly I'd like to find a fix rather than a delay, as this is happening with incoming messages at the same time.

All help is much appreciated! Thanks.

[DeTaiLs]

Is this an chatbot or an binary bot?

can u also post your join code

[Stealth]

Choose a character, such as 0x00 (null), that cannot be duplicated in account names. Separate the usernames with it:

"Billy" & chr(0) & "Bob" & chr(0) & "Jane" & chr(0)

Then your client should extract the names based on the positions of the delimiter character.

[Eli_1]

Is this an chatbot or an binary bot?

can u also post your join code

It's not a bot; it's a server.


I would seperate it with vbCrLfs (plural vbCrLf) so people can telnet to your server and still have the data displayed correctly.

Code Select Expand


.SendData "Billy" & vbCrLf & "Bob" & vbCrLf & "Jack" & vbCrLf



Or simulate b.net's CHAT protocol.

Code Select Expand


Winsock1.SendData _
"1001 USER Billy 0010 [CHAT]" & vbCrLf & _
"1001 USER Bob 0010 [CHAT]" & vbCrLf & _
"1001 USER Jack 0010 [CHAT]" & vbCrLf


[Anubis]

Those are good ideas, but similar to bnet when two people send at about the same millisecond the text merges also (if one person typed "Hi" and the other typed "Bob" it would send "HiBob" to everyone). I tried adding delays and such to the rebroadcasting of data and it works, but the only way I can get it to work makes person #1 in the list have a delay of 250ms, person #2 have 500ms, ect (250x2...3...). Is there a way to convince Winsock to not merge the packets? (For example I am using Winsock.SendData "Whatever" and another line uses Winsock.SendData "Whatever" [I think its SendData] and it ends up sending both messages at the same time -> "WhateverWhatever")

[Stealth]

If your client's data handler works properly, packet merging is OK. Create a length field that denotes the number of bytes in the packet as Battle.net does and you are sure to not overstep the bounds of each packet.

[Anubis]

By using Winsock.SendData, isn't the packet already sent and therefore shouldn't be able to be merged with another packet? The packets also aren't a set length. They range from 5 characters to 4+the length of the message characters. However, I'm already telling Winsock to send the packet and so it shouldn't cause any merging - the first packet should be long-gone before the next one.

[Adron]

By using Winsock.SendData, isn't the packet already sent and therefore shouldn't be able to be merged with another packet? The packets also aren't a set length. They range from 5 characters to 4+the length of the message characters. However, I'm already telling Winsock to send the packet and so it shouldn't cause any merging - the first packet should be long-gone before the next one.

TCP works with streams, not packets. Yes, it does send packets on the network, but the protocol is designed to manage data as streams.

There is the nagle algorithm, which will hold your sends, waiting for more data to build up before transmitting. This works to combine a lot of small sends into a bigger send, to save overhead on the network. It's implemented somewhere around winsock, and can be disabled, but I wouldn't recommend it.

Even if the data is sent as two packets, if they both reach the receiver before the receiver reads the first packet, they will be combined in the receiving buffer. You may think that by having a really fast receiver, and sending packets with wide spacing, you may be able to get around this, but you can't.

If there is congestion on the network, packets are delayed in transit. They can also be reordered in transit. If that's the case, and the second packet arrives before the first packet, TCP will hold the second packet until the first packet has received. TCP will then make both (i.e. the sequential stream) available to the receiving app.

You cannot read the data as packets, unless you manage the packets yourself, either by adding length information for each subunit, or by having some special separator between each subunit.

Or unless you switch to a packet based protocol such as message-based mailslots, or udp. But then you'll have other problems.

[Eli_1]

From what it sounds like now, I think you're sending it like this.

Code Select Expand


Winsock1.SendData "Billy"


Try appending (postpending, is that that right word?) some kind of delimiter to the end of it, so in the case they do get combinded, you'll be able to seperate them. Going back to my first example, add vbCrLf onto the names before you send them.

Code Select Expand


Winsock1.SendData "Billy" & vbCrLf



Here's an example of how your clients might go about handling something like this:

Code Select Expand


Dim Data as String
Dim MainParse() as String

Winsock1.GetData Data
MainParse = Split(Data, vbCrLf)
For x = 0 to UBound(MainParse)
   ' All the messages will be here.
   ' MainParse(0) might be "Billy"
   ' MainParse(1) might be "Bob"
   ' MainParse(2) might be "Hi"
   ' ect...
Next x


[Anubis]

I'll try some of these ways... Thanks for everyone's help

[Lycaon]

The problem is the Winsock control's wonderful way of receiving data.  If you don't .GetData RIGHT after it's sent and the control happens to receive more data before you get the last part, it'll buffer the two together.  As mentioned before, the best way would be to use some sort of termination marker like Chr(0) or vbCrLf, which should work fine for most applications, or go the BNet way and prefix each packet with a byte that indicates it's length (or two bytes if you want to allow messages larger than 255 bytes).  That way if you get more data crammed into your buffer faster than you can get it out you've got some sort of reference point to indicate the end (or length) of a packet.

I've got a simple buffer class that is built to solve the same problem you're having (it was for an irc client, the dalnet servers chunk together lines all the time).  If you'd like it, holler.

Edit:  Blah, I didn't read the few posts above mine, so I just basically reiterated what's already been said    Class is still available if you want it though.

[TheMinistered]

You need a way of delimeting packets.  You can then split all data up into seperate packets... and store incomplete packets until you receive the rest of the incomplete packet.

i.e. if you use &H0 as a packet delimeter here is what a packet like you are talking about would look like:

randomdata&H0randomdata2&H0randomdata3&H0rando

you could break these packets up into seperate packets by the nulls.... now if data on the end of the buffer isn't terminated by a null-- it's an incomplete packet and should be saved so that when the rest of the packet comes you can process it

so as long as you are sending packets with a delimeter here is how the client should handle the data-- I've seen someone else on the boards handle it this way before I did-- should be similar to his either way-- and it works...

Code Select Expand


'// Private member variables
Private m_strBuffer as String

Public Sub AddToBuffer(ByVal strData as String)
   m_strBuffer = strData
End Sub

Public Sub ProcessBuffer()

   Dim lngPosition As Long, strLine As String
   Const Delimiter as String = vbCrLf

   lngPosition = InStr(m_strBuffer, Delimiter)
   
   Do While (lngPosition > 0)
       strLine = Left$(m_strBuffer, lngPosition - 1)
       m_strBuffer = Mid$(m_strBuffer, lngPosition + 2)

       'DispatchMessage should handle individual packets
       DispatchMessage strLine
       lngPosition = InStr(m_strBuffer, Delimiter)
   Loop

   m_strBuffer = vbNullString
End Sub


Then, everytime you receive data call AddToBuffer then ProcessData!  Summing this all up, as long as your server sends packets with a unique delimeter -- then the server doesn't have to delay or do anything extra!  The client handles splitting packets up...  now data you send to the server from the client might also have to go through a similar process...

[Grok]

By using Winsock.SendData, isn't the packet already sent and therefore shouldn't be able to be merged with another packet? The packets also aren't a set length. They range from 5 characters to 4+the length of the message characters. However, I'm already telling Winsock to send the packet and so it shouldn't cause any merging - the first packet should be long-gone before the next one.

TCP works with streams, not packets. Yes, it does send packets on the network, but the protocol is designed to manage data as streams.

There is the nagle algorithm, which will hold your sends, waiting for more data to build up before transmitting. This works to combine a lot of small sends into a bigger send, to save overhead on the network. It's implemented somewhere around winsock, and can be disabled, but I wouldn't recommend it.

"Nagle says that when a TCP connection has outstanding data that has not yet been acknowledged, small segments cannot be sent until the outstanding data is acknowledged.  Instead, small amounts of data are collected by TCP and sent in a single segment when the acknowledgment arrives."
 -TCP/IP Illustrated, Vol 1., Richard Stevens.

I understand from Stevens description that Nagle only holds your sends when there's a outstanding ACK.

[Adron]

I understand from Stevens description that Nagle only holds your sends when there's a outstanding ACK.

Yes, when there's an outstanding ACK, and there's less data than a full segment. It sounds about right to be the cause of the effects he's seeing. He mentioned a delay of 250 ms fixing it, and 250 ms should be about enough time to make sure an ACK always gets back before he tries to send the next "packet".