Quick strtok question

[Eli_1]

Making a simple client for bnet using CHAT protocol. I want to split the data received by b.net using strtok.

Code Select Expand


char *ptok = strtok(buffer, "\r\n");
   while (ptok != NULL) {
       printf("%s\n", ptok);
       ptok = strtok(NULL, "\r\n");
   }

It works, to an extent, but the output is kinda weird.
Output in BW client:

<some_loser is a Statistics Bot>
<Eli_1> hi
<Eli_1> h2
<Eli_1> hi2
<Eli_1> hi22

Weird output:

1005 TALK Eli_1 0010 "hi"
tistics Bot"
1005 TALK Eli_1 0010 "h2"
tistics Bot"
1005 TALK Eli_1 0010 "hi2"
istics Bot"
1005 TALK Eli_1 0010 "hi22"
stics Bot"

Anyone have an idea why this is happening?

[K]

I don't know the internal workings of strok, but it looks like it's not putting a null terminator after the substring it returns, so you're seeing whats left at the end of the buffer.

[Adron]

Looks like you're not putting a \0 at the end of buffer before calling strtok.

[Eli_1]

I wasn't before. I changed it, but the output still isn't changing.

Code Select Expand


      buffer[strlen(buffer)] = 0;
      ptok = strtok(buffer, "\r\n");
      while (ptok != NULL) {
         printf("%s\n", ptok);
         ptok = strtok(NULL, "\r\n");
      }


[Mephisto]

I don't think he's talking about setting it to 0...NULL terminator perhaps?

[Adron]

0 or '\0' or NULL; all the same.

However, you're missing the point. Your problem is that you have a buffer containing text. The buffer is larger than the actual data in it. You have some length variable telling you how much data there is in the buffer. You have not put a string termination character at the end of the valid data.

Your recent addition

Code Select Expand


buffer[strlen(buffer)] = 0;

does nothing useful. By definition, the character buffer[strlen(buffer)] is zero already. That's what strlen checks for to find the length. And what strtok uses to find the end of the string.

[Skywing]

Note that NULL is typically defined as (void*)0 with C.  At least in GCC, you'll get warnings about assigning pointers to integers if you assign NULL to a char.

[Adron]

That's strange. You shouldn't be able to assign a void* to any other pointer type without a warning/error.

[Eli_1]

Ahh, thank you very very much Adron. I totally forgot recv() returns the number of bytes received,  and I can just use that to put a null-terminator at the end of the data. Works like a charm now.  

I don't think he's talking about setting it to 0...NULL terminator perhaps?

Character arrays don't hold the actual character. Instead, they hold the number representation of that char.

Code Select Expand


str[5] = '\0x00';
str[5] = 0x00;
str[5] = '\0';
str[5] = 0;
str[5] = '(char)0';

Would all be null-terminators.

[Skywing]

That's strange. You shouldn't be able to assign a void* to any other pointer type without a warning/error.

I think the rule about 0 implicitly casting still applies...

[Adron]

That's strange. You shouldn't be able to assign a void* to any other pointer type without a warning/error.

I think the rule about 0 implicitly casting still applies...

Code Select Expand


   char *a = (void*)0;


templatetest.cpp: In function `int main ()':
templatetest.cpp:8: cannot convert `void *' to `char *' in initialization

[K]

I was under the impression that NULL was a (void*)0 in C and simply 0 in C++

(C++ Standard:)

4.10 - Pointer conversions
-1- A null pointer constant is an integral constant expression rvalue of integer type that evaluates to zero. A null pointer constant can be converted to a pointer type; the result is the null pointer value of that type and is distinguishable from every other value of pointer to object or pointer to function type. Two null pointer values of the same type shall compare equal. The conversion of a null pointer constant to a pointer to cv-qualified type is a single conversion, and not the sequence of a pointer conversion followed by a qualification conversion (conv.qual).

[mynameistmp]

"Never use these functions." - Linux strtok (and strtok_r) man file.