Wrong length

[peoro]

Hi!

I'm studying the Battle.Net protocol to try and write a simple Warcraft 3 banlist for Linux.
have written a little battle.net packets sniffer, and i've encountered a problem that I couldn't find here or on bnetdocs:
Sometimes i receive a SID_GETADVLISTEX ( 0x09 ) from the server splitted in more TCP packets. The problem is that the "Message Length" written in third and fourth byte of the message (in the first packet) is greater than real data i receive.
Why does this happen?

Here three sample packets:

Code Select Expand

First packet
FF09 -> SID_GETADVLISTEX
e10b -> 3041
This first packet is 1434 bytes.

ff09 e10b 1600 0000 0000 0000 0000 0100    ................
0200 17e0 53fa e0fe 0000 0000 0000 0000    ....S...........
0400 0000 2b00 0000 6865 726f 206c 696e    ....+...hero lin
6520 7761 7220 6575 206f 6e6c 7900 0062    e war eu only..b
3131 3030 3030 3030 0103 4907 0101 4101    11000000..I...A.
f961 0139 45c9 c14d cb61 7173 5d45 6f77    .a.9E..M.aqs]Eow
196f 6d6f 6165 5d49 ab65 736f 4d69 6f65    .omoae]I.esoMioe
2757 6173 5737 2f37 952f 3721 656f 6961    'WasW7/7./7!eoia
cd6f 6365 652f 7733 f179 0177 6963 3931    .ocee/w3.y.wic91
0339 0101 0001 0000 0000 0001 0002 0017    .9..............
e051 b409 0e00 0000 0000 0000 0004 0000    .Q..............
002b 0000 0061 7265 6e61 2070 726f 6f6f    .+...arena prooo
6f6f 6f6f 6f6f 6f6f 6f6f 6f6f 6f6f 6f6f    oooooooooooooooo
6f6f 6f6f 0000 6235 3030 3030 3030 3001    oooo..b50000000.
0349 0701 01b5 0181 9501 49d5 6d91 4dcb    .I........I.m.M.
6171 735d 656f 7799 6f6d 6f61 655d 416d    aqs]eow.omoae]Am
6f67 656d 5f41 735b 656f 615f 595f 779d    ogem_As[eoa_Y_w.
395b 315d 2f31 5fa1 3331 3137 5f2f 7723    9[1]/1_.3117_/w#
3379 0145 616f 6f0f 5939 3901 0100 0100    3y.Eaoo.Y99.....
0000 0000 0100 0200 17e0 dd85 16b3 0000    ................
0000 0000 0000 0400 0000 2b00 0000 444f    ..........+...DO
5441 2056 4e20 484f 5354 2045 4d54 5241    TA VN HOST EMTRA
4920 4444 4545 0000 3961 3030 3030 3030    I DDEE..9a000000
3001 0349 0701 0175 01c9 7501 61ad 9131    0..I...u..u.a..1
4dcb 6171 735d 656f 7719 6f6d 6f61 655d    M.aqs]eow.omoae]
452b 6f75 4121 416d 6d2b 7375 6173 7321    E+ouA!Amm+suass!
77d1 372f 3531 2f77 3399 7901 654d 5573    w.7/51/w3.y.eMUs
6103 6901 0100 0100 0000 0000 0100 0200    a.i.............
17e0 5578 995f 0000 0000 0000 0000 0400    ..Ux._..........
0000 2b00 0000 646f 7461 2035 7635 202d    ..+...dota 5v5 -
6172 656d 2072 6f00 0039 3430 3030 3030    arem ro..9400000
3030 0103 4907 0101 7501 c975 0161 ad91    00..I...u..u.a..
314d cb61 7173 5d45 6f77 596f 6d6f 6165    1M.aqs]EowYomoae
735d 5545 6f75 4121 416d 556d 7375 6173    s]UEouA!AmUmsuas
7321 a177 372f 3531 2f77 b333 7901 6d65    s!.w7/51/w.3y.me
7561 716d 636d 6f63 6b01 0101 0001 0000    uaqmcmock.......
0000 0001 0002 0017 e056 7a3f 4100 0000    .........Vz?A...
0000 0000 0004 0000 002b 0000 002d 6172    .........+...-ar
2052 4f20 6d61 7374 6572 7320 2120 626c     RO masters ! bl
6f6e 0000 3933 3030 3030 3030 3001 0349    on..930000000..I
0701 0175 01c9 7501 61ad 9131 4dcb 6171    ...u..u.a..1M.aq
735d 456f 7719 6f6d 6f61 655d 452b 6f75    s]Eow.omoae]E+ou
4121 416d 6d2b 7375 6173 7321 77d1 372f    A!Amm+suass!w.7/
3531 2f77 3399 7901 6d69 757b 6107 5f31    51/w3.y.miu{a._1
0101 0001 0000 0000 0001 0002 0017 e0c9    ................
e64f 3300 0000 0000 0000 0004 0000 002b    .O3............+
0000 0044 6f74 6120 4c61 7469 6e6f 2033    ...Dota Latino 3
2076 7320 335f 4152 206e 6f20 6b69 7465     vs 3_AR no kite
7273 0000 3931 3030 3030 3030 3001 0349    rs..910000000..I
0701 0175 01c9 7501 61ad 9131 4dcb 6171    ...u..u.a..1M.aq
735d 456f 7719 6f6d 6f61 655d 452b 6f75    s]Eow.omoae]E+ou
4121 416d 6d2b 7375 6173 7321 77d1 372f    A!Amm+suass!w.7/
3531 2f77 33a9 7901 4773 6f6f 3301 0101    51/w3.y.Gsoo3...
0001 0000 0000 0001 0002 0017 e03e b16a    .............>.j
af00 0000 0000 0000 0004 0000 002b 0000    .............+..
004d 6166 6172 617a 7a6f 2054 4420 2121    .Mafarazzo TD !!
2121 2121 0000 3738 3030 3030 3030 3001    !!!!..780000000.
0349 0701 0149 01a3 4701 29c1 9b3f 4dcb    .I...I..G.)..?M.
6171 735d 456f 7799 6f6d 6f61 655d 4d2b    aqs]Eow.omoae]M+
6167 6173 617b 7b13 6f21 476f 7321 5195    agasa{{.o!Gos!Q.
736f 214f 6f6d 79cd 2f77 3379 0153 63f7    so!Oomy./w3y.Sc.
6f75 7367 652d 6301 7b01 0100 0100 0000    ousge-c.{.......
0000 0100 0200 17e3 3abb b8a4 0000 0000    ........:.......
0000 0000 0400 0000 2b00 0000 646f 7461    ........+...dota
2076 6e6e 6e6e 6e6e 6e6e 6e6e 0000 3932     vnnnnnnnnnn..92
3030 3030 3030 3001 0349 0701 0175 01c9    0000000..I...u..
7501 61ad 9131 4d4b 6171 735d 5541 5d55    u.a..1MKaqs]UA]U
456f 7541 2141 6d55 6d73 7561 7373 21a1    EouA!AmUmsuass!.
7737 2f35 312f 7763 3379 0175 6965 6f7f    w7/51/wc3y.uieo.
6375 6167 6169 0101 0100 0100 0000 0000    cuagai..........
0100 0200 17e0 5309 3e3b 0000 0000 0000    ......S.>;......
0000 0400 0000 2b00 0000 464f 4f54 4d45    ......+...FOOTME
4e20 554c 5449 4d41 5445 2121 2121 2121    N ULTIMATE!!!!!!
2121 2121 2100 0062 6530 3030 3030 3030    !!!!!..be0000000
0103 4907 0101 5101 c34f 01e9 6f63 234d    ..I...Q..O..oc#M
cb61 7173 5d45 6f77 996f 6d6f 6165 5d55    .aqs]Eow.omoae]U
b96d 7569 6d61 7565 d921 476f 6f75 6d65    .muimaue.!Gooume
c16f 2177 312f 3739 9b63 2f77 3379 016b    .o!w1/79.c/w3y.k
2b61 6d69 6365 7335 0135 0101 0001 0000    +amices5.5......
0000 0001 0002 0017 e056 7d8e df00 0000    .........V}.....
0000 0000 0004 0000 002b 0000 0065 6c65    .........+...ele
6d65 6e74 616c 2072 7067 2052 4f20 6f6e    mental rpg RO on
6c79 2061 6e64 206e 6f63 6b65 0000 6232    ly and nocke..b2
3030 3030 3030 3081 0349 0701 0101 01fd    0000000..I......
0101 8f95 7d1f 4dcb 6171 735d 456f 7799    ....}.M.aqs]Eow.
6f6d 6f61 655d 459d 6d65 6d65 6f75 6121    omoae]E.memeoua!
6d21 5351 4721 33a1 2f31 2147 696f 6119    m!SQG!3./1!Gioa.
6d2f 7733 7901 6557 6f63 756f 735f 7901    m/w3y.eWocuos_y.
0101 0001 0000 0000 0001 0002 0017 e0cb    ................
d2e7 7200 0000 0000 0000                   ..r.......


Second packet
This second packet is 1434 bytes.

0000 0044 4f54 4120 484f 5354 0000 3932    ...DOTA HOST..92
3030 3030 3030 3001 0349 0701 0175 01c9    0000000..I...u..
7501 61ad 9131 4dcb 6171 735d 456f 7719    u.a..1M.aqs]Eow.
6f6d 6f61 655d 452b 6f75 4121 416d 6d2b    omoae]E+ouA!Amm+
7375 6173 7321 77d1 372f 3531 2f77 33e1    suass!w.7/51/w3.
7901 636d 6163 6b3b 616f 6775 7301 0100    y.cmack;aogus...
0100 0000 0000 0100 0200 17e0 5b7f 1b72    ............[..r
0000 0000 0000 0000 0400 0000 2b00 0000    ............+...
3576 3520 2d61 7020 4555 0000 3934 3030    5v5 -ap EU..9400
3030 3030 3001 0349 0701 0175 01c9 7501    00000..I...u..u.
61ad 9131 4dcb 6171 735d 456f 7719 6f6d    a..1M.aqs]Eow.om
6f61 655d 452b 6f75 4121 416d 6d2b 7375    oae]E+ouA!Amm+su
6173 7321 77d1 372f 3531 2f77 33e1 7901    ass!w.7/51/w3.y.
4573 6167 6f01 6f6f 6f01 0100 0100 0000    Esago.ooo.......
0000 0100 0200 17e0 567e 3a4e 0000 0000    ........V~:N....
0000 0000 0400 0000 2b00 0000 6d61 6661    ........+...mafa
2065 7520 6f6e 6c79 2070 726f 0000 6238     eu only pro..b8
3030 3030 3030 3001 0349 0701 0155 01d3    0000000..I...U..
4b01 2381 39f3 4d6b 6171 735d 6d61 675b    K.#.9.Mkaqs]mag[
615d 4d61 6761 7333 617b 7b6f 5f55 4573    a]Magas3a{{o_UEs
5f77 335b 315d 2f1b 3567 5f45 7975 7357    _w3[1]/.5g_EyusW
615f 4969 755f 51a7 6f69 6f75 732f 7733    a_Iiu_Q.oious/w3
3379 014d 6173 65cf 756b 5f55 6965 5f07    3y.Mase.uk_Uie_.
476f 6501 0100 0100 0000 0000 0100 0200    Goe.............
17e0 5989 fed0 0000 0000 0000 0000 0400    ..Y.............
0000 2b00 0000 726f 2044 4f54 4120 3420    ..+...ro DOTA 4
6e6f 6f62 7320 2d61 7200 0039 3330 3030    noobs -ar..93000
3030 3030 0103 4907 0101 7501 c975 0161    0000..I...u..u.a
ad91 314d 8b61 7173 5d47 736f 857b 656f    ..1M.aqs]Gso.{eo
5569 736f 756f 655d 5363 656f 9b61 7369    Uisouoe]Sceo.asi
6f5d 456f 9575 4121 416d 6d73 1575 6173    o]Eo.uA!Amms.uas
7321 7737 692f 3531 2f77 3379 1501 3531    s!w7i/51/w3y..51
3531 0101 0001 0000 0000 0001 0002 0017    51..............
e0d3 1e4a e000 0000 0000 0000 0004 0000    ...J............
002b 0000 003e 3e3e 2054 4478 3320 5052    .+...>>> TDx3 PR
4f20 4f4e 4c59 2121 2120 3c3c 3c00 0062    O ONLY!!! <<<..b
3130 3030 3030 3030 0103 4907 0101 5501    10000000..I...U.
d135 018f 3dc7 8f4d cb61 7173 5d45 6f77    .5..=..M.aqs]Eow
196f 6d6f 6165 5d55 6945 7933 2157 6173    .omoae]UiEy3!Was
5373 2177 352f 332f e777 3379 0175 6d61    Ss!w5/3/.w3y.uma
ff6b 656d 6579 6177 016f 0101 0001 0000    .kemeyaw.o......
0000 0001 0002 0017 e0d2 d592 ad00 0000    ................
0000 0000 0004 0000 002b 0000 0031 7631    .........+...1v1
524d 4b00 0039 3330 3030 3030 3030 0103    RMK..930000000..
4907 0101 7501 c975 0161 ad91 314d cb61    I...u..u.a..1M.a
7173 5d45 6f77 196f 6d6f 6165 5d45 2b6f    qs]Eow.omoae]E+o
7541 2141 6d6d 2b73 7561 7373 2177 5137    uA!Amm+suass!wQ7
2f35 317f 312f 6777 3379 016b 3939 0135    /51.1/gw3y.k99.5
3533 3301 0100 0100 0000 0000 0100 0200    533.............
17e0 3c30 cf26 0000 0000 0000 0000 0400    ..<0.&..........
0000 2b00 0000 726d 6b20 646f 7461 2061    ..+...rmk dota a
7020 6d79 2f73 6720 746f 7020 666f 7220    p my/sg top for
6672 6965 6e00 0039 3930 3030 3030 3030    frien..990000000
0103 4907 0101 7501 c975 0161 ad91 314d    ..I...u..u.a..1M
cb61 7173 5d45 6f77 196f 6d6f 6165 5d45    .aqs]Eow.omoae]E
2b6f 7541 2141 6d6d 2b73 7561 7373 2177    +ouA!Amm+suass!w
d137 2f35 312f 7733 f979 0153 7563 6365    .7/51/w3.y.Succe
0773 7301 0100 0100 0000 0000 0100 0200    .ss.............
17e0 524d a069 0000 0000 0000 0000 0400    ..RM.i..........
0000 2b00 0000 646f 7461 2072 6f20 6e6f    ..+...dota ro no
6f62 7300 0039 3130 3030 3030 3030 0103    obs..910000000..
4907 0101 7501 b975 0183 e785 874d cb61    I...u..u.....M.a
7173 5d45 6f77 196f 6d6f 6165 5d45 2b6f    qs]Eow.omoae]E+o
7541 2141 6d6d 2b73 7561 7373 2177 c137    uA!Amm+suass!w.7
2f35 312f 7733 a979 0145 6d69 7969 0173    /51/w3.y.Emiyi.s
0101 0001 0000 0000 0001 0002 0017 e058    ...............X
6581 eb00 0000 0000 0000 0004 0000 002b    e..............+
0000 004d 4153 5445 522e 2e2e 4d41 4e59    ...MASTER...MANY
2e2e 2e47 414d 4500 0062 3130 3030 3030    ...GAME..b100000
3030 0103 4907 0101 4f01 eb7d 0151 35f1    00..I...O..}.Q5.
f54d cb61 7173 5d45 6f77 196f 6d6f 6165    .M.aqs]Eow.omoae
5d49 5b65 736f 4373 6167 5575 2d33 2d6d    ]I[esoCsagUu-3-m
6975 3365 2f35 2d6f 7175 cd2f 7733 7901    iu3e/5-oqu./w3y.
4d61 cb73 7565 732f 4d61 056f 7901 0100    Ma.sues/Ma.oy...
0100 0000 0000 0100 0200 17e0 3a45 40f4    ............:E@.
0000 0000 0000 0000 0400 0000 2b00 0000    ............+...
646f 7461 2070 696e 6f79 206c 6e67 0000    dota pinoy lng..
3931 3030 3030 3030 3001 0349 0701 0175    910000000..I...u
01b9 7501 83e7 8587 4dcb 6171 735d 456f    ..u.....M.aqs]Eo
7719 6f6d 6f61 655d 452b 6f75 4121 416d    w.omoae]E+ouA!Am
6d2b 7375 6173 7321 77c1 372f 3531 2f77    m+suass!w.7/51/w
3349 7901 6773 3973 7557 5f61 7363 6965    3Iy.gs9suW_ascie
7301 0101 0001 0000 0000 0001 0002 0017    s...............
e1de fde6 f200 0000 0000 0000 0004 0000    ................
002b 0000 0064 6f74 6120 766e 205b 5052    .+...dota vn [PR
4f5d 2072 6d6b 3330 0000 3937 3030 3030    O] rmk30..970000
3030 3001 0349 0701 0175 01c9 7501 61ad    000..I...u..u.a.
9131 4dcb 6171 735d 456f 7719 6f6d 6f61    .1M.aqs]Eow.omoa
655d 452b 6f75 4121 416d 6d2b 7375 6173    e]E+ouA!Amm+suas
7321 77d1 372f 3531 2f77                   s!w.7/51/w


Third (and last packet)
This third packet is 161 bytes.

2d57 4fef 2d5f 2d4f 6577 5305 7561 7301    -WO.-_-OewS.uas.
0100 0100 0000 0000 0100 0200 17e0 3aba    ..............:.
4a73 0000 0000 0000 0000 0400 0000 2b00    Js............+.
0000 5448 414e 4760 204e 414f 6020 4e47    ..THANG` NAO` NG
5520 4d4f 4927 204b 4f20 4455 4120 5845    U MOI' KO DUA XE
0000 6262 3030 3030 3030 3001 0349 0701    ..bb0000000..I..
0155 0189 5501 5b41 3f9b 4dcb 6171 735d    .U..U.[A?.M.aqs]
456f 7719 6f6d 6f61 655d 55e1 4755 2153    Eow.omoae]U.GU!S
4143 4529 2147 696f 616d 2f27 7733 7901    ACE)!Gioam/'w3y.
5b77 6fab 5d63 6163 796f 6f03 6f63 0101    [wo.]cacyoo.oc..
00                                         .


The length declared in the header was 3041, but 1434 + 1434 + 161 = 3029

[UserLoser]

Chances are Battle.net isn't sending you a bad length.  Every single person who has ever come here with this issue results in a bad data handler, and Battle.net has never been known to mess up like this.  Show us some code on how you're handling the messages (specifically ones that come in chunks).

[peoro]

The program I've written to study packets is in Python, it uses the pcapy module, which uses libpcap.

My code (the part which gets packets and print them):

Code Select Expand

        self.pcap.loop( 1, self.packetHandler )

    def packetHandler( self, hdr, data ):
        global clientAddr
       
        e = self.decoder.decode(data)
        ip = e.child()
       
        srcaddress = ( None, None )
        dstaddress = ( None, None )
        data = None
       
        # get port and address of client and server
        if ip.get_ip_p() == ImpactPacket.UDP.protocol:
          packet = "UDP"
          udp = ip.child()
         
          srcaddress = ( ip.get_ip_src(), udp.get_uh_sport() )
          dstaddress = ( ip.get_ip_dst(), udp.get_uh_dport() )
          data = udp.child()
       
        elif ip.get_ip_p() == ImpactPacket.TCP.protocol:
          packet = "TCP"
          tcp = ip.child()
         
          srcaddress = ( ip.get_ip_src(), tcp.get_th_sport() )
          dstaddress = ( ip.get_ip_dst(), tcp.get_th_dport() )
          data = tcp.child()
       
        else:
          return
       
        # check if this is a batttle.net packet
        if ( srcaddress[1] < 6112 or srcaddress[1] > 6119 ) and\
           ( dstaddress[1] < 6112 or dstaddress[1] > 6119 ):
          return
       
        # get the sender of the packet
        if srcaddress[0] == clientAddr:
          sender = client
        elif dstaddress[0] == clientAddr:
          sender = server
        else:
          return
       
        # ignore empty packets
        if not data or not data.get_size():
          return
       
        print data

Almost everything is done by pcapy module.
I get these problems only with SID_GETADVLISTEX messages, which are splitted in more packets, for the other messages everything is fine.

I'm not doing these tests on an official battle.net server, i'm using a PvPGN one, eurobattle.net, to not risk to get banned.

[UserLoser]

Without analyzing your code, I am jumping to the conclusion that PvPGN is at fault.  PvPGN is a horrible source of Battle.net information to write clients with.

I assure you, you won't get banned from using libpcap on a Blizzard client on Battle.net

[peoro]

Damn it!
Found the bug...
I set the max number of bytes to capture to 1500. These UDP packets used 66 bytes for headers, so the last 4 bytes got cut
I thought it was battle.net server's fault, but after your message i've used wireshark to monitor these packets and...
Well, thanks a lot

About PvPGN...
I thought it could be better than bilzzard's servers, since it isn't interested to crypt and encode messages like blizzard is, but I trust you
...What should I do when I'll need to send packages instead of just sniff them?

[UserLoser]

Hmm?  Battle.net doesn't use UDP for game listings, it's TCP and there's a 24 byte TCP header

[peoro]

Yes, you're right, UDP was a mistake, I meant TCP.
About 66 bytes, they're the 32 bytes of TCP header + 20 bytes of IP header + 14 of Ethernet header...

[UserLoser]

Yes, you're right, UDP was a mistake, I meant TCP.
About 66 bytes, they're the 32 bytes of TCP header + 20 bytes of IP header + 14 of Ethernet header...

There you go