Valhalla Legends Archive

Programming => Battle.net Bot Development => Topic started by: topaz on June 01, 2006, 08:30 PM

Title: [Python] BnFTP File Downloader
Post by: topaz on June 01, 2006, 08:30 PM
import socket
import struct

class packetbuffer:

    def __init__(self):
        self.buffer = list()
        #declares self.buffer as a list
   
    def insertData(self, data):
        self.buffer.append(data)

    def sendPacket(self, sock):

        tmp = ''
       
        for i in self.buffer: tmp += str(i)

        tmp = struct.pack('H', len(tmp) + 2) + tmp

        sock.send(tmp)
        print sock.recv(1024)

        self.clear()

    def clear(self):
        #clears the buffer
        self.buffer = list()

sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)

sock.connect(('useast.battle.net', 6112))

sock.send(chr(2))

pBuffer = packetbuffer()

pBuffer.insertData(0)
pBuffer.insertData(1)
pBuffer.insertData('68XIRATS')
pBuffer.insertData(0)
pBuffer.insertData(0)
pBuffer.insertData(0)
pBuffer.insertData(0)
pBuffer.insertData(0)
pBuffer.insertData('icons.bni')
pBuffer.sendPacket(sock)

print sock.recv(1024)


Problem is, though, it simply sits there. Does anyone have any ideas as to what I did wrong?

see below
Title: Re: [Python] BnFTP File Downloader
Post by: l2k-Shadow on June 01, 2006, 09:11 PM
I'm not very familiar with python but it seems to me that you are not waiting for it to actually connect, you just tell it to connect and send your chr(2) right away.. without getting a callback that the socket connected.
Title: Re: [Python] BnFTP File Downloader
Post by: K on June 01, 2006, 09:24 PM
most likely the default connect() is blocking, so that's not the issue.
Title: Re: [Python] BnFTP File Downloader
Post by: topaz on June 01, 2006, 09:29 PM
Quote from: l2k-Shadow on June 01, 2006, 09:11 PM
I'm not very familiar with python but it seems to me that you are not waiting for it to actually connect, you just tell it to connect and send your chr(2) right away.. without getting a callback that the socket connected.

It would throw an error if the socket wasn't connected.
Title: Re: [Python] BnFTP File Downloader
Post by: Yegg on June 02, 2006, 02:08 PM
Try doing something like:

while 1:
    received = sock.recv(1024)
    print received
Title: Re: [Python] BnFTP File Downloader
Post by: topaz on June 02, 2006, 02:44 PM
Quote from: Yegg on June 02, 2006, 02:08 PM
Try doing something like:

while 1:
    received = sock.recv(1024)
    print received


Useless...
Title: Re: [Python] BnFTP File Downloader
Post by: Yegg on June 02, 2006, 03:16 PM
Quote from: Topaz on June 02, 2006, 02:44 PM
Quote from: Yegg on June 02, 2006, 02:08 PM
Try doing something like:

while 1:
    received = sock.recv(1024)
    print received


Useless...

Wasn't thinking :).
Title: Re: [Python] BnFTP File Downloader
Post by: topaz on October 08, 2006, 09:51 PM
Hah! Been a couple of months since I've looked at this, and fixed all the bugs (it was the '.' over '\x00' issue I had in another project, since I used the same packet sending buffer)

bnftp.py

"""
Notes:
    This is BnFTP v1, which means it'll work for all products
    except WAR3 and W3XP. When ctypes starts to want to work with
    BNCSUtil, I'll write one for BnFTP v2.

Thanks to:
    #python on irc.freenode.net,
        for their help with my problem with integer division (and
        the resulting use of __future__ to get the desired results)
"""

__author__ = 'topaz'
__copyright__ = 'BSD License'

from __future__ import division
import pbuffer
import socket
import sys

class bnftpv1:
    def __init__(self):
        self.socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)

    def connect(self, server_address, file_name):
        self.socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
        self.server_address = (server_address, 6112,)
        self.file_name = file_name

        self.socket.connect(self.server_address)
        self.socket.send(chr(2))

        print 'Connecting to BnFTP (server: %s)...' %self.socket.getpeername()[0]
       
        send_buffer = pbuffer.send_buffer(self.socket)
        send_buffer.insert_byte(0)
        send_buffer.insert_byte(1)
        send_buffer.insert_string('68XIRATS')
        send_buffer.insert_dword(0)
        send_buffer.insert_dword(0)
        send_buffer.insert_dword(0)
        send_buffer.insert_dword(0)
        send_buffer.insert_dword(0)
        send_buffer.insert_nt_string(self.file_name)
        send_buffer.send_packet()

    def parse_socket(self):
        while True:
            data = self.socket.recv(1024)

            if data.find(file_name) <> -1:
                downloaded_file = open('./' + self.file_name, 'w')
                total_downloaded = 0

                recv_buffer = pbuffer.recv_buffer(data)
                recv_buffer.next_word()
                recv_buffer.next_word()
                file_size = recv_buffer.next_dword()
                file_time = recv_buffer.next_data(22)
                recv_buffer.jump(22)
                recv_buffer.next_string()
                data = recv_buffer.get_rest()

                print 'Filesize for %s is %d bytes' %(self.file_name, file_size)
               
            try:
                total_downloaded = total_downloaded + len(data)
            except NameError:               
                print 'Cannot download file %s (does not exist?).' %self.file_name
                print 'Connection to BnFTP (server: %s) closed.' %self.socket.getpeername()[0]
                self.socket.close()
               
                break
           
            downloaded_file.write(data)

            print 'Downloaded & wrote %d bytes to file "%s": %.1f%%.' %(total_downloaded, self.file_name, (total_downloaded / file_size) * 100)

            if total_downloaded == file_size:
                print 'Download complete!'
               
                downloaded_file.close()
                self.socket.close()
                break

def main(file_name, server_address):
    bnftp = bnftpv1()

    bnftp.connect(server_address, file_name)

    bnftp.parse_socket()


pbuffer.py
import struct

class send_buffer:
    def __init__(self, sock):
        self.buffer = []
        self.sock = sock
       
    def insert_data(self, data):
        self.buffer.append(data)

    def insert_string(self, data):
        self.buffer.append(data)

    def insert_nt_string(self, data):
        self.buffer.append(data + chr(0))

    def insert_dword(self, data):
        data = self.make_dword(data)
        self.buffer.append(data)
   
    def insert_dword_list(self, data):
    self.buffer.extend(data)

    def insert_byte(self, data):
        self.buffer.append(chr(data))

    def make_dword(self, data):
        return struct.pack('I', data)

    def make_word(self, data):
        return struct.pack('H', data)
   
    def send_packet(self):
        tmp = "".join([str(x) for x in self.buffer])
        packet_len = self.make_word(len(tmp) + 2)

        self.sock.send(packet_len)
        self.sock.send(tmp)
       
        self.clear()
       
    def clear(self):
        self.buffer = []

class recv_buffer:
    def __init__(self, raw_buffer):
        self.raw_buffer = raw_buffer
        self.position = 0
       
    def get_dword(self, data):
        return struct.unpack('<I', data)

    def get_word(self, data):
        return struct.unpack('<H', data)

    def increment(self):
        self.position = self.position + 1

    def jump(self, bytes):
        self.position += bytes

    def back(self, bytes):
        self.position -= bytes

    def set_pos(self, position):
        self.position = position

    def next_byte(self):
        param = self.raw_buffer[self.position:self.position + 1]
        byte = struct.unpack('b', param)[0]

        self.increment()
       
        return byte

    def next_string(self):
        pos = self.raw_buffer.find(chr(0), self.position)
        string = self.raw_buffer[self.position:pos]
       
        self.set_pos(pos + 1)
       
        return string
   
    def next_dword(self):
        param = self.raw_buffer[self.position:self.position + 4]
        dword = struct.unpack('I', param)[0]
       
        self.jump(4)

        return dword

    def next_word(self):
        param = self.raw_buffer[self.position:self.position + 2]
        word = struct.unpack('H', param)[0]
       
        self.jump(2)

        return word

    def next_data(self, bytes):
        return self.raw_buffer[self.position: self.position + bytes]

    def get_rest(self):
        data = self.raw_buffer[self.position:]
        self.set_pos(0)

        return data


http://advancedcontent.net/topaz/sources/bnftp-python.zip

Edit:

Cleaned up the code a bit, more verbose, error handling for nonexistent files on the FTP.

sample output:
QuoteConnecting to BnFTP (server: 63.240.202.129)...
Filesize for ver-ix86-0.mpq is 6894 bytes
Downloaded and wrote 0 bytes to file "ver-ix86-0.mpq": 0.0% complete
Downloaded and wrote 536 bytes to file "ver-ix86-0.mpq": 7.8% complete
Downloaded and wrote 1560 bytes to file "ver-ix86-0.mpq": 22.6% complete
Downloaded and wrote 2144 bytes to file "ver-ix86-0.mpq": 31.1% complete
Downloaded and wrote 3168 bytes to file "ver-ix86-0.mpq": 46.0% complete
Downloaded and wrote 3752 bytes to file "ver-ix86-0.mpq": 54.4% complete
Downloaded and wrote 4776 bytes to file "ver-ix86-0.mpq": 69.3% complete
Downloaded and wrote 4824 bytes to file "ver-ix86-0.mpq": 70.0% complete
Downloaded and wrote 5848 bytes to file "ver-ix86-0.mpq": 84.8% complete
Downloaded and wrote 6432 bytes to file "ver-ix86-0.mpq": 93.3% complete
Downloaded and wrote 6894 bytes to file "ver-ix86-0.mpq": 100.0% complete
Download complete!

2:

QuoteConnecting to BnFTP (server: 63.240.202.129)...
Cannot download file Fefe.mpq (does not exist?).
Connection to BnFTP (server: 63.240.202.129) closed.

Edit 3:

put all the code together into a class, can be used like such:

if __name__ == '__main__':
    try:
        file_name, server_address = sys.argv[1:3]
    except ValueError:
        print 'Error: probably not enough arguments:'
        print 'Format is bnftp.py file server (ex: bnftp.py icons.bni useast.battle.net)'

    main(file_name, server_address)


and then:

Quote
bnftp.py lockdown-IX86-00.mpq useast.battle.net
Title: Re: [Python] BnFTP File Downloader
Post by: Joe[x86] on October 09, 2006, 07:11 AM
I just noticed a weird thing in the BNFTP protocol.

send_buffer = pbuffer.send_buffer(sock)
send_buffer.insert_byte(0)
send_buffer.insert_byte(1)
send_buffer.insert_string('68XIRATS')
send_buffer.insert_dword(0)
send_buffer.insert_dword(0)
send_buffer.insert_dword(0)
send_buffer.insert_dword(0)
send_buffer.insert_dword(0)
send_buffer.insert_nt_string(file_name)
send_buffer.send_packet()


That must be horrid on their processors for DWORD alignment, although the two bytes in the beginning followed by "icons.bni\x00" do line up to 8 bytes.
Title: Re: [Python] BnFTP File Downloader
Post by: Kp on October 15, 2006, 04:18 PM
Quote from: topaz on October 08, 2006, 09:51 PM
main.py

__author__ = 'topaz'
__copyright__ = 'BSD License'

Is that BSD-with-attribution or revised BSD?
Title: Re: [Python] BnFTP File Downloader
Post by: topaz on October 15, 2006, 04:25 PM
Quote from: Kp on October 15, 2006, 04:18 PM
Quote from: topaz on October 08, 2006, 09:51 PM
main.py

__author__ = 'topaz'
__copyright__ = 'BSD License'

Is that BSD-with-attribution or revised BSD?

http://www.gnu.org/licenses/info/BSD_3Clause.html
Title: Re: [Python] BnFTP File Downloader
Post by: MyndFyre on October 15, 2006, 04:37 PM
Quote from: Joex86] link=topic=15108.msg159591#msg159591 date=1160395916]
That must be horrid on their processors for DWORD alignment, although the two bytes in the beginning followed by "icons.bni\x00" do line up to 8 bytes.
Have you noticed that most of the Battle.net protocol isn't word-aligned?

The thing about this protocol is that it's not like there's a time-sensitive requirement on it.  A nuclear reactor isn't going to fail, nor do we need to worry about a dropped frame if the structure isn't padded precisely.

The processor doesn't do anything while it's waiting for aligned memory.  The memory controller is what's doing the work for non-aligned memory.