Image recognition/processing (VB6)

[Tazo]

Ok, this is a pretty complex problem. I currently use PostMessage to send mouse clicks to Internet Explorer to manipulate a game. However, every now and then (it's random), a "security check" will show up. It displays an image with boxes in it and asks you to click a button corresponding to the number of boxes (see http://img106.imageshack.us/img106/1058/boxesgj6.png). I already have the child handle of the IE window I need (obviously), but first I need to know how to either retrieve the image or the image's name/location on the server. After that, I need VB to process the number of white squares.

I am assuming that the game tells  IE to re-dimension the image because if I open the image's location, it is only an 8x8 bmp (see http://img106.imageshack.us/img106/9905/1185938089iw6.png). They are actually bitmaps but imageshack conveniently made it a png.

I am not sure if the server changes these images, so it'd be best to retrieve it directly from IE rather than comparing saved security data with the image's name (ex. 37366123.bmp).

Any help on either of these 2 tasks is appreciated.

[Spht]

Task 1, if I understand correctly, is to download the bitmap from a website? You seem to already know where to find the image, so this is relatively easy.  All you need to do is connect to the site and save the binary file to memory, which can be accomplished in a few lines of code.  See Google

As for task 2, since it's a 8x8 black and white bitmap you're analyzing, and each box is one pixel, this is very easy. The easiest method is to just count the amount of 0xffffff (white) in the file after the header, and that's how many white boxes/dots there are

[Banana fanna fo fanna]

See also ImageMagick

[Tazo]

Task 1, if I understand correctly, is to download the bitmap from a website? You seem to already know where to find the image, so this is relatively easy.  All you need to do is connect to the site and save the binary file to memory, which can be accomplished in a few lines of code.  See Google

As for task 2, since it's a 8x8 black and white bitmap you're analyzing, and each box is one pixel, this is very easy. The easiest method is to just count the amount of 0xffffff (white) in the file after the header, and that's how many white boxes/dots there are

I actually can't retrieve where the image is stored. Finding the image's location was listed as an objective above. I found the one listed above by manually right clicking.

Also, I'm planning on using GetPixel on the image since it's only 8x8 and counting the number of white pixels.

Once I find a way to grab the image from internet explorer or the image's location on the server, I will be able to accomplish the original goal. My current methods of being able to do this are anything that involves the hWnd, so I can do it using SHDocVw (Microsoft Internet Controls) or by using post/sendmessage/any API call that requires hwnd. I already use SHDocVw to retrieve status bar text.

[Banana fanna fo fanna]

I would fetch the web page using HTTP and grep for it and submit the form yourself.

[Tazo]

I would fetch the web page using HTTP and grep for it and submit the form yourself.

Put simply: can't.

[l2k-Shadow]

if IE can't do it, why can't you?

EDIT: meant if IE can do it

[Tazo]

if IE can't do it, why can't you?

The game requires logins and has a ton of frames and shit.. see for yourself, www.racewarkingdoms.com.

I can, however, (I believe) retrieve all of the html using SHDocVw, seeing as I read the status bar text fine doing this. But, like I said, the page has frames and I'm not sure how to figure out how which image is the security one (if it is present. This method would not only check if a security image is present but attempt to read it, too).

EDIT::

OK. Viewing the innerHTML gives me this

Code Select Expand

<FRAME name=main marginWidth=0 marginHeight=0 src="start.htm" noResize><FRAME name=poll marginWidth=0 marginHeight=0 src="poll.htm" noResize scrolling=no><FRAME name=poll2 marginWidth=0 marginHeight=0 src="pol.htm" noResize scrolling=no>
Frames.

Code Select Expand


    Dim SWs As New SHDocVw.ShellWindows
    Dim IE As SHDocVw.InternetExplorer
    Dim objBody As HTMLBody
   
           
    For Each IE In SWs
        If IE.hwnd = hWndParent Then
            Set objBody = IE.Document.body
            Debug.Print objBody.innerText
        End If
    Next

this is how the website posts the security check

Code Select Expand


function security(secnum)
{
top.frames.main.skipform.action.value="security";
top.frames.main.skipform.target.value=secnum;
pollzero(top.frames.main.skipform,0);
}

(http://racewarkingdoms.com/v.txt)

So, basically, I need to extract frame "main" and it's html/images using SHDocVw. Open to all suggestions.

[Tazo]

Alright, I've managed to extract the frame I'm searching for ("main"), I believe. But every time I try to display the innerHTML or src so I can search for images, I get run-time error 438, object does not support this property or method.

Code Select Expand


    Dim SWs As New SHDocVw.ShellWindows
    Dim IE As SHDocVw.InternetExplorer
    Dim objDoc As HTMLDocument
    Dim objFrame As MSHTMLCtl.HTMLFrameElement
    Dim i As Integer
           
    For Each IE In SWs
        If IE.hwnd = hWndParent Then

            Set objDoc = IE.Document
           
            For i = 0 To objDoc.frames.length - 1

                If objDoc.frames.Item(i).Name = "main" Then
                    Set objFrame = objDoc.frames.Item(i)
                    Debug.Print objFrame.src                                     'both of these statements throw the error
                    Debug.Print objFrame.innerhtml
                End If

            Next i
           
        End If
    Next

If anyone could tell me what the problem is or what I'm doing wrong, it'd be greatly appreciated.

[Tazo]

problem solved after many many many failed attempts

Code Select Expand


    Dim SWs As New SHDocVw.ShellWindows
    Dim IE, fra As SHDocVw.InternetExplorer
    Dim fDoc, doc As HTMLDocument
    Dim x As Integer

    For Each IE In SWs

        Set fDoc = IE.Document
       
        For x = 0 To fDoc.All.length - 1
            If TypeName(fDoc.All(x)) = "HTMLFrameElement" Then
                Set fra = fDoc.All(x)
                Set doc = fra.Document
                Debug.Print doc.body.outerHTML
            End If
        Next
   
        Set fDoc = Nothing
        Set fra = Nothing
        Set doc = Nothing

    Next