Im just curious as me and my friends may attempt this but with your current knowledge would it be possible to make a bot that crawls bnets channels somewhat like the googlebot crawls the web?
Googlebot? Try being more specific
huh? u mean channel to channel retrieving user info or just chan 2 chan
Finding all the channels and indexing them, say putting them onto a website as a list... like bnet does with wc3 users..and keeping them current and updated. Maybe going as far stating how many users are in each channel @ the time it went there.
Googlebot - is google.com's bot that moves from website to website gathering information.
Receiving a complete private channel list is impossible.
But there are no links to crawl...
Quote from: NuLL on March 15, 2004, 08:25 PM
Receiving a complete private channel list is impossible.
There is another way of going throught a list of channels someone has made.
uh elaborate , receiving a complete server side channel list is AFAIK impossible.
if ur really desperate theres a really slow and boring way of doing it...
Quote from: Ecelectic on March 15, 2004, 08:20 PMFinding all the channels and indexing them
have ur bot do a /whois command on a bunch of possible channels listed in some dictionary
so...
have ur bot do a "/who a" and if it gets a msg like "That channel does not exist." then dont do anything, if it gets a different msg then the channel exists, and u can save it to a text file or something. after that continue thru the alphabet, once getting to "/who z" start over and go 2 digits "/who aa" "/who ab" and so on. repeat the process till ur done "/who zzzzzzzzzzzzzzzz" somewhere along there u would have found actual channels. dont forget to add numbers and stuff in the rotation and do it over again for op and clan channels. having multiple bots doing multiple parts of the dictionary would speed things up a bit but not much.
(like i said, slow and boring, not to mention not worth the effort)
Quote from: Ecelectic on March 15, 2004, 08:20 PMMaybe going as far stating how many users are in each channel @ the time it went there.
if the channel exists (and ur not banned) u can count the users in the channel from the list bnet sends u from the /who command.
this is a really disgusting way of doing it if u ask me but right now its the only method i can think of.
In other words a brute force approach on finding all possible channels. Doing it this way could take days, and would most def. be inaccurate by the time it finished... a better, yet still slow way, would be to have a pre-defined list of known channels that usually always have people in them...
Not sure what the purpose of this is, but you could roam public Warcraft III channels and make note of every Warcraft III user that's in a clan, that would get you list of clan channels. As you get more clan channels, you would also add those to the list of channels to roam to find more people in clans. In an hour you'd probably collect over 100 channels.
Quote from: St0rm.iD on March 15, 2004, 08:25 PM
But there are no links to crawl...
profile a user and a extract channels?
most clan members mention their home channel
in their profile..
whether it be "Op " or "Clan "
That would be the closest thing to a link.
And/Or you could setup so
the bot "analyses" the username
and generates a list of possible channels
based on the username's tag if any.. 2 to 3 letter tags.
I don't know if it will work well..
but it is an option to consider
Quote from: Eli_1 on March 15, 2004, 09:53 PM
In other words a brute force approach on finding all possible channels. Doing it this way could take days, and would most def. be inaccurate by the time it finished... a better, yet still slow way, would be to have a pre-defined list of known channels that usually always have people in them...
It wouldn't take
days. Doing it this way would take years and eons. It seems probable that you wouldn't be done before the sun runs out.
One idea I've seen for a follower bot (i.e. automatically trails you as you roam, but does not crawl independently) is based off the fact that mutual friends always know where each other are. So make the bot a mutual friend, have it "/f l" whenever you leave channel, and then use the resulting friendslist to track you down. Not exactly a channel crawler, but it is an automated way of locating channels not known in advance.
I would just have a database of channels. Users could submit via the web or command. Although you would either have to sort through these using what's real/what's not, or if the bot joins the channel and it's empty, it deletes that channel and goes about it's business.
:edit: grammar/punctuation.
As adron said, it would take more then days to obtain a list, even then by the time your finished the list will be inaccurate. There are also unlimited combinations for channels, a three digit channel has afew
example:
123
231
321
312
213
132
image even a six digit channel. May i also ask what the purpose of this is? Why would one want to know all the channels?
Edit:
You also have people that hang in ASCII channels, i think obtaining such a list is virtually impossible unless ofcourse you ask blizzard. But they will respond with something like: "We can not give you this list due to the privacy act." Something like that, believe me ;)
It's not unlimited.
Channels are limited to 31 characters total.
The reason why Adron was against this
was because the number of
possible of combinations is daunting
and you would have to be out of your mind
to attempt something such as this.
Permutations.. ouch.
Well if you looking specifically for clan channels, you could easily idle a bot in "Clan Recruitment", it could record text while also take profile's of people who join the channel.....
It's the most practical approach
Wouldn't be THAT hard if you were only looking for clan channels and you had a decent amount of keys and proxies at your disposal, since clan channels can only be 2-4 characters long.
Quote from: LoRd[nK] on March 19, 2004, 09:11 PM
Wouldn't be THAT hard if you were only looking for clan channels and you had a decent amount of keys and proxies at your disposal, since clan channels can only be 2-4 characters long.
Clan channels must be >= 7 characters long
Quote from: UserLoser. on March 19, 2004, 09:24 PM
Quote from: LoRd[nK] on March 19, 2004, 09:11 PM
Wouldn't be THAT hard if you were only looking for clan channels and you had a decent amount of keys and proxies at your disposal, since clan channels can only be 2-4 characters long.
Clan channels must be >= 7 characters long
Yes, but since "Clan " never changes, it massively affects the number of possible combinations.
You could make something similar to a database of all channels visited by the bot, and recording other various information about that channel, similar to a list that records every user the bot makes contact with.
Tested my method today, for 10 minutes on USEast and 10 minutes on USWest. Here are the channels the bot collected over that time period:
USEast:
clan eG
clan ZiXz
clan FF4e
clan NoA
clan RIP
clan Tos
clan Ez
clan Kali
clan GRRR
clan NaMe
clan NWTR
clan TiTz
clan NAoC
clan ShAs
clan SkY
clan FaWo
clan Unit
clan UCA
clan Holy
clan GM
clan xTFx
clan xCKx
clan DKOG
clan PIGS
clan wwp
clan fg
clan oPKo
clan xEPx
clan AnR
clan b2rk
clan LoH
clan SGL
clan AnTi
clan DsFx
clan EFX
clan ub3R
clan FwH
clan XxX
clan PVVP
clan EDyn
clan Maps
clan TtHb
clan xxBA
clan SeX
clan MRCY
clan FARP
clan bZrK
clan JD
clan LotF
clan seeD
clan xKAx
clan GanG
clan 46n2
clan xRx
clan SOTA
clan SNiP
clan PotM
clan XXXX
clan nswt
clan gr
clan AiX
clan X69X
clan EW
clan PoC
clan AGTS
clan Roma
clan AM
clan SPFR
clan DLx
USWest:
clan GAZ
clan TidE
clan RTG
clan tha
clan SLAY
clan AUSN
clan DotL
clan FWG
clan KOCL
clan LoVE
clan EnT
clan xP
clan BoA
clan iaC
clan GaNg
clan LGoC
clan FAK3
clan kota
clan BloG
clan aWp
clan DGNB
clan Wolf
clan UDNE
clan FAKE
clan RPE
clan GotL
clan SBLM
clan HYPE
clan REFI
clan GIRL
clan caps
clan nKh
clan DaG
clan GDS
clan DRAG
clan TOXI
clan KgZ
clan PnY
clan BsKr
clan rT
clan AUSA
clan LGnD
clan EmTs
clan GHO
clan POJO
clan KotU
clan EotB
clan 0zR
clan duma
clan SK8
clan xL
clan WMD
clan WCRs
clan MTD
clan Syn
clan SKK
clan Bms
clan ansn
clan TDO
clan DARK
Quote from: FuzZ on March 16, 2004, 04:18 PM
I would just have a database of channels. Users could submit via the web or command. Although you would either have to sort through these using what's real/what's not, or if the bot joins the channel and it's empty, it deletes that channel and goes about it's business.
:edit: grammar/punctuation.
That's what I was going to say :)
That, combined with Spht's idea, would get a reasonably big list :)
I actually made this exact bot, heres how it worked:
It would start with a small set of reliable feeder channels, these need to be high and diversly trafficed, i used clan w@r, Open Tech, Town Square and sex.
multiple bots would idle in these channels listening to the chat and parse what was said for possible channel names using a set of pattern matches.
Upon finding a possible channel the bot would inspect its list of known channels, if the channel was not charted, it would go to it and idle awaiting chat. It had a timeout of like 30 mins in which it would return to its feeder channel.
If the new found channel was active, it would go from there to new channels apon hearing them, then return to the secondary channel if idle.
This worked pretty well, i got a list of over 500 channels as i recall before i lost it due to a corrupt harddrive (ReiserFS can die please)
anyway thats my .02$
Hmm.. I personally would set the timeout rate a little lower.
question though, did it successfully parse channels with spaces?
also, I would have a bot just designated to the use of joining the channels found (This way you don't miss any possible other channels from the 'feeder bot')
I'll be willing to host/write the php script to add a channel to a MySQL database which will be available to the public through another script if someone is willing to write the bot ( I have enough bot projects going )
Reply/IM/PM/Email me~
:EDIT:
I've been going over this idea, it would work well, i'll probably need some help designing/writing some of the php if anyone is interested in doing this
OR! Since blizzard.com publically posts updates on clans all over battle.net why not just make "googlebot" that roams around on the war3 pages on blizzard.com? I am sure there is a link on each page to take and add. All the war3 channels are public and able to be seen...
ok, and how do you intend to get op channels from Blizzard.com ??
it didn't get added because no one from Clan zer0 joined the channel while the bot was present?
it might be handy for someone that's new to battle.net or someone that's looking for a clan to checkout a page that has a list of channels.
I suppose they could use blizzards Clan pages but it would leave out op chans along with other chans.
All in all, it's rather pointless.
Why don't you just release a chatbot that reports to a main server for every populated channel it finds?
I think Sphts idea is smart.
Quote from: Fr0z3N on March 30, 2004, 11:30 PM
I think Sphts idea is smart.
I concur.
Make it like those illegal scanners, except join the channel
Then get the statstring for each war3 player, if they have a clan, get the clan, add the clan to a text document.
Quote from: Adron on March 16, 2004, 11:14 AM
Quote from: Eli_1 on March 15, 2004, 09:53 PM
In other words a brute force approach on finding all possible channels. Doing it this way could take days, and would most def. be inaccurate by the time it finished... a better, yet still slow way, would be to have a pre-defined list of known channels that usually always have people in them...
It wouldn't take days. Doing it this way would take years and eons. It seems probable that you wouldn't be done before the sun runs out.
I was bored in chemistry yesterday, and I decided to try to figure out how much time it would take. I threw away my notes on it so I'll just try to type my results by memory :P.
Variables:
The idle time was equaled to 4 seconds in this
The max amount of characters in a channel was 31
The test uses all 255 chars, so it could be cut down to take less time if you took out using capital letters.
I might have done this wrong, but this is how I did it:
255^2 * 31 = combos
combos * 4 = amount of time in seconds
then divide to get the amount of months...
If nothing changes, and nothing goes wrong, it would take ~3.1 months to do this with 1 bot doing all the work.
So this idea isn't really that far-fetched if you have say, 10 CDKeys and split the dictionary (which will be frickin huge, btw) into 10 different sections. Also, the idle time could be made smaller, and so could the dictionary considering no one in their right mind uses all 31 chars. for a channel (I think 15-20 would find almost all channels)
I'm afraid your math is quite wrong. It would actually be
255 ^ 31 = 4.0063196158689474245553792846195e+74 combinations.
Assuming your 4 seconds / combination, it would take 1.6025278463475789698221517138478e+75 seconds.
Which is 2.6708797439126316163702528564117e+73 minutes
Which is 4.4514662398543860272837547606861e+71 hours
Which is 1.8547775999393275113682311502859e+70 days
Which is ~5.0781043119488775123017964415767e+67 years.
Now, the universe is, rounding up, 20 billion years old. Let's see the universe is going to last a total of 200 billion years, which is a pretty high estimate. If that's true, it will take
2.5390521559744387561508982207883e+56 times the age of the universe to complete.
omg wow :o! Thanks for correcting me, I was all getting ready to make one :P.