Remote Bot Linking

[Smarter]

I am currently working on Nerve a current project of mine, and I am attempting to create a system for linking bot's to share necessary information (such as a Global Ban Queue, "shitlist", etc.). After speaking to numerous people about the best method in which to link the bot's (IP > IP, Server, IRC, etc.), I have decided to have all bot's connect to a mutual server which would hold for example a Global Ban Queue, where each bot would add to the list for whichever reason (detected a load, loadbanning command, etc.), the Server would be sure to check for dupes of names as it received it. Each bot would request periodic updates to their local banQueue where they would specify an amount of user's to update by each time (locally set), at which time the server would automatically remove each name from the banQueue believing they have been banned. The server will also use a callback to start banning among all bot's for example:

If Bot1 was to request an update, but the server returned null, because it's banQueue was empty at that instance, the bot wouldn't request again as it would assume the banning is done, but if Bot2 was to detect a load and add users a second later, the server would initialize banning the second it starts receiving new additions to the banQueue .

Example Coding:

Code Select Expand


        public Dictionary<string, int> Access = new Dictionary<string, int>();
        public List<string> banQueue = new List<string>();

        public Form1()
        {
            InitializeComponent();
        }

        public string[] getNext(int Amount)
        {
            List<string> temp = new List<string>();
            for (int i = 0; i <= Amount; i++)
            {
                temp.Add(banQueue[i]);
                banQueue.RemoveAt(i);
            }
            return temp.ToArray();
        }

        public void addUser(string user)
        {
            if (!banQueue.Contains(user))
            {
                banQueue.Add(user);
                //Call banMe Callback.
            }
        }
    }


What i'm looking for is any comments/suggestions that would make this system more secure and efficient? :-D

[Mystical]

uhm.... so basicly same thing as botnet, n have differnt passworded databases. duh.

[Smarter]

No, this is a private server, to link groups of bots, not all bots? It would be packaged with the Bot's exe.

[Mystical]

thats why botnet has seperate databases? private server? how would you and someone else link your bots then?

you build one server, have certain bots login to a certain database with the supplied password.

[Smarter]

Why would I be so nice as to host one big server for anyone who uses the bot to use, this bot is actually made for personal reasons lol but I'm going to release it, and I don't want to take the time to code databases and all that shit, so it's easier to let people run local servers and link just the bot's they wish together .

[l2k-Shadow]

yay for C# servers

[Smarter]

yay for C# servers

What's wrong with that?

[BaDaSs]

Why not try a remote SQL server? One that runs on a 24/7 Web Server? Unless im going in the wrong direction...

[Mystical]

Why not try a remote SQL server? One that runs on a 24/7 Web Server? Unless im going in the wrong direction...

that'd work for like a bot database user list or somthing, but he wants his server to alse parse cmds, Uhm, Hmm, i think that should be done client side.

[Banana fanna fo fanna]

This is a basic shared state problem.

Just have a networked shared SQL (MySQL, Postgres, MSSQL, Oracle etc) database server that all the bots connect to and use. Instant shared state and guaranteed integrity. Also allows you to duplicate standalone functionality without changing your code. If you really want, wrap it in a web service.

If you need to push messages to different bots (to send announcements and the like), go with a message queue (http://en.wikipedia.org/wiki/Message_queue). If that functionality is too advanced/annoying, go with XMPP (Jabber, which is very similar to IRC). Keep in mind that if you go with Jabber you won't have any guarantees that your messages will be received if a connection fails (with a message queue server, you get this guarantee). http://www.foo.be/mqs/ looks interesting, though I haven't used it.

[Smarter]

That could work too, but what would be faster, MySQL Server (getting CONSTANT updates), or a server program?

[Camel]

Using non-prepared statements against any SQL database will probably be slower than a custom solution.

It's certainly possible that you could fuck it up, though.

[Hell-Lord]

MySQL Server would be the way to go since the data ain't static so a server application is going to be not so good performance wise.

[Banana fanna fo fanna]

That could work too, but what would be faster, MySQL Server (getting CONSTANT updates), or a server program?

Depends. Probably a custom solution would be faster. I dunno how fast MySQL's network protocol is or code...but probably faster than C# and probably pretty concise, however MySQL does a lot of stuff that your solution may not need.

However, I HIGHLY doubt you are going to be able to implement a foolproof ACID solution that is reasonably efficient in your spare time. If you don't know what ACID means:

Atomicity - each transaction (a conceptual block of statements that execute together, aka a group of create, read, update, and delete commands) either executes all at once, or none at all. You can have two writes, and if the first write concludes successfully but the second one fails, you are guaranteed that the database will roll back the previous write, and the database will look like nothing had happened (and be in a consistent state).

Consistency - one can set criteria that absolutely cannot be violated, ever.

Isolation - one transaction will never see the intermediate state of the database while another transaction is executing. That is, if  transaction writes to table X at time 1, and later writes to table X again at time 3, any transaction will not be able to access the data in table X at time 2 (row level locking may apply for maximal concurrency).

Durability - once the application has been notified that the transaction completed successfully, you are guaranteed that the data is actually there and is resilient to any subsequent system failures.

If you can do all of that while maximizing concurrency (aka not naively locking out more people than you need to), go for it. I'd suggest you have your bots directly talk to the database or wrap the database in XML-RPC calls or something similar.

Tangent:

In my albeit limited experience in the CS field, I've found that when presented with a problem it's best to,

a) avoid real concurrency if you don't need to use it. sometimes problems are better solved using conceptual concurrency rather than real concurrency; in that case, use cooperative "concurrency" (coroutines)
b) avoid sharing data among separate concurrent processes as much as possible (queues are your friend)
c) use message passing when you need different processes to communicate
d) when you need to share data, either you better be really fucking careful, or let someone else deal with the problem and put it in a RDBMS (if your application supports it).

See the Wikipedia entry on ACID.

[Smarter]

Well the custom solution idea I think is much more efficent than an MySQL server, also because the amount of data being transfered would overload a SQL DB, for example:

Bot1 Detects Load
Bot1 > Server: user1
Bot1 > Server: user2
Bot1 > Server: user3
Bot2 Dectects Same Load
Bot2 > Server: user1
Bot2 > Server: user2
Bot2 > Server: user3
etc. for each bot in the channel, each bot would be sending all these names to the server. The server of course would check for duplicate entrys and discard them, then each bot would at the same instance begin requesting queue:
Bot1 > Server: get(10)
Server > Bot1: user1, user2, user3, etc (to 10)
Bot2 > Server: get(15)
etc.

At which point the server would be removing these names from the queue as they are sent out, now thinking of the amount of data being sent at that speed, none of these solutions are sounding very efficient, although this is SMALL amounts of information that is being sent, so now you got me thinking lol.... hmm Anyone have an alternate solution?