P2P Project?

[Banana fanna fo fanna]

Would anyone be interested in working on my P2P project with me? I already have a swarming file download protocol created.

I'm using UDP for all communications so it willwork over NATs. When a peer is idle, it downloads random chunks of random files and contributes to the swarm.

I still need a way of searching for files; I'm thinking of a sort of spider, where a peer asks a bunch of other peers for files, and if it doesn't have one, it queries that peer's neighbors etc.

What do you think?

[Grok]

How does it prevent packet storms for requests?  I mean, if A asks B asks C asks D, how does D know to not ask A?

[warz]

I still need a way of searching for files; I'm thinking of a sort of spider, where a peer asks a bunch of other peers for files, and if it doesn't have one, it queries that peer's neighbors etc.

Looks to me like he hasn't created it yet. That'd probably be a future delimma though.

[dev invisible]

I still need a way of searching for files; I'm thinking of a sort of spider, where a peer asks a bunch of other peers for files, and if it doesn't have one, it queries that peer's neighbors etc.

If you have 4 users on.  They are all each others neighbors.  You send a request from A to B.  B doesnt have it so he sends requests to everyone except the person who sent it.  C and D both don't have it so they send requests to everyone but the person who sent it.  A, C and A, D both receive it.. A discards it because he sent it out in the first place... but the other two nodes keep asking each other in a circle.

Any means of avoiding this?  It seems the only way would for each node to know their neighbors neighbors and etc...   But with enough nodes it would still just cycle along.   Maybe have each node know the "hops" away from its neighbors.  When a node receives a search request, only send out to those nodes that are farther away from you than the .. i dunno just a dragging on thought

[Kp]

You could try adopting the solution IRC servers use: an acyclic spanning tree.  Essentially, all nodes advertise who they represent, and you don't talk with anybody who represents someone you've already met.  Again, this could become memory/CPU intensive on a large scale, so it may not be practical.  It works for IRC because it only applies to the server side, and the number of IRC servers on a given net is comparatively small.

[RyanIdium]

your using udp, connectionless right? what happens if the packet doesn't arrive?

[drivehappy]

your using udp, connectionless right? what happens if the packet doesn't arrive?

That's what I see a problem with. I suppose he could try to have some sort of ID for each data chunk on the file, the recieving end then could ask for a chunk if it doesn't have it (and keep asking if it still doesn't recieve it).

Searching could be done as he suggested, tag the packet with something like a GUID and any peer who's searched that ID within the last 5 minutes can ignore.

[dev invisible]

... the recieving end then could ask for a chunk if it doesn't have it (and keep asking if it still doesn't recieve it)....

I smell DoS

[RyanIdium]

or you could make a better bit torrent. call it YaBT.

[Banana fanna fo fanna]

Nah

Send one data packet with an ID, send ack, send next data packet, etc.

[drivehappy]

... the recieving end then could ask for a chunk if it doesn't have it (and keep asking if it still doesn't recieve it)....

I smell DoS

Well, say after 3 fails on an ID the program calls it quits and tells the user their connection sucks.

[MyndFyre]

What if in the requests, we propogate who is asking and who has been asked:

For example, in a four-client network, A asks B asks C asks D.  Well what prevents C from asking A as well?

In a little network simulator I put together to emulate a network developed by a grad student, each request carried with it the nodes that it has already visited.  So, say we include a table:

Req: A
A: NA
B: NO
C: NO
D: NO

So, A asks B:
Req: A
A: NA
B: NO

B passes to C, since it knows not to pass to A.

B asks C:
Req: A
A: NA
B: NO
C: NO

C passes to D, since it knows not to pass to A or B.

C asks D:
(Let's say D has it)
Req: A
A: NA
B: NO
C: NO
D: YES

Rather than transferring back through the other clients, since there's a Request Originator (Req) included in the table, D connects directly with A.

Just a thought about how to solve the problem posed above.  

[Arta]

What if there are 100,000 nodes?

[MyndFyre]

What if there are 100,000 nodes?

Then the network needs to adapt and provide a better solution.  

I believe the way that KaZaA does it is similar to what I just described, except that rather than each computer acting as its own node, some computers act as a SuperNode.  Each SuperNode tracks files that are shared on each computer connected to it, and so if we have a factor of 50:1 (computers to supernodes), we can have 1,000,000 computers to 20,000 supernodes (which is still excessive, but if you had 100,000 clients as in your example, 2,000 is not terribly excessive).

If we instead make each SuperNode a dedicated node (rather than a client to the network), we can increase load much, much more without hurting someone's performance -- probably to a factor of 200:1 or more.

If the SuperNode is set up properly -- with indexing inside of a database, for example -- then there shouldn't be a terribly horrific problem with usage on the server, because the only communication it would be doing is the coordination of client-to-client requests, not actually transferring data.

I admit, I'm a I33t n3wb when it comes to programming stuff like this, but I don't think that this is a terribly inefficient system, either.

[edit]
Once you hit a million users, the chances of something not being available gets to be pretty slim, and only the searches for rare media will propogate throughout the entire network.  Just another consideration.  
[/edit]

[Banana fanna fo fanna]

My favorite has to be the flat P2P idea. Google for it, it was on the p2p-hackers list.