Can CIC handle this?

[Sphynx]

In an effort to promote WC and encourage companies out there to produce decent space combat sims, I've been doing some publicity for what's going on here at the CIC, but I want to make sure I'm not getting us in over our heads.

I need a consultation with the people who really know about the CIC and what it is able to handle as far as hits and download requests are concerned. I told one of my friends, a hardcore programmer, about Standoff, WC Saga, and the Privateer remake, and he sent me this reply:

"What news sites have you posted this to? There's one I read frequently
(slashdot.org) that I'm tempted to submit this to. I'm hesitant to post an article there if you're not ready to handle about a half million hits and software download requests in the space of about an hour or two. If you're not familiar with the "Slashdot effect" then let me assure you that having it happen to your
webserver is a bad way to find out about it.

On the topic of handling bandwidth issues, I've looked into the big
BitTorrent tracker sites, and many of them have essentially shut down
until the legal issues are worked out. The good news is that if you're
interested you can set up a tracker on your own. There are several
options available:

The original bittorrent tracker software, written in C and Python:
http://sourceforge.net/projects/bittorrent/
http://bittorrent.com/

BNBT, a port of the original bittorrent software to C++, claims to be
faster:
http://sourceforge.net/projects/bnbt/
http://bnbt.depthstrike.com/

BNBT EasyTracker, a port of BNBT to windows with easy-to-use installer:
http://sourceforge.net/projects/bnbteasytracker/
http://bnbteasytracker.sourceforge.net/

ByteMonsoon, a torrent tracker written for PHP and MySQL (fits in well
with a LAMP (Linux/Apache/MySQL/PHP) setup):
http://sourceforge.net/projects/bytemonsoon/
http://bytemonsoon.sourceforge.net/

Extended BitTorrent Client and Tracker, another C++ client/tracker, uses
MySQL as secondary storage (needs a PHP front end):
http://sourceforge.net/projects/xbtt/
http://xbtt.sourceforge.net/

From my poking around to make the list above it appears that
registering your files with the tracker can be a pain, so there's a tool
called MakeTorrent that runs under either windows or Linux to do the
work for you:
http://krypt.dyndns.org:81/torrent/maketorrent/v1.html (Linux)
http://krypt.dyndns.org:81/torrent/maketorrent/index.phtml (Win32)


I can tell from looking at Netcraft.com that you're using Apache on
Linux, so most of these should be simple to implement. If you have a
windows box available then EasyTracker will pretty much set itself up
for you.

I just don't want to send a rabid horde of Slashdot readers at your page
if you're not ready for it. I'd host the torrents myself, but I don't
have a computer I'd feel good about hosting it on (I bring my work
computer home at night, and my home connection is a bit shaky).

I guess what I'm really getting at is this: do you want me to hold off
on submitting an article on Slashdot until you can brace for the impact,
or should I go ahead and post it now? Let me know. Thanks for letting
me know about this, I think it's really exciting that you've managed to
pull this off!

-Vaughn

PS - If you're already aware of Slashdot and BitTorrent, please don't
think I'm trying to be condescending or anything like that; I just
haven't had time to talk with you much about this stuff, and I don't
know where you stand. This letter is probably too much information for
someone who's already well informed and too little for someone who's
not, so please accept my apologies either way. Feel free to email me
with requests for clarification/polite requests to keep my meddling to
myself. =)"


So, what do you think, should I have him post it on slashdot.org? It sounds like he expects the CIC to have a major influx if he does, and he's worried about whether the site can handle it.

I wanted to consult with those of you who know before I say "yea" or "nay" to him. From what he's told me, though, he thinks this community might double in size if he posts it there. That could be good or it could be bad. If the CIC WC games have enormous numbers of downloads, that's a great message to game designers that space flight sims are worth their time and money investments. However, we don't want a page that is overloaded where downloads are impossible.

Hence, I'm not willing to make the decision without your input.

 

[KrisV]

I wanted to consult with those of you who know before I say "yea" or "nay" to him.

This is the sort of thing that should be discussed by e-mail (news@wcnews.com or any of the other addresses on our contact page).

That said, the CIC has been mentioned on Slashdot's frontpage before. The many thousands of page requests were handled just fine, until hundreds of Slashdot visitors found their way to our files and music sections. So all downloads would be suspended as a precautionary measure if we were ever Slashdotted again.

You would definitely need to use bittorrent for the downloads, since no normal file server could possibly handle all those file requests. We could consider seeding from the server, but the tracker would have to be hosted elsewhere. If someone wants to set up trackers at Filerush or elsewhere and get some seeders organised, then we'd welcome the exposure.

 

[Sphynx]

Thanks for the guidance. I wasn't sure of the protocol for this sort of thing. I'll have him contact you guys through e-mail.

 

[PeteyG]

I think you're a lot less likely to get a server-crushing slashdot effect if the story is only posted in the 'games' section, which isn't displayed by default for most slashdotters.

 

[TC]

The CIC's been in the games section before, it would still kill things if everyone started hitting the large downloads.

 

[Worf]

Azureus is a very nice bittorrent client that can serve as a tracker too. Its only real drawback is that it uses Java, so is a bit icky to get started. In fact, one of the uses is that Azureus can "poll" a BT tracker for active clients. If there's an active client and it can seed, it'll connect and start seeding. If not, it sits idle and sees if other files need serving. Now, for day-to-day use it's just a regular server, but when it gets busy, it'll serve up a limited set of files until it's seeded properly, then switch to another set.

Then again, slashdottings of this site happen so infrequently it's easier to just manually shut down the server. And some "Games" articles show up on the main website (like the 10th year anniversary of WC3). And the CIC handled it just fine.

 

[ChrisReid]

Then again, slashdottings of this site happen so infrequently it's easier to just manually shut down the server. And some "Games" articles show up on the main website (like the 10th year anniversary of WC3). And the CIC handled it just fine.

I was going to point that one out. The WC3 anniversary at Games.Slashdot last month was the first slashdotting since our most recent server upgrade, and we handled that without having to really take anything offline and without any interruption in service.

 

[Sphynx]

So, are you saying that if he posted it in the games section, we should be fine, or that he could even post it on the main page and we'd still be fine?

 

[KrisV]

In either scenario, we can't serve those large files.

 

[Sphynx]

I've got you. It would be too much, then, unless we can get something to handle the downloads, right?

 

[KrisV]

Right.

 

[Worf]

The easy way to handle it is to disable the large file for 36 hours after it's posted on slashdot. After that, re-enable it - the click-and-read everything crowd would be gone, and the more interested ones will wait around until the file's re-posted.

 

[Sphynx]

Questions:
1.) Does anyone have the resources and knowhow to set up a bittorent (or however you spell it) service for CIC.
2.) Would the people who run this page be willing to go along with Worf's suggestion?
3.) Ultimately, what should I tell my friend: "go" or "no go"?