Saturday, February 12, 2005

Server Re-tooling and upgrades

The servers we deploy within our efforts are being upgraded. More capacity, more utility, more focus. Hence, some of the pages won't be accessible for a few days, perhaps.

Thanks for your understanding.

It is okay to email me at:
Call: 412 298 3432.


Mark Rauterkus said...

That site was suspended for excessive php processes. Due to your circumstances, we have unsuspended his site in good faith. Please make sure you are not running any scripts that require excessive processing power to run.

This site was causing performance degredation for other users on the server. Sorry but if the server load once again increases due to the scripts on this site, we will have to suspend to reduce server load.

This site is now active.

Mark Rauterkus said...

We're breaking things. I love it. :)

I'd like to clarify that brief note I sent last night.

First, the server problem resulted from me repeatedly trying to convert
the Wiki to static HTML files. I'm having trouble getting the WGET
utility to behave properly in the following ways.

1) Excluding links: The utility tries to follow every link on every page -- including things like "history" and "edit". This means that the Wiki is converted into over 2000 files, when it should be closer to 200 files. I've had trouble telling it exactly which links to follow. As a result, the conversion takes a couple of hours and apparently places a huge load on the server (more on that later)

2) Making relative links, so that a local version works: I thought that
I had this figured out, and I have some small snapshots of the Wiki
where this does work. However, when I ran the real thing, backing up the
entire wiki, this didn't work. We may be able to look at these files and
fix the problem. I don't think that it is feasible to retrieve all of
the data from the server again due to the overload problems.

I have composed the following note for tech-support. It explains the
source of the overload and suggests ways to avoid it:

Dear Tech Support,

Thank you for reactivating my account the other day. I am sorry that I
did not consider the effect that my behavior would have on other users
of the server, and I will not make that mistake again.

I am using the server to maintain a wiki, which stores information in a
database and converts it to HTML as the users request information. The
problem arose because I was attempting to convert the wiki to static
HTML for use off-line. A program on my home computer was automatically
following all of the links on the webpages in order to copy all of the
pages. I would like to know if this can be done in a way that avoids
overloading the server.

The first solution that I can think of is to introduce a delay between
requests for data from the server. Would 30 seconds be enough of a delay?

I could also try making the backup late at night and early in the
morning in order to avoid high traffic periods.

Anyway, I will not make another backup in the near future, and will wait
for your suggestions as to how I can do it without overloading the
server. If this is not an acceptable use of the server, I can probably
figure out how to make the conversion on my home computer, but it would
be a complicated process for me.

Thank you,