Project Overload

\n studio-striking\n

Message boards : Number crunching : Project Overload
Message board moderation

To post messages, you must log in.

AuthorMessage
BarryAZ

Send message
Joined: 20 Jun 11
Posts: 34
Credit: 5,041,652,374
RAC: 6,756,735
Message 2106 - Posted: 5 Jan 2012, 18:31:15 UTC

I suspect a number of folks here have noticed (but not commented on) reporting lags and validation lags that have become the norm over the past week or so.

Part of that may have been a batch of new users attracted by the 'Santa Claus' credit boost (which has now been corrected it seems).

Another piece of the suspected load increase might be folks temporarily migrating from MilkyWay as they move to a new server and then figure out what it is they need to do to have it communicate properly.

Unlike MW though, Moo is a very light project in terms of support and communications people (person singular?) so we shouldn't expect to see much information here about what is going on.

Further, even more than MW, with the relatively low user count here, traffic on the message boards tends to be quite sparse.
ID: 2106 · Rating: 0 · rate: Rate + / Rate - Report as offensive
John Clark

Send message
Joined: 27 Jul 11
Posts: 342
Credit: 252,653,488
RAC: 0
Message 2107 - Posted: 5 Jan 2012, 21:23:59 UTC

Barry

I am quite surprised to see fairly regular posts here, despite the low cruncher numbers compared to other projects.

I think you are right about theSanta Claus bonus, now becoming a distant memory. At least Milkyway is up and running again.
ID: 2107 · Rating: 0 · rate: Rate + / Rate - Report as offensive
Profile Zydor
Avatar

Send message
Joined: 5 May 11
Posts: 233
Credit: 351,414,150
RAC: 0
Message 2111 - Posted: 5 Jan 2012, 23:51:13 UTC

Teemu provides a good service as admin, particularly noteworthy as there is only one physical server. He always jumps on the truely significant issues where possible, leaving the not important / cosmetic / epeen for later updates. It is one of the most stable projects in BOINC with rare downtime. There are a couple of significant issues he cannot do anything about (aka fragmentation from Upstream, incorrect driver timing reporting, and CPU loading in later 11.XX drivers; the latter two AMD have resolved in 12.XX ).

Recently the incoming wave of credit chasers resulting from the Santa bonus stretched the server to its limit - 50% more than its normal load, and it took the hit well, until the last few days when it filled up to capacity. Its eased off 20% and still reducing now that Santa's tail lights are a dim memory, so I anticipate back to mormal good service as chasers move on.

There is always light traffic on the boards in non credit chaser times as there is usually little of genuine Project related significance to yack about - a good sign :)

Regards
Zy
ID: 2111 · Rating: 0 · rate: Rate + / Rate - Report as offensive
Profile Teemu Mannermaa
Project administrator
Project developer
Project tester

Send message
Joined: 20 Apr 11
Posts: 388
Credit: 822,356,221
RAC: 0
Message 2116 - Posted: 6 Jan 2012, 11:29:14 UTC - in response to Message 2111.  

I suspect a number of folks here have noticed (but not commented on) reporting lags and validation lags that have become the norm over the past week or so.


I've noticed them too. :) I've done some configuration changes to hopefully ease the load but like you noted, these problems most likely correct themself now that Santa Magic is vanishing completely today. (10% increase still left.)

One change was to switch to FastCGI so that the Apache doesn't need to spawn so many processes all the time. Second was to switch to matchmaker job selector in scheduler, which doesn't scan a huge array like the previous, deprecated, selector did.

There's still some investigation I need do to see where the scheduler spends it time and why it crashes from time to time.

..communications people (person singular?) so we shouldn't expect to see much information here about what is going on.


Yeah, it's pretty much me only. I'd appreciate any help, though. Especially on following this forum. Some of you have done a good job already here (answering questions or pinging me for any serious problems), so a big thank you for that!

Teemu provides a good service as admin, particularly noteworthy as there is only one physical server.


Thanks! :)

I actually have three physical servers that I can use for Moo related things at the moment. One primary one, secondary that currently only has the DB replicated to for backups and a third one used for big ro DB queries.

I plan to move main DB to the secondary server leaving the primary server to only handle the backend jobs and answering to BOINC Clients. This way memory usage is spready more evenly. I'll need to move the DB backup to the third server because otherwise that'll block the main DB.

-w
ID: 2116 · Rating: 0 · rate: Rate + / Rate - Report as offensive
BarryAZ

Send message
Joined: 20 Jun 11
Posts: 34
Credit: 5,041,652,374
RAC: 6,756,735
Message 2124 - Posted: 6 Jan 2012, 18:45:33 UTC - in response to Message 2116.  

Teemu -- thanks for the extensive reply -- it is definitely appreciated.

I expect load here will back off -- not only with credits returning to normal, but also with MW going back online.

Also, POEM is just about to have production running GPU support as well (currently it is supporting folks willing to do some additional handling so that GPU processes for them). Though over at POEM I expect they may lament the change over, they already had some problems having enough work to hand out, and with GPU support, that available work may really be a constraint.

There was a temporary burp here about an hour ago (9:30AM Pacific time in the US) -- I suspect you were doing some work on the servers over there.
ID: 2124 · Rating: 0 · rate: Rate + / Rate - Report as offensive

Message boards : Number crunching : Project Overload


 
Copyright © 2011-2024 Moo! Wrapper Project