Zydor currently has the top host ~ With what looks like Dual 5970's or Quad 5870s. !? Which is it.

I have two boxes with Dual 5970s and if 1200w+ Power Supplies weren't so darned expensive I'd put all 4 of them in one host and go for top computer.

I found cheap 5970s and think I might get a few more but not sure yet, anyone got 6870s or 6990?

I had a 6990 for a few days and it wasn't working well (I didn't know about the fragmentation then and havent found any cheap 6990s since)

I believe Zydor's top host makes use of 2xHD5970s on the same AMD PC. Also, but not sure, I get the impression he uses an app_info.xml file to get each of the 4 GPUs to run 1 WU each.

I believe Zydor's top host makes use of 2xHD5970s on the same AMD PC. Also, but not sure, I get the impression he uses an app_info.xml file to get each of the 4 GPUs to run 1 WU each.

I've been doing the same thing, same setup, but I'm stuck getting "normal" wu's right now, so I'm working on that.

(2) 5870s are good for 750k/day - I really figured 2 5970s with overclock to at least 5870 core clock would give 1.4m/day

one lonely 6770 in the corral

Zydor currently has the top host ~ With what looks like Dual 5970's or Quad 5870s. !? Which is it.

Its dual 5970s

I have two boxes with Dual 5970s and if 1200w+ Power Supplies weren't so darned expensive I'd put all 4 of them in one host and go for top computer.

I found cheap 5970s and think I might get a few more but not sure yet, anyone got 6870s or 6990?

I had a 6990 for a few days and it wasn't working well (I didn't know about the fragmentation then and havent found any cheap 6990s since)

Take a breath rofl :) The 79XX series is deploying, on time as such, and the 7990 is due april/may. When that deploys, it will push down prices of other kit, especially 5970s and 6990s. There's much to think about as soon as you slam two of the XX90s into a box, dont knee jerk it, else you could end up with an expensive mistake. Non of it is rocket science, but its more than just slamming the cards in the box.

First off theres power, 1200w will eat 3x5970s and 3x6990s, maybe four depends on the o/c level and how much the CPU is overclocked. Remember dont pust power to the limit of the PSU unless you want long term trouble, keep a "reserve" there - say - 150w.

Second - think - real hard - about cooling. You can fry these things quicker than an egg if you get blasee with them, they can be unforgiving. Particularly the 5970s which have the dreaded design fault on the memory voltage regulator causing overheating if you are not carefull. To o/c the 5970 and 6990, you MUST have top notch cooling in the case with good airflow, pressure fans are a real good idea .... also think through other cooling methods like third party fans for the cards, water etc .... one solution doesnt fit all, but whatever you do think carefully, I've seen far too many 5970s get fried at BOINC just because individuals decided to ignore warnings re heat and the design fault - dont join them :)

Third is software, as MW showed, its a new world surfacing all on OpenCL, and much water to go under the bridge. There maybe other implications for 7990s that we are not yet aware of, and will not be until it deploys. That will affect the availability and price of second hand 5970s and 6990s.

Overall the 79XX series is very power efficient compared to 5970 and 6990, especially the latter, and is clealry far more powerfull. 2x7970s will take out 4x5870s or two 5970s any day (even 3x5970s will have a hard time and probably lose) - at MW 2x7970s will take on 4x6970s, so if you are going for records/top slots, dont even think of 5970s, you will need at the least 6990s, and even 2x6990s will have problems set against 3x7970s, let alone the upcoming 7990.

So my earnest advice is pause .... think through it much more ... there's a lot to work through before purchase, made more complex by the effects of the release of 79XX. NVidia is out of the equation for now in your context as their 690 will not see the light of day until mid-late Q3, my bet is not until just before Xmas at the earliest. So your choices are 6990s, 7970s or 7990s - but as hopefully the above shows ... some thought is needed.

Regards
Zy

Yeah cooling has been a pain, but most of mine are hosted in a Data Center that's 60 degrees and I can run 250cfm fans in the cases to move air. But even stacked cards heat up.

I've had various issues with several boards not supporting the 5970 except in the first pci-e slot.

I built another z68 based intel vt-d machine that has 3 slots and I'm looking now at something like a pci-e extender. I've downclocked most of the cpu's running boinc at this point and really no longer do CPU tasks on them, seems pointless to me.

I'm getting the 5970's for sub 300 at this point, so they aren't that bad on price, but they hog some power.

I really like the 7970s, especially the 1000mhz ones. I hope the 7990's aren't too detuned.

I have 6 amd 5770 gpu's and 4 amd 5870 gpu's. Two of the 5770's are in one machine while all the rest are spread out into one machine each. Most of my machines do not have dual pci-e slots so upgrading the system is the only way to do dual cards. I also have another AMD 5770 gpu on the shelf waiting for my newest machines memory to come in, so I do have some more head room to go yet and I still put out about about 2.3 million rac yesterday.

I have several gpu's on the shelf that have been replaced by newer gpu's...I have a couple of Nvidia cards and some AMD cards too.

Yeah cooling has been a pain, but most of mine are hosted in a Data Center that's 60 degrees and I can run 250cfm fans in the cases to move air. But even stacked cards heat up.

The issue with 5970s lay in the design of the VRMs (voltage regulators), they are badly placed on the board to the extent that they heat up more than the designers hoped. You will often get a 30+ degree difference in temperature between the GPU and the VRMs. The practical point is that many are so set in mindset, that they think only GPU temps need looking at, many dont bother, or even realise about the VRMs. Then when the GPU hits 90 degrees plus, conventional wisdom says ... "well thats hot, but its inside GPU armagedeon" (and it is) "so all is well".

Its not well, because its likely the VRMs are at 125+ by this stage, and the absolute maximum for physical integrity of the VRMs is 120degrees, just one degree over and the interconnects on the card start to slowly evapourate. It takes around 3 months to degrade, then, one day there's nothing left to melt and the card goes *poof*. Watch VRMs like a hawk with 5970s, dont rely on standard GPU cooling wisdom, they have an issue and solution all their own.

I've had various issues with several boards not supporting the 5970 except in the first pci-e slot

Most atx size boards in the past had two PCI-E x16 Lanes, and one is always designated as the primary (on the board usually known as "slot zero"). The driver expected the card driving the monitor to be in slot zero on dual or more setups. Less stringent now with better driver sets, but one to watch if piece part building with older driver sets.

I built another z68 based intel vt-d machine that has 3 slots and I'm looking now at something like a pci-e extender. I've downclocked most of the cpu's running boinc at this point and really no longer do CPU tasks on them, seems pointless to me.

If you are really into play time ...... play with this if you have the finance (pricey as such, but at the end of the day the box itself is not *that* pricey compared to the cost of the cards). Salivate over the bigger configurations of 16-32 GPUs working over a PCI-E bus per rack mounted box, via several rack mounted boxes, to the main machine rofl :)

One of the Magma offerings:
http://www.magma.com/expressbox16basic.asp
Have a nose around their site.

Google "PCI Express Expansion Chassis" as a starter, and follow your nose from there. Recently the market has matured and smaller size options are becoming common place as the base PCI-E technology matures. I suspect there will be a rash of offerings of this hardware for 2-4 slots when PCI-E 3.0 is common place. Its not a silver bullet, has its downside, but is a neat solution for the right niche, and can often be better solutions than using riser cards - think on a PCI-E 3.0 capable laptop hooked up to one :)

I'm getting the 5970's for sub 300 at this point, so they aren't that bad on price, but they hog some power.

Veritable power hogs - all designed prior to the world going "green" :)

I hope the 7990's aren't too detuned.

They will be - the issue is as old as the hills, they still need toi get inside the 300w PCI certification limit, and without PCI certification a board or a card is heading to commercial suicide. Withe the double GPU cards, they end up having to declock formally else they dont get certification. However, they always leave it open to overclocking, even unofficially support that. Hence the faster the GPU technology gets at its core, the more overclock room gets built into the card.

7970s have massive overclock room compared to official figures because of this, and I expect 7990 to similarly be hugely "overclockable". Downside is power, it will always draw relatively more power over default settings per watt of output when overclocking from official ratings, so particular attention needs paying to PSU, else the PSU will devolt cause all kinds of wiredness (symptoms often mistakenly attributed to other things .... never forget PSU if pushing hardware hard, and the PSU is near to its limits - keep at least 15% "spare" on PSU rating, and rate it on Overclocked power draw, not the official defaullt settings power draw. Guru3D is always excellent for checking on this in their card reviews - the latter are a "must read", they are good.

Regards
Zy

Great Discussion!

I'm constantly watching the VRM temps, I've had two cards so far I had to RMA (I'm not into changing pads and regreasing with 3yr/warranty in place)

As soon as I get the time to work with it I'm picking up a x8 Extender for the rack but we haven't had time to test it with Xen/VMWare and just running it extended yet. It's definitely the way of the future for multi GPU systems.


Even with the ATI Catalyst Drivers 12.1 and 12.2 still had some issues with getting various boards to see more then 1 5970 when only using 2 of them in a system.



I got a XFX 7970 Black Edition for my main gaming rig last night so we'll see how that does.

Even with the ATI Catalyst Drivers 12.1 and 12.2 still had some issues with getting various boards to see more then 1 5970 when only using 2 of them in a system.

Ahhh .... there's a little "slight of hand" to beat the thing into submission :)

Doesnt always work .... but .... more often than not.

Do all the usual clean outs re control panel deinstall & Driver Sweeper cleanout, and preferably a registry cleaner after those are done+reboot. All then pristine clean.

Connect the Crossfire Strap to the 5970s, load the driver - it will load ...

Trick is sometimes you dont get both cards visible in CCC, but actually they are there. Alaways have a look at the start of the Notes in BAM where the card first booted up - it should show 4 separate GPUs, even if not visible in CCC, if they are, crunch away, use afterburner to control them.

Back to main point .... driver now loaded with crossfire strap attatched. Make sure Crossfire is enabled in CCC (if visible), either its not (thats life), or its there. In the latter case disable crossfire, that will force the internal AMD card software to load the relevant driver set for each GPU as independent entities.

Power down, take off the crossfire strap, power up, and inside CCC there will be two cards showing (it will not show all GPUs when not in crossfire, just the cards.

Last but not least, always have a go with afterburner, it will often see the card when CCC doesnt. Give it a whirl, the chances afterburner it will control it fine. Another check within a check, inside CCC information click hardware, click the adaptor drop down at the bottom of primary adaptor - it will be marked down as either "Enabled Adaptor" or "Disabled adaptior". Dont let that put you off, give afterburner a twirl.

Its a long way oif saying with CCC all is sometimes not what it seems - multiple GPU software never been AMD/ATI strongpoint - getting better, but more wayer yet to go under the bridge.

Overall force driver loading by disabling crossfire, having just loaded the driver with strap attatched, dont believe CCC in this regard and give Afterburner a whirl.

Its a hell of a mouthful that lot, but its really much simpler than it will sound - they will go in easy enough, just focus nitially on driver loading with strap attatched, then disable crossfire to force internal software loads, power down, strap off - your in :)

Regards
Zy

I got a XFX 7970 Black Edition for my main gaming rig last night so we'll see how that does.

Load 12.1 / 7.0.15 with that straight away. That combination is solid as a rock with the 7970, it runs all projects that I have tried (that have the OpenCL app - thats MW, POEM, Collatz, and Donate@Home, they are the only ones at present, lots others converting to OpenCL at the moment). Be aware, by going to 7.X.X, there's no going back to 6.X.X without a 100% COMPLETE BOINC and Projects installs - everything.

Wack up the CCC sliders to 1125 GPU and the power slider to +20%, dont hesitate, the card is massively overclockable, and much much more to come via overvolting. You are dead safe staying inside CCC, and running it at CCC shown maximum. Theres a whole lot more hidden under the hood that you will only tap by overvolting - Afterburner Unofficial Mode has been tweeked for use with 7970s (get latest Beta - RC12 - from Guru3D). You MUST have the power slider at 20% inside CCC even if overvolting - the power slider is a TDP slider, not actual power as such, without that TDP increase, overvolting will not happen.

Its a Beast of a card, hugely overclockable.

Regards
Zy

What are you running for voltage on Afterbuner with the aforementioned things in CCC?

The 7970 card will take 1.3v, dont even bother, its pushing it way too hard. You will find a Law of Diminishing Returns after 1.22v, could push it to 1.23v maybe but its no really worth going past 1.22v. The card default is 1.174v

After 1.22v you have the card straining real hard for tangible gains, thats not worth the stress all round. It maybe as the software gets matured, it will be worth it, but right now it is not.

I suggest setting it at 1225 GPU / 1375 Memory 1.218v. No point messing with memory voiltage, the card memory voltage regulator is the "intelligent" type, it treats your entered value as "Target Voltage" it will set Real Voltage to whatever it thinks best for the card use, no matter what you say, or the utilitiy purports to say. So by all means set a target voltage, just dont expect to get it, no matter what afterburner or any other utility says it is on screen.

Regards
Zy

I can't get past 1125 Core, the slider has no more to go, Same with Afterburner.

Its not well, because its likely the VRMs are at 125+ by this stage, and the absolute maximum for physical integrity of the VRMs is 120degrees, just one degree over and the interconnects on the card start to slowly evapourate. It takes around 3 months to degrade, then, one day there's nothing left to melt and the card goes *poof*.

Ouch, I bet this is why my 5970 now hangs if I put any workload on it. :( I was working on getting the VRM temps under control when it decided to stop working totally. I was seeing over 100+ temps easily on VRMs while it started to enter heat regulation mode (which is why I noticed it first place).

-w

100+ is ok for VRMs, nice to get it down, but they will survive.

Its 120+ thats the real no-no. As you go over 120, the minute melt starts, not before, its the temperatue threashold of the metal used in the interconnects.

However you only mentioned 100+, if so, its likely all is well.

Its only if its constantly over 120+ for an accumulated 3-4 months of time, that it will go.

Regards
Zy

I can't get past 1125 Core, the slider has no more to go, Same with Afterburner.

You cant do it inside the slider above 1125, you have to do it by overvolting. You can do it with afterburner, but only by going to the afterburner unofficial mode which is made for voltage changes.

Afterburner Unofficial Mode must only be used by those used to voltage tweeking, the latter can be dangerous to the card (big time) if incorrectly applied, and must not be attempted by the unitiated. Afterburner Unofficial Mode is not for use by those not used to voltage changes.

Regards
Zy

What drivers do you use for your 5970s?

I'm having issues with the (4) 5970 box, the installer causes the screen to just fade to a black screen and never come back, I've tried a myriad of things to load the drivers and nothing's working 100% right.

I've been trying the 11.10, 12.1, 12.2.

I'm currently downloading the 11.4 and 11.5 for testing.

They are on 12.1 at present.

Do you use a Registry Cleaner - a good one - not a freebie extravaganza :)

Its always a good idea to clean up the registry before loading up the driver. Just rebooting without cleaning the registry after a reboot when recovering from a crash or blue screen will invariably leave bad entries in the registry. Using a cleaner gets rid of those bad entries. Its possible you have a build up of bad entries causing confusion. Its not a silver bullet, but its worth a go.

I would recommend the utility Suite "Advanced System Optimiser" http://www.systweak.com/

or

the registry cleaner "Advanced Registry Optimiser" http://www.sammsoft.com/AroDetails.aspx

Cant guarantee it will work as I'm "flying blind" here, but its worth a try, you should have a Registry Optimiser in your toolkit anyway.

Regards
Zy

I completely rebuilt the box 4 times today from scratch for each version of the Driver I was trying.

I took it to a whole different level.

But normally I use Driver Sweeper and CCCleaner with a combination of safe boots and manual verification.

;)

I'm trying to get rid of the distributed.net has stopped working messages.