Bulldozer on 990FXA-UD5: issues with all 4 RAM slots populated

[protocall]

Hi all. First time poster, long time Gigabyte fan. I've been building and tweaking systems since the 386 days. Currently have three different Gigabyte/AMD systems happily running issue free and spanning generations from socket754 to 790GX AM2. Never had problems until the new Bulldozer rig, which I'm prepared to accept is new-ish territory and I'm willing to try whatever it takes to get this working right.

I've read numerous threads about issues when all 4 ram sticks are populated, most notably these two:
http://forum.giga-byte.co.uk/index.php/topic,7190.0.html
http://forums.tweaktown.com/gigabyte/46600-overclocking-bulldozer-ga-990fxa-ud5.html

I'm sharing my own experiences here for troubleshooting, sanity check, and to add to the community in hopes that fixes may be possible. In contrast to other posts, I am not overclocking and I am now using RAM that shows up on the approved list for this board. Just trying to get everything to run at rated 1866MHz.

Hardware -
brains: AMD Bulldozer FX-8120
guts: Gigabyte GA-990FXA-UD5
psu: Enermax maxrevo 1350W

RAM: (two full sets for testing)
first - 4x4GB G.skill Ripjaws F3-14900CL9Q-16GBXL rated 1866 9-10-9-28 @1.5V
second - 4x4GB Corsair Vengeance CMZ8GX3M2A1866C9 rated 1866 9-10-9-27 @1.5V

Settings and details -
BIOS at latest release: F6
All clocks at stock/auto - 3.1GHz CPU, 2200 NB, 2600 HT, 200MHz base clock
Turbo/CPB disabled
Memory channels set to UNGANGED

Initially bought G.Skill Ripjaws as they were a 16GB matched set and generally reviewed higher than the Corsair Vengeance. Would not post at 1866 even with ONE stick alone in ANY slot:

- Clear CMOS, Load Optimized defaults, Load Profile1, save/reboot. Eventually comes back to POST screen at 1333 saying my clock settings didn't work, or requires me to clear CMOS again to get it to POST.

- Clear and try again with manual timings. Avoid Profile1 setting. Change DRAM timings to 9-10-9-28 by hand and set multiplier to 9.33 = 1866. Same results.

- Raised DRAM voltage to 1.65V. Raised NB VID to 1.3V. Raised CPU VID to 1.4-1.45V. Same results.

- Clear CMOS and change multiplier to 8.0. Adjust CPU base clock to reach an effective 1866. Will POST but will not boot (never gets past "loading OS" or gives me "writing recovery to backup bios" or whatever error) at same voltages.

This is with one stick. Change to 1600MHz (8.0 DRAM multiplier) and everything works fine. POSTs fine with 4 sticks at 1600. Ran memtest overnight on 1 stick, then 2 sticks at 1600 = clean. Switched pairs, switched slots, repeat memtest = clean. Ran memtest overnight on 4 sticks @ 1600 = thousands of errors by test #5/6.

RMA'ed this kit. Since I cannot get even 1 stick to post at 1866 I figured it was a compatibility issue since 4gb g.skill 1866 dimms are not on the Gigabyte tested list.


Try again with Corsair Vengeance. This is the ONLY 4gb 1866 kit on the approved RAM list for this board. Same general approach as above. Clear CMOS, load optimized defaults, load profile1, reboot.
- 4 sticks, 1866 with profile1, does not post
- 4 sticks, 1600 with profile1, posts fine
- 1 stick, 1866, passes memtest in any slot with any stick at rated timings
- 2 sticks, 1866, posts fine. Memtest runs overnight with no errors (about 3 passes depending when I wake up). Swap sticks, same. Swap slots, same.
- 4 sticks, rated timings, 1600 - memtest has thousands of errors by the time I wake up.

In any case where I've tried 4 sticks, I've also tried raising the DRAM and NB VID to 1.65V and 1.3V, respectively. I've also tried extremely loose timings at 14-14-14-36 and upped tRFC values to 300ns (as I know from the old days this can cause issues between banks with multi-gig setups). Nothing changes. Still will not post at 1866 nor run clean at 1600 with 4 sticks.

Am I doing something wrong here? I don't think I'm asking the impossible. The CPU, board, and RAM are all designed to run at 1866 stock. There are 4 slots on the board. If 1866 only works with 1 DIMM per memory channel then I'd expect this to be disclosed by either AMD or Gigabyte or there might as well only be 2 slots on the board (yes I understand this is not a Gigabyte design flaw, but asking for all available slots to run at rated speed doesn't seem far fetched and seems like something that would have been tested prior to release).

Next I will memtest two sticks in adjacent slots (DIMM 1 and 3, or 2 and 4) to see if it really is an issue with 2 sticks per channel.

Some errata I've noticed while testing and would like to report and/or get a sanity check on:
- after clear cmos, CPU voltage defaults to 1.45V+. Turn off turbo/CPB and vcore drops to 1.36ish. Temperature changes by 5-10 degrees. Is this normal?
- if DRAM E.O.C.P. is used at all, including to load Profile1 timings, HT multiplier goes to 11 (insert spinal tap joke) which is 2200. FX-8120 HT stock is 2600. IF YOU SAVE AND REBOOT at this point, memory set to 9.33/1866, POST screen shows DRAM is 1875. Manually change HT back to 2600 (or auto) and POST screen shows correct 1866. Is this just an error with text on the POST screen, or are the HT defaults really screwing up the multiplier math?

So, anyone who has read this novel: ideas? suggestions? do I have a bad CPU? I am open to further troubleshooting (as I've been at it for days now). I refuse to believe I can't run 16GB of ram at even 1600 cleanly, let alone 1866.  I guess I'll be trying 4x4 @ 1333 next.

(edited because I accidentally hit post before finishing)

[absic]

Hi prtocall and welcome to the forum.

If it helps your sanity your're not the only one who seems to have noticed this and it's not just on the UD5 as the UD3 also seems to be having some issues with this. http://forum.giga-byte.co.uk/index.php/topic,7190.0.html

And there is a very similar post over at TweakTown: http://forums.tweaktown.com/gigabyte/46600-overclocking-bulldozer-ga-990fxa-ud5.html with a four slots populated BOOT issue.

The best advice I can offer is that you contact Gigabyte Technical support on your side of the pond via this link: http://ggts.gigabyte.com/tech.asp?ClassID=2&Country=U.S.A.&SourceWeb=B2C as I feel they will be the best area of expertise to help you resolve this issue.

[protocall]

Cheers from the other side of the pond.

I've submitted a tech support request per your instructions.  Will update this thread as anything develops.

[protocall]

UPDATE

Some progress. In swapping between memory kits/brands, I've noticed the DRAM config menu's 'AUTO' column seems pretty intelligent, detecting and wildly changing timings between kits, when more than 2 DIMMS are installed, and most importantly when changing the DRAM multiplier. I also noticed both kits will only boot four DIMMS at 1333MHz if timings are set to AUTO, which ends up being 9-9-9-24. They will not boot (CMOS setup menu hangs after a few seconds, even) if set to 1333 at Profile1 timings or rated 9-10-9-etc. All Profile1 timings are higher than AUTO when set to 1333, except Trfc. So I raised that to the 160ns value and it boots at 1333.

With this, I decided to try a few OCD timing adjustments with all 4 sticks. I cleared CMOS and loaded Profile1 timings. I then edited the DRAM config and changed settings based on the AUTO column - I chose the greater value of Profile1 or AUTO with multiplier set to 8=1600MHz. This leaves me with:
2T
11-11-11-29
7T
160ns
14T
7T
48T
7T

It has just completed its 3rd error-free memtest pass at 1600MHz. This is the first time 1600 was stable with 4 sticks. The above timing approach was repeated for the g.skill kit and also ran 3 error-free passes 1600. Albeit at the world's most ridiculous timings.

Unfortunately, the same approach did not get me to 1866. I chose the higher of AUTO or Profile1 again, set to 1866, which gave me:
2T
13-13-13-34
8T
160ns
16T (auto says 15 but there are only even numbers to select here)
8T
48T
8T

The Corsair kit will not post. The g.skill kit will post and boot, but memtest starts counting thousands of errors before 40% completion (I stopped there).

I then jacked up all the DRAM timings to their max (left drive strength alone). No change. Changed DRAM to 1.6V, CPU NB VID to 1.3V, and Vcore to 1.45V. No change.

So, I can get 1600 stable now, but 1866 still seems like a far cry after maxing out the timings. So my question to the experts now becomes "what other settings should I be changing?" DQS is on. Interleaves are on. Drive strengths and all the other lower-menu settings are auto. Any recommendations on changing these values to help the cause?

[protocall]

Update - WINNING, sort of.

Well I've given up on my stubborn "this should work at rated speeds out of box" approach. I'll accept that it's electrically more difficult to drive two DIMMs per memory channel. What I can't accept is that seemingly the board has no way of compensating for this via timing or drive strength settings, or maybe I still haven't found the right combination. I raised all the drive strength settings to their max and adjusted some of the fine timings - basically maxed out the DRAM menu. Then raised NB VID, DRAM voltage, etc. No change from the above posts.

So I tried this the old school way and just jacked up the base clock. Left the DRAM multiplier at 8 and raised the base (HTT) clock to 225. This gives me 1800MHz RAM and a mild overclock on the other busses/CPU. It has been running memtest clean for 10 hours, currently on its fourth pass.

Unfortunately, setting the base clock to 233 x8 = 1866 (1864 actually) is not much different than 200 x9.33 = 1866. Board will POST and boot, but memtest chokes on test #5 as usual.

I may try some additional timing adjustments at 233x8 or raise the voltages a bit. Hell, I may buy a DDR3-2133 kit just to see if it'll follow suit and run clean at 1866 with four modules. Per the overclocking post on tweaktown it looks like he's having a little trouble with 230+ HTT speeds as well.

After a week of testing, I may just live with my 1800. Better luck to those who end up here via google.

[DarkFoss]

Man sorry to see your having so many issues..I've just oc'ed my 8120 on the UD5 but I'm only using the Vengence 16g 1600 kit..
I'd stay away from using the Dram EOCP and Profile 1..It will use the XMP profile settings and those are for Intel chipsets..There are a few AMD Black Profile ram out there but those are mostly for the PhenomII's..also according to Aida64 the dram is being undervolted..mine showed 1.488 instead of 1.5 and I had to up the Dram Voltage Control to 1.525 to get Aida to show 1.504 without drooping.

If you ever start over I'd recommend using just 2 sticks then manually set up the correct timings and multiplier timings from here http://www.corsair.com/vengeance-8gb-dual-channel-ddr3-memory-kit-cmz8gx3m2a1866c9.html are 9-9-9-24 the rest should be copied from auto if any need to be changed Cr of 2 recommended and change the system Voltage control to manual as well and set the Dram Voltage Control to 1.525 multiplier of 9.33.. As you noted earlier that one dram setting of 160ns is a must..2 sticks should post and run memtest error free.

For 4 sticks I'd use the tested Latency settings of 9-10-9-9-27 Cr2 ...Corsair doesn't recommend or support mixing 2 8g kits even if they are the same speed..sometimes you'll even get 2 different revision numbers when bought at the same time and place ..from personal experience that rarely works..

I had to do the above myself to get all 4 sticks working at 1600 speeds btw.

I'll be making a post in the oc'ing section with about my 4.5 oc and will post my settings and a few results links..just looking over my notes and I forgot my ram timings lol..

[twohype]

Sorry to hear about your issues, I'm currently running all 4 slots (see my sign) and I'm loving my AMD system since it's running perfect 24/7. Maybe your issue is due to mixed ram models. The only time I've every had issues is when I mixed ram, oh well I hope can fix it.

[protocall]

Hey thanks guys.

Funny, first response from Gigabyte tech support told me to use EOCP exclusively.  I don't suppose it matters much either way, the numbers are what they are. In my testing, EOCP or manual didn't produce much difference.

Good tip on the voltages...I'll mess around and install a few monitoring tools.

So far I am clean at 1600 and 1800 at 9-10-9-27 with 2t and trfc=160ns.
1866 still refuses to live. Will post update if I get this working.

Anyone know what the AMD equivalent of VCCIO and/or VCCSA is?  CPU NB VID cover this? Seen some intel posts about raising these voltages for 4x sticks.

[gunite]

Hate to burst everyones bubble but Bulldozer chip has a limitation on running 1866 and above speeds to just two banks. If you don't believe me email AMD and they will tell you themselves. Not something they advertise due to reasons understandable. I went through this for a week and finally found out on a German website. I was able to get my GSkill F3-14900CL9Q-16GBXL running at 16gb @ 1600 CL8 8 8 24 @ 1.55v or 8gb of it @ 1866 CL9 10 9 28 @1.5v. good luck 

[sdunnin]

Was able to get my 990FXA-UD5 / FX-8150 running first try @1600 with 4 sticks of GSKILL (F3-14900CL9D-8GBSR) using the auto timings so far but have encountered same problem as noted by protocall and  have had no luck getting them running at 1866 

Along the way I again experienced the weirdness with this board of disappearing drives http://forum.giga-byte.co.uk/index.php/topic,7288.0.html after making BIOS changes. This is really an unrelated issue to the RAM speed problem but I thought that I'd mention just in case anyone else has ever experienced that type of issue, otherwise I may just have a flaky board.

[gunite]

I forgot to mention my CR was at 2T in both situations. This is what was given to me so I will show you as well. I truly believe AMD and ALL MOBO manufacturers should do a better job to properly disclose that information. You sure can run 1866 memory,(whispering) but only two sticks of it hehehe is what it seems like to me... That is shady and it cost me money by not buying the appropriate memory from the get go. Whether it was to buy just two sticks @ 1866 or all four sticks @1600. If I would have known I would have gone with four sticks of 1600 with the lowest case latency I could have found seeing how there aren't any single 8gb 1866( or higher) sticks of memory yet. Just my two cents

[jimdot]

protocall,
You say you were successfull at getting your mem to run at 1800Mhz in your post above? I'm a noob at this stuff so I really don't know what the heck I'm doing with all these settings changes . If your system is still running stable would you mind listing exactly each settings & what you've changed the to ?? I would be quite happy if I could get my G.Skill 1866MHz 16Gb kit running at the 1800 Mhz that you've been able to get.

You mentiond that you'd raised the base clock(HTT) to 225. What's HTT? I didn't see anything in BIOS labled HTT. All this terminology is just clear as mud to me

Any help advice you could give me would be a help & thanks in advance,

Jim

[obxeye]

i recently built 990fxa-ud5 with amd 8150 and Corsair 16 GB Vengeance Low Profile 1600mhz PC3-12800 240-pin Dual Channel DDR3 Memory Kit 16 Dual Channel Kit - CML16GX3M4A1600C9 with no problems.  i do not overclock.  just loaded on board and everything auto in bios.

[gunite]

The problem isn't DDR3- 1600. You can run all 4 banks full to whatever the max is allowed on the board. The problem is DDR3-1866 and above. You can run only 2 banks, not all 4 banks, whether it be in unganged or ganged mode and no matter how much or little the amount is due to a Bulldozer processor limitation.

[protocall]

protocall,
You say you were successfull at getting your mem to run at 1800Mhz in your post above? I'm a noob at this stuff so I really don't know what the heck I'm doing with all these settings changes . If your system is still running stable would you mind listing exactly each settings & what you've changed the to ?? I would be quite happy if I could get my G.Skill 1866MHz 16Gb kit running at the 1800 Mhz that you've been able to get.

You mentiond that you'd raised the base clock(HTT) to 225. What's HTT? I didn't see anything in BIOS labled HTT. All this terminology is just clear as mud to me

Any help advice you could give me would be a help & thanks in advance,

Jim

Sure, Jim. First, disable turbo, save and reboot, go back into setup. Enable profile 1 from the EOCP settings. Then, change the HT link and Northbridge back to AUTO, as the EOCP profile jacks with the HT setting and I get mixed results. Best to go with stock. As for CPU multiplier, overclock as you wish, I've been sitting on 3.5GHz stock voltage with a mild aftermarket tower heatsink. CPU Host Clock control is how you get to 1800. Change to manual and set the frequency to 225.

Now change your DRAM multiplier to 8.0 Check your manual timings, which should have carried over when you enable profile1. Ensure command rate is 2T. My best stable timings are 9-10-9-27 with the rest at whatever profile1 chose, and trfc=160ns for both banks IS A MUST.

Be sure to check your exact kit to see if it needs 1.5V, 1.65V, etc.  I just picked up another kit, Gskill ripjaws Z 16gb 2133MHz. They're 1.6V rated, but I'm sure that's just to get to 2133 with fat rated timings of 11's. I get the exact same results with this new kit. 1866 is no good.

I'm with gunite here. After 3 kits the same, all evidence points to the memory controller. There isn't any magic BIOS setting to make 1866 stable. Half these 1866+ kits are just binned stable at looser timings and/or higher voltages. Save the headache and get a low latency 1600 kit. Until there's a new stepping...