More on this.
Eventually problems still appeared with the GA-H87M-D3H and this Kingston KHX16C10B1K2 DDR1600 RAM, even given some settings tweaks as described above.
I think the problem is related to the motherboard incorrectly detecting or differentiating between JEDEC RAM timings and XMP. My build was for a home server and I'm not interested in overclocking, as such, I made sure that XMP RAM timings were turned off in the BIOS. Its my understanding that DDR3 standard JEDEC timings top out at DDR1333, everything above this is an XMP profile. Yes, of course, memory is sold advertising DDR1600 or 2000 or 2400 or whatever, but I think this is referring to high spec memory guaranteed to work at XMP timing profiles, not std JEDEC timings (I'm open to correction on this though, things might have changed in the last year).
In any event, I had another of my servers motherboards fail (unrelated) and it was time therefore for a full rebuild on thatmachine. Bought a GA-H97-HD3 this time and some pretty standard Kingston ValueRAM to go with it (2x8GB KHX1600C10D3/8GX). This is 11/11/11 at 1600, so, lower spec than the previously purchased Hyper-X which is 10/10/10.
As a test, I popped this new Kingston RAM into the problematic motherboard ... still getting detected as DDR1600 without any obvious way to change it to the JEDEC std DDR1333 but, of course, its timings are slower (11/11/11) and so far it seems stable.
More telling perhaps is that I stuck the original problematic Hyper-X into my new board (the GA-H97-HD3) and low and behold, its _correctly_ detected as JEDEC 1333 - also stable in the new board.
My guess .... the H87M-D3H needs a bios tweek, by Gigabyte, to correctly determine the JEDEC RAM settings as opposed to the XMP settings. I'm sure these problems would go away as a result. Either that or allow underclocking the RAM directly from the BIOS (a feature gigabyte boards used to have, but no longer it seems).
For the time being and/or presuming a BIOS update doesn't arrive, I'd suggest _only_ putting DDR1333 RAM into these boards. The reason I bought 1600 RAM even though I'm not interested in overclocking was that unbelievably both times I purchased the 1600 was cheaper than the 1333.