AMD Epyc 7H12 Power Readings

[Skillz]

I happen to have a really expensive power meter for measuring power use on my hosts. I will be making threads as I test the hosts to see what kind of power they draw at the wall.

7H12 - 128 tasks of Universe - ~305W at the wall - 2.5Ghz

[Skillz]

Private GFN full 128 threads - 290W at the wall - 2.2Ghz

[StefanR5R]

I would expect Private GFN Server to pull more power than Universe@Home. One possible reason could be a cache bottleneck at Private GFN Server.

I had a quick look at some random hosts at Private GFN Server, and they show the following FFT lengths in current workunits:
"LLR2 testing" .......... 120K --> 0.94 MB cache footprint
"PRST Testing" ..... 2x288K --> 4.5 MB cache footprint?
"Sandbox 2" ............ 2x36K --> 0.6 MB cache footprint?
So there should be enough cache to run single-threaded LLR 2 testing and Sandbox 2 on all logical CPUs. But PRST Testing perhaps need to be restricted to 2 tasks at once per CCX (on Zen 2 CPUs, such as the 7H12), and forced with CCX-aligned CPU affinities.

If you want the operating system to retain all threads of an individual task within one and the same CCX for much of the time (rather than setting CPU affinities yourself), you can enable the BIOS option "ACPI SRAT L3 Cache as NUMA Domain". Works well with Linux at least. I don't know if Windows takes this up properly, but it should.

SMT may or may not be beneficial to either of these three applications. If SMT doesn't help, it may be especially unhelpful in Windows, compared to Linux.

--------

On the other hand, the fact that CPU clocks dropped from 2.5 GHz for Universe to 2.2 GHz for PGFN makes it look as if the CPU firmware did in fact encounter some sort of power limit within the cores, to clock them down accordingly.

[Skillz]

I was wondering if it was the VRMs that got hot for the project causing the CPU to throttle. The AsRock board has no way to monitor VRM temps and I don't know of a way to check through the OS (Linux) to see VRM temps. I'm assuming the AsRock board doesn't have a temp sensor on the VRM since the IPMI/BMC sensor list doesn't list them like the Supermicro dual board I have does.

[crashtech]

Perhaps server boards are expecting the VRMs to be in a wind tunnel and therefore aren't an item of concern.

[mmonnin]

Like many PG CPU apps, PRST slows down with too many concurrent tasks on my 5950s so I limit the # of threads and let the other threads run something else.

Sandbox 2?

[Skillz]

The only thing I am getting are the LLR2 Testing tasks. I'm not even sure what Sandbox 2 is.

I think I might set a box fan up next to the host to move air across the whole board. See if that helps the cooling and gets it to boost higher.

[mmonnin]

Damn, I missed the end of PRST tasks notification on the PGFN main page. Like 4k hours away from my 26k WUProp hours goal. I have like 2/3 of the remaining tasks but it's not enough hours once they timeout. No wonder there have been none.

[StefanR5R]

I had a quick look at some random hosts at Private GFN Server, and they show the following FFT lengths in current workunits:
"LLR2 testing" .......... 120K --> 0.94 MB cache footprint
"PRST Testing" ..... 2x288K --> 4.5 MB cache footprint?
"Sandbox 2" ............ 2x36K --> 0.6 MB cache footprint?
So there should be enough cache to run single-threaded LLR 2 testing and Sandbox 2 on all logical CPUs. But PRST Testing perhaps need to be restricted to 2 tasks at once per CCX (on Zen 2 CPUs, such as the 7H12), and forced with CCX-aligned CPU affinities.

Well, it's too late now due to the end of the "PRST testing" subproject, but for posterity:
The recipe to configure multithreading in PRST was posted by stream in the project's thread,
https://www.primegrid.com/forum_thread. ... rue#159303

Code: Select all

<app_config>
  <app_version>
    <app_name>prst</app_name>
    <cmdline>-t 2</cmdline>
    <avg_ncpus>2</avg_ncpus>
  </app_version>
</app_config>

…for 2-threaded tasks. If you put two concurrent tasks onto each Zen 2 CCX, try 2-threaded and 4-threaded tasks, i.e. MT unused/ SMT used. Either way, you should force affinities because otherwise you still bog the CPU down with memory accesses.

[Skillz]

Primegrid: GFN18 (CPU Only) - 32 tasks all pinned to 4 threads - 310W - 2.4Ghz clock speed