err="no CPU power status has been recorded" on M3

[chenpfeisoo]

Host operating system: output of uname -a

Darwin MacBook-Pro.local 23.1.0 Darwin Kernel Version 23.1.0: Mon Oct 9 21:32:11 PDT 2023; root:xnu-10002.41.9~7/RELEASE_ARM64_T6030 x86_64

node_exporter version: output of node_exporter --version

node_exporter, version 1.7.0 (branch: HEAD, revision: 7333465)
build user: root@192f292aac5e
build date: 20231112-23:56:56
go version: go1.19.12
platform: darwin/amd64
tags: netgo osusergo static_build

node_exporter command line flags

What did you expect to see?

What did you see instead?

[dongjiang1989]

--no-collector.thermal disable on M1, M2, M3

dongjiang@MacBook Pro:prometheus-operator $ pmset -g thermlog
Note: No thermal warning level has been recorded
Note: No performance warning level has been recorded
Note: No CPU power status has been recorded

ref: #2218

[rexagod]

I was able to reproduce this issue on master, however, I don't think this bug is within the scope of node_exporter, but instead, needs to be addressed from the IOPMCopyCPUPowerStatus side of things (the underlying API the thermal collector relies on for this data).

It's also worth mentioning here that all recommended practices mentioned in the same documentation are being followed in the repository (such as releasing allocated CFDictionaryRef memory and checking for returned error codes).

A simplified version of the C-based collection logic shows that while node_exporter is responsible for exposing the data, it's the response from the API that is unexpected in nature.

package main

/*
#cgo LDFLAGS: -framework IOKit -framework CoreFoundation
#include <stdio.h>
#include <CoreFoundation/CoreFoundation.h>
#include <IOKit/IOKitLib.h>
#include <IOKit/pwr_mgt/IOPMLib.h>
#include <IOKit/pwr_mgt/IOPM.h>

struct ref_with_ret {
    CFDictionaryRef ref;
    IOReturn ret;
};

struct ref_with_ret FetchThermal();

struct ref_with_ret FetchThermal() {
    CFDictionaryRef ref;
    IOReturn ret;
    ret = IOPMCopyCPUPowerStatus(&ref);
    struct ref_with_ret result = {
            ref,
            ret,
    };
    return result;
}
*/
import "C"
import "fmt"

func main() {
	result := C.FetchThermal()
	/*
		    kIOPMCPUPowerLimitProcessorSpeedKey: 100, // cpu_scheduler_limit_ratio
		    kIOPMCPUPowerLimitProcessorCountKey: 4,   // cpu_available_cpu
		    kIOPMCPUPowerLimitSchedulerTimeKey: 100   // cpu_speed_limit_ratio
	*/
	fmt.Println(result.ref)                        // 0
	fmt.Println(result.ret == C.kIOReturnNotFound) // true
	fmt.Println(result.ret != C.kIOReturnSuccess)  // true
}

Additionally, ioreg -n IOPMrootDomain -r | grep IOPMCPUPowerLimitProcessorSpeedKey fails as well.

[SaiPisey2]

I'd like to pick this up, with a narrower scope than the original report.

@rexagod is right that the missing data itself can't be fixed from here — IOPMCopyCPUPowerStatus genuinely returns kIOReturnNotFound on Apple Silicon and there's nothing node_exporter can do about that. But there's a separate problem on our side that I think is worth fixing: because Update returns that error immediately, the temperature sensors read further down are never collected either.

Reproduced on an M5 Pro (darwin/arm64, current master):

fetchCPUPowerStatus()      -> status=map[] err=no CPU power status has been recorded
Update()                   -> err=no CPU power status has been recorded ; metrics emitted=0
IsNoDataError(err)         -> false
updateTemperatures() alone -> err=<nil> ; temperature metrics available=52

So there are 52 working temperature sensors on this machine that never make it out, and since the error isn't ErrNoData it's logged at error level on every scrape rather than debug.

The change I have locally treats kIOReturnNotFound as "this system doesn't report CPU power status" instead of a failure: skip the three CPU power metrics, log at debug, and carry on to the temperatures. Any other non-success return code is still returned as an error as before.

cpuPowerStatus, err := fetchCPUPowerStatus()
switch {
case err == nil:
    // ... emit the three CPU power metrics as before
case errors.Is(err, errNoCPUPowerStatus):
    // Apple Silicon does not report CPU power status. The temperature
    // sensors collected below are still available, so this must not abort
    // the collector.
    c.logger.Debug("No CPU power status reported by the system, skipping CPU power metrics")
default:
    return err
}

return c.updateTemperatures(ch)

After the change, Update() on the same machine returns no error and emits all 52 temperature metrics. On Intel Macs, where the CPU power status is available, nothing changes.

I've added a regression test (collector/thermal_darwin_test.go) that fails on master and passes with the fix. go build ./..., go vet ./collector/, gofmt and the full ./collector/ suite are all clean.

Worth noting this doesn't close #2218 or make the CPU power metrics appear on Apple Silicon — those still won't exist. It just stops their absence from taking the temperature metrics down with them. Happy to adjust the scope if you'd rather handle it differently.