Skip to content

Conversation

@Artoria2e5
Copy link

Summary

Description

  • Add accounting for current-running processes via CPU load. The queue length would otherwise be not comparable to Unix systems.
  • Initialize at current load value instead of 0.
  • Use a new sampling interval of 4.615 seconds per Ripke.

The disk queue length is still not implemented as it is not required to match most of Unix and would require another counter handle.

@Artoria2e5
Copy link
Author

And that, kids, is why you don't signal things with NANs and then not handle it...

@Artoria2e5
Copy link
Author

Just one newline missing. Might as well let it finish the checks... Or not. The new push is gonna go through checks too.

@Artoria2e5
Copy link
Author

Perhaps someone should make a clang-format config and add it to lint too, for the C files. I mean, my changes look reasonably close to what is already here, but perhaps a linter can do better...

@Artoria2e5
Copy link
Author

@giampaolo This is good to go, I think. Is there supposed to be a "request review" button somewhere? Ah whatever, @ works too.

Hey what's this copilot button? They gave me some usage for free, might as well...


This pull request significantly improves the accuracy and behavior of the getloadavg() function on Windows to better match Unix behavior, updates its documentation, and refines its internal implementation. The changes include immediate initialization of load averages, inclusion of CPU utilization as an option, a more precise refresh interval to reduce artifacts, and improved thread safety and error handling.

Windows load average improvements:

  • The Windows implementation of getloadavg() now accounts for currently running processes by incorporating CPU utilization, closely matching Unix behavior. The function initializes with the current load value rather than starting at zero, providing meaningful results immediately. The refresh interval is changed to 4.615 seconds to minimize Moire artifacts with processes firing on integer-second intervals. An optional include_util parameter allows users to control whether CPU utilization is included in the load calculation. [1] [2] [3]

  • The internal C implementation in wmi.c is refactored to use exponentially weighted moving averages for both processor queue length and CPU utilization, with improved thread safety and error reporting. The sampling interval and EWMA factors are updated for greater accuracy. [1] [2]

  • The number of logical CPUs is now determined using a new exported function psutil_get_num_cpus, improving accuracy in load calculations. [1] [2]

Documentation updates:

  • The documentation for getloadavg() in index.rst is updated to describe the new Windows behavior, including the include_util parameter, immediate initialization, and the new refresh interval.

Metadata and attribution:

  • Copyright notices are updated in relevant files to include the new contributor. [1] [2]

Changelog:

  • The HISTORY.rst file is updated to document the new Windows load average behavior and the rationale for the refresh interval change.

@Artoria2e5
Copy link
Author

Artoria2e5 commented Oct 26, 2025

On second thought disk queue length is actually useful because Windows does have this tendency of freezing the entire explorer when you insert a broken drive. Alright it's just one more handle, how hard can it be.

It would very likely be beneficial to mutex the whole initialization process, so that we don't do it twice by accident. (So: mutex the context.)

That would unfortunately not remove the need for mutex on the tracking variables since the read-function actually resets error tracking, so it makes a write and has the potential to make things inconsistent! (Just reading would be fine, being stuck between two updates isn't that wrong.) Of course I could regress back to when there's no error tracking, but that's not cool.

On second thought the last-error thing already steps over the last-last error, so what's more stepping on a foot? Plus the data size is so small we could use atomics if we really cared…

@Artoria2e5
Copy link
Author

There's no need for initialization mutex in C because (1) it's already held by a lock the Python part (2) it finishes one initialization in the main thread before handing it off to the timer. So it's always only gonna be one thread spawned. So why am I even doing HeapAlloc or bothering to protect any part of context?

Mutex on the load and error variables is not strictly necessary for "okay" operation but might as well keep it. Bumping into it as held is going to be rare.

@Artoria2e5
Copy link
Author

Aha, there is clang-format now. Just gotta do some merging...

@Artoria2e5
Copy link
Author

Merges, builds, and runs ok, will rebase later

* Add accounting for current-running processes via CPU load. The queue
  length would otherwise be not comparable to Unix systems.
* Initialize at current load value instead of 0.
* Use a new sampling interval of 4.615 seconds per Ripke.

The disk queue length is still not implemented as it is not required to
match most of Unix and would require another counter handle.

Amend 1: Never return NaN or Infinity from util calculation. If things
are about to go wrong, return a signaling negative value, set error
variable, and skip this update tick.

Amend 2: Make black happy, missed an empty line. Might as well fix up C
formatting.

Amend 3: User (or rather, webpage)-visible documentation.

Amend 4: Instead of giving up update altogether, just regurgitate 1m
util value. Would help 5m/15m continue to converge. Is also simpler.

Optional disk queue for Windows loadavg.

This helps admins who are more familiar with Linux-style as opposed to Unix-style calculation. Including the IO wait time complicates the "CPU overload" interpretation, but some admins prefer to use it as it does give a better idea of overall wait times. It's also common for the Windows explorer to just hang when you plug in a slow or failing disk, so there is definitely relevance.

The disk queue is not included by default. Users must explicitly request it in the load-average sum using the ``selector`` parameter.

In addition, add a way to fetch the last-tick load, similar to what one would obtain from running `ps -Max -o stat | grep ^[DR] | wc -l` on Linux.
@Artoria2e5
Copy link
Author

I turned on IndentPPDirectives to remove one common reason for turning off c-f.

@Artoria2e5
Copy link
Author

What the heck is a dprint? Okay..

@giampaolo
Copy link
Owner

giampaolo commented Oct 29, 2025

Hi,
this PR aims at doing too much. I suggest to remove the disk queue length stuff, and for now only focus on these 2 items from #2664:

  • The 5 second interval is very prone to moire artifacts, see https://ripke.com/loadavg/moire. I recommend using (60/13).
  • Queue length do not include current running count, only the waiting-count, unlike Unix load average. You gotta add processor util% to match

Or to put it another way, let's not change the function signature to accept new args (getloadavg(selector="qu", instant=False)). Only focus on returning a more accurate value for now, since you seem to suggest that the current one is unrealistic. The minimal the changes, the better.

And once that is done I'll probably ask @ammaraskar if he can take a look, because this part of the code is obscure to me (and the new changes even more so).

@Artoria2e5
Copy link
Author

By default it returns the qu, so I think it is fine. I can split it into a different API if you want.

@Artoria2e5
Copy link
Author

More specifically, minimization will not be fruitful beyond a change of the API interface only.

  • The disk queue code is of the same sturcture as the processor queue code, so very little will be deleted.
  • Much of the addition is for correctness. Like the goto destructor for freeing resources. Like the processor util thing which has to look like that.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Windows] getloadavg() improvements

2 participants