-
-
Notifications
You must be signed in to change notification settings - Fork 1.7k
fix(fold): use buffered read to prevent hang on infinite streams like /dev/zero #9966
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
|
GNU testsuite comparison: |
CodSpeed Performance ReportMerging #9966 will degrade performance by 6.42%Comparing Summary
Benchmarks breakdown
Footnotes
|
…ev/zero also fixed failure testcases
…ev/zero also fixed failure testcases after cargo fmt
|
GNU testsuite comparison: |
|
GNU testsuite comparison: |
| writer.write_all(&output)?; | ||
| output.clear(); | ||
| match std::str::from_utf8(chunk) { | ||
| Ok(s) => process_utf8_line(s, &mut ctx)?, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This has the same issue as the following PR: #10396 where multi byte characters that are at the end of a buffer are getting split into multiple characters.
Description
Fixes #9845
The previous implementation of
foldattempted to read input line-by-line usingread_until. When processing infinite streams without newlines (such as/dev/zero), this caused the internal buffer to grow indefinitely, leading to a hang or OOM (Out of Memory) crash.Changes
fold_fileto use a fixed-size stack buffer (8KB) andRead::readinstead ofBufRead::read_until./dev/null(guarded by#[cfg(unix)]to avoid Windows CI failures).Verification
fold /dev/zerono longer hangs and streams output correctly.cargo test --package uu_foldpasses.