Implement uring in `AsyncWrite for fs::File` #7713

RedKinda · 2025-10-27T14:20:38Z

Motivation

Currently IO operations for fs::File is implemented using spawn_blocking to perform IO, which is not great for an async runtime. This PR implements IO operations using io_uring. The fs::write function already uses io_uring for async operations, and this PR extends the uring functionality into impl AsyncWrite for fs::File.

For full discussion see here #7684 and subsequently on discord https://discord.com/channels/500028886025895936/810724255046172692/1427735341003051079

Solution

The fs::File struct already implements an internal buffer when writing using spawn_blocking, which is similar to what io_uring requires. Therefore in poll_write, we can check if io_uring is initialized and supported, and if not fall back to the spawn_blocking implementation, without requiring any API changes.

This PR only implements AsyncWrite, and once this PR is accepted, I am happy to also open a PR for AsyncRead for File using the same structure.

tokio/src/fs/file.rs

This reverts commit 863a69c.

RedKinda · 2025-11-12T11:34:04Z

This should be ready for merging i think, is there anything else blocking it?

Darksonn · 2025-11-12T11:35:14Z

Sorry I've been too focused on #7696. I will try to review this PR too.

tokio/src/fs/file.rs

Darksonn · 2025-11-17T19:06:25Z

tokio/src/fs/file.rs

+#[derive(Debug)]
+enum JoinHandleInner<T> {
+    Blocking(JoinHandle<T>),
+    #[cfg(all(
+        tokio_unstable,
+        feature = "io-uring",
+        feature = "rt",
+        feature = "fs",
+        target_os = "linux"
+    ))]
+    Async(BoxedOp<T>),


Here is an idea:

What if you perform the io-uring logic in a tokio::spawn task? That way, you can use JoinHandle in both cases.

I think it will simply code a lot.

The Blocking(JoinHandle) is crate::blocking::JoinHandle not tokio::runtime::task::join::JoinHandle so some enum would be required anyway, and at that point might as well just have this kind of a boxed future

That is a re-export of the same thing.

tokio/tokio/src/blocking.rs

Line 9 in 4714ca1

pub(crate) use crate::task::JoinHandle;

One reason that I suggest this is that the io-uring logic performs multiple writes in a loop to write the entire buffer, which requires the end-user to await flush or write on the file to make progress. However, the implementation we have today does not require the user to interact with the file to make progress - it happens in the background automatically.

Ah got it, I'll change it.

Out of curiosity, general rule is that for a future to keep making progress, user needs to keep polling it. I feel like not having to do that specifically for fs::File is just an implementation detail, and users shouldn't rely on that always being the case. So i guess my question is how much is it worth preserving old behavior like this when implementing new features, if it comes at the cost of (small) performance overhead? relevant XKCD

Had to adjust mocks to accommodate non-blocking spawning https://github.com/tokio-rs/tokio/actions/runs/19464918802/job/55697554687 b91836c - can you doublecheck im doing it right?

This kind of thing is always a hard question. Though I would say that I'm pretty sure people rely on writes continuing even if the file is dropped, and it would be very hard to change that.

tokio/src/io/uring/write.rs

tokio/src/fs/write.rs

tokio/src/fs/file.rs

RedKinda · 2025-12-03T21:31:15Z

If someone can doublecheck that what I added to tokio/src/fs/mocks.rs makes sense, i think this should be ready for merging?

Also looks like GCP is breaking the CI :/

Darksonn · 2025-12-05T09:40:07Z

tokio/src/fs/file.rs

+                        // Handle not present in some tests?
+                        if let Ok(handle) = Handle::try_current() {


Surely if the handle is not present, then spawn_mandatory_blocking would also fail. What tests is this?

Darksonn · 2025-12-05T09:43:56Z

tokio/src/io/uring/write.rs

+    /// Issue a write at `file_offset` from the provided `buf`. To use current file cursor, set `file_offset` to `-1` or `u64::MAX`.
+    pub(crate) fn write_at(fd: ArcFd, buf: blocking::Buf, file_offset: u64) -> Self {


This seems to be a feature that relies on a flag IORING_FEAT_RW_CUR_POS being present. We should probably make sure to check this feature somehow.

Darksonn · 2025-12-05T09:47:22Z

tokio/src/io/uring/utils.rs

+use std::sync::Arc;
 use std::{ffi::CString, io, path::Path};

+pub(crate) type ArcFd = Arc<dyn AsRawFd + Send + Sync + 'static>;


Instead of introducing this, could we make the Write op generic instead? This avoids allocating a useless Arc in fs::write.

Darksonn · 2025-12-05T09:49:43Z

tokio/src/fs/file.rs

+                    #[allow(unused_mut)]
+                    let mut task_join_handle = inner.poll_write_inner((std, buf), seek)?;


Suggested change

#[allow(unused_mut)]

let mut task_join_handle = inner.poll_write_inner((std, buf), seek)?;

let task_join_handle = inner.poll_write_inner((std, buf), seek)?;

Darksonn · 2025-12-05T09:51:06Z

tokio/src/fs/file.rs

+                                    let handle = spawn(async move {
+                                        loop {
+                                            let op = Op::write_at(fd, buf, u64::MAX);
+                                            let (r, _buf, _fd) = op.await;
+                                            buf = _buf;
+                                            fd = _fd;
+                                            match r {
+                                                Ok(_) if buf.is_empty() => {
+                                                    break (Operation::Write(Ok(())), buf);
+                                                }
+                                                Ok(0) => {
+                                                    break (
+                                                        Operation::Write(Err(
+                                                            io::ErrorKind::WriteZero.into(),
+                                                        )),
+                                                        buf,
+                                                    );
+                                                }
+                                                Ok(_) => continue, // more to write
+                                                Err(e) => break (Operation::Write(Err(e)), buf),
+                                            }
+                                        }
+                                    });


This is indented like 8 or 9 times. Can we move the loop into a separate function to reduce this?

Darksonn · 2025-12-05T09:52:12Z

tokio/src/fs/file.rs

+                    }

-                    inner.state = State::Busy(blocking_task_join_handle);
+                    inner.state = State::Busy(task_join_handle.unwrap());


The existing code returns io::Error::new(io::ErrorKind::Other, "background task failed") instead of panicking here. Please follow that pattern. If there are not tests for this, please add a test that would have caught this mistake.

this isn't a panic on the io error, its an unwrap for task_join_handle: Option<_>, which should always be present. It's an Option to allow falling back from uring to regular writes

Darksonn · 2025-12-05T09:53:45Z

tokio/src/fs/file.rs

+    fn poll_write_inner(
+        &self,
+        data: (Arc<StdFile>, Buf),
+        seek: Option<SeekFrom>,
+    ) -> io::Result<JoinHandle<(Operation, Buf)>> {


There is a lot of duplication between normal and vectored writes. The logic that actually writes is the same in both cases, so let's avoid this duplication. Please refactor to avoid this.

RedKinda added 3 commits October 27, 2025 14:56

implement uring AsyncWrite for File

d9d2ddc

add basic uring File write tests

093070a

formatting

6856207

ADD-SP added C-enhancement Category: A PR with an enhancement or bugfix. A-tokio Area: The main tokio crate M-fs Module: tokio/fs labels Oct 27, 2025

martin-g reviewed Oct 28, 2025

View reviewed changes

tokio/src/fs/file.rs Outdated Show resolved Hide resolved

tokio/src/fs/file.rs Outdated Show resolved Hide resolved

tokio/src/fs/file.rs Outdated Show resolved Hide resolved

RedKinda added 3 commits October 29, 2025 14:30

remove sync bound from BoxedOp

863a69c

make sure to write the full buffer in uring file writes

ea4040d

Revert "remove sync bound from BoxedOp"

226fcf8

This reverts commit 863a69c.

Darksonn reviewed Nov 17, 2025

View reviewed changes

move part of poll_write to its own function

937da2d

martin-g reviewed Nov 18, 2025

View reviewed changes

tokio/src/io/uring/write.rs Outdated Show resolved Hide resolved

tokio/src/fs/write.rs Outdated Show resolved Hide resolved

tokio/src/fs/file.rs Outdated Show resolved Hide resolved

RedKinda added 7 commits November 18, 2025 12:42

implement review comment suggestions

c30a4dd

add non-blocking spawn to mocks

b91836c

fix type issue in mock spawn

58e9b0b

fix spawn signature 2.0

9504e53

appease clippy

daa0532

Merge branch 'master' into uring_file

8cf4ca3

Merge branch 'master' into uring_file

a72bf97

Darksonn mentioned this pull request Dec 4, 2025

Support io_uring for file I/O #7266

Open

17 tasks

Merge branch 'master' into uring_file

9c7387f

Darksonn reviewed Dec 5, 2025

View reviewed changes

		// Handle not present in some tests?
		if let Ok(handle) = Handle::try_current() {

		/// Issue a write at `file_offset` from the provided `buf`. To use current file cursor, set `file_offset` to `-1` or `u64::MAX`.
		pub(crate) fn write_at(fd: ArcFd, buf: blocking::Buf, file_offset: u64) -> Self {

		#[allow(unused_mut)]
		let mut task_join_handle = inner.poll_write_inner((std, buf), seek)?;

	#[allow(unused_mut)]
	let mut task_join_handle = inner.poll_write_inner((std, buf), seek)?;
	let task_join_handle = inner.poll_write_inner((std, buf), seek)?;

Uh oh!

Implement uring in AsyncWrite for fs::File #7713

Are you sure you want to change the base?

Implement uring in AsyncWrite for fs::File #7713

Conversation

RedKinda commented Oct 27, 2025

Motivation

Solution

Uh oh!

Uh oh!

Uh oh!

Uh oh!

RedKinda commented Nov 12, 2025

Uh oh!

Darksonn commented Nov 12, 2025

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

RedKinda commented Dec 3, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Implement uring in `AsyncWrite for fs::File` #7713

Implement uring in `AsyncWrite for fs::File` #7713