SysadminNews

Linux 7.3 Brings RWF_DONTCACHE to Block Devices

On this page
  1. The flag, briefly
  2. Why block devices needed separate work
  3. Where the 65 percent number comes from
  4. The rest of the merge
  5. What we would do
  6. Sources and further reading

The block and io_uring changes merged for Linux 7.3 include one that anyone running a database on a raw device should note: RWF_DONTCACHE now works on block devices. The flag gives you buffered I/O that prunes its pages from the page cache as soon as the operation completes, so you get the simplicity of buffered reads and writes without leaving behind a cache full of data nobody will read again. Tal Zussman enabled the block device path and reported read and write gains from testing a single device in a virtual machine. The same merge brings dynamic area provisioning for io_uring zero copy receive, plus NVMe and MD RAID work.

The short answer

Linux 7.3 extends RWF_DONTCACHE, the uncached buffered I/O flag, to raw block devices. The flag runs ordinary buffered I/O but prunes the pages it touched as soon as the operation finishes, so heavy workloads stop filling the page cache with data nobody will read again. Tal Zussman enabled the block device path and reported read and write gains from a single device in a virtual machine. The obvious audience is databases that talk to a device directly. The same merge adds dynamic area provisioning for io_uring zero copy receive, parallel NVMe RDMA queue setup and MD RAID work.

Linux 6.14when RWF_DONTCACHE first arrived
Linux 7.3when block devices finally get it
65%throughput gain in the original filesystem work
Answer card summarising the Linux 7.3 block layer changes: RWF_DONTCACHE uncached buffered I/O now supported on block devices thanks to Tal Zussman, which suits databases running on raw devices, alongside dynamic area provisioning for io_uring zero copy receive and NVMe RDMA queue setup parallelisation.
What landed in the Linux 7.3 block layer. PNG

There is a specific frustration in running a database that manages its own cache and then watching the kernel diligently build a second copy of everything in the page cache. Linux 7.3 removes one of the remaining reasons to reach for O_DIRECT to escape it.

The flag, briefly

RWF_DONTCACHE is buffered I/O that cleans up after itself. The operation goes through the page cache exactly as usual, and the ranges it touched are pruned once it completes.

That sounds minor and is not. The cost of buffered I/O on fast storage is rarely the copy, it is what the leftover pages do to reclaim afterwards. Fill enough RAM with pages nobody will read again and kswapd goes to work, and then your predictable I/O times stop being predictable. Pruning on completion removes the cause rather than tuning the symptom.

You set it per operation, with preadv2 or pwritev2, or in io_uring by putting the flag in sqe->rw_flags. There is no mount option and no global switch, which is the right design: the application knows which reads will be reused and the kernel does not.

Why block devices needed separate work

The original implementation was built around filesystem I/O. The iomap layer provides most of the machinery, so filesystems that use iomap fully picked up the flag nearly for free, while ones doing more of their own work, ext4 included, needed extra effort.

Raw block device access does not go through a filesystem at all, so it was never covered. Tal Zussman closed that gap for Linux 7.3, and reported read and write benefits from testing on a single block device in a virtual machine.

The audience is narrow and knows who it is. If you run PostgreSQL or another database on a filesystem, this is not your change. If you run something that addresses a device directly, it is.

Checklist card for the Linux 7.3 block and io_uring merge: RWF_DONTCACHE now works on raw block devices, the flag is set per operation through preadv2, pwritev2 or io_uring rather than globally, io_uring zero copy receive gains dynamic area provisioning, NVMe RDMA queue allocation and start up are parallelised, and a caution that the 65 percent figure came from the original filesystem work rather than from block device testing.
What to take from this merge, and what not to assume. PNG

Where the 65 percent number comes from

It is worth being precise about this, because the figure circulates without its context.

Jens Axboe first proposed an uncached buffered mode as RWF_UNCACHED in 2019 and the effort stalled. He revived it in November 2024, and RWF_DONTCACHE landed in Linux 6.14. In that work he reported roughly 65 percent better throughput for both reads and writes, fully predictable I/O timing, and a substantial CPU reduction with no kswapd reclaim activity at all.

Those numbers describe his test setup on the filesystem path in early 2025. They are not a measurement of the new block device support, and they are not a promise about your storage. Treat them as evidence that the mechanism works, then measure your own workload.

The rest of the merge

The io_uring side is focused on zero copy receive, where the notable change is dynamic area provisioning. Memory areas can now be allocated and provisioned on demand instead of being pre-allocated statically, which helps anyone who had to choose between wasting memory and capping throughput at provisioning time.

On the block side there is Clang context analysis support for the NVMe host driver, parallelised allocation and start up of NVMe RDMA I/O queues, which shortens the time to bring a fabric attached target online, fixes for Apple NVMe hardware, and a set of MD RAID enhancements.

Storage has had a strong cycle overall in 7.3, including direct I/O and fsync work in Btrfs, DAX support for fabric attached memory and Time Machine interoperability in ksmbd.

What we would do

If you maintain an application that reads or writes large volumes it will never touch again, log ingestion, backup, checksum passes, bulk restores, this flag is worth a day of work regardless of whether you are on a block device or a filesystem. It is a per operation flag, so you can apply it to the paths you know are streaming and leave everything else alone.

If you were using O_DIRECT solely to keep the page cache clean, this is the moment to re-examine that decision. You may be carrying alignment handling and application complexity that a flag now removes.

And if you run a database on a raw device, wait for the kernel to reach your distribution, then benchmark it properly, on your storage and your access pattern. The mechanism is sound. The size of the win is yours to find out.

Sources and further reading

Frequently asked questions

What does RWF_DONTCACHE actually do?

It gives you buffered I/O that cleans up after itself. A normal buffered read or write goes through the page cache and leaves its pages there, which is exactly what you want when the data will be read again and exactly what you do not want when it will not. RWF_DONTCACHE performs the same page cache I/O but prunes the touched ranges once the operation completes, so the cache does not fill with data that has no future value. The practical effect is that reclaim has far less work to do, which is where the performance comes from. You set it per operation with preadv2 or pwritev2, or in io_uring by putting the flag in sqe->rw_flags.

How is this different from O_DIRECT?

O_DIRECT bypasses the page cache entirely, and it makes you pay for that with alignment requirements on offsets, lengths and buffers, awkward interactions with buffered access to the same file, and a fair amount of application complexity. RWF_DONTCACHE keeps you on the buffered path, with the kernel handling synchronisation as usual, and simply drops the pages afterwards. So you get the main benefit people reach for O_DIRECT to obtain, no page cache pollution, without inheriting the constraints. It is not a replacement for O_DIRECT in every case, particularly where an application wants to manage its own cache with strict control, but it removes a common reason to use it.

Why did block device support need doing separately?

Because the original implementation was built around the filesystem I/O paths. The iomap layer supplies most of the infrastructure, so filesystems that use iomap fully picked the flag up almost for free when it landed, while filesystems doing more of their own work, ext4 among them, needed extra effort. Raw block device access does not go through a filesystem at all, so it was simply not covered. Tal Zussman's work in Linux 7.3 fills that gap, and the audience is specific: databases and other applications that talk to a device directly rather than through a filesystem.

Where did the flag come from and what performance did it show originally?

Jens Axboe first proposed an uncached buffered mode as RWF_UNCACHED in 2019, and that attempt stalled. He revived it in November 2024 as the uncached buffered IO series, and RWF_DONTCACHE arrived in Linux 6.14. In that original work he reported roughly a 65 percent throughput improvement for both reads and writes, fully predictable I/O timing, and a substantial CPU reduction with no kswapd reclaim activity at all. Those numbers describe the filesystem path in his test setup, not the new block device support, so do not carry them over to your own case without measuring.

What else is in the Linux 7.3 block and io_uring merge?

On the io_uring side the focus is zero copy receive, and the notable addition is dynamic area provisioning, which lets memory areas be allocated and provisioned on demand instead of being pre-allocated statically. That matters for anyone sizing a service where the static allocation was either wasteful or a ceiling. On the block side there is Clang context analysis support for the NVMe host driver, parallelised allocation and start up of NVMe RDMA I/O queues, which shortens the time to bring a fabric attached target online, fixes for Apple NVMe hardware, and a set of MD RAID enhancements.