linux-ext4 - Re: [RFC 0/9] ext4: Add direct-io atomic write support using fsawu

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date: Wed, 6 Mar 2024 11:22:44 +0000
From: John Garry <john.g.garry@...cle.com>
To: "Ritesh Harjani (IBM)" <ritesh.list@...il.com>,
        linux-fsdevel@...r.kernel.org, linux-ext4@...r.kernel.org
Cc: Ojaswin Mujoo <ojaswin@...ux.ibm.com>, Jan Kara <jack@...e.cz>,
        Theodore Ts'o <tytso@....edu>, Matthew Wilcox <willy@...radead.org>,
        "Darrick J . Wong" <djwong@...nel.org>,
        Luis Chamberlain
 <mcgrof@...nel.org>, linux-kernel@...r.kernel.org
Subject: Re: [RFC 0/9] ext4: Add direct-io atomic write support using fsawu

On 02/03/2024 07:41, Ritesh Harjani (IBM) wrote:
> Hello all,
> 
> This RFC series adds support for atomic writes to ext4 direct-io using
> filesystem atomic write unit. It's built on top of John's "block atomic
> write v5" series which adds RWF_ATOMIC flag interface to pwritev2() and enables
> atomic write support in underlying device driver and block layer.
> 
> This series uses the same RWF_ATOMIC interface for adding atomic write support
> to ext4's direct-io path. One can utilize it by 2 of the methods explained below.
> ((1)mkfs.ext4 -b <BS>, (2) with bigalloc).
> 
> Filesystem atomic write unit (fsawu):
> ============================================
> Atomic writes within ext4 can be supported using below 3 methods -
> 1. On a large pagesize system (e.g. Power with 64k pagesize or aarch64 with 64k pagesize),
>     we can mkfs using different blocksizes. e.g. mkfs.ext4 -b <4k/8k/16k/32k/64k).
>     Now if the underlying HW device supports atomic writes, than a corresponding
>     blocksize can be chosen as a filesystem atomic write unit (fsawu) which
>     should be within the underlying hw defined [awu_min, awu_max] range.
>     For such filesystem, fsawu_[min|max] both are equal to blocksize (e.g. 16k)
> 
>     On a smaller pagesize system this can be utilized when support for LBS is
>     complete (on ext4).
> 
> 2. EXT4 already supports a feature called bigalloc. In that ext4 can handle
>     allocation in cluster size units. So for e.g. we can create a filesystem with
>     4k blocksize but with 64k clustersize. Such a configuration can also be used
>     to support atomic writes if the underlying hw device supports it.
>     In such case the fsawu_min will most likely be the filesystem blocksize and
>     fsawu_max will mostly likely be the cluster size.
> 
>     So a user can do an atomic write of any size between [fsawu_min, fsawu_max]
>     range as long as it satisfies other constraints being laid out by HW device
>     (or by software stack) to support atomic writes.
>     e.g. len should be a power of 2, pos % len should be naturally
>     aligned and [start | end] (phys offsets) should not straddle over
>     an atomic write boundary.

JFYI, I gave this a quick try, and it seems to work ok. Naturally it 
suffers from the same issue discussed at 
https://lore.kernel.org/linux-fsdevel/434c570e-39b2-4f1c-9b49-ac5241d310ca@oracle.com/ 
with regards to writing to partially written extents, which I have tried 
to address properly in my v2 for that same series.

Thanks,
John