linux-ext4 - Re: How capacious and well-indexed are ext4, xfs and btrfs directories?

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:   Tue, 25 May 2021 22:26:17 +0100
From:   Matthew Wilcox <willy@...radead.org>
To:     Andreas Dilger <adilger@...ger.ca>
Cc:     Josh Triplett <josh@...htriplett.org>,
        David Howells <dhowells@...hat.com>,
        Theodore Ts'o <tytso@....edu>,
        "Darrick J. Wong" <djwong@...nel.org>, Chris Mason <clm@...com>,
        Ext4 Developers List <linux-ext4@...r.kernel.org>,
        xfs <linux-xfs@...r.kernel.org>,
        linux-btrfs <linux-btrfs@...r.kernel.org>,
        linux-cachefs@...hat.com,
        linux-fsdevel <linux-fsdevel@...r.kernel.org>,
        NeilBrown <neilb@...e.com>
Subject: Re: How capacious and well-indexed are ext4, xfs and btrfs
 directories?

On Tue, May 25, 2021 at 03:13:52PM -0600, Andreas Dilger wrote:
> Definitely "-o discard" is known to have a measurable performance impact,
> simply because it ends up sending a lot more requests to the block device,
> and those requests can be slow/block the queue, depending on underlying
> storage behavior.
> 
> There was a patch pushed recently that targets "-o discard" performance:
> https://patchwork.ozlabs.org/project/linux-ext4/list/?series=244091
> that needs a bit more work, but may be worthwhile to test if it improves
> your workload, and help put some weight behind landing it?

This all seems very complicated.  I have chosen with my current laptop
to "short stroke" the drive.  That is, I discarded the entire bdev,
then partitioned it roughly in half.  The second half has never seen
any writes.  This effectively achieves the purpose of TRIM/discard;
there are a lot of unused LBAs, so the underlying flash translation layer
always has plenty of spare space when it needs to empty an erase block.

Since the steady state of hard drives is full, I have to type 'make clean'
in my build trees more often than otherwise and remember to delete iso
images after i've had them lying around for a year, but I'd rather clean
up a little more often than get these weird performance glitches.

And if I really do need half a terabyte of space temporarily, I can
always choose to use the fallow range for a while, then discard it again.