linux-ext4 - Re: [PATCH 2/2] Add batched discard support for ext4

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:	Tue, 27 Apr 2010 17:29:06 +0200
From:	Edward Shishkin <edward@...hat.com>
To:	Lukas Czerner <lczerner@...hat.com>
CC:	Jeff Moyer <jmoyer@...hat.com>, Ric Wheeler <rwheeler@...hat.com>,
	Jan Kara <jack@...e.cz>,
	Greg Freemyer <greg.freemyer@...il.com>,
	Eric Sandeen <sandeen@...hat.com>,
	Mark Lord <kernel@...savvy.com>, linux-ext4@...r.kernel.org,
	Edward Shishkin <eshishki@...hat.com>,
	Eric Sandeen <esandeen@...hat.com>,
	Christoph Hellwig <hch@...radead.org>
Subject: Re: [PATCH 2/2] Add batched discard support for ext4 - using rbtree

Lukas Czerner wrote:
> On Mon, 26 Apr 2010, Lukas Czerner wrote:
>
>   
>> Create an ioctl which walks through all the free extents in each
>> allocating group and discard those extents. As an addition to improve
>> its performance one can specify minimum free extent length, so ioctl
>> will not bother with shorter extents.
>>
>> This of course means, that with each invocation the ioctl must walk
>> through whole file system, checking and discarding free extents, which
>> is not very efficient. The best way to avoid this is to keep track of
>> deleted (freed) blocks. Then the ioctl have to trim just those free
>> extents which were recently freed.
>>
>> In order to implement this I have created new structure
>> ext4_deleted_data which represents deleted extent in per-group rbtree.
>> When blocks are freed, extents are stored in the tree (possibly merged
>> with other extents). The ioctl then can walk through the tree, take out
>> extents from the tree, compare them with the free extents and possibly
>> trim them.
>>
>> Note that there is support for setting minimum extent length in ioctl
>> call, so it will not bother with shorter extents. Also, when the
>> previously deleted range is taken from the tree and it is not entirely
>> free, the free fragments are discarded and extents shorter than minlen
>> are NOT returned back to the tree to avoid fragmentation of the tree
>> which could lead to the big memory consumption.
>>
>> But you may notice, that there is one problem. bb_bitmap_deleted does
>> not survive umount. To bypass the problem the first ioctl call have to
>> walk through whole file system trimming all free extents.
>>
>> Signed-off-by: Lukas Czerner <lczerner@...hat.com>
>>     
>
> For now it just ignores the small extents to avoid fragmentation. As I
> said before, I agree that they should not be ignored, I just need to
> figure out the way to do this efficiently.

I suggest to not start with optimisations: let's first take a look
what is going on in the simple case. Benchmarks are our friends..

Edward.

>  
>
> Also it was not properly tested yet.
>
>
> -Lukas.
>   

--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html