Date:   Wed, 20 Jul 2022 12:42:11 -0400
From:   Jeff Layton <jlayton@...nel.org>
To:     Lukas Czerner <lczerner@...hat.com>
Cc:     tytso@....edu, adilger.kernel@...ger.ca,
        linux-ext4@...r.kernel.org,
        linux-fsdevel <linux-fsdevel@...r.kernel.org>,
        Benjamin Coddington <bcodding@...hat.com>
Subject: Re: should we make "-o iversion" the default on ext4 ?

On Wed, 2022-07-20 at 17:22 +0200, Lukas Czerner wrote:

>  But not zero, at least
> every time the inode is loaded from disk it is scheduled for i_version
> update on the next attempted increment. Could that have an effect on
> some particular common workload you can think of?
> 

FWIW, it's doubtful that you'd even notice this. You'd almost certainly
be updating the mtime or ctime on the next change anyway, so updating
the i_version in that case is basically free. You will probably need to
do some a few extra atomic in-memory operations, but that's probably not
noticeable in something I/O constrained.

> 
> Could you provide some performance numbers for iversion case?
> 

I'm writing to a LVM volume on a no-name-brand ssd I have sitting
around. fio jobfile is here:

[global]
name=fio-seq-write
filename=fio-seq-write
rw=write
bs=4k
direct=0
numjobs=1
time_based
runtime=300

[file1]
size=1G
ioengine=libaio
iodepth=16

iversion support disabled:

$ fio ./4k-write.fio
file1: (g=0): rw=write, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=16
fio-3.27
Starting 1 process
file1: Laying out IO file (1 file / 1024MiB)
Jobs: 1 (f=1): [W(1)][100.0%][w=52.5MiB/s][w=13.4k IOPS][eta 00m:00s]
file1: (groupid=0, jobs=1): err= 0: pid=10056: Wed Jul 20 12:28:21 2022
  write: IOPS=96.3k, BW=376MiB/s (394MB/s)(110GiB/300001msec); 0 zone resets
    slat (nsec): min=1112, max=5727.5k, avg=1917.70, stdev=1300.30
    clat (nsec): min=1112, max=2146.5M, avg=156067.38, stdev=15568002.13
     lat (usec): min=3, max=2146.5k, avg=158.03, stdev=15568.00
    clat percentiles (usec):
     |  1.00th=[   36],  5.00th=[   36], 10.00th=[   37], 20.00th=[   37],
     | 30.00th=[   38], 40.00th=[   38], 50.00th=[   38], 60.00th=[   39],
     | 70.00th=[   39], 80.00th=[   40], 90.00th=[   42], 95.00th=[   44],
     | 99.00th=[   52], 99.50th=[   59], 99.90th=[   77], 99.95th=[   88],
     | 99.99th=[  169]
   bw (  KiB/s): min=15664, max=1599456, per=100.00%, avg=897761.07, stdev=504329.17, samples=257
   iops        : min= 3916, max=399864, avg=224440.26, stdev=126082.33, samples=257
  lat (usec)   : 2=0.01%, 4=0.01%, 10=0.01%, 20=0.01%, 50=98.80%
  lat (usec)   : 100=1.18%, 250=0.02%, 500=0.01%
  lat (msec)   : 10=0.01%, 2000=0.01%, >=2000=0.01%
  cpu          : usr=5.45%, sys=23.92%, ctx=78418, majf=0, minf=14
  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=100.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.1%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=0,28889786,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16

Run status group 0 (all jobs):
  WRITE: bw=376MiB/s (394MB/s), 376MiB/s-376MiB/s (394MB/s-394MB/s), io=110GiB (118GB), run=300001-300001msec

Disk stats (read/write):
    dm-7: ios=0/22878, merge=0/0, ticks=0/373254, in_queue=373254, util=43.89%, aggrios=0/99746, aggrmerge=0/9246, aggrticks=0/1406831, aggrin_queue=1408420, aggrutil=73.56%
  sda: ios=0/99746, merge=0/9246, ticks=0/1406831, in_queue=1408420, util=73.56%

mounted with -o iversion:

$ fio ./4k-write.fio
file1: (g=0): rw=write, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=16
fio-3.27
Starting 1 process
Jobs: 1 (f=1): [W(1)][100.0%][eta 00m:00s]                          
file1: (groupid=0, jobs=1): err= 0: pid=10369: Wed Jul 20 12:33:57 2022
  write: IOPS=96.2k, BW=376MiB/s (394MB/s)(110GiB/300001msec); 0 zone resets
    slat (nsec): min=1112, max=1861.5k, avg=1994.58, stdev=890.78
    clat (nsec): min=1392, max=2113.3M, avg=156252.71, stdev=15409487.99
     lat (usec): min=3, max=2113.3k, avg=158.30, stdev=15409.49
    clat percentiles (usec):
     |  1.00th=[   37],  5.00th=[   38], 10.00th=[   38], 20.00th=[   38],
     | 30.00th=[   39], 40.00th=[   39], 50.00th=[   40], 60.00th=[   40],
     | 70.00th=[   41], 80.00th=[   42], 90.00th=[   43], 95.00th=[   45],
     | 99.00th=[   53], 99.50th=[   60], 99.90th=[   79], 99.95th=[   90],
     | 99.99th=[  174]
   bw (  KiB/s): min=  304, max=1540000, per=100.00%, avg=870727.42, stdev=499371.78, samples=265
   iops        : min=   76, max=385000, avg=217681.82, stdev=124842.94, samples=265
  lat (usec)   : 2=0.01%, 4=0.01%, 10=0.01%, 20=0.01%, 50=98.49%
  lat (usec)   : 100=1.48%, 250=0.02%, 500=0.01%
  lat (msec)   : 2=0.01%, 2000=0.01%, >=2000=0.01%
  cpu          : usr=5.71%, sys=24.49%, ctx=52874, majf=0, minf=18
  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=100.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.1%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=0,28856695,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16

Run status group 0 (all jobs):
  WRITE: bw=376MiB/s (394MB/s), 376MiB/s-376MiB/s (394MB/s-394MB/s), io=110GiB (118GB), run=300001-300001msec

Disk stats (read/write):
    dm-7: ios=1/16758, merge=0/0, ticks=2/341817, in_queue=341819, util=47.93%, aggrios=1/98153, aggrmerge=0/5691, aggrticks=2/1399496, aggrin_queue=1400893, aggrutil=73.42%
  sda: ios=1/98153, merge=0/5691, ticks=2/1399496, in_queue=1400893, util=73.42%

-- 
Jeff Layton <jlayton@...nel.org>

Powered by blists - more mailing lists