lists.openwall.net   lists  /  announce  owl-users  owl-dev  john-users  john-dev  passwdqc-users  yescrypt  popa3d-users  /  oss-security  kernel-hardening  musl  sabotage  tlsify  passwords  /  crypt-dev  xvendor  /  Bugtraq  Full-Disclosure  linux-kernel  linux-netdev  linux-ext4  linux-hardening  linux-cve-announce  PHC 
Open Source and information security mailing list archives
 
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Date:   Sat, 16 Mar 2019 12:57:57 -0600
From:   Andreas Dilger <adilger@...ger.ca>
To:     Burke Harper <go88team@...il.com>
Cc:     linux-ext4@...r.kernel.org
Subject: Re: Should Never Happen: Resize Inode Corrupt

You could kill e2fsck and disable the resize_inode feature?  There is a different resize mechanism available now (meta_bg) that doesn't need it. 

Cheers, Andreas

> On Mar 16, 2019, at 11:11, Burke Harper <go88team@...il.com> wrote:
> 
> I updated to 1.44.5.
> 
> It's been going along now for about 21 hours.  There is some
> noticeable io this time around.  Looks like it's still trying to
> recreate the resize inode.
> 
>> On Fri, Mar 15, 2019 at 3:19 PM Andreas Dilger <adilger@...ger.ca> wrote:
>> 
>> Kill your e2fsck and upgrade to the latest version 1.44.5, as it has a lot of fixes over 1.42.13.
>> 
>> If you have the ability, make a "dd" copy of the filesystem first, or a snapshot, and run e2fsck on that first.
>> 
>> Cheers, Andreas
>> 
>>> On Mar 15, 2019, at 00:38, Burke Harper <go88team@...il.com> wrote:
>>> 
>>> Over the past weekend, I added 2 more drives to my /dev/md0 array:
>>> 
>>> sudo mdadm --detail /dev/md0
>>> /dev/md0:
>>>       Version : 1.2
>>> Creation Time : Sat Dec 16 18:32:08 2017
>>>    Raid Level : raid6
>>>    Array Size : 54697266176 (52163.38 GiB 56010.00 GB)
>>> Used Dev Size : 7813895168 (7451.91 GiB 8001.43 GB)
>>>  Raid Devices : 9
>>> Total Devices : 9
>>>   Persistence : Superblock is persistent
>>> 
>>> Intent Bitmap : Internal
>>> 
>>>   Update Time : Mon Mar 11 05:13:12 2019
>>>         State : clean
>>> Active Devices : 9
>>> Working Devices : 9
>>> Failed Devices : 0
>>> Spare Devices : 0
>>> 
>>>        Layout : left-symmetric
>>>    Chunk Size : 512K
>>> 
>>>          Name : powerhouse:0  (local to host powerhouse)
>>>          UUID : 19b5c7a5:59e4bd00:b4b1c18c:089df9bd
>>>        Events : 45981
>>> 
>>>   Number   Major   Minor   RaidDevice State
>>>      0       8        0        0      active sync   /dev/sda
>>>      1       8       16        1      active sync   /dev/sdb
>>>      2       8       32        2      active sync   /dev/sdc
>>>      3       8       48        3      active sync   /dev/sdd
>>>      5       8      144        4      active sync   /dev/sdj
>>>      4       8      128        5      active sync   /dev/sdi
>>>      6       8      112        6      active sync   /dev/sdh
>>>      8       8       80        7      active sync   /dev/sdf
>>>      7       8       64        8      active sync   /dev/sde
>>> 
>>> Afterwards I did an fsck:
>>> 
>>> sudo fsck.ext4 -f /dev/md0
>>> e2fsck 1.42.13 (17-May-2015)
>>> Pass 1: Checking inodes, blocks, and sizes
>>> Pass 2: Checking directory structure
>>> Pass 3: Checking directory connectivity
>>> Pass 4: Checking reference counts
>>> Pass 5: Checking group summary information
>>> /dev/md0: 70089/1220923392 files (3.6% non-contiguous),
>>> 7726498938/9767368960 blocks
>>> 
>>> Following that, I tried to perform an offline resize:
>>> 
>>> sudo resize2fs /dev/md0
>>> resize2fs 1.42.13 (17-May-2015)
>>> Resizing the filesystem on /dev/md0 to 13674316544 (4k) blocks.
>>> Should never happen: resize inode corrupt!
>>> 
>>> After having done that, it looks like it should have been an online
>>> resize from reading a thread on here from 2015.
>>> 
>>> After trying the resize I tried to do another fsck:
>>> 
>>> sudo fsck.ext4 -f /dev/md0
>>> e2fsck 1.42.13 (17-May-2015)
>>> ext2fs_check_desc: Corrupt group descriptor: bad block for inode table
>>> fsck.ext4: Group descriptors look bad... trying backup blocks...
>>> Superblock has an invalid journal (inode 8).
>>> Clear<y>? yes
>>> *** ext3 journal has been deleted - filesystem is now ext2 only ***
>>> 
>>> Resize inode not valid.  Recreate<y>? yes
>>> 
>>> It's been stuck here for days, with:
>>> 
>>> 14827 root      20   0  141796 121044   2688 R  93.8  0.4   5546:26 fsck.ext4
>>> 
>>> It's been running at around 100% the whole time.  I don't see any disk
>>> io happening either.
>>> 
>>> sudo dumpe2fs -h /dev/md0
>>> dumpe2fs 1.42.13 (17-May-2015)
>>> Filesystem volume name:   <none>
>>> Last mounted on:          /Media10
>>> Filesystem UUID:          d36119d5-e3ec-47f7-b93e-124eb4598367
>>> Filesystem magic number:  0xEF53
>>> Filesystem revision #:    1 (dynamic)
>>> Filesystem features:      has_journal ext_attr resize_inode dir_index
>>> filetype extent 64bit flex_bg sparse_super large_file huge_file
>>> uninit_bg dir_nlink extra_isize
>>> Filesystem flags:         signed_directory_hash
>>> Default mount options:    user_xattr acl
>>> Filesystem state:         clean with errors
>>> Errors behavior:          Continue
>>> Filesystem OS type:       Linux
>>> Inode count:              1709293568
>>> Block count:              13674316544
>>> Reserved block count:     683715825
>>> Free blocks:              5886063280
>>> Free inodes:              1709223479
>>> First block:              0
>>> Block size:               4096
>>> Fragment size:            4096
>>> Group descriptor size:    64
>>> Blocks per group:         32768
>>> Fragments per group:      32768
>>> Inodes per group:         4096
>>> Inode blocks per group:   256
>>> RAID stride:              128
>>> RAID stripe width:        256
>>> Flex block group size:    16
>>> Filesystem created:       Sun Dec 17 10:10:08 2017
>>> Last mount time:          Sat Mar  9 17:58:06 2019
>>> Last write time:          Mon Mar 11 05:48:14 2019
>>> Mount count:              0
>>> Maximum mount count:      -1
>>> Last checked:             Mon Mar 11 05:16:14 2019
>>> Check interval:           0 (<none>)
>>> Lifetime writes:          29 TB
>>> Reserved blocks uid:      0 (user root)
>>> Reserved blocks gid:      0 (group root)
>>> First inode:              11
>>> Inode size:               256
>>> Required extra isize:     28
>>> Desired extra isize:      28
>>> Journal inode:            8
>>> Default directory hash:   half_md4
>>> Directory Hash Seed:      23fd4260-aee9-4f36-8406-240f3b7a39d2
>>> Journal backup:           inode blocks
>>> Journal superblock magic number invalid!
>>> 
>>> 
>>> Should I let the fsck continue, or is it safe to exit to try something else.
>>> 
>>> I recently did an offline resize a few weeks ago on the same array and
>>> it worked out just fine.  I'm not sure what happened this time, I
>>> followed the same steps.
>>> 
>>> Thanks for any help.

Powered by blists - more mailing lists

Powered by Openwall GNU/*/Linux Powered by OpenVZ