linux-hardening - Re: Fwd: Waking up from resume locks up on sr device

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:   Mon, 12 Jun 2023 12:09:29 +0900
From:   Damien Le Moal <dlemoal@...nel.org>
To:     Bart Van Assche <bvanassche@....org>,
        Bagas Sanjaya <bagasdotme@...il.com>,
        Pavel Machek <pavel@....cz>
Cc:     "Rafael J. Wysocki" <rafael@...nel.org>,
        Len Brown <len.brown@...el.com>,
        Greg Kroah-Hartman <gregkh@...uxfoundation.org>,
        Kees Cook <keescook@...omium.org>,
        Tony Luck <tony.luck@...el.com>,
        "Guilherme G. Piccoli" <gpiccoli@...lia.com>,
        Thorsten Leemhuis <linux@...mhuis.info>,
        "James E.J. Bottomley" <jejb@...ux.ibm.com>,
        "Martin K. Petersen" <martin.petersen@...cle.com>,
        Phillip Potter <phil@...lpotter.co.uk>,
        Joe Breuer <linux-kernel@...reuer.net>,
        Linux Power Management <linux-pm@...r.kernel.org>,
        Linux Kernel Mailing List <linux-kernel@...r.kernel.org>,
        Linux Hardening <linux-hardening@...r.kernel.org>,
        Linux Regressions <regressions@...ts.linux.dev>,
        Linux SCSI <linux-scsi@...r.kernel.org>,
        Alan Stern <stern@...land.harvard.edu>,
        Dan Williams <dan.j.williams@...el.com>,
        Hannes Reinecke <hare@...e.com>,
        Adrian Hunter <adrian.hunter@...el.com>,
        Martin Kepplinger <martin.kepplinger@...i.sm>,
        Kai-Heng Feng <kai.heng.feng@...onical.com>
Subject: Re: Fwd: Waking up from resume locks up on sr device

On 6/11/23 00:03, Bart Van Assche wrote:
> On 6/10/23 06:27, Bagas Sanjaya wrote:
>> On 6/10/23 15:55, Pavel Machek wrote:
>>>>> #regzbot introduced: v5.0..v6.4-rc5 https://bugzilla.kernel.org/show_bug.cgi?id=217530
>>>>> #regzbot title: Waking up from resume locks up on SCSI CD/DVD drive
>>>>>
>>>> The reporter had found the culprit (via bisection), so:
>>>>
>>>> #regzbot introduced: a19a93e4c6a98c
>>> Maybe cc the authors of that commit?
>>
>> Ah! I forgot to do that! Thanks anyway.
> 
> Hi Damien,
> 
> Why does the ATA code call scsi_rescan_device() before system resume has
> finished? Would ATA devices still work with the patch below applied?

I do not know the PM code well at all, need to dig into it. But your patch
worries me as it seems it would prevent rescan of the device on a resume, which
can be an issue if the device has changed.

I am not yet 100% clear on the root cause for this, but I think it comes from
the fact that ata_port_pm_resume() runs before the sci device resume is done, so
with scsi_dev->power.is_suspended still true. And ata_port_pm_resume() calls
ata_port_resume_async() which triggers EH (which will do reset + rescan)
asynchronously. So it looks like we have scsi device resume and libata EH for
rescan fighting each others for the scan mutex and device lock, leading to deadlock.

Trying to recreate this issue now to confirm and debug further. But I suspect
the solution to this may be best implemented in libata, not in scsi.
This looks definitely related to this thread:

https://lore.kernel.org/linux-scsi/7b553268-69d3-913a-f9de-28f8d45bdb1e@acm.org/

Similaraly to your comment on that thread, having to look at
dev->power.is_suspended is not ideal I think. What we need is to have ata and
scsi pm resume be synchronized, but I am not yet 100% clear on the scsi layer side.

> 
> Thanks,
> 
> Bart.
> 
> 
> diff --git a/drivers/scsi/scsi_scan.c b/drivers/scsi/scsi_scan.c
> index 6a959c993dd8..be3971b7fd27 100644
> --- a/drivers/scsi/scsi_scan.c
> +++ b/drivers/scsi/scsi_scan.c
> @@ -1629,6 +1629,20 @@ void scsi_rescan_device(struct device *dev)
>   {
>   	struct scsi_device *sdev = to_scsi_device(dev);
> 
> +#ifdef CONFIG_PM_SLEEP
> +	/*
> +	 * The ATA subsystem may call scsi_rescan_device() before resuming has
> +	 * finished. If this happens, prevent a deadlock on the device_lock()
> +	 * call by skipping rescanning.
> +	 */
> +	if (dev->power.is_suspended)
> +		return;
> +#endif
> +
> +	/*
> +	 * Serialize scsi_driver.rescan() calls and scsi_driver.gendrv.remove()
> +	 * calls.
> +	 */
>   	device_lock(dev);
> 
>   	scsi_attach_vpd(sdev);
> 

-- 
Damien Le Moal
Western Digital Research