Date: Wed, 30 Jul 2025 14:29:37 +0200 From: Kristof Provost <kp@FreeBSD.org> To: Mark Johnston <markj@FreeBSD.org>, Hareshx Sankar Raj <hareshx.sankar.raj@intel.com> Cc: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org Subject: Re: git: ded037e65e52 - main - qat: driver updates to improve code and fix bugs Message-ID: <9DB885DF-8BDD-451F-B928-0C10E881C4CF@FreeBSD.org> In-Reply-To: <202506061357.556Dv9Zc084087@gitrepo.freebsd.org> References: <202506061357.556Dv9Zc084087@gitrepo.freebsd.org>
next in thread | previous in thread | raw e-mail | index | archive | help
--=_MailMate_B08AB6DB-7748-4F2A-B2B9-63ABCCFE9199_= Content-Type: text/plain; charset=UTF-8; format=flowed; markup=markdown Content-Transfer-Encoding: quoted-printable On 6 Jun 2025, at 15:57, Mark Johnston wrote: > The branch main has been updated by markj: > > URL: = > https://cgit.FreeBSD.org/src/commit/?id=3Dded037e65e5239671b1292ec987a2= e0894b217b5 > > commit ded037e65e5239671b1292ec987a2e0894b217b5 > Author: Hareshx Sankar Raj <hareshx.sankar.raj@intel.com> > AuthorDate: 2025-05-07 09:38:21 +0000 > Commit: Mark Johnston <markj@FreeBSD.org> > CommitDate: 2025-06-06 13:43:54 +0000 > > qat: driver updates to improve code and fix bugs > > Bug fixes and improvements are done for the qat code base > to improve code quality. > > Reviewed by: markj, ziaee > MFC after: 2 weeks > Sponsored by: Intel Corporation > Differential Revision: https://reviews.freebsd.org/D50379 > --- =E2=80=A6 > diff --git a/sys/dev/qat/qat_hw/qat_c3xxx/adf_c3xxx_hw_data.h = > b/sys/dev/qat/qat_hw/qat_c3xxx/adf_c3xxx_hw_data.h > index bfc5db1f5e5c..cddfc3f84853 100644 > --- a/sys/dev/qat/qat_hw/qat_c3xxx/adf_c3xxx_hw_data.h > +++ b/sys/dev/qat/qat_hw/qat_c3xxx/adf_c3xxx_hw_data.h > @@ -1,11 +1,11 @@ > /* SPDX-License-Identifier: BSD-3-Clause */ > -/* Copyright(c) 2007-2022 Intel Corporation */ > +/* Copyright(c) 2007-2025 Intel Corporation */ > #ifndef ADF_C3XXX_HW_DATA_H_ > #define ADF_C3XXX_HW_DATA_H_ > > /* PCIe configuration space */ > -#define ADF_C3XXX_PMISC_BAR 0 > -#define ADF_C3XXX_ETR_BAR 1 > +#define ADF_C3XXX_PMISC_BAR 1 > +#define ADF_C3XXX_ETR_BAR 2 > #define ADF_C3XXX_RX_RINGS_OFFSET 8 > #define ADF_C3XXX_TX_RINGS_MASK 0xFF > #define ADF_C3XXX_MAX_ACCELERATORS 3 We=E2=80=99re seeing panics loading the QAT driver on Atom Processor C300= 0 = (with pfSense). I believe this change is the trigger. This causes us to look for the ETR registers in the third BAR, but there = are only two on that hardware: # pciconf -l -b -vV pci0:1:0:0 none0@pci0:1:0:0: class=3D0x0b4000 rev=3D0x11 hdr=3D0x00 vendor=3D0x8086= = device=3D0x19e2 subvendor=3D0x8086 subdevice=3D0x19e2 vendor =3D 'Intel Corporation' device =3D 'Atom Processor C3000 Series QuickAssist Technology' class =3D processor bar [18] =3D type Memory, range 64, base 0x80700000, size 262144, = enabled bar [20] =3D type Memory, range 64, base 0x80740000, size 262144, = enabled That produces this backtrace: Fatal trap 12: page fault while in kernel mode cpuid =3D 1; apic id =3D 18 fault virtual address =3D 0x10 fault code =3D supervisor read data, page not present instruction pointer =3D 0x20:0xffffffff83c4c25a stack pointer =3D 0x28:0xfffffe00681c26b0 frame pointer =3D 0x28:0xfffffe00681c26b0 code segment =3D base 0x0, limit 0xfffff, type 0x1b =3D DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags =3D interrupt enabled, resume, IOPL =3D 0 current process =3D 3075 (kldload) rdi: 0000000000000000 rsi: 0000000000000000 rdx: 0000000000000000 rcx: 0000000000000000 r8: fffffe00681c2715 r9: 000000000000000a rax: 00000000000001ff rbx: 0000000000000000 rbp: fffffe00681c26b0 r10: 0000000000000000 r11: 0000000000000001 r12: 0000000000000000 r13: fffff80003a1a000 r14: 0000000000000000 r15: 0000000000000008 trap number =3D 12 panic: page fault cpuid =3D 1 time =3D 1753859856 KDB: enter: panic [ thread pid 3075 tid 100160 ] Stopped at kdb_enter+0x33: movq $0,0x1b55532(%rip) db> bt Tracing pid 3075 tid 100160 td 0xfffff80100d4a780 kdb_enter() at kdb_enter+0x33/frame 0xfffffe00681c2530 panic() at panic+0x43/frame 0xfffffe00681c2590 trap_pfault() at trap_pfault+0x3c9/frame 0xfffffe00681c25e0 calltrap() at calltrap+0x8/frame 0xfffffe00681c25e0 --- trap 0xc, rip =3D 0xffffffff83c4c25a, rsp =3D 0xfffffe00681c26b0, rb= p =3D = 0xfffffe00681c26b0 --- write_csr_ring_config() at write_csr_ring_config+0xa/frame = 0xfffffe00681c26b0 adf_init_etr_data() at adf_init_etr_data+0x36a/frame 0xfffffe00681c2840 adf_dev_init() at adf_dev_init+0xc1/frame 0xfffffe00681c28f0 adf_attach() at adf_attach+0x4e1/frame 0xfffffe00681c2950 device_attach() at device_attach+0x43d/frame 0xfffffe00681c29a0 pci_driver_added() at pci_driver_added+0xf2/frame 0xfffffe00681c29e0 devclass_driver_added() at devclass_driver_added+0x29/frame = 0xfffffe00681c2a10 devclass_add_driver() at devclass_add_driver+0x11e/frame = 0xfffffe00681c2a50 module_register_init() at module_register_init+0x85/frame = 0xfffffe00681c2a80 linker_load_module() at linker_load_module+0xc0f/frame = 0xfffffe00681c2d80 kern_kldload() at kern_kldload+0x165/frame 0xfffffe00681c2dd0 sys_kldload() at sys_kldload+0x59/frame 0xfffffe00681c2e00 amd64_syscall() at amd64_syscall+0x126/frame 0xfffffe00681c2f30 fast_syscall_common() at fast_syscall_common+0xf8/frame = 0xfffffe00681c2f30 --More-- Because we=E2=80=99ve tried to use a pci_bars[2] entry with virt_addr =3D= =3D 0 = (because it=E2=80=99s outside of the BARs we set up). Reverting just the above hunk of the patch fixes the panic for us. Best regards, Kristof --=_MailMate_B08AB6DB-7748-4F2A-B2B9-63ABCCFE9199_= Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable <!DOCTYPE html> <html> <head> <meta http-equiv=3D"Content-Type" content=3D"text/xhtml; charset=3Dutf-8"= > </head> <body><div style=3D"font-family: sans-serif;"><div class=3D"markdown" sty= le=3D"white-space: normal;"> <p dir=3D"auto">On 6 Jun 2025, at 15:57, Mark Johnston wrote:</p> <blockquote style=3D"margin: 0 0 5px; padding-left: 5px; border-left: 2px= solid #136BCE; color: #136BCE;"> <p dir=3D"auto">The branch main has been updated by markj:</p> <p dir=3D"auto">URL: <a href=3D"https://cgit.FreeBSD.org/src/commit/?id=3D= ded037e65e5239671b1292ec987a2e0894b217b5">https://cgit.FreeBSD.org/src/co= mmit/?id=3Dded037e65e5239671b1292ec987a2e0894b217b5</a></p> <p dir=3D"auto">commit ded037e65e5239671b1292ec987a2e0894b217b5<br> Author: Hareshx Sankar Raj <a href=3D"mailto:hareshx.sankar.raj@intel= =2Ecom">hareshx.sankar.raj@intel.com</a><br> AuthorDate: 2025-05-07 09:38:21 +0000<br> Commit: Mark Johnston <a href=3D"mailto:markj@FreeBSD.org">markj@Free= BSD.org</a><br> CommitDate: 2025-06-06 13:43:54 +0000</p> <pre style=3D"margin-left: 15px; margin-right: 15px; padding: 5px; border= : thin solid gray; overflow-x: auto; max-width: 90vw; background-color: #= E4E4E4;"><code style=3D"padding: 0 0.25em; background-color: #E4E4E4;">qa= t: driver updates to improve code and fix bugs Bug fixes and improvements are done for the qat code base to improve code quality. Reviewed by: markj, ziaee MFC after: 2 weeks Sponsored by: Intel Corporation Differential Revision: https://reviews.freebsd.org/D50379 </code></pre> <hr> </blockquote> <p dir=3D"auto">=E2=80=A6</p> <blockquote style=3D"margin: 0 0 5px; padding-left: 5px; border-left: 2px= solid #136BCE; color: #136BCE;"> <p dir=3D"auto">diff --git a/sys/dev/qat/qat_hw/qat_c3xxx/adf_c3xxx_hw_da= ta.h b/sys/dev/qat/qat_hw/qat_c3xxx/adf_c3xxx_hw_data.h<br> index bfc5db1f5e5c..cddfc3f84853 100644<br> --- a/sys/dev/qat/qat_hw/qat_c3xxx/adf_c3xxx_hw_data.h<br> +++ b/sys/dev/qat/qat_hw/qat_c3xxx/adf_c3xxx_hw_data.h<br> @@ -1,11 +1,11 @@<br> /* SPDX-License-Identifier: BSD-3-Clause <em>/<br> -/</em> Copyright(c) 2007-2022 Intel Corporation <em>/<br> +/</em> Copyright(c) 2007-2025 Intel Corporation */<br> #ifndef ADF_C3XXX_HW_DATA_H_<br> #define ADF_C3XXX_HW_DATA_H_</p> <p dir=3D"auto">/* PCIe configuration space */<br> -#define ADF_C3XXX_PMISC_BAR 0<br> -#define ADF_C3XXX_ETR_BAR 1<br> +#define ADF_C3XXX_PMISC_BAR 1<br> +#define ADF_C3XXX_ETR_BAR 2<br> #define ADF_C3XXX_RX_RINGS_OFFSET 8<br> #define ADF_C3XXX_TX_RINGS_MASK 0xFF<br> #define ADF_C3XXX_MAX_ACCELERATORS 3</p> </blockquote> <p dir=3D"auto">We=E2=80=99re seeing panics loading the QAT driver on Ato= m Processor C3000 (with pfSense).</p> <p dir=3D"auto">I believe this change is the trigger.<br> This causes us to look for the ETR registers in the third BAR, but there = are only two on that hardware:</p> <pre style=3D"margin-left: 15px; margin-right: 15px; padding: 5px; border= : thin solid gray; overflow-x: auto; max-width: 90vw; background-color: #= E4E4E4;"><code style=3D"padding: 0 0.25em; background-color: #E4E4E4;"># = pciconf -l -b -vV pci0:1:0:0 none0@pci0:1:0:0: class=3D0x0b4000 rev=3D0x11 hdr=3D0x00 vendor=3D0x8086 = device=3D0x19e2 subvendor=3D0x8086 subdevice=3D0x19e2 vendor =3D 'Intel Corporation' device =3D 'Atom Processor C3000 Series QuickAssist Technology' class =3D processor bar [18] =3D type Memory, range 64, base 0x80700000, size 262144, e= nabled bar [20] =3D type Memory, range 64, base 0x80740000, size 262144, e= nabled </code></pre> <p dir=3D"auto">That produces this backtrace:</p> <pre style=3D"margin-left: 15px; margin-right: 15px; padding: 5px; border= : thin solid gray; overflow-x: auto; max-width: 90vw; background-color: #= E4E4E4;"><code style=3D"padding: 0 0.25em; background-color: #E4E4E4;">Fa= tal trap 12: page fault while in kernel mode cpuid =3D 1; apic id =3D 18 fault virtual address =3D 0x10 fault code =3D supervisor read data, page not present instruction pointer =3D 0x20:0xffffffff83c4c25a stack pointer =3D 0x28:0xfffffe00681c26b0 frame pointer =3D 0x28:0xfffffe00681c26b0 code segment =3D base 0x0, limit 0xfffff, type 0x1b =3D DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags =3D interrupt enabled, resume, IOPL =3D 0 current process =3D 3075 (kldload) rdi: 0000000000000000 rsi: 0000000000000000 rdx: 0000000000000000 rcx: 0000000000000000 r8: fffffe00681c2715 r9: 000000000000000a rax: 00000000000001ff rbx: 0000000000000000 rbp: fffffe00681c26b0 r10: 0000000000000000 r11: 0000000000000001 r12: 0000000000000000 r13: fffff80003a1a000 r14: 0000000000000000 r15: 0000000000000008 trap number =3D 12 panic: page fault cpuid =3D 1 time =3D 1753859856 KDB: enter: panic [ thread pid 3075 tid 100160 ] Stopped at kdb_enter+0x33: movq $0,0x1b55532(%rip) db> bt Tracing pid 3075 tid 100160 td 0xfffff80100d4a780 kdb_enter() at kdb_enter+0x33/frame 0xfffffe00681c2530 panic() at panic+0x43/frame 0xfffffe00681c2590 trap_pfault() at trap_pfault+0x3c9/frame 0xfffffe00681c25e0 calltrap() at calltrap+0x8/frame 0xfffffe00681c25e0 --- trap 0xc, rip =3D 0xffffffff83c4c25a, rsp =3D 0xfffffe00681c26b0, rbp= =3D 0xfffffe00681c26b0 --- write_csr_ring_config() at write_csr_ring_config+0xa/frame 0xfffffe00681c= 26b0 adf_init_etr_data() at adf_init_etr_data+0x36a/frame 0xfffffe00681c2840 adf_dev_init() at adf_dev_init+0xc1/frame 0xfffffe00681c28f0 adf_attach() at adf_attach+0x4e1/frame 0xfffffe00681c2950 device_attach() at device_attach+0x43d/frame 0xfffffe00681c29a0 pci_driver_added() at pci_driver_added+0xf2/frame 0xfffffe00681c29e0 devclass_driver_added() at devclass_driver_added+0x29/frame 0xfffffe00681= c2a10 devclass_add_driver() at devclass_add_driver+0x11e/frame 0xfffffe00681c2a= 50 module_register_init() at module_register_init+0x85/frame 0xfffffe00681c2= a80 linker_load_module() at linker_load_module+0xc0f/frame 0xfffffe00681c2d80= kern_kldload() at kern_kldload+0x165/frame 0xfffffe00681c2dd0 sys_kldload() at sys_kldload+0x59/frame 0xfffffe00681c2e00 amd64_syscall() at amd64_syscall+0x126/frame 0xfffffe00681c2f30 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe00681c2f3= 0 --More-- </code></pre> <p dir=3D"auto">Because we=E2=80=99ve tried to use a pci_bars[2] entry wi= th virt_addr =3D=3D 0 (because it=E2=80=99s outside of the BARs we set up= ).</p> <p dir=3D"auto">Reverting just the above hunk of the patch fixes the pani= c for us.</p> <p dir=3D"auto">Best regards,<br> Kristof</p> </div> </div> </body> </html> --=_MailMate_B08AB6DB-7748-4F2A-B2B9-63ABCCFE9199_=--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?9DB885DF-8BDD-451F-B928-0C10E881C4CF>