From owner-freebsd-virtualization@freebsd.org Wed Jun 5 16:11:30 2019 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2E33815B2576 for ; Wed, 5 Jun 2019 16:11:30 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id B403089437 for ; Wed, 5 Jun 2019 16:11:29 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id 6C10015B2575; Wed, 5 Jun 2019 16:11:29 +0000 (UTC) Delivered-To: virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2C3E815B2574 for ; Wed, 5 Jun 2019 16:11:29 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B5AFF89431 for ; Wed, 5 Jun 2019 16:11:28 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id B6943C3EB for ; Wed, 5 Jun 2019 16:11:27 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id x55GBR3q037661 for ; Wed, 5 Jun 2019 16:11:27 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id x55GBRrG037660 for virtualization@FreeBSD.org; Wed, 5 Jun 2019 16:11:27 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: virtualization@FreeBSD.org Subject: [Bug 238333] bhyve random crash in rfb.c on FreeBSD current (after r346011) Date: Wed, 05 Jun 2019 16:11:27 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: bin X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: crash, needs-qa, regression X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: cem@freebsd.org X-Bugzilla-Status: Open X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: virtualization@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Jun 2019 16:11:30 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D238333 --- Comment #2 from Conrad Meyer --- Ok, here's what rfb.c tries to do: 1. Allocate a single zstream per rfb_softc. (fine) 2. Handle a single connection at a time per rfb_softc. (fine) 3. Periodically re-send the screen from a co-thread. (fine? could just be done from main rfb_handle thread) 4. Serialize screen send using a mutex + flag. (fine) Here is some dubious stuff: 1. Client commands, including "set encoding" (i.e., RAW, ZLIB, RESIZE) are = not serialized against periodic worker thread. a. Also including things like "change dimensions of the screen" and "ena= ble ZLIB". 2. The zstream is never reinitialized, but reused. This may be desirable (i.e., better compression is achieved if the client can reliably inflate references to historical stream data). 3. Client is allowed to arbitrarily update softc "height" and "width" field= s, although amusingly these are never used? 4. zbuf isn't really sized for totally incompressible full size maximal screens? 2000*1200*4 is 9.1 MB and 16 bytes slop is definitely not enou= gh for zlib --fast overhead. --- The line number in deflate() and stack in flush_pending() suggest that assumptions in rfb.c were violated. rfb initiates every single operation w= ith Z_SYNC_FLUSH, which indicates that all input should be processed and nothing should be left pending in zstream's output buffer. However, it does nothin= g to *verify* this assumption (avail_out >0). I'm not immediately seeing how th= is leads to this crash, though. E.g.: $ dd if=3D/dev/urandom bs=3D$((2000*1200*4)) count=3D1 of=3D./randomscree= n.dat 9600000 bytes transferred in 0.026229 secs (366010901 bytes/sec) $ gzip --fast --no-name ./randomscreen.dat $ ls -l ./randomscreen.dat.gz ... 9602938 Jun 5 09:04 ./randomscreen.dat.gz So that's 2938 bytes of zlib overhead for a maximally sized, incompressible screen. Subtract a handful of bytes for gzip header/trailer (maybe 32) and another 16 for the manual slop in rfb.c and you're still short 2.8 kB re: unchecked assumptions made in rfb.c zbuf sizing. (And to what end? zlib c= an operate efficiently with much smaller buffers.) Can you also print `*stream` and `*s`? What are avail_out, etc? What size= of framebuffer are you attempting to use? Do you have a core you can share or repro steps I could follow? Thanks! --=20 You are receiving this mail because: You are the assignee for the bug.=