From owner-freebsd-current@FreeBSD.ORG Sun Mar 1 00:36:06 2015 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 2D678C99 for ; Sun, 1 Mar 2015 00:36:06 +0000 (UTC) Received: from mail-yk0-f178.google.com (mail-yk0-f178.google.com [209.85.160.178]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id EC03A123 for ; Sun, 1 Mar 2015 00:36:05 +0000 (UTC) Received: by ykr200 with SMTP id 200so10390840ykr.12 for ; Sat, 28 Feb 2015 16:35:58 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=9X78PZr5UsWLQdXyd4S0c952rFwv+442fY3lP9JSs3Y=; b=NWLoGwIJJo6xXvdBRD5d1T4/1G/GBA7rmjAd0PnY1en3UYBBVCZXcj70/HcdbDWKgl 9pEOMfAJNnNmDU8g1wtfSQ2I+gnU6cwjmXK3q/ZrMjywu5ZJGnT1opv9CPTK4zpkF9DW e4N9hQUO02PV6nPy2Brph2L802EDKjx6xFRqgemr4cpqMDwv8Jq3s2ji7IUKWhIMcr8S acx1W0AhPXDkFnxFZdJcn4GotJSQgOg+1ue4rQU0xoQHZduK65DK2cQajpjlEn6BaHTi fBg/OxxI4B2Esxfy0ZfOFGNMRhc89xjiYANOtakR4V9NL9mI2eVly7hNV5NDrqTSTrm/ lJFA== X-Gm-Message-State: ALoCoQlSP+DGoIEv8tKfWULbliEzbamUM+3Tl1Izs5rUn2bs7rDKmFNJN2cFzdhnukBg/Hqz9k3n MIME-Version: 1.0 X-Received: by 10.236.31.33 with SMTP id l21mr19809283yha.181.1425170158726; Sat, 28 Feb 2015 16:35:58 -0800 (PST) Received: by 10.170.56.134 with HTTP; Sat, 28 Feb 2015 16:35:58 -0800 (PST) In-Reply-To: <372ECBB0-2491-4995-8A9C-65ECF7E675AD@gmail.com> References: <2C7266F4-70C0-4AAE-876D-A03FA2E91FE4@gmail.com> <7455B5E3-EAE6-4D5D-B0DA-373F2D75397B@gmail.com> <372ECBB0-2491-4995-8A9C-65ECF7E675AD@gmail.com> Date: Sun, 1 Mar 2015 01:35:58 +0100 Message-ID: Subject: Re: atkbd.c not compiling? From: Oliver Pinter To: Garrett Cooper Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: FreeBSD Current , Ryan Stone X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 01 Mar 2015 00:36:06 -0000 On Sat, Feb 28, 2015 at 11:19 PM, Garrett Cooper wr= ote: > On Feb 28, 2015, at 14:15, Garrett Cooper wrote: > >> On Feb 28, 2015, at 13:56, Ryan Stone wrote: >> >>> On Sat, Feb 28, 2015 at 4:51 PM, Garrett Cooper = wrote: >>>> I=E2=80=99m not sure about key_map =E2=80=94 are you building with sys= cons or vt? >>> >>> I have no idea. I'm just running make tinderbox. So far >>> _.sparc64.LINT, _.i386.LINT-NOINET and _.i386.LINT-VIMAGE have failed, >>> among others. >>> >>> i386.LINT and sparc64.LINK have both "device sc" and "device vt" from >>> what I can see >> >> I think I figured it out. Is it because MK_VT !=3D =E2=80=9Cno=E2=80=9D = and MK_LEGACY_CONSOLE =3D=3D =E2=80=9Cno=E2=80=9D? > > =E2=80=A6 or because MK_SYSCONS =3D=3D no? No, when you try to compile vt enabled kernel on system which running on syscons, or the versa, then you got this problem. Try to compile with the us.pc-ctrl or us.ctrl keyboard layout. In bugzilla exists some patch, that fixes this issue, by altering the font search path. The other remaining problem is, when use try to use the VT, and it required the kbdmux layer. The kbdmux layer then discards the custom keyboard layout, which is configured at ukbd or atkbd level. Fix already exists in bugzilla too. From owner-freebsd-current@FreeBSD.ORG Sun Mar 1 02:01:34 2015 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 77975BCC for ; Sun, 1 Mar 2015 02:01:34 +0000 (UTC) Received: from mail-ie0-x229.google.com (mail-ie0-x229.google.com [IPv6:2607:f8b0:4001:c03::229]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 3EA7EB94 for ; Sun, 1 Mar 2015 02:01:34 +0000 (UTC) Received: by iecrp18 with SMTP id rp18so40013985iec.1 for ; Sat, 28 Feb 2015 18:01:33 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=IZyS8Gd6saEarwa3dtsMXiiLm4VkZi0t2rH2sT9Uk8s=; b=Iq60cuv64COYkTtJKyJ66BRvRrs6F+XcdVZcm5D8WO3gD2OcMlfArKo2qVfa1QoHmZ wLukivw2iU8P/O5F4ciYA5OOsv8AexiW6+tRoQdwCEGA91U0HIyQz8j5587uETGi9dtb U+TzjpZEZ7lYZQpv8HUlPtJx3qXBN637KYcGuOLVvX97+1dslVFmaWR/4ucgMi2cY0s7 /S/Y6BKf3BSrFDzOwH+tD2qCtYcxb9pRch1mneVyYAJqE6BfPAT7iHN7HE/QncKdo2Oj p+LN/zhyDQ85FNy8H8H+Dy51xvTgQhOu8v/n907+vZkM48kGWS8bQsRhRZf6sovCG19L a0PA== MIME-Version: 1.0 X-Received: by 10.42.150.130 with SMTP id a2mr23800219icw.69.1425175293563; Sat, 28 Feb 2015 18:01:33 -0800 (PST) Received: by 10.107.156.75 with HTTP; Sat, 28 Feb 2015 18:01:33 -0800 (PST) Date: Sat, 28 Feb 2015 21:01:33 -0500 Message-ID: Subject: HEADS UP: PCI SR-IOV infrastructure has been committed to head From: Ryan Stone To: FreeBSD Current Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 01 Mar 2015 02:01:34 -0000 I've just finished committing support for PCI Single Root I/O Virtualization in the pci subsystem to head. This should be a no-op for everyone right now, but there were some minor refactorings in the pci code that could have a lingering bug. I did make sure to test that it boots on a variety of systems (but only i386/amd64, as that's all that I have access to). What's been committed to head is only the pci subsystem side of things, along with the userland tools to configure SR-IOV (along with, I'm happy to say, a full set of man pages). What's not in head yet are any drivers making use of the infrastructure. Full support for ixl(4) is complete and I've sent the patch to jfv@; I hope to see the driver support committed soon. I don't have any word on timelines for getting support in other drivers. Unfortunately adding SR-IOV support to a driver is not trivial as the standard leaves a lot of the details up to particular implementations (in the same way the the PCIe standard does not define how to send a packet from a NIC; instead defining how the PCIe device will expose its registers and whatnot, and its up to the PCIe device and driver to understand how to poke at the registers to send a packet). I have heard anecdotally that a number of driver maintainers have been very interested in this work so I hope that to see more drivers supported SR-IOV in the near future. I encourage all driver maintainers to read over the new manpages and contact me if they have any questions about the new infrastructure. Anybody interested in using SR-IOV should try to attend BSDCan 2015, as I will be giving a talk on the subject. I intend to focus more on the system administration side of configuring and using SR-IOV rather than the details of implementing an SR-IOV driver. If anybody did an "svn up" half-way through my muddled series of commits, sorry about the temporary breakage. My buildworld/buildkernel on r279466 just completed successfully so please make sure that you have at least that revision. If you still have problems, please let me know. I do want to thank John Baldwin for advice about the PCI Subsystem and newbus and Jack Vogel for his help with the Fortville NIC, including getting me early access to the VF driver for testing purposes. Thanks to everybody who reviewed the changes. Specially thanks to Mark Johnston and Sean Mahood, who literally spent hours with me in a meeting room reviewing the entire patch series last summer (thankfully, those hours at least weren't consecutive). Above all, thanks to Sandvine Inc. for sponsoring this work. This is definitely the biggest contribution we've ever made to FreeBSD and I hope to see this kind of thing continue. From owner-freebsd-current@FreeBSD.ORG Sun Mar 1 01:31:42 2015 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 8C2AC963; Sun, 1 Mar 2015 01:31:42 +0000 (UTC) Received: from jenkins-9.freebsd.org (jenkins-9.freebsd.org [8.8.178.209]) by mx1.freebsd.org (Postfix) with ESMTP id 76124900; Sun, 1 Mar 2015 01:31:42 +0000 (UTC) Received: from jenkins-9.freebsd.org (localhost [127.0.0.1]) by jenkins-9.freebsd.org (Postfix) with ESMTP id 542EAA44; Sun, 1 Mar 2015 01:31:42 +0000 (UTC) Date: Sun, 1 Mar 2015 01:31:35 +0000 (GMT) From: jenkins-admin@freebsd.org To: jenkins-admin@FreeBSD.org, freebsd-current@freebsd.org, loos@FreeBSD.org, rstone@FreeBSD.org, arybchik@FreeBSD.org, markj@FreeBSD.org, adrian@FreeBSD.org, ngie@FreeBSD.org, alc@FreeBSD.org, kib@FreeBSD.org, kan@FreeBSD.org Message-ID: <2106876404.49.1425173501970.JavaMail.jenkins@jenkins-9.freebsd.org> In-Reply-To: <1887508547.48.1425163799584.JavaMail.jenkins@jenkins-9.freebsd.org> References: <1887508547.48.1425163799584.JavaMail.jenkins@jenkins-9.freebsd.org> Subject: Build failed in Jenkins: FreeBSD_HEAD #2473 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Jenkins-Job: FreeBSD_HEAD X-Jenkins-Result: FAILURE X-Mailman-Approved-At: Sun, 01 Mar 2015 04:10:03 +0000 X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 01 Mar 2015 01:31:42 -0000 See Changes: [rstone] Add macros to make code compile in kernel Make it possible to compile libnv in the kernel. Mostly this involves wrapping functions that have a different signature in the kernel and in userland (e.g. malloc()) in a macro that will conditionally expand to the right API depending on whether the code is being compiled for the kernel or not. I have also #ifdef'ed out all of file descriptor-handling code, as well as the unsafe varargs functions. Differential Revision:=09=09https://reviews.freebsd.org/D1882 Reviewed by:=09=09=09jfv MFC after:=09=09=091 month Sponsored by:=09=09=09Sandvine Inc [rstone] Allow Illumos code to co-exist with nv(9) Differential Revision:=09=09https://reviews.freebsd.org/D1881 Reviewed by:=09=09=09jfv, will Suggested by:=09=09=09pjd MFC after:=09=09=091 month Sponsored by:=09=09=09Sandvine Inc [rstone] Prevent creation of an invalid nvlist If an nvlist is set as a child of another nvlist with nvlist_move_nvlist then fail the operation and set the parent nvlist to the error state. Differential Revision:=09=09https://reviews.freebsd.org/D1880 Reviewers:=09=09=09jfv MFC after:=09=09=091 month Sponsored by:=09=09=09Sandvine Inc [rstone] Don't allocate memory for operations that do not insert Almost every operation performed on an nvlist was allocating a new string to hold the key name. The nvlist_exists* family of functions would always return false if they failed to allocate the string. The rest of the functions would outright abort(). Fix the non-varargs variants of the functions to perform the requested operations directly and the varargs versions to allocate the string and call into the non-varargs versions. The varargs versions are still broken and really can't be fixed, so we might consider axing them entirely. However, now the non- varargs functions are always safe to call. Differential Revision:=09=09https://reviews.freebsd.org/D1879 Reviewed by:=09=09=09pjd, jfv MFC after:=09=09=091 month Sponsored by:=09=09=09Sandvine Inc. [rstone] Add function to force an nvlist into the error state Add an nvlist_set_error() function that can be used to force an nvlist into the error state. This is useful both for writing tests and for writing APIs that use nvlists internally. Differential Revision:=09=09https://reviews.freebsd.org/D1878 Reviewed by:=09=09=09pjd, jfv MFC After:=09=09=091 month Sponsored by:=09=09=09Sandvine Inc. [rstone] Implement asprintf in libkern Differential Revision:=09=09https://reviews.freebsd.org/D1877 Reviewed by:=09=09=09pjd, jfv MFC After:=09=09=091 month Sponsored by:=09=09=09Sandvine Inc. [rstone] Extend the unit test to fix the bug caught in r277925 Differential Revision:=09=09https://reviews.freebsd.org/D1888 MFC After:=09=09=091 month Sponsored by:=09=09=09Sandvine Inc. [rstone] Add tests for dnvlist_take_* Differential Revision:=09=09https://reviews.freebsd.org/D1876 Reviewed by:=09=09=09jfv, pjd MFC after:=09=09=091 month Sponsored by:=09=09=09Sandvine Inc. [rstone] Add tests for dnv_get_* Differential Revision:=09=09https://reviews.freebsd.org/D1875 Reviewed by:=09=09=09jfv, pjd MFC after:=09=09=091 month Sponsored by:=09=09=09Sandvine Inc. [rstone] Add tests for nvlist_free* functions Differential Revision:=09=09https://reviews.freebsd.org/D1874 Reviewed by:=09=09=09jfv, pjd MFC after:=09=09=091 month Sponsored by:=09=09=09Sandvine Inc. [rstone] Add tests for nvlist_take_* Differential Revision:=09=09https://reviews.freebsd.org/D1873 Reviewed by:=09=09=09jfv, pjd MFC after:=09=09=091 month Sponsored by:=09=09=09Sandvine Inc. [rstone] Add test cases for nvlist_move_* Differential Revision:=09=09https://reviews.freebsd.org/D1872 Reviewed by:=09=09=09jfv, pjd MFC after:=09=09=091 month Sponsored by:=09=09=09Sandvine Inc. [rstone] Add tests for nvlist_pack/unpack Differential Revision:=09=09https://reviews.freebsd.org/D1871 Reviewed by:=09=09=09jfv, pjd MFC after:=09=09=091 month Sponsored by:=09=09=09Sandvine Inc. [rstone] Add tests for nvlist_clone Differential Revision:=09=09https://reviews.freebsd.org/D1870 Reviewed by:=09=09=09pjd, jfv MFC after:=09=09=091 month Sponsored by:=09=09=09Sandvine Inc [rstone] Tests of basic nvlist add functions Differential Revision:=09=09https://reviews.freebsd.org/D1869 Reviewed by:=09=09=09jfv, pjd MFC after:=09=09=091 month Sponsored by:=09=09=09Sandvine Inc. [rstone] Revert r279422. My "apply patch and commit" script wasn't adding new files properly. Pointy hat to: rstone [rstone] Tests of basic nvlist add functions Differential Revision:=09=09https://reviews.freebsd.org/D1869 Reviewed by:=09=09=09jfv, pjd MFC after:=09=09=091 month Sponsored by:=09=09=09Sandvine Inc. [rstone] Make libnv headers includable from C++ Differential Revision:=09=09https://reviews.freebsd.org/D1868 Reviewed by:=09=09=09jfv, pjd MFC after:=09=09=091 month Sponsored by:=09=09=09Sandvine Inc. [adrian] Add another register definition for the AR8327. Obtained from:=09OpenWRT [markj] Remove the old DTrace test suite makefile - it was somewhat primiti= ve and mostly unmaintained, and it has been superseded by the infrastructure added in r279418. Reviewed by:=09ngie Sponsored by:=09EMC / Isilon Storage Divison [markj] Add infrastructure to integrate the DTrace test suite with Kyua. For each test category, we generate a script containing ATF test cases for the tests under that category. Each test case simply runs dtest.pl (the upstream test harness) with the corresponding test files. The exclude.sh script is used to record info about tests which should be skipped or are expected to fail; it is used to generate atf_skip and atf_expect_fail calls= . The genmakefiles.sh script can be used to regenerate the test makefiles whe= n new tests are brought it from upstream. The test suite is currently not connected to the build as there is a small number of lingering test issues which still need to be worked out. In the meantime however, the test suite can be easily built and installed manually from cddl/usr.sbin/dtrace/tests. Reviewed by:=09ngie Sponsored by:=09EMC / Isilon Storage Division [markj] Use the -shared option to create a shared library. MFC after:=091 week [markj] Remove a leading tab that causes a make(1) error when running the t= est. MFC after:=091 week [markj] Only kill sleep processes that were forked from the test script. MFC after:=091 week [markj] Stop hard-coding an incorrect path to rm(1). MFC after:=091 week [rstone] Add a missing include of an options header. watchdog.c does an #ifdef DDB but does not #include "opt_ddb.h". Fixing this turned up a missing include file. MFC after:=091 week X-MFC-With:=09r261495, r279410 [kan] Do not leak 'copy' buffer if bmap_truncate_indirect fails. Reported by: Brainy Code Scanner, by Maxime Villard. MFC after: 2 weeks [ngie] Unbreak 'make depend' with sfxge by removing debugging code activate= d in the INET || INET6 case X-MFC with: r279398 Pointyhat to: arybchik [rstone] Correct the use of an unitialized variable in sendfind_getobj() When sendfile_getobj() is called on a DTYPE_SHM file, it never initializes error, which is eventually returned to the caller. Differential Revision:=09=09https://reviews.freebsd.org/D1989 Reviewed by:=09=09=09kib Reported by: =09=09=09Brainy Code Scanner, by Maxime Villard. ------------------------------------------ [...truncated 156850 lines...] atfu_tc_nvlist_add_string__single_insert ^ :1407:1: error: redefinition of 'atfu_tcptr_nvlist_add_string__single_ins= ert' ATF_TEST_CASE_WITHOUT_HEAD(nvlist_add_string__single_insert); ^ :46:30: note: expanded from macro 'ATF_TES= T_CASE_WITHOUT_HEAD' static atfu_tc_ ## name* atfu_tcptr_ ## name; \ ^ :3:1: note: expanded from here atfu_tcptr_nvlist_add_string__single_insert ^ :161:1: note: previous definition is here ATF_TEST_CASE_WITHOUT_HEAD(nvlist_add_string__single_insert); ^ :46:30: note: expanded from macro 'ATF_TES= T_CASE_WITHOUT_HEAD' static atfu_tc_ ## name* atfu_tcptr_ ## name; \ ^ :76:1: note: expanded from here atfu_tcptr_nvlist_add_string__single_insert ^ :1407:1: error: redefinition of 'atfu_tc_nvlist_add_string__single_insert= ' ATF_TEST_CASE_WITHOUT_HEAD(nvlist_add_string__single_insert); ^ :47:23: note: expanded from macro 'ATF_TES= T_CASE_WITHOUT_HEAD' atfu_tc_ ## name::atfu_tc_ ## name(void) : atf::tests::tc(#name, false)= {} \ ^ :3:1: note: expanded from here atfu_tc_nvlist_add_string__single_insert ^ :161:1: note: previous definition is here ATF_TEST_CASE_WITHOUT_HEAD(nvlist_add_string__single_insert); ^ :47:23: note: expanded from macro 'ATF_TES= T_CASE_WITHOUT_HEAD' atfu_tc_ ## name::atfu_tc_ ## name(void) : atf::tests::tc(#name, false)= {} \ ^ :78:1: note: expanded from here atfu_tc_nvlist_add_string__single_insert ^ fatal error: too many errors emitted, stopping now [-ferror-limit=3D] --- sbin.all__D --- =3D=3D=3D> sbin/geom/class/virstor (all) --- secure.all__D --- --- des_old2.po --- cc -pg -O2 -pipe -DTERMIOS -DANSI_SOURCE -I -I -I -DOPENSSL_THREADS -DDSO_DLFCN -D= HAVE_DLFCN_H -DL_ENDIAN -DOPENSSL_IA32_SSE2 -DAES_ASM -DBSAES_ASM -DVPAES_A= SM -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_MONT5 -DOPENSSL_BN_ASM_GF2m -DMD5= _ASM -DGHASH_ASM -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DWHIRLPOOL_ASM -I -I -I -std=3Dgnu89 -fstack-protector -Wno-pointer-sign -Wno= -empty-body -Wno-string-plus-int -Wno-unused-const-variable -Wno-tautologic= al-compare -Wno-unused-value -Wno-parentheses-equality -Wno-unused-function= -Wno-enum-conversion -Wno-switch -Wno-switch-enum -Wno-knr-promoted-parame= ter -Wno-parentheses -Qunused-arguments -c -o des_old2.po --- sbin.all__D --- --- binstream.So --- cc -fpic -DPIC -O2 -pipe -I -std=3Dgnu99 -fstack-protector -Wsystem= -headers -Werror -Wall -Wno-format-y2k -W -Wno-unused-parameter -Wstrict-pr= ototypes -Wmissing-prototypes -Wpointer-arith -Wreturn-type -Wcast-qual -Ww= rite-strings -Wswitch -Wshadow -Wunused-parameter -Wcast-align -Wchar-subsc= ripts -Winline -Wnested-externs -Wredundant-decls -Wold-style-definition -W= no-pointer-sign -Wthread-safety -Wno-empty-body -Wno-string-plus-int -Wno-u= nused-const-variable -Qunused-arguments -c -o binstream.So --- secure.all__D --- --- ecb3_enc.po --- cc -pg -O2 -pipe -DTERMIOS -DANSI_SOURCE -I -I -I -DOPENSSL_THREADS -DDSO_DLFCN -D= HAVE_DLFCN_H -DL_ENDIAN -DOPENSSL_IA32_SSE2 -DAES_ASM -DBSAES_ASM -DVPAES_A= SM -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_MONT5 -DOPENSSL_BN_ASM_GF2m -DMD5= _ASM -DGHASH_ASM -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DWHIRLPOOL_ASM -I -I -I -std=3Dgnu89 -fstack-protector -Wno-pointer-sign -Wno= -empty-body -Wno-string-plus-int -Wno-unused-const-variable -Wno-tautologic= al-compare -Wno-unused-value -Wno-parentheses-equality -Wno-unused-function= -Wno-enum-conversion -Wno-switch -Wno-switch-enum -Wno-knr-promoted-parame= ter -Wno-parentheses -Qunused-arguments -c -o ecb3_enc.po --- sbin.all__D --- --- g_virstor_md.So --- cc -fpic -DPIC -O2 -pipe -I -std=3Dgnu99 -fstack-protector -Wsystem= -headers -Werror -Wall -Wno-format-y2k -W -Wno-unused-parameter -Wstrict-pr= ototypes -Wmissing-prototypes -Wpointer-arith -Wreturn-type -Wcast-qual -Ww= rite-strings -Wswitch -Wshadow -Wunused-parameter -Wcast-align -Wchar-subsc= ripts -Winline -Wnested-externs -Wredundant-decls -Wold-style-definition -W= no-pointer-sign -Wthread-safety -Wno-empty-body -Wno-string-plus-int -Wno-u= nused-const-variable -Qunused-arguments -c -o g_virstor_md.So --- secure.all__D --- --- ecb_enc.po --- cc -pg -O2 -pipe -DTERMIOS -DANSI_SOURCE -I -I -I -DOPENSSL_THREADS -DDSO_DLFCN -D= HAVE_DLFCN_H -DL_ENDIAN -DOPENSSL_IA32_SSE2 -DAES_ASM -DBSAES_ASM -DVPAES_A= SM -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_MONT5 -DOPENSSL_BN_ASM_GF2m -DMD5= _ASM -DGHASH_ASM -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DWHIRLPOOL_ASM -I -I -I -std=3Dgnu89 -fstack-protector -Wno-pointer-sign -Wno= -empty-body -Wno-string-plus-int -Wno-unused-const-variable -Wno-tautologic= al-compare -Wno-unused-value -Wno-parentheses-equality -Wno-unused-function= -Wno-enum-conversion -Wno-switch -Wno-switch-enum -Wno-knr-promoted-parame= ter -Wno-parentheses -Qunused-arguments -c -o ecb_enc.po --- sbin.all__D --- --- geom_virstor.So --- cc -fpic -DPIC -O2 -pipe -I -std=3Dgnu99 -fstack-protector -Wsystem= -headers -Werror -Wall -Wno-format-y2k -W -Wno-unused-parameter -Wstrict-pr= ototypes -Wmissing-prototypes -Wpointer-arith -Wreturn-type -Wcast-qual -Ww= rite-strings -Wswitch -Wshadow -Wunused-parameter -Wcast-align -Wchar-subsc= ripts -Winline -Wnested-externs -Wredundant-decls -Wold-style-definition -W= no-pointer-sign -Wthread-safety -Wno-empty-body -Wno-string-plus-int -Wno-u= nused-const-variable -Qunused-arguments -c -o geom_virstor.So --- secure.all__D --- --- ede_cbcm_enc.po --- cc -pg -O2 -pipe -DTERMIOS -DANSI_SOURCE -I -I -I -DOPENSSL_THREADS -DDSO_DLFCN -D= HAVE_DLFCN_H -DL_ENDIAN -DOPENSSL_IA32_SSE2 -DAES_ASM -DBSAES_ASM -DVPAES_A= SM -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_MONT5 -DOPENSSL_BN_ASM_GF2m -DMD5= _ASM -DGHASH_ASM -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DWHIRLPOOL_ASM -I -I -I -std=3Dgnu89 -fstack-protector -Wno-pointer-sign -Wno= -empty-body -Wno-string-plus-int -Wno-unused-const-variable -Wno-tautologic= al-compare -Wno-unused-value -Wno-parentheses-equality -Wno-unused-function= -Wno-enum-conversion -Wno-switch -Wno-switch-enum -Wno-knr-promoted-parame= ter -Wno-parentheses -Qunused-arguments -c -o ede_cbcm_enc.po --- share.all__D --- =3D=3D=3D> share/i18n/esdb/DEC (all) --- sbin.all__D --- --- subr.So --- cc -fpic -DPIC -O2 -pipe -I -std=3Dgnu99 -fstack-protector -Wsystem= -headers -Werror -Wall -Wno-format-y2k -W -Wno-unused-parameter -Wstrict-pr= ototypes -Wmissing-prototypes -Wpointer-arith -Wreturn-type -Wcast-qual -Ww= rite-strings -Wswitch -Wshadow -Wunused-parameter -Wcast-align -Wchar-subsc= ripts -Winline -Wnested-externs -Wredundant-decls -Wold-style-definition -W= no-pointer-sign -Wthread-safety -Wno-empty-body -Wno-string-plus-int -Wno-u= nused-const-variable -Qunused-arguments -c -o subr.So --- share.all__D --- =3D=3D=3D> share/i18n/esdb/EUC (all) --- secure.all__D --- --- enc_read.po --- cc -pg -O2 -pipe -DTERMIOS -DANSI_SOURCE -I -I -I -DOPENSSL_THREADS -DDSO_DLFCN -D= HAVE_DLFCN_H -DL_ENDIAN -DOPENSSL_IA32_SSE2 -DAES_ASM -DBSAES_ASM -DVPAES_A= SM -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_MONT5 -DOPENSSL_BN_ASM_GF2m -DMD5= _ASM -DGHASH_ASM -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DWHIRLPOOL_ASM -I -I -I -std=3Dgnu89 -fstack-protector -Wno-pointer-sign -Wno= -empty-body -Wno-string-plus-int -Wno-unused-const-variable -Wno-tautologic= al-compare -Wno-unused-value -Wno-parentheses-equality -Wno-unused-function= -Wno-enum-conversion -Wno-switch -Wno-switch-enum -Wno-knr-promoted-parame= ter -Wno-parentheses -Qunused-arguments -c -o enc_read.po --- share.all__D --- =3D=3D=3D> share/i18n/esdb/EBCDIC (all) --- secure.all__D --- --- enc_writ.po --- cc -pg -O2 -pipe -DTERMIOS -DANSI_SOURCE -I -I -I -DOPENSSL_THREADS -DDSO_DLFCN -D= HAVE_DLFCN_H -DL_ENDIAN -DOPENSSL_IA32_SSE2 -DAES_ASM -DBSAES_ASM -DVPAES_A= SM -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_MONT5 -DOPENSSL_BN_ASM_GF2m -DMD5= _ASM -DGHASH_ASM -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DWHIRLPOOL_ASM -I -I -I -std=3Dgnu89 -fstack-protector -Wno-pointer-sign -Wno= -empty-body -Wno-string-plus-int -Wno-unused-const-variable -Wno-tautologic= al-compare -Wno-unused-value -Wno-parentheses-equality -Wno-unused-function= -Wno-enum-conversion -Wno-switch -Wno-switch-enum -Wno-knr-promoted-parame= ter -Wno-parentheses -Qunused-arguments -c -o enc_writ.po --- share.all__D --- =3D=3D=3D> share/i18n/esdb/GB (all) --- sbin.all__D --- --- gvirstor.8.gz --- gzip -cn > gvirstor.8.gz --- geom_virstor.so --- building shared library geom_virstor.so --- share.all__D --- =3D=3D=3D> share/i18n/esdb/GEORGIAN (all) --- secure.all__D --- --- fcrypt.po --- cc -pg -O2 -pipe -DTERMIOS -DANSI_SOURCE -I -I -I -DOPENSSL_THREADS -DDSO_DLFCN -D= HAVE_DLFCN_H -DL_ENDIAN -DOPENSSL_IA32_SSE2 -DAES_ASM -DBSAES_ASM -DVPAES_A= SM -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_MONT5 -DOPENSSL_BN_ASM_GF2m -DMD5= _ASM -DGHASH_ASM -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DWHIRLPOOL_ASM -I -I -I -std=3Dgnu89 -fstack-protector -Wno-pointer-sign -Wno= -empty-body -Wno-string-plus-int -Wno-unused-const-variable -Wno-tautologic= al-compare -Wno-unused-value -Wno-parentheses-equality -Wno-unused-function= -Wno-enum-conversion -Wno-switch -Wno-switch-enum -Wno-knr-promoted-parame= ter -Wno-parentheses -Qunused-arguments -c -o fcrypt.po --- share.all__D --- =3D=3D=3D> share/i18n/esdb/ISO-2022 (all) --- sbin.all__D --- --- all_subdir_ggate --- =3D=3D=3D> sbin/ggate (all) --- _sub.all --- =3D=3D=3D> sbin/ggate/ggatec (all) --- share.all__D --- =3D=3D=3D> share/i18n/esdb/ISO-8859 (all) --- sbin.all__D --- --- ggatec.o --- cc -O2 -pipe -DMAX_SEND_SIZE=3D32768 -DLIBGEOM -I -std=3Dgnu99 -fstack= -protector -Wsystem-headers -Werror -Wall -Wno-format-y2k -W -Wno-unused-pa= rameter -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Wreturn-t= ype -Wcast-qual -Wwrite-strings -Wswitch -Wshadow -Wunused-parameter -Wcast= -align -Wchar-subscripts -Winline -Wnested-externs -Wredundant-decls -Wold-= style-definition -Wno-pointer-sign -Wmissing-variable-declarations -Wthread= -safety -Wno-empty-body -Wno-string-plus-int -Wno-unused-const-variable -Qu= nused-arguments -c --- secure.all__D --- --- ofb64ede.po --- cc -pg -O2 -pipe -DTERMIOS -DANSI_SOURCE -I -I -I -DOPENSSL_THREADS -DDSO_DLFCN -D= HAVE_DLFCN_H -DL_ENDIAN -DOPENSSL_IA32_SSE2 -DAES_ASM -DBSAES_ASM -DVPAES_A= SM -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_MONT5 -DOPENSSL_BN_ASM_GF2m -DMD5= _ASM -DGHASH_ASM -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DWHIRLPOOL_ASM -I -I -I -std=3Dgnu89 -fstack-protector -Wno-pointer-sign -Wno= -empty-body -Wno-string-plus-int -Wno-unused-const-variable -Wno-tautologic= al-compare -Wno-unused-value -Wno-parentheses-equality -Wno-unused-function= -Wno-enum-conversion -Wno-switch -Wno-switch-enum -Wno-knr-promoted-parame= ter -Wno-parentheses -Qunused-arguments -c -o ofb64ede.po --- share.all__D --- =3D=3D=3D> share/i18n/esdb/ISO646 (all) --- secure.all__D --- --- ofb64enc.po --- cc -pg -O2 -pipe -DTERMIOS -DANSI_SOURCE -I -I -I -DOPENSSL_THREADS -DDSO_DLFCN -D= HAVE_DLFCN_H -DL_ENDIAN -DOPENSSL_IA32_SSE2 -DAES_ASM -DBSAES_ASM -DVPAES_A= SM -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_MONT5 -DOPENSSL_BN_ASM_GF2m -DMD5= _ASM -DGHASH_ASM -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DWHIRLPOOL_ASM -I -I -I -std=3Dgnu89 -fstack-protector -Wno-pointer-sign -Wno= -empty-body -Wno-string-plus-int -Wno-unused-const-variable -Wno-tautologic= al-compare -Wno-unused-value -Wno-parentheses-equality -Wno-unused-function= -Wno-enum-conversion -Wno-switch -Wno-switch-enum -Wno-knr-promoted-parame= ter -Wno-parentheses -Qunused-arguments -c -o ofb64enc.po --- ofb_enc.po --- cc -pg -O2 -pipe -DTERMIOS -DANSI_SOURCE -I -I -I -DOPENSSL_THREADS -DDSO_DLFCN -D= HAVE_DLFCN_H -DL_ENDIAN -DOPENSSL_IA32_SSE2 -DAES_ASM -DBSAES_ASM -DVPAES_A= SM -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_MONT5 -DOPENSSL_BN_ASM_GF2m -DMD5= _ASM -DGHASH_ASM -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DWHIRLPOOL_ASM -I -I -I -std=3Dgnu89 -fstack-protector -Wno-pointer-sign -Wno= -empty-body -Wno-string-plus-int -Wno-unused-const-variable -Wno-tautologic= al-compare -Wno-unused-value -Wno-parentheses-equality -Wno-unused-function= -Wno-enum-conversion -Wno-switch -Wno-switch-enum -Wno-knr-promoted-parame= ter -Wno-parentheses -Qunused-arguments -c -o ofb_enc.po --- share.all__D --- =3D=3D=3D> share/i18n/esdb/KAZAKH (all) --- sbin.all__D --- --- ggate.o --- cc -O2 -pipe -DMAX_SEND_SIZE=3D32768 -DLIBGEOM -I -std=3Dgnu99 -fstack= -protector -Wsystem-headers -Werror -Wall -Wno-format-y2k -W -Wno-unused-pa= rameter -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Wreturn-t= ype -Wcast-qual -Wwrite-strings -Wswitch -Wshadow -Wunused-parameter -Wcast= -align -Wchar-subscripts -Winline -Wnested-externs -Wredundant-decls -Wold-= style-definition -Wno-pointer-sign -Wmissing-variable-declarations -Wthread= -safety -Wno-empty-body -Wno-string-plus-int -Wno-unused-const-variable -Qu= nused-arguments -c --- share.all__D --- =3D=3D=3D> share/i18n/esdb/KOI (all) --- secure.all__D --- --- pcbc_enc.po --- cc -pg -O2 -pipe -DTERMIOS -DANSI_SOURCE -I -I -I -DOPENSSL_THREADS -DDSO_DLFCN -D= HAVE_DLFCN_H -DL_ENDIAN -DOPENSSL_IA32_SSE2 -DAES_ASM -DBSAES_ASM -DVPAES_A= SM -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_MONT5 -DOPENSSL_BN_ASM_GF2m -DMD5= _ASM -DGHASH_ASM -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DWHIRLPOOL_ASM -I -I -I -std=3Dgnu89 -fstack-protector -Wno-pointer-sign -Wno= -empty-body -Wno-string-plus-int -Wno-unused-const-variable -Wno-tautologic= al-compare -Wno-unused-value -Wno-parentheses-equality -Wno-unused-function= -Wno-enum-conversion -Wno-switch -Wno-switch-enum -Wno-knr-promoted-parame= ter -Wno-parentheses -Qunused-arguments -c -o pcbc_enc.po --- share.all__D --- =3D=3D=3D> share/i18n/esdb/MISC (all) =3D=3D=3D> share/i18n/esdb/TCVN (all) --- secure.all__D --- --- qud_cksm.po --- cc -pg -O2 -pipe -DTERMIOS -DANSI_SOURCE -I -I -I -DOPENSSL_THREADS -DDSO_DLFCN -D= HAVE_DLFCN_H -DL_ENDIAN -DOPENSSL_IA32_SSE2 -DAES_ASM -DBSAES_ASM -DVPAES_A= SM -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_MONT5 -DOPENSSL_BN_ASM_GF2m -DMD5= _ASM -DGHASH_ASM -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DWHIRLPOOL_ASM -I -I -I -std=3Dgnu89 -fstack-protector -Wno-pointer-sign -Wno= -empty-body -Wno-string-plus-int -Wno-unused-const-variable -Wno-tautologic= al-compare -Wno-unused-value -Wno-parentheses-equality -Wno-unused-function= -Wno-enum-conversion -Wno-switch -Wno-switch-enum -Wno-knr-promoted-parame= ter -Wno-parentheses -Qunused-arguments -c -o qud_cksm.po --- share.all__D --- =3D=3D=3D> share/i18n/esdb/UTF (all) --- sbin.all__D --- --- ggatec.8.gz --- gzip -cn > ggatec.8.gz --- ggatec --- cc -O2 -pipe -DMAX_SEND_SIZE=3D32768 -DLIBGEOM -I -std=3Dgnu99 -fstack= -protector -Wsystem-headers -Werror -Wall -Wno-format-y2k -W -Wno-unused-pa= rameter -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Wreturn-t= ype -Wcast-qual -Wwrite-strings -Wswitch -Wshadow -Wunused-parameter -Wcast= -align -Wchar-subscripts -Winline -Wnested-externs -Wredundant-decls -Wold-= style-definition -Wno-pointer-sign -Wmissing-variable-declarations -Wthread= -safety -Wno-empty-body -Wno-string-plus-int -Wno-unused-const-variable -Qu= nused-arguments -o ggatec ggatec.o ggate.o -lgeom -lutil -lpthread --- secure.all__D --- --- rand_key.po --- cc -pg -O2 -pipe -DTERMIOS -DANSI_SOURCE -I -I -I -DOPENSSL_THREADS -DDSO_DLFCN -D= HAVE_DLFCN_H -DL_ENDIAN -DOPENSSL_IA32_SSE2 -DAES_ASM -DBSAES_ASM -DVPAES_A= SM -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_MONT5 -DOPENSSL_BN_ASM_GF2m -DMD5= _ASM -DGHASH_ASM -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DWHIRLPOOL_ASM -I -I -I -std=3Dgnu89 -fstack-protector -Wno-pointer-sign -Wno= -empty-body -Wno-string-plus-int -Wno-unused-const-variable -Wno-tautologic= al-compare -Wno-unused-value -Wno-parentheses-equality -Wno-unused-function= -Wno-enum-conversion -Wno-switch -Wno-switch-enum -Wno-knr-promoted-parame= ter -Wno-parentheses -Qunused-arguments -c -o rand_key.po --- sbin.all__D --- =3D=3D=3D> sbin/ggate/ggated (all) --- share.all__D --- =3D=3D=3D> share/keys (all) --- lib.all__D --- 20 errors generated. --- share.all__D --- --- _sub.all --- =3D=3D=3D> share/keys/pkg (all) --- lib.all__D --- *** [nv_tests.o] Error code 1 make[6]: stopped in 1 error make[6]: stopped in *** [nv_tests] Error code 2 make[5]: stopped in 1 error make[5]: stopped in *** [_sub.all] Error code 2 make[4]: stopped in 1 error make[4]: stopped in *** [all_subdir_libnv] Error code 2 make[3]: stopped in 1 error make[3]: stopped in --- sbin.all__D --- A failure has been detected in another branch of the parallel make make[5]: stopped in *** [_sub.all] Error code 2 make[4]: stopped in --- lib.all__D --- *** [lib.all__D] Error code 2 make[2]: stopped in --- sbin.all__D --- 1 error make[4]: stopped in *** [all_subdir_ggate] Error code 2 make[3]: stopped in 1 error make[3]: stopped in *** [sbin.all__D] Error code 2 make[2]: stopped in --- secure.all__D --- A failure has been detected in another branch of the parallel make make[5]: stopped in *** [_sub.all] Error code 2 make[4]: stopped in 1 error make[4]: stopped in *** [_sub.all] Error code 2 make[3]: stopped in 1 error make[3]: stopped in *** [secure.all__D] Error code 2 make[2]: stopped in --- share.all__D --- A failure has been detected in another branch of the parallel make make[5]: stopped in *** [_sub.all] Error code 2 make[4]: stopped in 1 error make[4]: stopped in *** [_sub.all] Error code 2 make[3]: stopped in 1 error make[3]: stopped in *** [share.all__D] Error code 2 make[2]: stopped in 4 errors make[2]: stopped in *** [everything] Error code 2 make[1]: stopped in 1 error make[1]: stopped in *** [buildworld] Error code 2 make: stopped in 1 error make: stopped in Build step 'Execute shell' marked build as failure From owner-freebsd-current@FreeBSD.ORG Sun Mar 1 01:33:28 2015 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id CD9FA981; Sun, 1 Mar 2015 01:33:28 +0000 (UTC) Received: from mail-ie0-x230.google.com (mail-ie0-x230.google.com [IPv6:2607:f8b0:4001:c03::230]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 9160C907; Sun, 1 Mar 2015 01:33:28 +0000 (UTC) Received: by iecrd18 with SMTP id rd18so39886464iec.5; Sat, 28 Feb 2015 17:33:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=jl0TN2MZ71kptoHZfXOUflL71O1lO4OopefqlyuDGuc=; b=cIQBgWfe+Uvhx5qjdrhqOHtJKpd8CCl0CpHZvjwlyKpfDhiT2oSV/X6qeBmlo1bQXS IJ4g+wyIoy1Q21mIHRD1qBQ4x4PqpxQwfpiRGAeeAGF0d8bxxNPuHJLJhhkt79EQpvaF jx2XaD39KePKL20UmTI/R+6kH+P2wNfHW7l7GIPh8nMsuFpvywbEp0nCUub5m6eKHLUM Cn/nY4L0UCXUV9vKf/nIwtP23GU/gv6i112e7b+veaeVqG0nza5rdWGVTWUrt3bDAjCz NJWrw+KF5voPL32ud4YcGBARj3IG0G8vNl7/MN4dD74B63eHI9J6HIQTjxDiSdqILb3S /nZA== MIME-Version: 1.0 X-Received: by 10.43.150.10 with SMTP id km10mr23413289icc.83.1425173607922; Sat, 28 Feb 2015 17:33:27 -0800 (PST) Received: by 10.107.156.75 with HTTP; Sat, 28 Feb 2015 17:33:27 -0800 (PST) In-Reply-To: <2106876404.49.1425173501970.JavaMail.jenkins@jenkins-9.freebsd.org> References: <1887508547.48.1425163799584.JavaMail.jenkins@jenkins-9.freebsd.org> <2106876404.49.1425173501970.JavaMail.jenkins@jenkins-9.freebsd.org> Date: Sat, 28 Feb 2015 20:33:27 -0500 Message-ID: Subject: Re: Build failed in Jenkins: FreeBSD_HEAD #2473 From: Ryan Stone To: jenkins-admin@freebsd.org Content-Type: text/plain; charset=UTF-8 X-Mailman-Approved-At: Sun, 01 Mar 2015 04:10:23 +0000 Cc: kan@freebsd.org, Adrian Chadd , loos@freebsd.org, Ryan Stone , alc@freebsd.org, FreeBSD Current , Andrew Rybchenko , kib@freebsd.org, ngie@freebsd.org, Mark Johnston X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 01 Mar 2015 01:33:29 -0000 *sigh*, this was bad timing by Jenkins. This is already fixed in r279440 (it's good to see that Jenkins will catch this kind of thing so quickly though) From owner-freebsd-current@FreeBSD.ORG Sun Mar 1 08:35:47 2015 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id CFFE0A68 for ; Sun, 1 Mar 2015 08:35:47 +0000 (UTC) Received: from mail-we0-x22d.google.com (mail-we0-x22d.google.com [IPv6:2a00:1450:400c:c03::22d]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 581DDF81 for ; Sun, 1 Mar 2015 08:35:47 +0000 (UTC) Received: by wesw55 with SMTP id w55so27742685wes.5 for ; Sun, 01 Mar 2015 00:35:45 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=DbxXHmQyHJztsQBrVoHe7JKp5lEOcepJiwR1QYRaAU0=; b=CeY7UWsf19A9y/8lu/xr1BbYzhwt9hZX7PpR/BYpdVaS2GYar4wlWz82YLkumLi6RV V2cBFbY3oSrsvMMRs7vNlU78RYdUsk4HjUMVnrTMUD2rEvF2VzXKMQL2v7X9Obbl9JwZ CnIEngoPgtS2XBXsFXKImAnAIuh8/Ko7p6Nh0JphLEVSi/BFPj+4Ci1IZy/V9DCJp3fD YCw7JTC5LLNkXiBr1IpWI3/VdtHdMPY3/w41qxQl9gGuwaEQFgOhEpuZimPYNY8R0o51 St60brdFuhXT349fnktO1MIsRGPmz7mLac0idOQlI7oH34S5yJTqFGqq0sAoC0f6BRse gQEA== MIME-Version: 1.0 X-Received: by 10.181.27.199 with SMTP id ji7mr23391164wid.76.1425198945695; Sun, 01 Mar 2015 00:35:45 -0800 (PST) Received: by 10.194.101.103 with HTTP; Sun, 1 Mar 2015 00:35:45 -0800 (PST) In-Reply-To: References: Date: Sun, 1 Mar 2015 00:35:45 -0800 Message-ID: Subject: Re: HEADS UP: PCI SR-IOV infrastructure has been committed to head From: Jack Vogel To: Ryan Stone , "Greer, Shan A" , "Ronciak, John" Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 Cc: FreeBSD Current X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 01 Mar 2015 08:35:47 -0000 Awesome news my friend. I got distracted with some fire drills this week, but I'll make it a priority to get the ixl changes committed next week. Thanks to you and everyone involved for getting this done. Jack On Sat, Feb 28, 2015 at 6:01 PM, Ryan Stone wrote: > I've just finished committing support for PCI Single Root I/O > Virtualization in the pci subsystem to head. This should be a no-op > for everyone right now, but there were some minor refactorings in the > pci code that could have a lingering bug. I did make sure to test > that it boots on a variety of systems (but only i386/amd64, as that's > all that I have access to). > > What's been committed to head is only the pci subsystem side of > things, along with the userland tools to configure SR-IOV (along with, > I'm happy to say, a full set of man pages). What's not in head yet > are any drivers making use of the infrastructure. Full support for > ixl(4) is complete and I've sent the patch to jfv@; I hope to see the > driver support committed soon. I don't have any word on timelines for > getting support in other drivers. Unfortunately adding SR-IOV support > to a driver is not trivial as the standard leaves a lot of the details > up to particular implementations (in the same way the the PCIe > standard does not define how to send a packet from a NIC; instead > defining how the PCIe device will expose its registers and whatnot, > and its up to the PCIe device and driver to understand how to poke at > the registers to send a packet). I have heard anecdotally that a > number of driver maintainers have been very interested in this work so > I hope that to see more drivers supported SR-IOV in the near future. > I encourage all driver maintainers to read over the new manpages and > contact me if they have any questions about the new infrastructure. > > Anybody interested in using SR-IOV should try to attend BSDCan 2015, > as I will be giving a talk on the subject. I intend to focus more on > the system administration side of configuring and using SR-IOV rather > than the details of implementing an SR-IOV driver. > > If anybody did an "svn up" half-way through my muddled series of > commits, sorry about the temporary breakage. My > buildworld/buildkernel on r279466 just completed successfully so > please make sure that you have at least that revision. If you still > have problems, please let me know. > > I do want to thank John Baldwin for advice about the PCI Subsystem and > newbus and Jack Vogel for his help with the Fortville NIC, including > getting me early access to the VF driver for testing purposes. Thanks > to everybody who reviewed the changes. Specially thanks to Mark > Johnston and Sean Mahood, who literally spent hours with me in a > meeting room reviewing the entire patch series last summer > (thankfully, those hours at least weren't consecutive). > > Above all, thanks to Sandvine Inc. for sponsoring this work. This is > definitely the biggest contribution we've ever made to FreeBSD and I > hope to see this kind of thing continue. > _______________________________________________ > freebsd-current@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-current > To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org" > From owner-freebsd-current@FreeBSD.ORG Sun Mar 1 06:57:53 2015 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3C324C3D; Sun, 1 Mar 2015 06:57:53 +0000 (UTC) Received: from jenkins-9.freebsd.org (jenkins-9.freebsd.org [8.8.178.209]) by mx1.freebsd.org (Postfix) with ESMTP id 285583CE; Sun, 1 Mar 2015 06:57:53 +0000 (UTC) Received: from jenkins-9.freebsd.org (localhost [127.0.0.1]) by jenkins-9.freebsd.org (Postfix) with ESMTP id D1726B11; Sun, 1 Mar 2015 06:57:50 +0000 (UTC) Date: Sun, 1 Mar 2015 06:57:44 +0000 (GMT) From: jenkins-admin@freebsd.org To: jenkins-admin@FreeBSD.org, freebsd-current@freebsd.org, loos@FreeBSD.org, rstone@FreeBSD.org, neel@FreeBSD.org, arybchik@FreeBSD.org, markj@FreeBSD.org, adrian@FreeBSD.org, ngie@FreeBSD.org, alc@FreeBSD.org, kib@FreeBSD.org, kan@FreeBSD.org Message-ID: <1045828539.50.1425193069707.JavaMail.jenkins@jenkins-9.freebsd.org> In-Reply-To: <2106876404.49.1425173501970.JavaMail.jenkins@jenkins-9.freebsd.org> References: <2106876404.49.1425173501970.JavaMail.jenkins@jenkins-9.freebsd.org> Subject: Jenkins build is back to normal : FreeBSD_HEAD #2474 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Jenkins-Job: FreeBSD_HEAD X-Jenkins-Result: SUCCESS X-Mailman-Approved-At: Sun, 01 Mar 2015 12:29:47 +0000 X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 01 Mar 2015 06:57:53 -0000 See From owner-freebsd-current@FreeBSD.ORG Sun Mar 1 13:34:24 2015 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 85A3857C for ; Sun, 1 Mar 2015 13:34:24 +0000 (UTC) Received: from mail-we0-x22f.google.com (mail-we0-x22f.google.com [IPv6:2a00:1450:400c:c03::22f]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 17866F49 for ; Sun, 1 Mar 2015 13:34:24 +0000 (UTC) Received: by wesu56 with SMTP id u56so28514448wes.10 for ; Sun, 01 Mar 2015 05:34:22 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:message-id:date:from:user-agent:mime-version:to:subject :content-type:content-transfer-encoding; bh=MFWk79UtiLDYhG0RyoM+tebM2FBu7ZNLz9CL6tvOcac=; b=juQ+7xMTtFvOtnovZ42PANBKg7j3bfXnFafBIHJWvj6DJPiuIZQVmokVBjkZ68VSrP d6sOjZXWi3LJwjWI/FrudW19XV631GLJYmy2fUUW7RP5UDS7sT6CULN/koRsK25ADnzh IEl4M4/YsTs8Zrs0xMkF46DNiTP67S9g2PzgsXqoWEOZTAaGH2E3jl85zrNgrgOwaip8 JAQJLvfvcmxl2CR5XQWsXLxvbOyKd5d6MkBNNdQgBz5ZDVZa7syeqeGOVw987G/rxPXy VyuYHM/PdEnQrwytlaa3JL3VHCTr4+hbgZXJt8skd2NOHgaT1ASX44ouf5pCeyjPmOqq 6lWw== X-Received: by 10.180.91.79 with SMTP id cc15mr17912655wib.37.1425216862420; Sun, 01 Mar 2015 05:34:22 -0800 (PST) Received: from ketas-laptop.mydomain (65-38-190-90.dyn.estpak.ee. [90.190.38.65]) by mx.google.com with ESMTPSA id hs7sm11555907wib.4.2015.03.01.05.34.20 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Sun, 01 Mar 2015 05:34:21 -0800 (PST) Sender: Sulev-Madis Silber Message-ID: <54F31510.7050607@hot.ee> Date: Sun, 01 Mar 2015 15:33:04 +0200 From: "Sulev-Madis Silber (ketas)" User-Agent: Mozilla/5.0 (X11; FreeBSD i386; rv:15.0) Gecko/20120912 Thunderbird/15.0.1 MIME-Version: 1.0 To: freebsd-current Subject: Massive libxo-zation that breaks everything X-TagToolbar-Keys: D20150301153303381 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 01 Mar 2015 13:34:24 -0000 Hello. First, I would be happy to have JSON and XML output about filesystems, users, routes... but I don't like how it makes code of df, w, netstat hard to read/maintain and often broken. I don't think it would be good to continue with this. Maybe the effort should be put to creating new layer/library and then something on top of it that actually outputs JSON and XML. Or, if that's too difficult... maybe just regular df/w/netstat could be copied to somewhere else and made code libxo-output-only. And original df/w/netstat changes reverted and left alone. Then, maybe later, df/w/netstat/... could be updated to this new layer/library. Or maybe this should be just left as it is. That would mean having two netstat's in system, which could be both good (separation) and bad (maintaining). Just some ideas... I don't know how to solve this issue fully. I'm also not likely the one who would write code for all this. Hell, those aren't even all my ideas here. I just worry that system drop-in xo-zation is bad for overall health of base. Oh and, it makes rescue larger and more complex, too? On that, there was suggestion to maybe create separate "first aid kit" and "emergency room" types of system rescue utils/methods. Thanks. From owner-freebsd-current@FreeBSD.ORG Sun Mar 1 16:26:10 2015 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7D9FC2EA for ; Sun, 1 Mar 2015 16:26:10 +0000 (UTC) Received: from mx1.scaleengine.net (beauharnois2.bhs1.scaleengine.net [142.4.218.15]) by mx1.freebsd.org (Postfix) with ESMTP id 5643F2F5 for ; Sun, 1 Mar 2015 16:26:09 +0000 (UTC) Received: from [192.168.1.2] (Seawolf.HML3.ScaleEngine.net [209.51.186.28]) (Authenticated sender: allanjude.freebsd@scaleengine.com) by mx1.scaleengine.net (Postfix) with ESMTPSA id 42E7F9366D for ; Sun, 1 Mar 2015 16:26:03 +0000 (UTC) Message-ID: <54F33D99.2060102@freebsd.org> Date: Sun, 01 Mar 2015 11:26:01 -0500 From: Allan Jude User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:31.0) Gecko/20100101 Thunderbird/31.5.0 MIME-Version: 1.0 To: freebsd-current@freebsd.org Subject: Re: Massive libxo-zation that breaks everything References: <54F31510.7050607@hot.ee> In-Reply-To: <54F31510.7050607@hot.ee> Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="ji7pKcvd5vk4sMLRScIx7QugpqU7ONcEc" X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 01 Mar 2015 16:26:10 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --ji7pKcvd5vk4sMLRScIx7QugpqU7ONcEc Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable On 2015-03-01 08:33, Sulev-Madis Silber (ketas) wrote: > Hello. >=20 > First, I would be happy to have JSON and XML output about filesystems, > users, routes... but I don't like how it makes code of df, w, netstat > hard to read/maintain and often broken. >=20 > I don't think it would be good to continue with this. Maybe the effort > should be put to creating new layer/library and then something on top o= f > it that actually outputs JSON and XML. >=20 > Or, if that's too difficult... maybe just regular df/w/netstat could be= > copied to somewhere else and made code libxo-output-only. And original > df/w/netstat changes reverted and left alone. >=20 > Then, maybe later, df/w/netstat/... could be updated to this new > layer/library. Or maybe this should be just left as it is. >=20 > That would mean having two netstat's in system, which could be both goo= d > (separation) and bad (maintaining). >=20 > Just some ideas... I don't know how to solve this issue fully. I'm also= > not likely the one who would write code for all this. Hell, those aren'= t > even all my ideas here. I just worry that system drop-in xo-zation is > bad for overall health of base. >=20 > Oh and, it makes rescue larger and more complex, too? On that, there wa= s > suggestion to maybe create separate "first aid kit" and "emergency room= " > types of system rescue utils/methods. >=20 >=20 > Thanks. > _______________________________________________ > freebsd-current@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-current > To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.o= rg" >=20 Is there a specific bug you are running in to? So far the only bugs I've seen with the xo-ification have been ones where the JSON output was not always well formed. --=20 Allan Jude --ji7pKcvd5vk4sMLRScIx7QugpqU7ONcEc Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (MingW32) iQIcBAEBAgAGBQJU8z2cAAoJEJrBFpNRJZKfuOcQAJ1IoB9K9juB+0MRW0c7A7v8 O1gM+IKyzTHuCagmUABH3LE3fR2VFhg+ZSsjdMj8AcstWCgfOk/KpcGXV0o1edJz U0T+e72D2rwfLpSxLbqF41f+fwDNOUMaGl2FUF11GlXs1EuKsAl99OWbobb8X0FG WvObM47TclFJS+CrA+1+FsQIm8YROVq8H1TAb2Qi1Ob6Vv/vWg1FQqWcsMQ5A5GG 7fBYJhomjC6uHfWoGA78yAwAPvdPyZKFnVCMRIQILN+8pVDPxXr1kreVlDwOjvPh 2B2RfrCBaWkbg4z2G2fCBOSi6UvThlIYoELwW3xFtutxWLE/KWv1kc5YtMdhr5sc y6VN6YL7M5cGW1BRPjwTBmtARz74uNfrUAqBME6BEklbcd4YQ3qyZst7aLzf4MfU ewbNfVgKOuC3EQfiLipUAojoxjrkJWtlDqfNyC+W5irZqp372T8UDyoB9PHrsUbj edpMUESPo/0N4ZRd3QXJ0t26wrlzeoy0exHPrygx+9Lk8BbKf9hU7Q5HmlZbdPhC mc+L2b23GyzOWWJYN9FwlqgUwYWQi5GdKHDX+et+Ibw7DVzywG77wmlS8WknKcNI fSTKk8ApIPEuPSW0RFSJ1Ea2atEjjjyMYan9mU55zA8iV+Jr3+qNcgoGkNVRntNB n5itg1rqTOLQJtbS2yVm =dZHx -----END PGP SIGNATURE----- --ji7pKcvd5vk4sMLRScIx7QugpqU7ONcEc-- From owner-freebsd-current@FreeBSD.ORG Sun Mar 1 16:46:50 2015 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A5056666; Sun, 1 Mar 2015 16:46:50 +0000 (UTC) Received: from troutmask.apl.washington.edu (troutmask.apl.washington.edu [128.95.76.21]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "troutmask", Issuer "troutmask" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 834CF6BE; Sun, 1 Mar 2015 16:46:50 +0000 (UTC) Received: from troutmask.apl.washington.edu (localhost [127.0.0.1]) by troutmask.apl.washington.edu (8.14.9/8.14.9) with ESMTP id t21GkiKm069675 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Sun, 1 Mar 2015 08:46:44 -0800 (PST) (envelope-from sgk@troutmask.apl.washington.edu) Received: (from sgk@localhost) by troutmask.apl.washington.edu (8.14.9/8.14.9/Submit) id t21Gkil4069674; Sun, 1 Mar 2015 08:46:44 -0800 (PST) (envelope-from sgk) Date: Sun, 1 Mar 2015 08:46:44 -0800 From: Steve Kargl To: Allan Jude Subject: Re: Massive libxo-zation that breaks everything Message-ID: <20150301164644.GA69625@troutmask.apl.washington.edu> References: <54F31510.7050607@hot.ee> <54F33D99.2060102@freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <54F33D99.2060102@freebsd.org> User-Agent: Mutt/1.5.23 (2014-03-12) Cc: freebsd-current@freebsd.org X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 01 Mar 2015 16:46:50 -0000 On Sun, Mar 01, 2015 at 11:26:01AM -0500, Allan Jude wrote: > On 2015-03-01 08:33, Sulev-Madis Silber (ketas) wrote: > > Is there a specific bug you are running in to? So far the only bugs I've > seen with the xo-ification have been ones where the JSON output was not > always well formed. > Given the poor quality of the manpages, it does lead one to wonder about the quality of the library code. -- Steve From owner-freebsd-current@FreeBSD.ORG Sun Mar 1 16:54:51 2015 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7D45F8C1 for ; Sun, 1 Mar 2015 16:54:51 +0000 (UTC) Received: from mail.grem.de (outcast.grem.de [213.239.217.27]) by mx1.freebsd.org (Postfix) with SMTP id B6E327F7 for ; Sun, 1 Mar 2015 16:54:49 +0000 (UTC) Received: (qmail 56407 invoked by uid 89); 1 Mar 2015 16:54:41 -0000 Received: from unknown (HELO bsd64.grem.de) (mg@grem.de@88.217.180.225) by mail.grem.de with ESMTPA; 1 Mar 2015 16:54:41 -0000 Date: Sun, 1 Mar 2015 17:54:36 +0100 From: Michael Gmelin To: Allan Jude Subject: Re: Massive libxo-zation that breaks everything Message-ID: <20150301175436.2da04674@bsd64.grem.de> In-Reply-To: <54F33D99.2060102@freebsd.org> References: <54F31510.7050607@hot.ee> <54F33D99.2060102@freebsd.org> X-Mailer: Claws Mail 3.11.1 (GTK+ 2.24.25; amd64-portbld-freebsd10.0) X-Face: $wrgCtfdVw_H9WAY?S&9+/F"!41z'L$uo*WzT8miX?kZ~W~Lr5W7v?j0Sde\mwB&/ypo^}> +a'4xMc^^KroE~+v^&^#[B">soBo1y6(TW6#UZiC]o>C6`ej+i Face: iVBORw0KGgoAAAANSUhEUgAAADAAAAAwBAMAAAClLOS0AAAAJFBMVEWJBwe5BQDl LASZU0/LTEWEfHbyj0Txi32+sKrp1Mv944X8/fm1rS+cAAAACXBIWXMAAAsTAAAL EwEAmpwYAAAAB3RJTUUH3wESCxwC7OBhbgAAACFpVFh0Q29tbWVudAAAAAAAQ3Jl YXRlZCB3aXRoIFRoZSBHSU1QbbCXAAAAAghJREFUOMu11DFvEzEUAGCfEhBVFzuq AKkLd0O6VrIQsLXVSZXoWE5N1K3DobBBA9fQpRWc8OkWouaIjedWKiyREOKs+3PY fvalCNjgLVHeF7/3bMtBzV8C/VsQ8tecEgCcDgrzjekwKZ7TwsJZd/ywEKwwP+ZM 8P3drTsAwWn2mpWuDDuYiK1bFs6De0KUUFw0tWxm+D4AIhuuvZqtyWYeO7jQ4Aea 7jUqI+ixhQoHex4WshEvSXdood7stlv4oSuFOC4tqGcr0NjEqXgV4mMJO38nld4+ xKNxRDon7khyKVqY7YR4d+Cg0OMrkWXZOM7YDkEfKiilCn1qYv4mighZiynuHHOA Wq9QJq+BIES7lMFUtcikMnkDGHUoncA+uHgrP0ctIEqfwLHzeSo+eUA66AqzwN6n 2ZHJhw6Qh/PoyC/QENyEyC/AyNjq74Bs+3UH0xYwzDUC4B97HgLocg1QLYgDDO1v f3UX9Y307Ew4AHh67YAFFsxEpkXwpXY3eIgMhAAE3R19L919nNnuD2wlPcDE3UeT L2ytEICQib9BXgS2fU8PrD82ToYO1OEmMSnYTjSqSv9wdC0tPYC+rQRQD9ESnldF CyqfmiYW+tlALt8gH2xrMdC/youbjzPXEun+/ReXsMCDyve3dZc09fn2Oas8oXGc Jj6/fOeK5UmSMPmf/jL+GD8BEj0k/Fn6IO4AAAAASUVORK5CYII= MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Cc: freebsd-current@freebsd.org X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 01 Mar 2015 16:54:51 -0000 On Sun, 01 Mar 2015 11:26:01 -0500 Allan Jude wrote: > On 2015-03-01 08:33, Sulev-Madis Silber (ketas) wrote: > > Hello. > > > > First, I would be happy to have JSON and XML output about > > filesystems, users, routes... but I don't like how it makes code of > > df, w, netstat hard to read/maintain and often broken. > > > > I don't think it would be good to continue with this. Maybe the > > effort should be put to creating new layer/library and then > > something on top of it that actually outputs JSON and XML. > > > > Or, if that's too difficult... maybe just regular df/w/netstat > > could be copied to somewhere else and made code libxo-output-only. > > And original df/w/netstat changes reverted and left alone. > > > > Then, maybe later, df/w/netstat/... could be updated to this new > > layer/library. Or maybe this should be just left as it is. > > > > That would mean having two netstat's in system, which could be both > > good (separation) and bad (maintaining). > > > > Just some ideas... I don't know how to solve this issue fully. I'm > > also not likely the one who would write code for all this. Hell, > > those aren't even all my ideas here. I just worry that system > > drop-in xo-zation is bad for overall health of base. > > > > Oh and, it makes rescue larger and more complex, too? On that, > > there was suggestion to maybe create separate "first aid kit" and > > "emergency room" types of system rescue utils/methods. > > > > > > Thanks. > > _______________________________________________ > > freebsd-current@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-current > > To unsubscribe, send any mail to > > "freebsd-current-unsubscribe@freebsd.org" > > > > Is there a specific bug you are running in to? So far the only bugs > I've seen with the xo-ification have been ones where the JSON output > was not always well formed. > There was one where uptime(1) wouldn't emit any output when called in a pipeline (missing call to xo_finish, that wouldn't affect the output on a tty due to implicit flush on newline). -- Michael Gmelin From owner-freebsd-current@FreeBSD.ORG Sun Mar 1 17:27:35 2015 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E341922E for ; Sun, 1 Mar 2015 17:27:35 +0000 (UTC) Received: from mailout.easymail.ca (mailout.easymail.ca [64.68.201.169]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 986D3B3E for ; Sun, 1 Mar 2015 17:27:35 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by mailout.easymail.ca (Postfix) with ESMTP id 677E5E414 for ; Sun, 1 Mar 2015 12:27:34 -0500 (EST) X-Virus-Scanned: Debian amavisd-new at mailout.easymail.ca X-Spam-Flag: NO X-Spam-Score: -3.853 X-Spam-Level: X-Spam-Status: No, score=-3.853 required=5 tests=[ALL_TRUSTED=-1.8, AWL=-0.146, BAYES_00=-2.599, DNS_FROM_AHBL_RHSBL=0.692] Received: from mailout.easymail.ca ([127.0.0.1]) by localhost (easymail-mailout.easydns.vpn [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 9ysd7DNmsjAN for ; Sun, 1 Mar 2015 12:27:34 -0500 (EST) Received: from bsddt1241.lv01.astrodoggroup.com (unknown [40.141.24.126]) (using TLSv1 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by mailout.easymail.ca (Postfix) with ESMTPSA id 3D8E1E3AB for ; Sun, 1 Mar 2015 12:27:32 -0500 (EST) Message-ID: <54F34B6E.2040809@astrodoggroup.com> Date: Sun, 01 Mar 2015 09:25:02 -0800 From: Harrison Grundy User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:31.0) Gecko/20100101 Thunderbird/31.3.0 MIME-Version: 1.0 To: freebsd-current@freebsd.org Subject: Re: Massive libxo-zation that breaks everything References: <54F31510.7050607@hot.ee> In-Reply-To: <54F31510.7050607@hot.ee> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-Mailman-Approved-At: Sun, 01 Mar 2015 17:52:22 +0000 X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 01 Mar 2015 17:27:36 -0000 On 03/01/15 05:33, Sulev-Madis Silber (ketas) wrote: > Hello. > > First, I would be happy to have JSON and XML output about > filesystems, users, routes... but I don't like how it makes code of > df, w, netstat hard to read/maintain and often broken. > > I don't think it would be good to continue with this. Maybe the > effort should be put to creating new layer/library and then > something on top of it that actually outputs JSON and XML. > > Or, if that's too difficult... maybe just regular df/w/netstat > could be copied to somewhere else and made code libxo-output-only. > And original df/w/netstat changes reverted and left alone. > > Then, maybe later, df/w/netstat/... could be updated to this new > layer/library. Or maybe this should be just left as it is. > > That would mean having two netstat's in system, which could be both > good (separation) and bad (maintaining). > > Just some ideas... I don't know how to solve this issue fully. I'm > also not likely the one who would write code for all this. Hell, > those aren't even all my ideas here. I just worry that system > drop-in xo-zation is bad for overall health of base. > > Oh and, it makes rescue larger and more complex, too? On that, > there was suggestion to maybe create separate "first aid kit" and > "emergency room" types of system rescue utils/methods. > > > Thanks. _______________________________________________ > freebsd-current@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-current To > unsubscribe, send any mail to > "freebsd-current-unsubscribe@freebsd.org" > Forgive my ignorance about exactly what this is, and possibly reviving a long-settled debate, but this sounds like something that would be a great deal more useful as a port/package, rather than in base. Due to the lack of XML parsing code in -base, the difficulty in maintaining yet another interface, and the overhead involved in doing it, I don't quite see where one would really want XML output *and* be entirely opposed to ports or packages. If someone could summarize what this is, I'd greatly appreciate it. --- Harrison From owner-freebsd-current@FreeBSD.ORG Sun Mar 1 18:09:02 2015 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 56120D9C for ; Sun, 1 Mar 2015 18:09:02 +0000 (UTC) Received: from st11p02mm-asmtp002.mac.com (st11p02mm-asmtpout002.mac.com [17.172.220.237]) (using TLSv1.2 with cipher DHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 29A58F0A for ; Sun, 1 Mar 2015 18:09:01 +0000 (UTC) Received: from fukuyama.hsd1.ca.comcast.net (unknown [73.162.13.215]) by st11p02mm-asmtp002.mac.com (Oracle Communications Messaging Server 7.0.5.35.0 64bit (built Dec 4 2014)) with ESMTPSA id <0NKJ00A3SP2CXT10@st11p02mm-asmtp002.mac.com> for freebsd-current@freebsd.org; Sun, 01 Mar 2015 18:08:38 +0000 (GMT) X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:5.13.68,1.0.33,0.0.0000 definitions=2015-03-01_03:2015-02-27,2015-03-01,1970-01-01 signatures=0 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 suspectscore=0 phishscore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=7.0.1-1412110000 definitions=main-1503010198 Content-type: text/plain; charset=us-ascii MIME-version: 1.0 (Mac OS X Mail 8.2 \(2070.6\)) Subject: Re: Massive libxo-zation that breaks everything From: Rui Paulo In-reply-to: <54F34B6E.2040809@astrodoggroup.com> Date: Sun, 01 Mar 2015 10:08:36 -0800 Content-transfer-encoding: quoted-printable Message-id: <7F7D08BE-7A18-4FEA-A79A-F9278524AE34@me.com> References: <54F31510.7050607@hot.ee> <54F34B6E.2040809@astrodoggroup.com> To: Harrison Grundy X-Mailer: Apple Mail (2.2070.6) X-Mailman-Approved-At: Sun, 01 Mar 2015 18:12:46 +0000 Cc: freebsd-current@freebsd.org X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 01 Mar 2015 18:09:02 -0000 On Mar 1, 2015, at 09:25, Harrison Grundy = wrote: > Due to the lack of XML parsing code in -base, the difficulty in > maintaining yet another interface, and the overhead involved in doing > it, I don't quite see where one would really want XML output *and* be > entirely opposed to ports or packages. You're already too late. Everything's already in base: man libbsdxml man libxo -- Rui Paulo From owner-freebsd-current@FreeBSD.ORG Sun Mar 1 18:31:14 2015 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id DFF427B4 for ; Sun, 1 Mar 2015 18:31:14 +0000 (UTC) Received: from mail-la0-x231.google.com (mail-la0-x231.google.com [IPv6:2a00:1450:4010:c03::231]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5F46425E for ; Sun, 1 Mar 2015 18:31:14 +0000 (UTC) Received: by labgf13 with SMTP id gf13so1684715lab.5 for ; Sun, 01 Mar 2015 10:31:11 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=JAMfvERgaBEXo0a6/EKixIdWQC4QTbYFj6RitUrP8jg=; b=Ys5W6Z2rvNFk825i6K70hrEE2+Tqd07r/K0rcWmaXI2NTHtLHr+7hDOFE9KW0BPpyO yGbLV1drD2fNfVK6LkgpGIvzzNPCl3K2yIhTLFxwqFVbqouicdGRn25vf9s/g7s9B/+y She53E/+9mYdaXSACyywKk1j4R+Hq+2WkXWVezh5xubpaGKec/Moo4vLjy22dpSgg5Wk r7qRRydP99528Uf07jOI9W6a3lleg+/TutrQLEIQSoAlja9M83HS0Ivzt2N+QltX67Fo 3tAdEkzmw58Fa8//ceXl7rkatxU+aXaNRUGe5rpOHhJohCaA0EKK/heZbEx+uD+saPd7 qU2w== MIME-Version: 1.0 X-Received: by 10.152.1.170 with SMTP id 10mr20978209lan.89.1425234671491; Sun, 01 Mar 2015 10:31:11 -0800 (PST) Sender: crodr001@gmail.com Received: by 10.112.82.164 with HTTP; Sun, 1 Mar 2015 10:31:11 -0800 (PST) In-Reply-To: <54F34B6E.2040809@astrodoggroup.com> References: <54F31510.7050607@hot.ee> <54F34B6E.2040809@astrodoggroup.com> Date: Sun, 1 Mar 2015 10:31:11 -0800 X-Google-Sender-Auth: j-tUg1AXlYlExo7_-D7ZVyP177s Message-ID: Subject: Re: Massive libxo-zation that breaks everything From: Craig Rodrigues To: Harrison Grundy Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 Cc: freebsd-current Current X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 01 Mar 2015 18:31:15 -0000 On Sun, Mar 1, 2015 at 9:25 AM, Harrison Grundy < harrison.grundy@astrodoggroup.com> wrote: > > > If someone could summarize what this is, I'd greatly appreciate it. > https://lists.freebsd.org/pipermail/freebsd-arch/2014-July/015633.html From owner-freebsd-current@FreeBSD.ORG Sun Mar 1 18:51:48 2015 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 46C7FC69 for ; Sun, 1 Mar 2015 18:51:48 +0000 (UTC) Received: from mailout.easymail.ca (mailout.easymail.ca [64.68.201.169]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 0833066B for ; Sun, 1 Mar 2015 18:51:47 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by mailout.easymail.ca (Postfix) with ESMTP id 694D3E3F9 for ; Sun, 1 Mar 2015 13:51:46 -0500 (EST) X-Virus-Scanned: Debian amavisd-new at mailout.easymail.ca X-Spam-Flag: NO X-Spam-Score: -3.852 X-Spam-Level: X-Spam-Status: No, score=-3.852 required=5 tests=[ALL_TRUSTED=-1.8, AWL=-0.145, BAYES_00=-2.599, DNS_FROM_AHBL_RHSBL=0.692] Received: from mailout.easymail.ca ([127.0.0.1]) by localhost (easymail-mailout.easydns.vpn [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id hKHFw7o3426s for ; Sun, 1 Mar 2015 13:51:46 -0500 (EST) Received: from bsddt1241.lv01.astrodoggroup.com (unknown [40.141.24.126]) (using TLSv1 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by mailout.easymail.ca (Postfix) with ESMTPSA id 082C3E3F4 for ; Sun, 1 Mar 2015 13:51:45 -0500 (EST) Message-ID: <54F35F29.4000603@astrodoggroup.com> Date: Sun, 01 Mar 2015 10:49:13 -0800 From: Harrison Grundy User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:31.0) Gecko/20100101 Thunderbird/31.3.0 MIME-Version: 1.0 To: freebsd-current@freebsd.org Subject: Re: Massive libxo-zation that breaks everything References: <54F31510.7050607@hot.ee> <54F34B6E.2040809@astrodoggroup.com> In-Reply-To: Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-Mailman-Approved-At: Sun, 01 Mar 2015 18:57:27 +0000 X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 01 Mar 2015 18:51:48 -0000 Thanks! That does seem useful, but I'm not sure I see the reasoning behind putting into base, over a port or package, since processing XML in base is a pain, and it can't serve up JSON or HTML without additional utilities anyway. (If I'm reviving a long-settled thing, let me know and I'll drop it. I'm trying to understand the use case for this.) --- Harrison On 03/01/15 10:31, Craig Rodrigues wrote: > On Sun, Mar 1, 2015 at 9:25 AM, Harrison Grundy < > harrison.grundy@astrodoggroup.com> wrote: > >> >> >> If someone could summarize what this is, I'd greatly appreciate it. >> > > https://lists.freebsd.org/pipermail/freebsd-arch/2014-July/015633.html > _______________________________________________ > freebsd-current@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-current > To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org" > From owner-freebsd-current@FreeBSD.ORG Sun Mar 1 19:10:43 2015 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 76D7A1BE for ; Sun, 1 Mar 2015 19:10:43 +0000 (UTC) Received: from mx1.scaleengine.net (beauharnois2.bhs1.scaleengine.net [142.4.218.15]) by mx1.freebsd.org (Postfix) with ESMTP id 4D810832 for ; Sun, 1 Mar 2015 19:10:42 +0000 (UTC) Received: from [192.168.1.2] (Seawolf.HML3.ScaleEngine.net [209.51.186.28]) (Authenticated sender: allanjude.freebsd@scaleengine.com) by mx1.scaleengine.net (Postfix) with ESMTPSA id C104193906 for ; Sun, 1 Mar 2015 19:10:41 +0000 (UTC) Message-ID: <54F36431.30506@freebsd.org> Date: Sun, 01 Mar 2015 14:10:41 -0500 From: Allan Jude User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:31.0) Gecko/20100101 Thunderbird/31.5.0 MIME-Version: 1.0 To: freebsd-current@freebsd.org Subject: Re: Massive libxo-zation that breaks everything References: <54F31510.7050607@hot.ee> <54F34B6E.2040809@astrodoggroup.com> <54F35F29.4000603@astrodoggroup.com> In-Reply-To: <54F35F29.4000603@astrodoggroup.com> Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="5I1KuTN0BdWEfaLX6iffIWpnkSXKhuTUD" X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 01 Mar 2015 19:10:43 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --5I1KuTN0BdWEfaLX6iffIWpnkSXKhuTUD Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable On 2015-03-01 13:49, Harrison Grundy wrote: > Thanks! >=20 > That does seem useful, but I'm not sure I see the reasoning behind > putting into base, over a port or package, since processing XML in base= > is a pain, and it can't serve up JSON or HTML without additional > utilities anyway. >=20 > (If I'm reviving a long-settled thing, let me know and I'll drop it. I'= m > trying to understand the use case for this.) >=20 > --- Harrison >=20 > On 03/01/15 10:31, Craig Rodrigues wrote: >> On Sun, Mar 1, 2015 at 9:25 AM, Harrison Grundy < >> harrison.grundy@astrodoggroup.com> wrote: >> >>> >>> >>> If someone could summarize what this is, I'd greatly appreciate it. >>> >> >> https://lists.freebsd.org/pipermail/freebsd-arch/2014-July/015633.html= >> _______________________________________________ >> freebsd-current@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-current >> To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.= org" >> > _______________________________________________ > freebsd-current@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-current > To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.o= rg" >=20 I think you're missing the important bit here. This isn't about adding a parser for anything, this is about making the tools in base, like netstat, wc, uptime, etc, output in JSON or XML, so you can use the data programmatically. Your scripts no longer have to rely on awk/sed/grep magic to get a specific bit of information out of the uptime command, the command can just output the data in a structured machine readable format. I am not sure how you can put netstat into the ports tree. --=20 Allan Jude --5I1KuTN0BdWEfaLX6iffIWpnkSXKhuTUD Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (MingW32) iQIcBAEBAgAGBQJU82QzAAoJEJrBFpNRJZKfZPcQAJH1H90OdW2/RWkPtaxl9WvB TeKL3C5bFfR8bbXj934BPDQnkxgOmbTq8lk4G/mYgME5wYrH48Vm20W5KPAqNLas dPd4hExFfCdGLfP+OUwdkdbvJIeDZaU9nmNzBHM7q0LvtDqopvXuhZ62ycO1ywdq 9YT7pS+rSzz2TfuePeS0C/sCYRnnHT8rG4Wbyjd3iJlfD3VTafm/QUy/TJTrZyg4 U6+VUizlwNjCqz+HQID8zECfnh44Z6nGlNZsSyKDGlFLCr/7jvoxko1jEwROsP2d uSs0tHN9a85Xsl1uT9mY9L0f9K7+Y+8SqehYxMGFF/RxeFgJYZwaGUwrdd19EN4V w9HTYaIhPkAFovqk5M6z2MP0QjRmXu9EUNUPN8c2qE0nLUrHrqhDkaGAxi+qjmzd ZELY6dqUxuu7rIiKePDGnLMvx0URIUQ4aSjmr9ZV2OtZ2hL/5iQbp8b+OoJH+7LN 0yQIm6+Mwp+SWq6Z/aokBOED3AcbsVYSvDFGSCO9ZqHT19f6VUMfnhkc0+ZJPpjg +yud72e635I0wYNhX2MA7TZfsr1R+6aowFMCQFFDp9xCO38dAdvNc6/oX5o9lqWm q91yTYrVrdTrtabNDc6EkclRFLh9zTd+QbNIshlERceEbCRi4na5bzmxlCIZhjvW MD1ySR1au3xTXV3mJnIT =i9Qf -----END PGP SIGNATURE----- --5I1KuTN0BdWEfaLX6iffIWpnkSXKhuTUD-- From owner-freebsd-current@FreeBSD.ORG Sun Mar 1 19:12:03 2015 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D88A52CC for ; Sun, 1 Mar 2015 19:12:03 +0000 (UTC) Received: from theravensnest.org (theraven.freebsd.your.org [216.14.102.27]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "cloud.theravensnest.org", Issuer "StartCom Class 1 Primary Intermediate Server CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 9992B8D6 for ; Sun, 1 Mar 2015 19:12:02 +0000 (UTC) Received: from [192.168.0.7] (cpc16-cmbg15-2-0-cust60.5-4.cable.virginm.net [86.5.162.61]) (authenticated bits=0) by theravensnest.org (8.15.1/8.14.9) with ESMTPSA id t21JBp45019846 (version=TLSv1 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Sun, 1 Mar 2015 19:11:54 GMT (envelope-from theraven@FreeBSD.org) X-Authentication-Warning: theravensnest.org: Host cpc16-cmbg15-2-0-cust60.5-4.cable.virginm.net [86.5.162.61] claimed to be [192.168.0.7] Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 8.2 \(2070.6\)) Subject: Re: Massive libxo-zation that breaks everything From: David Chisnall In-Reply-To: <54F35F29.4000603@astrodoggroup.com> Date: Sun, 1 Mar 2015 19:11:45 +0000 Content-Transfer-Encoding: quoted-printable Message-Id: References: <54F31510.7050607@hot.ee> <54F34B6E.2040809@astrodoggroup.com> <54F35F29.4000603@astrodoggroup.com> To: Harrison Grundy X-Mailer: Apple Mail (2.2070.6) Cc: freebsd-current@freebsd.org X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 01 Mar 2015 19:12:03 -0000 On 1 Mar 2015, at 18:49, Harrison Grundy = wrote: >=20 > That does seem useful, but I'm not sure I see the reasoning behind > putting into base, over a port or package, since processing XML in = base > is a pain, and it can't serve up JSON or HTML without additional > utilities anyway. How would it be in a port? It involves modifying core utilities (some = of which, like ifconfig, rely on kernel APIs that change between = releases) to emit structured output. Maintaining two copies of each = utility, one in the base system with plain-text output only and another = in ports with XML/JSON output would be very painful. The goal of having machine-readable output in the base system is that = people building systems atop FreeBSD will be able to expect a stable, = machine-parsable, extensible, output from these tools. If you're = building a web admin GUI or automated administration tool for FreeBSD = 11, or improving integration in your favourite DE, then you should be = able to rely on the output from base system tools, without having to = depend on anything external. My only concern with libxo at present is that many of the modified tools = are not emitting self-describing output (e.g. not specifying units for = things), but that's something that we have a year to shake out before = 11. David From owner-freebsd-current@FreeBSD.ORG Sun Mar 1 19:50:01 2015 Return-Path: Delivered-To: current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 929EDA35; Sun, 1 Mar 2015 19:50:01 +0000 (UTC) Received: from clavin1.langille.org (clavin.langille.org [162.208.116.86]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "clavin.langille.org", Issuer "StartCom Class 2 Primary Intermediate Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 52786BDC; Sun, 1 Mar 2015 19:50:00 +0000 (UTC) Received: from (clavin1.int.langille.org (clavin1.int.unixathome.org [10.4.7.7]) (Authenticated sender: hidden) with ESMTPSA id 57A1EA1F ; Sun, 1 Mar 2015 19:49:57 +0000 (UTC) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 8.2 \(2070.6\)) Subject: Re: sa(4) driver changes available for test From: Dan Langille In-Reply-To: Date: Sun, 1 Mar 2015 14:49:58 -0500 Content-Transfer-Encoding: quoted-printable Message-Id: <1C3D78F7-98C3-40E6-BA20-378A74E0BB35@langille.org> References: <20150214003232.GA63990@mithlond.kdm.org> <20150219001347.GA57416@mithlond.kdm.org> <45551872-C9F9-4597-BB67-5E853D2CB324@langille.org> <20150228231521.GA53017@mithlond.kdm.org> To: "Kenneth D. Merry" X-Mailer: Apple Mail (2.2070.6) Cc: "current@freebsd.org" , "scsi@freebsd.org" X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 01 Mar 2015 19:50:01 -0000 > On Feb 28, 2015, at 6:42 PM, Dan Langille wrote: >=20 >> On Feb 28, 2015, at 6:15 PM, Kenneth D. Merry = wrote: >>=20 >>> On Sat, Feb 28, 2015 at 17:29:48 -0500, Dan Langille wrote: >>>=20 >>>> On Feb 18, 2015, at 7:13 PM, Kenneth D. Merry = wrote: >>>>=20 >>>>=20 >>>> I have updated the patches. >>>>=20 >>>> I have removed the XPT_DEV_ADVINFO changes from the patches to = head, since >>>> I committed those separately. >>>>=20 >>>> I have (hopefully) fixed the build for the stable/10 patches by = MFCing >>>> dependencies. (One of them mav did for me, thanks!) >>>>=20 >>>> Rough draft commit message: >>>>=20 >>>> http://people.freebsd.org/~ken/sa_changes_commitmsg.20150218.1.txt >>>=20 >>>=20 >>> I have current installed and running with Bacula, but I have not = tried the tape drive yet. >>=20 >> Thanks for all your work on this! >>=20 >>> It seems like your changes are in there from about 5 days ago. >>=20 >> Yes, that is correct. >>=20 >>> Having solved my server hardware issues, I'm now having issues with = the autochanger mechanism=20 >>> of the tape library. =20 >>=20 >> Does it work with chio(1)? >>=20 >> Does it look like hardware or software? (If it is software, I can = help >> with that.) >>=20 >>=20 >=20 > Hardware. The screw drive for the tape robot is sticky. It can be = moved by hand, but the motor in this DL 891 cannot. It might need lube. = It was suspect last time I used it. Worst case, I will remove ins of = the two DLT 8000 tape drives and load tapes manually.=20 A bit of lube, and we have POST: = https://twitter.com/DLangille/status/572117570997919744 More, soon. =E2=80=94=20 Dan Langille http://langille.org/ From owner-freebsd-current@FreeBSD.ORG Sun Mar 1 19:49:29 2015 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 78B45A2F for ; Sun, 1 Mar 2015 19:49:29 +0000 (UTC) Received: from mailout.easymail.ca (mailout.easymail.ca [64.68.201.169]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 3095ABD9 for ; Sun, 1 Mar 2015 19:49:28 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by mailout.easymail.ca (Postfix) with ESMTP id 8E138E1B4 for ; Sun, 1 Mar 2015 14:49:24 -0500 (EST) X-Virus-Scanned: Debian amavisd-new at mailout.easymail.ca X-Spam-Flag: NO X-Spam-Score: -3.852 X-Spam-Level: X-Spam-Status: No, score=-3.852 required=5 tests=[ALL_TRUSTED=-1.8, AWL=-0.145, BAYES_00=-2.599, DNS_FROM_AHBL_RHSBL=0.692] Received: from mailout.easymail.ca ([127.0.0.1]) by localhost (easymail-mailout.easydns.vpn [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 98MWP9h8HNP8 for ; Sun, 1 Mar 2015 14:49:24 -0500 (EST) Received: from bsddt1241.lv01.astrodoggroup.com (unknown [40.141.24.126]) (using TLSv1 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by mailout.easymail.ca (Postfix) with ESMTPSA id 34292E0E1 for ; Sun, 1 Mar 2015 14:49:24 -0500 (EST) Message-ID: <54F36CA9.6090108@astrodoggroup.com> Date: Sun, 01 Mar 2015 11:46:49 -0800 From: Harrison Grundy User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:31.0) Gecko/20100101 Thunderbird/31.3.0 MIME-Version: 1.0 To: freebsd-current@freebsd.org Subject: Re: Massive libxo-zation that breaks everything References: <54F31510.7050607@hot.ee> <54F34B6E.2040809@astrodoggroup.com> <54F35F29.4000603@astrodoggroup.com> In-Reply-To: Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-Mailman-Approved-At: Sun, 01 Mar 2015 20:17:17 +0000 X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 01 Mar 2015 19:49:29 -0000 On 03/01/15 11:11, David Chisnall wrote: > On 1 Mar 2015, at 18:49, Harrison Grundy > wrote: >> >> That does seem useful, but I'm not sure I see the reasoning >> behind putting into base, over a port or package, since >> processing XML in base is a pain, and it can't serve up JSON or >> HTML without additional utilities anyway. > > How would it be in a port? It involves modifying core utilities > (some of which, like ifconfig, rely on kernel APIs that change > between releases) to emit structured output. Maintaining two > copies of each utility, one in the base system with plain-text > output only and another in ports with XML/JSON output would be very > painful. > > The goal of having machine-readable output in the base system is > that people building systems atop FreeBSD will be able to expect a > stable, machine-parsable, extensible, output from these tools. If > you're building a web admin GUI or automated administration tool > for FreeBSD 11, or improving integration in your favourite DE, then > you should be able to rely on the output from base system tools, > without having to depend on anything external. > > My only concern with libxo at present is that many of the modified > tools are not emitting self-describing output (e.g. not specifying > units for things), but that's something that we have a year to > shake out before 11. > > David Thinking about the kernel interfaces, I can see the need for it in base... but then it boils down to why, say, ifconfig in particular needs to support this behavior. It seems easier/safer to implement this independently of the existent commands using the various APIs. By putting this in the actual base utilities, an API that I would assume needs to remain relatively static is being introduced that may have some fairly large consequences for how those utilities need to operate. It just seems odd to me to have the same command one would use to, say, join a wireless network and set an IP also spit out HTML reports on interfaces. It seems like the needs of those programs may diverge over time, or that functionality needed for some new ifconfig thing would then require picking (and being stuck with) an XML schema for that information at the same time. --- Harrison From owner-freebsd-current@FreeBSD.ORG Sun Mar 1 20:43:46 2015 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 14B8988A for ; Sun, 1 Mar 2015 20:43:46 +0000 (UTC) Received: from mail-pd0-x22a.google.com (mail-pd0-x22a.google.com [IPv6:2607:f8b0:400e:c02::22a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id CD87E155 for ; Sun, 1 Mar 2015 20:43:45 +0000 (UTC) Received: by pdbfl12 with SMTP id fl12so3102079pdb.5 for ; Sun, 01 Mar 2015 12:43:45 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=content-type:mime-version:subject:from:in-reply-to:date:cc :message-id:references:to; bh=Nb073sCFWgzQOdA71H6RqaMQJFJcD8xTJk/ngNxj1Sk=; b=y1WX1EE6ZEO2vxvbK7r3VR8Df3XbLSI133TFRo61Sjuc+jZXAvcVz7x4mtye0SXSQ4 1rFDslCU/xPHpZ4JNts1jhdycvV05CbSDExS8Dm7acLqQ9j8167qXwBDpUIy8qsp9igg W2DsyQktzVS42foJ29rjF1BDZIg/46UOm0dbAU2gcYcYDRJWrnImuQrx0GOi2hgWYxQm 0d89BWs8nWX5UE7mmTaxONyyxG3dIFs3oTwpZjIn5qaq/J0kdbPhyDK8zrvFCN20oYMr Hr64ocNOE0yjbNsoeY/4DPPodGkKBVwFfS31TUQyS+qQrrQMzzJ3tZhUGxmKY6fIkVlI eVzw== X-Received: by 10.68.95.130 with SMTP id dk2mr41571459pbb.51.1425242625374; Sun, 01 Mar 2015 12:43:45 -0800 (PST) Received: from ?IPv6:2601:8:ab80:7d6:b571:b5ed:888f:2793? ([2601:8:ab80:7d6:b571:b5ed:888f:2793]) by mx.google.com with ESMTPSA id hr3sm9902572pbb.13.2015.03.01.12.43.44 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Sun, 01 Mar 2015 12:43:44 -0800 (PST) Content-Type: multipart/signed; boundary="Apple-Mail=_45536404-150D-48A2-8D02-91C80A38318C"; protocol="application/pgp-signature"; micalg=pgp-sha512 Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.6\)) Subject: Re: Massive libxo-zation that breaks everything From: Garrett Cooper In-Reply-To: <54F31510.7050607@hot.ee> Date: Sun, 1 Mar 2015 12:43:42 -0800 Message-Id: References: <54F31510.7050607@hot.ee> To: "Sulev-Madis Silber (ketas)" X-Mailer: Apple Mail (2.1878.6) Cc: freebsd-current X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 01 Mar 2015 20:43:46 -0000 --Apple-Mail=_45536404-150D-48A2-8D02-91C80A38318C Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=windows-1252 On Mar 1, 2015, at 5:33, Sulev-Madis Silber (ketas) = wrote: > Hello. >=20 > First, I would be happy to have JSON and XML output about filesystems, > users, routes... but I don't like how it makes code of df, w, netstat > hard to read/maintain and often broken. >=20 > I don't think it would be good to continue with this. Maybe the effort > should be put to creating new layer/library and then something on top = of > it that actually outputs JSON and XML. >=20 > Or, if that's too difficult... maybe just regular df/w/netstat could = be > copied to somewhere else and made code libxo-output-only. And original > df/w/netstat changes reverted and left alone. >=20 > Then, maybe later, df/w/netstat/... could be updated to this new > layer/library. Or maybe this should be just left as it is. >=20 > That would mean having two netstat's in system, which could be both = good > (separation) and bad (maintaining). >=20 > Just some ideas... I don't know how to solve this issue fully. I'm = also > not likely the one who would write code for all this. Hell, those = aren't > even all my ideas here. I just worry that system drop-in xo-zation is > bad for overall health of base. >=20 > Oh and, it makes rescue larger and more complex, too? On that, there = was > suggestion to maybe create separate "first aid kit" and "emergency = room" > types of system rescue utils/methods. Hi Sulev, Could you please cite instances where things were broken with = converting utilities (netstat seems to be a sticking point =97 do you = have data to support this?) over to libxo? The big problem (I think) is = lack of tests with these utilities to ensure that output doesn=92t break = unknowingly. I definitely see the wisdom in having /rescue be de-libxoified. = It seems to go against the purpose of having a small =93rescue image=94 = binary =97 whereas the bsdxml integration into geom(3)/geom(4) is a = necessary evil. Thanks! --Apple-Mail=_45536404-150D-48A2-8D02-91C80A38318C Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- Comment: GPGTools - https://gpgtools.org iQEcBAEBCgAGBQJU83n+AAoJEMZr5QU6S73eexYH/1MFAqXiTyCYqTU6y7bYj9WI cTweLVA9tD/IPgjfcybcx8E1IX7tiBPsUpFd19knGQXuEIcTJYKu/amceaouixIe p8trz58rpfoIc8lAUsyDZ41nUzTcREHpiqM5SfURoHpLQemwjfrIbOLG8kLBVKPc DVTigpRNjSTBNCBtQ842qc15A27RZk75HHGltMKoLtFIJrHS9gFiIV3HUIkLeSs6 RbwxJAkBW365egC6JAby6O/ItmER63PSv4pjU/gbWytaYebKMZwBOsH7a1b19ajs 0GvM1mib5r9iRiCyULE9qa80Pr3tOmfs3keyUYjHTPgTP7tHMnCxsDedjcawOks= =i689 -----END PGP SIGNATURE----- --Apple-Mail=_45536404-150D-48A2-8D02-91C80A38318C-- From owner-freebsd-current@FreeBSD.ORG Sun Mar 1 21:25:34 2015 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 0A3E3906 for ; Sun, 1 Mar 2015 21:25:34 +0000 (UTC) Received: from mail-la0-x230.google.com (mail-la0-x230.google.com [IPv6:2a00:1450:4010:c03::230]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 696FF809 for ; Sun, 1 Mar 2015 21:25:33 +0000 (UTC) Received: by labgf13 with SMTP id gf13so2128493lab.5 for ; Sun, 01 Mar 2015 13:25:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=/SiCg2hjstufv9mQdwXeVyb0PZx33I0k6ZjTPDboir0=; b=vXuMkTsXwnuInTcEAtXKaK+E3kUkYOi9y/TcGSWaJ9AA3ULO36AJ/8b+ostnRDmufh 0yjlbs7xrbJEkIC4amKSWDcL/oUl2x2m13NWmHdWDqjjUY7sPFcc6lVOJgqHnUoLY1xT oUoeS5TE/3dJOTViA6Ib2+NEHD3hO38hSK+AypZskUA148gw53VsWEcHMN7zNWgT7uyv BIb3IvEOcKe8ydTzLDL56FiEKFyy2khOWg9Jqc4roJ/YrktQeM3u1dEN6jPir3mWV3dr iMFLn+2srtLzVd/kf1Gk6b5/WvytMi0cf+oObr+ZJ7n/qtz68Dva39ViXO6NTsEamBD7 VmSA== MIME-Version: 1.0 X-Received: by 10.112.170.132 with SMTP id am4mr21369701lbc.89.1425245131089; Sun, 01 Mar 2015 13:25:31 -0800 (PST) Sender: crodr001@gmail.com Received: by 10.112.82.164 with HTTP; Sun, 1 Mar 2015 13:25:31 -0800 (PST) In-Reply-To: <54F35F29.4000603@astrodoggroup.com> References: <54F31510.7050607@hot.ee> <54F34B6E.2040809@astrodoggroup.com> <54F35F29.4000603@astrodoggroup.com> Date: Sun, 1 Mar 2015 13:25:31 -0800 X-Google-Sender-Auth: QU4t6jI6G4GgT2a7lMUgKqAah20 Message-ID: Subject: Re: Massive libxo-zation that breaks everything From: Craig Rodrigues To: Harrison Grundy Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 Cc: freebsd-current Current X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 01 Mar 2015 21:25:34 -0000 On Sun, Mar 1, 2015 at 10:49 AM, Harrison Grundy < harrison.grundy@astrodoggroup.com> wrote: > Thanks! > > That does seem useful, but I'm not sure I see the reasoning behind > putting into base, over a port or package > > , since processing XML in base is a pain, and it can't serve up JSON or > HTML without additional > utilities anyway. > I think if you take another pass at reading the entire thread of responses to see the discussion for the motivation behind libxo, you will get the idea: https://lists.freebsd.org/pipermail/freebsd-arch/2014-July/015633.html There are many people who are building products on top of FreeBSD. For these people, it is super useful for the base utilities in FreeBSD to emit output in properly formatted XML or JSON. That way, they do not need to write scripts to take the output of say, netstat, and use awk/sed/whatever scripts to take the human readable netstat output and convert it to a form which can be used in a script. There are many, many parsers for XML and JSON not in the base system. For people building products on top of FreeBSD, they don't care if these parsers are not in the base, since they can add these parsers on top of base FreeBSD. For example, languages like Python and Ruby have excellent parsers for JSON and XML. Many people build products using these languages. There are JSON and XML parsers in C, C++, and other languages as well. In addition to people building products, the other audience of people who benefit from libxo are devops people. These days, devops folks have no problem using Python, Ruby, Perl, whatever to write scripts to interact with Unix boxes and pull information off of it. Having the base utilities emit info in native JSON or XML greatly facilitates this. I talked to one person who is improving FreeBSD support for Saltstack (a devops framework). He told me that if more base utilities such as sysctl, could use libxo to emit output in JSON, that would greatly facilitate improving FreeBSD support for these devops frameworks. That is because Saltstack would require less FreeBSD-specific parsing code for getting info from base utilities. -- Craig From owner-freebsd-current@FreeBSD.ORG Sun Mar 1 22:06:40 2015 Return-Path: Delivered-To: current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7C5E26F2; Sun, 1 Mar 2015 22:06:40 +0000 (UTC) Received: from clavin1.langille.org (clavin.langille.org [162.208.116.86]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "clavin.langille.org", Issuer "StartCom Class 2 Primary Intermediate Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 33B2FBE2; Sun, 1 Mar 2015 22:06:38 +0000 (UTC) Received: from (clavin1.int.langille.org (clavin1.int.unixathome.org [10.4.7.7]) (Authenticated sender: hidden) with ESMTPSA id 3C4E0DF8 ; Sun, 1 Mar 2015 22:06:26 +0000 (UTC) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 8.2 \(2070.6\)) Subject: Re: sa(4) driver changes available for test From: Dan Langille In-Reply-To: <20150214003232.GA63990@mithlond.kdm.org> Date: Sun, 1 Mar 2015 17:06:24 -0500 Content-Transfer-Encoding: quoted-printable Message-Id: <4A478C5C-7965-498E-9F0F-80192265E310@langille.org> References: <20150214003232.GA63990@mithlond.kdm.org> To: "Kenneth D. Merry" X-Mailer: Apple Mail (2.2070.6) Cc: current@freebsd.org, scsi@freebsd.org X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 01 Mar 2015 22:06:40 -0000 > On Feb 13, 2015, at 7:32 PM, Kenneth D. Merry wrote: >=20 >=20 > I have a fairly large set of changes to the sa(4) driver and mt(1) = driver > that I'm planning to commit in the near future. >=20 > A description of the changes is here and below in this message. >=20 > If you have tape hardware and the inclination, I'd appreciate testing = and > feedback. >=20 > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > Rough draft commit message: >=20 > http://people.freebsd.org/~ken/sa_changes_commitmsg.20150213.3.txt >=20 > The patches against FreeBSD/head as of SVN revision 278706: >=20 > http://people.freebsd.org/~ken/sa_changes.20150213.3.txt >=20 > And (untested) patches against FreeBSD stable/10 as of SVN revision = 278721. >=20 > http://people.freebsd.org/~ken/sa_changes.stable_10.20150213.3.txt > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D >=20 > The intent is to get the tape infrastructure more up to date, so we = can > support LTFS and more modern tape drives: >=20 > http://www.ibm.com/systems/storage/tape/ltfs/ >=20 > I have ported IBM's LTFS Single Drive Edition to FreeBSD. The port = depends > on the patches linked above. It isn't fully cleaned up and ready for > redistribution. If you're interested, though, let me know and I'll = tell > you when it is ready to go out. You need an IBM LTO-5, LTO-6, TS1140 = or > TS1150 tape drive. HP drives aren't supported by IBM's LTFS, and = older > drives don't have the necessary features to support LTFS. >=20 > The commit message below outlines most of the changes. >=20 > A few comments: >=20 > 1. I'm planning to commit the XPT_DEV_ADVINFO changes separately. >=20 > 2. The XML output is similar to what GEOM and CTL do. It would be = nice to > figure out how to put a standard schema on it so that standard tools > could read it. I don't know how feasible that is, since I haven't > time to dig into it. If anyone has suggestions on whether that is > feasible or advisable, I'd appreciate feedback. >=20 > 3. I have tested with a reasonable amount of tape hardware (see below = for a > list), but more testing and feedback would be good. >=20 > 4. Standard 'mt status' output looks like this: >=20 > # mt -f /dev/nsa3 status -v > Drive: sa3: Serial Number: 101500520A > --------------------------------- > Mode Density Blocksize bpi Compression > Current: 0x5a:LTO-6 variable 384607 enabled (0xff) > --------------------------------- > Current Driver State: at rest. > --------------------------------- > Partition: 0 Calc File Number: 0 Calc Record Number: 0 > Residual: 0 Reported File Number: 0 Reported Record Number: 0 > Flags: BOP >=20 > 5. 'mt status -v' looks like this: >=20 > # mt -f /dev/nsa3 status -v > Drive: sa3: Serial Number: 101500520A > --------------------------------- > Mode Density Blocksize bpi Compression > Current: 0x5a:LTO-6 variable 384607 enabled (0xff) > --------------------------------- > Current Driver State: at rest. > --------------------------------- > Partition: 0 Calc File Number: 0 Calc Record Number: 0 > Residual: 0 Reported File Number: 0 Reported Record Number: 0 > Flags: BOP > --------------------------------- > Tape I/O parameters: > Maximum I/O size allowed by driver and controller (maxio): 1081344 = bytes > Maximum I/O size reported by controller (cpi_maxio): 5197824 bytes > Maximum block size supported by tape drive and media (max_blk): = 8388608 bytes > Minimum block size supported by tape drive and media (min_blk): 1 = bytes > Block granularity supported by tape drive and media (blk_gran): 0 = bytes > Maximum possible I/O size (max_effective_iosize): 1081344 bytes # mtx -f /dev/pass0 status Storage Changer /dev/pass0:2 Drives, 10 Slots ( 0 Import/Export ) Data Transfer Element 0:Empty Data Transfer Element 1:Empty Storage Element 1:Empty Storage Element 2:Empty Storage Element 3:Empty Storage Element 4:Full :VolumeTag=3DFAI260 = =20 Storage Element 5:Full :VolumeTag=3DFAI261 = =20 Storage Element 6:Full :VolumeTag=3DFAI262 = =20 Storage Element 7:Full :VolumeTag=3DFAI263 = =20 Storage Element 8:Empty Storage Element 9:Empty Storage Element 10:Empty It was at this point I spent the next 90 minute trying to get the tape=20= drive out of the tape library to free a stuck tape. Some of this was = spent attempting, and failing, to undo a stripped screw. I stopped the = attempt when I noticed the screw did need to be removed. :/ When I do this command, I hear the drive move a bit, to read the tape: # mt -f /dev/nsa1 status Drive: sa1: Serial Number: CXA09S1340 --------------------------------- Mode Density Blocksize bpi Compression Current: 0x1b:DLTapeIV(35GB) variable 85937 enabled (IDRC) --------------------------------- Current Driver State: at rest. --------------------------------- Partition: 0 Calc File Number: 0 Calc Record Number: 0 Residual: 0 Reported File Number: -1 Reported Record Number: -1 Flags: None # mt -f /dev/nsa1 ostatus =20 Mode Density Blocksize bpi Compression Current: 0x1b:DLTapeIV(35GB) variable 85937 IDRC ---------available modes--------- 0: 0x1b:DLTapeIV(35GB) variable 85937 IDRC 1: 0x1b:DLTapeIV(35GB) variable 85937 IDRC 2: 0x1b:DLTapeIV(35GB) variable 85937 IDRC 3: 0x1b:DLTapeIV(35GB) variable 85937 IDRC --------------------------------- Current Driver State: at rest. --------------------------------- File Number: 0 Record Number: 0 Residual Count 0 After doing a very small tar -c and tar -x, I have: # mt -f /dev/nsa1 /dev/nsa1 ostatus Mode Density Blocksize bpi Compression Current: 0x1b:DLTapeIV(35GB) variable 85937 IDRC ---------available modes--------- 0: 0x1b:DLTapeIV(35GB) variable 85937 IDRC 1: 0x1b:DLTapeIV(35GB) variable 85937 IDRC 2: 0x1b:DLTapeIV(35GB) variable 85937 IDRC 3: 0x1b:DLTapeIV(35GB) variable 85937 IDRC --------------------------------- Current Driver State: at rest. --------------------------------- File Number: 0 Record Number: 7 Residual Count 0 # mt -f /dev/nsa1 status -v Drive: sa1: Serial Number: CXA09S1340 --------------------------------- Mode Density Blocksize bpi Compression Current: 0x1b:DLTapeIV(35GB) variable 85937 enabled (IDRC) --------------------------------- Current Driver State: at rest. --------------------------------- Partition: 0 Calc File Number: 0 Calc Record Number: 7 Residual: 0 Reported File Number: -1 Reported Record Number: -1 Flags: None --------------------------------- Tape I/O parameters: Maximum I/O size allowed by driver and controller (maxio): 65536 bytes Maximum I/O size reported by controller (cpi_maxio): 0 bytes Maximum block size supported by tape drive and media (max_blk): = 16777214 bytes Minimum block size supported by tape drive and media (min_blk): 2 = bytes Block granularity supported by tape drive and media (blk_gran): 0 = bytes Maximum possible I/O size (max_effective_iosize): 65536 bytes I may not get to testing Bacula today. =20 Based on the above, is there any commands you'd like me to try? Read below regarding two tape drives >=20 > 6. Existing applications should work without changes. If not, please = let > me know. Hopefully they will move over time to the new interfaces. >=20 > 7. There are lots of additional features that could be added later. > Append-only support, encryption, more log pages, etc. >=20 > 8. I have SCSI READ ATTRIBUTE changes for camcontrol(8) that will go = in > separately. These changes allow displaying the contents of the MAM > (Medium Auxiliary Memory) chips on LTO, TS and other modern tape = drives. > These are good, and a future possible direction is adding attributes=20= > to the status XML from the sa(4) driver. >=20 > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > Significant upgrades to sa(4) and mt(1). >=20 > The primary focus of these changes is to modernize FreeBSD's > tape infrastructure so that we can take advantage of some of the > features of modern tape drives and allow support for LTFS. >=20 > Significant changes and new features include: >=20 > o sa(4) driver status and parameter information is now exported via an > XML structure. This will allow for changes and improvements later > on that will not break userland applications. The old MTIOCGET > status ioctl remains, so applications using the existing interface > will not break. >=20 > o 'mt status' now reports drive-reported tape position information > as well as the previously available calculated tape position > information. These numbers will be different at times, because > the drive-reported block numbers are relative to BOP (Beginning > of Partition), but the block numbers calculated previously via > sa(4) (and still provided) are relative to the last filemark. > Both numbers are now provided. 'mt status' now also shows the > drive INQUIRY information, serial number and any position flags > (BOP, EOT, etc.) provided with the tape position information. > 'mt status -v' adds information on the maximum possible I/O size, > and the underlying values used to calculate it. >=20 > o The extra sa(4) /dev entries (/dev/saN.[0-3]) have been removed. How does this affect a tape library with more than one tape drive? [root@cuppy:~] # camcontrol amcontrol devlist at scbus0 target 0 lun 0 (pass0,ch0) at scbus0 target 2 lun 0 (sa1,pass2) at scbus1 target 0 lun 0 (pass3,ada0) at scbus2 target 0 lun 0 (pass4,ada1) at scbus3 target 0 lun 0 (pass5,ses0) This system has two tapes drives and I can access them through the front = panel but: # ls -l /dev/*sa* crw-rw---- 1 root operator 0x65 Feb 28 22:04 /dev/esa1 crw-rw---- 1 root operator 0x64 Mar 1 22:43 /dev/nsa1 crw-rw---- 1 root operator 0x63 Feb 28 22:04 /dev/sa1 crw-rw---- 1 root operator 0x62 Feb 28 22:04 /dev/sa1.ctl ... only one tape drives shows up. >=20 > The extra devices were originally added as place holders for > density-specific device nodes. Some OSes (NetBSD, NetApp's OnTap > and Solaris) have had device nodes that, when you write to them, > will automatically select a given density for particular tape = drives. >=20 > This is a convenient way of switching densities, but it was never > implemented in FreeBSD. Only the device nodes were there, and that > sometimes confused users. >=20 > For modern tape devices, the density is generally not selectable > (e.g. with LTO) or defaults to the highest availble density when > the tape is rewritten from BOT (e.g. TS11X0). So, for most users, > density selection won't be necessary. If they do need to select > the density, it is easy enough to use 'mt density' to change it. >=20 > o Protection information is now supported. This is either a > Reed-Solomon CRC or CRC32 that is included at the end of each block > read and written. On write, the tape drive verifies the CRC, and > on read, the tape drive provides a CRC for the userland application > to verify. >=20 > o New, extensible tape driver parameter get/set interface. >=20 > o Density reporting information. For drives that support it, > 'mt getdensity' will show detailed information on what formats the > tape drive supports, and what formats the tape drive supports. >=20 > o Some mt(1) functionality moved into a new mt(3) library so that > external applications can reuse the code. >=20 > o The new mt(3) library includes helper routines to aid in parsing > the XML output of the sa(4) driver, and build a tree of driver > metadata. >=20 > o Support for the MTLOAD (load a tape in the drive) and MTWEOFI > (write filemark immediate) ioctls needed by IBM's LTFS > implementation. >=20 > o Improve device departure behavior for the sa(4) driver. The = previous > implementation led to hangs when the device was open. >=20 > o This has been tested on the following types of drives: > IBM TS1150 > IBM TS1140 > IBM LTO-6 > IBM LTO-5 > HP LTO-2 > Seagate DDS-4 > Quantum DLT-4000 > Exabyte 8505 > Sony DDS-2 >=20 > contrib/groff/tmac/doc-syms, > share/mk/bsd.libnames.mk, > lib/Makefile, > Add libmt. >=20 > lib/libmt/Makefile, > lib/libmt/mt.3, > lib/libmt/mtlib.c, > lib/libmt/mtlib.h, > New mt(3) library that contains functions moved from mt(1) and > new functions needed to interact with the updated sa(4) driver. >=20 > This includes XML parser helper functions that application = writers > can use when writing code to query tape parameters. >=20 > rescue/rescue/Makefile: > Add -lmt to CRUNCH_LIBS. >=20 > sys/cam/cam_ccb.h > Add a new flag value for the XPT_DEV_ADVINFO CCB, = CDAI_FLAG_NONE. >=20 > sbin/camcontrol/camcontrol.c, > sys/cam/scsi/scsi_da.c, > sys/cam/scsi/scsi_enc_ses.c, > sys/dev/mps/mps_sas.c: > Make sure the flags for the XPT_DEV_ADVINFO CCB are set = correctly. > This prevents unintended attempts to set advanced information > values when XPT_DEV_ADVINFO CCBs are not pre-zeroed. >=20 > src/share/man/man4/mtio.4 > Clarify this man page a bit, and since it contains what is > essentially the mtio.h header file, add new ioctls and structure > definitions from mtio.h. >=20 > src/share/man/man4/sa.4 > Update BUGS and maintainer section. >=20 > sys/cam/scsi/scsi_all.c, > sys/cam/scsi/scsi_all.h: > Add SCSI SECURITY PROTOCOL IN/OUT CDB definitions and CDB = building > functions. >=20 > sys/cam/scsi/scsi_sa.c > sys/cam/scsi/scsi_sa.h > Many tape driver changes, largely outlined above. >=20 > Increase the sa(4) driver read/write timeout from 4 to 32 > minutes. This is based on the recommended values for IBM LTO > 5/6 drives. This may also avoid timeouts for other tape > hardware that can take a long time to do retries and error > recovery. Longer term, a better way to handle this is to ask > the drive for recommended timeout values using the REPORT > SUPPORTED OPCODES command. Modern IBM and Oracle tape drives > at least support that command, and it would allow for more > accurate timeout values. >=20 > Add XML status generation. This is done with a series of > macros to eliminate as much duplicate code as possible. The > new XML-based status values are reported through the new > MTIOCEXTGET ioctl. >=20 > Add XML driver parameter reporting, using the new MTIOCPARAMGET > ioctl. >=20 > Add a new driver parameter setting interface, using the new > MTIOCPARAMSET and MTIOCSETLIST ioctls. >=20 > Add a new MTIOCRBLIM ioctl to get block limits information. >=20 > Add CCB/CDB building routines scsi_locate_16, scsi_locate_10, > and scsi_read_position_10(). >=20 > scsi_locate_10 implements the LOCATE command, as does the > existing scsi_set_position() command. It just supports > additional arguments and features. If/when we figure out a > good way to provide backward compatibility for older > applications using the old function API, we can just revamp > scsi_set_position(). The same goes for > scsi_read_position_10() and the existing scsi_read_position() > function. >=20 > Revamp sasetpos() to take the new mtlocate structure as an > argument. It now will use either scsi_locate_10() or > scsi_locate_16(), depending upon the arguments the user > supplies. As before, once we change position we don't have a > clear idea of what the current logical position of the tape > drive is. >=20 > For tape drives that support long form position data, we > read the current position and store that for later reporting > after changing the position. This should help applications > like Bacula speed tape access under FreeBSD once they are > modified to support the new ioctls. >=20 > Add a new quirk, SA_QUIRK_NO_LONG_POS, that is set for all > drives that report SCSI-2 or older, as well as drives that > report an Illegal Request type error for READ POSITION with > the long format. So we should automatically detect drives > that don't support the long form and stop asking for it after > an initial try. >=20 > Add a partition number to the sa(4) softc. >=20 > Improve device departure handling. The previous implementation > led to hangs when the device was open. >=20 > If an application had the sa(4) driver open, and attempted to > close it after it went away, the cam_periph_release() call in > saclose() would cause the periph to get destroyed because that > was the last reference to it. Because destroy_dev() was > called from the sa(4) driver's cleanup routine (sacleanup()), > and would block waiting for the close to happen, a deadlock > would result. >=20 > So instead of calling destroy_dev() from the cleanup routine, > call destroy_dev_sched_cb() from saoninvalidate() and wait for > the callback. >=20 > Acquire a reference for devfs in saregister(), and release it > in the new sadevgonecb() routine when all devfs devices for=09 > the particular sa(4) driver instance are gone. >=20 > Add a new function, sasetupdev(), to centralize setting > per-instance devfs device parameters instead of repeating the > code in saregister(). >=20 > Add an open count to the softc, so we know how many > peripheral driver references are a result of open > sessions. >=20 > Add the D_TRACKCLOSE flag to the cdevsw flags so > that we get a 1:1 mapping of open to close calls > instead of a N:1 mapping. >=20 > This should be a no-op for everything except the > control device, since we don't allow more than one > open on non-control devices. >=20 > However, since we do allow multiple opens on the > control device, the combination of the open count > and the D_TRACKCLOSE flag should result in an > accurate peripheral driver reference count, and an > accurate open count. >=20 > The accurate open count allows us to release all > peripheral driver references that are the result > of open contexts once we get the callback from devfs. >=20 > sys/sys/mtio.h: > Add a number of new mt(4) ioctls and the requisite data > structures. None of the existing interfaces been removed > or changed. >=20 > This includes definitions for the following new ioctls: >=20 > MTIOCRBLIM /* get block limits */ > MTIOCEXTLOCATE /* seek to position */ > MTIOCEXTGET /* get tape status */ > MTIOCPARAMGET /* get tape params */ > MTIOCPARAMSET /* set tape params */ > MTIOCSETLIST /* set N params */ >=20 > usr.bin/mt/Makefile: > mt(1) now depends on libmt, libsbuf and libbsdxml. >=20 > usr.bin/mt/mt.1: > Document new mt(1) features and subcommands. >=20 > usr.bin/mt/mt.c: > Implement support for mt(1) subcommands that need to > use getopt(3) for their arguments. >=20 > Implement a new 'mt status' command to replace the old > 'mt status' command. The old status command has been > renamed 'ostatus'. >=20 > The new status function uses the MTIOCEXTGET ioctl, and > therefore parses the XML data to determine drive status. > The -x argument to 'mt status' allows the user to dump out > the raw XML reported by the kernel. >=20 > The new status display is mostly the same as the old status > display, except that it doesn't print the redundant density > mode information, and it does print the current partition > number and position flags. >=20 > Add a new command, 'mt locate', that will supersede the > old 'mt setspos' and 'mt sethpos' commands. 'mt locate' > implements all of the functionality of the MTIOCEXTLOCATE > ioctl, and allows the user to change the logical position > of the tape drive in a number of ways. (Partition, > block number, file number, set mark number, end of data.) > The immediate bit and the explicit address bits are > implemented, but not documented in the man page. >=20 > Add a new 'mt weofi' command to use the new MTWEOFI ioctl. > This allows the user to ask the drive to write a filemark > without waiting around for the operation to complete. >=20 > Add a new 'mt getdensity' command that gets the XML-based > tape drive density report from the sa(4) driver and displays > it. This uses the SCSI REPORT DENSITY SUPPORT command > to get comprehensive information from the tape drive about > what formats it is able to read and write. >=20 > Add a new 'mt protect' command that allows getting and setting > tape drive protection information. The protection information > is a CRC tacked on to the end of every read/write from and to > the tape drive. >=20 > Sponsored by: Spectra Logic > MFC after: 1 month >=20 > Thanks, >=20 > Ken > --=20 > Kenneth Merry > ken@FreeBSD.ORG > _______________________________________________ > freebsd-scsi@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-scsi > To unsubscribe, send any mail to = "freebsd-scsi-unsubscribe@freebsd.org" =E2=80=94=20 Dan Langille http://langille.org/ From owner-freebsd-current@FreeBSD.ORG Sun Mar 1 21:30:04 2015 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A12C4A42; Sun, 1 Mar 2015 21:30:04 +0000 (UTC) Received: from st11p02mm-asmtp002.mac.com (st11p02mm-asmtpout002.mac.com [17.172.220.237]) (using TLSv1.2 with cipher DHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 732F6838; Sun, 1 Mar 2015 21:30:04 +0000 (UTC) Received: from fukuyama.hsd1.ca.comcast.net (unknown [73.162.13.215]) by st11p02mm-asmtp002.mac.com (Oracle Communications Messaging Server 7.0.5.35.0 64bit (built Dec 4 2014)) with ESMTPSA id <0NKJ000HDYDW0W20@st11p02mm-asmtp002.mac.com>; Sun, 01 Mar 2015 21:29:58 +0000 (GMT) X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:5.13.68,1.0.33,0.0.0000 definitions=2015-03-01_03:2015-02-27,2015-03-01,1970-01-01 signatures=0 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 suspectscore=0 phishscore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=7.0.1-1412110000 definitions=main-1503010238 Content-type: text/plain; charset=us-ascii MIME-version: 1.0 (Mac OS X Mail 8.2 \(2070.6\)) Subject: Re: Massive libxo-zation that breaks everything From: Rui Paulo In-reply-to: Date: Sun, 01 Mar 2015 13:29:56 -0800 Content-transfer-encoding: quoted-printable Message-id: <75C49F53-C675-4712-A446-370025EED037@me.com> References: <54F31510.7050607@hot.ee> <54F34B6E.2040809@astrodoggroup.com> <54F35F29.4000603@astrodoggroup.com> To: David Chisnall X-Mailer: Apple Mail (2.2070.6) X-Mailman-Approved-At: Sun, 01 Mar 2015 22:38:43 +0000 Cc: Harrison Grundy , freebsd-current@freebsd.org X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 01 Mar 2015 21:30:04 -0000 On Mar 1, 2015, at 11:11, David Chisnall wrote: > How would it be in a port? It involves modifying core utilities (some = of which, like ifconfig, rely on kernel APIs that change between = releases) to emit structured output. Maintaining two copies of each = utility, one in the base system with plain-text output only and another = in ports with XML/JSON output would be very painful. It would work fine if we had *libraries* for ifconfig/netstat/route/etc. = Obviously that's not the case and no one has stepped up to implement = them. I've also seen FreeBSD committers expressing their distaste for = libraries for "trivial" command line utilities, which implies they are = unaware of another world beyond the CLI. :-) -- Rui Paulo From owner-freebsd-current@FreeBSD.ORG Sun Mar 1 22:00:21 2015 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 244C753A for ; Sun, 1 Mar 2015 22:00:21 +0000 (UTC) Received: from mailout.easymail.ca (mailout.easymail.ca [64.68.201.169]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id BDEFAAD7 for ; Sun, 1 Mar 2015 22:00:20 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by mailout.easymail.ca (Postfix) with ESMTP id AABE7E355 for ; Sun, 1 Mar 2015 17:00:18 -0500 (EST) X-Virus-Scanned: Debian amavisd-new at mailout.easymail.ca X-Spam-Flag: NO X-Spam-Score: -3.851 X-Spam-Level: X-Spam-Status: No, score=-3.851 required=5 tests=[ALL_TRUSTED=-1.8, AWL=-0.144, BAYES_00=-2.599, DNS_FROM_AHBL_RHSBL=0.692] Received: from mailout.easymail.ca ([127.0.0.1]) by localhost (easymail-mailout.easydns.vpn [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id OmGPrd74X0f8 for ; Sun, 1 Mar 2015 17:00:18 -0500 (EST) Received: from bsddt1241.lv01.astrodoggroup.com (unknown [40.141.24.126]) (using TLSv1 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by mailout.easymail.ca (Postfix) with ESMTPSA id 50686E349 for ; Sun, 1 Mar 2015 17:00:18 -0500 (EST) Message-ID: <54F38B52.6090209@astrodoggroup.com> Date: Sun, 01 Mar 2015 13:57:38 -0800 From: Harrison Grundy User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:31.0) Gecko/20100101 Thunderbird/31.3.0 MIME-Version: 1.0 To: freebsd-current@freebsd.org Subject: Re: Massive libxo-zation that breaks everything References: <54F31510.7050607@hot.ee> <54F34B6E.2040809@astrodoggroup.com> <54F35F29.4000603@astrodoggroup.com> In-Reply-To: Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-Mailman-Approved-At: Sun, 01 Mar 2015 23:46:43 +0000 X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 01 Mar 2015 22:00:21 -0000 On 03/01/15 13:25, Craig Rodrigues wrote: > On Sun, Mar 1, 2015 at 10:49 AM, Harrison Grundy < > harrison.grundy@astrodoggroup.com> wrote: > >> Thanks! >> >> That does seem useful, but I'm not sure I see the reasoning behind >> putting into base, over a port or package >> > > >> , since processing XML in base is a pain, and it can't serve up JSON or >> HTML without additional >> utilities anyway. >> > > > I think if you take another pass at reading the entire thread of responses > to see the discussion for the motivation behind libxo, you will get the > idea: > https://lists.freebsd.org/pipermail/freebsd-arch/2014-July/015633.html > > > There are many people who are building products on top of FreeBSD. > For these people, it is super useful for the base utilities in FreeBSD > to emit output in properly formatted XML or JSON. > > That way, they do not need to write scripts to take the output of > say, netstat, and use awk/sed/whatever scripts to take the human readable > netstat output and convert it to a form which can be used in a script. > > There are many, many parsers for XML and JSON not in the base system. > For people building products on top of FreeBSD, they don't care > if these parsers are not in the base, since they can add these parsers on > top of base FreeBSD. > > For example, languages like Python and Ruby have excellent parsers > for JSON and XML. Many people build products using these languages. > There are JSON and XML parsers in C, C++, and other languages as well. > > In addition to people building products, the other audience of people who > benefit from libxo are devops people. > These days, devops folks have no problem using Python, Ruby, Perl, > whatever to write scripts to interact with Unix boxes and pull information > off of it. Having the base utilities emit info in native JSON or XML > greatly facilitates this. > > I talked to one person who is improving FreeBSD support for Saltstack (a > devops framework). He told me > that if more base utilities such as sysctl, could use libxo to emit output > in JSON, that would greatly facilitate improving FreeBSD support for > these devops frameworks. That is because Saltstack would require > less FreeBSD-specific parsing code for getting info from base utilities. I like the idea behind this... where I'm running into difficulty is why these bits of functionality need to be combined. What someone does with ifconfig on the command line, versus what someone wants to know about their network interfaces in an XML dump may be very distinct bits of information. (For instance, an XML dump of network information may want to incorporate data from netstat, route, and ifconfig, while ifconfig is really only designed to deal with the interfaces themselves.) I know it involves some notable reworking, but would there be any interest in seeing what "libifying" a handful of binaries and splitting the libxo and CLI functionality looks like? It seems like this may give FreeBSD the best of both worlds, so the XML output isn't logically limited by how the CLI needs to work, and the CLI won't have to deal with the complexity XML output can require. I'd be happy to get this set up for a few binaries so people can see how it might work and what it could look like if there's any interest in going that route. --- Harrison From owner-freebsd-current@FreeBSD.ORG Mon Mar 2 00:15:12 2015 Return-Path: Delivered-To: current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 234B3E25; Mon, 2 Mar 2015 00:15:12 +0000 (UTC) Received: from clavin1.langille.org (clavin.langille.org [162.208.116.86]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "clavin.langille.org", Issuer "StartCom Class 2 Primary Intermediate Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E0BC4A01; Mon, 2 Mar 2015 00:15:11 +0000 (UTC) Received: from (clavin1.int.langille.org (clavin1.int.unixathome.org [10.4.7.7]) (Authenticated sender: hidden) with ESMTPSA id 96945F61 ; Mon, 2 Mar 2015 00:15:08 +0000 (UTC) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 8.2 \(2070.6\)) Subject: Re: sa(4) driver changes available for test From: Dan Langille In-Reply-To: <20150217183645.GA30947@mithlond.kdm.org> Date: Sun, 1 Mar 2015 19:15:05 -0500 Content-Transfer-Encoding: quoted-printable Message-Id: <303C48AC-0DA8-447C-8DE7-0A1DD5DD7351@langille.org> References: <20150214003232.GA63990@mithlond.kdm.org> <7CA52DF3-E073-4F50-BE4E-01C51CCDF2C7@langille.org> <20150217183645.GA30947@mithlond.kdm.org> To: "Kenneth D. Merry" X-Mailer: Apple Mail (2.2070.6) Cc: current@freebsd.org, scsi@freebsd.org X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Mar 2015 00:15:12 -0000 > On Feb 17, 2015, at 1:36 PM, Kenneth D. Merry wrote: >=20 > On Sat, Feb 14, 2015 at 18:22:43 -0500, Dan Langille wrote: >>=20 >>> On Feb 13, 2015, at 7:32 PM, Kenneth D. Merry = wrote: >>>=20 >>>=20 >>> I have a fairly large set of changes to the sa(4) driver and mt(1) = driver >>> that I'm planning to commit in the near future. >>>=20 >>> A description of the changes is here and below in this message. >>>=20 >>> If you have tape hardware and the inclination, I'd appreciate = testing and >>> feedback. >>=20 >> I have a DLT 8000 and an SDLT 220. >>=20 >> I don't have anything running current, but I have a spare machine = which I could use for testing. >>=20 >> Do you see any value is tests with that hardware? I'd be testing it = via Bacula. >>=20 >> disclosure: I'm the sysutils/bacula-* maintainer and a Bacula = committer. >>=20 >=20 > Actually, yes. Bacula is a bit tricky to configure, so your trying it = out > would be helpful if you have the time. >=20 > In looking at the manuals for both the SDLT 220 and the DLT 8000, they = both > claim to support long position information for the SCSI READ POSITION > command. >=20 > You can see what I'm talking about by doing: >=20 > mt eod > mt status >=20 > On my DDS-4 tape drive, this shows: >=20 > # mt -f /dev/nsa3 status > Drive: sa3: Serial Number: HJ00YWY > --------------------------------- > Mode Density Blocksize bpi Compression > Current: 0x26:DDS-4 1024 bytes 97000 enabled (DCLZ) > --------------------------------- > Current Driver State: at rest. > --------------------------------- > Partition: 0 Calc File Number: -1 Calc Record Number: -1 > Residual: 0 Reported File Number: -1 Reported Record Number: -1 > Flags: None >=20 > But on an LTO-5, which will give long position information, I get: >=20 > [root@doc ~]# mt status > Drive: sa0: > --------------------------------- > Mode Density Blocksize bpi Compression > Current: 0x58:LTO-5 variable 384607 enabled (0x1) > --------------------------------- > Current Driver State: at rest. > --------------------------------- > Partition: 0 Calc File Number: 2 Calc Record Number: -1 > Residual: 0 Reported File Number: 2 Reported Record Number: = 32373 > Flags: None >=20 > That, in combination with the changes I made to the position = information > code in the driver, mean that even the old MTIOCGET ioctl should = return an > accurate file number at end of data. e.g., on the LTO-5: >=20 > [root@doc ~]# mt ostatus > Mode Density Blocksize bpi Compression > Current: 0x58:LTO-5 variable 384607 0x1 > ---------available modes--------- > 0: 0x58:LTO-5 variable 384607 0x1 > 1: 0x58:LTO-5 variable 384607 0x1 > 2: 0x58:LTO-5 variable 384607 0x1 > 3: 0x58:LTO-5 variable 384607 0x1 > --------------------------------- > Current Driver State: at rest. > --------------------------------- > File Number: 2 Record Number: -1 Residual Count -1 >=20 > So the thing to try, in addition to just making sure that Bacula = continues > to work properly, is to try setting this for the tape drive in > bacula-sd.conf: >=20 > Hardware End of Medium =3D yes >=20 > It looks like the Bacula tape program (btape) has a test mode, and it = would > be good to run through the tests on one of the tape drives and see = whether > they work, and whether the results are different before and after the > changes. I'm not sure how to enable the test mode. I have this in /usr/local/etc/bacula/bacula-sd.conf Device { Name =3D DLT Description =3D "QUANTUM DLT7000 1624" Media Type =3D DLT Archive Device =3D /dev/nsa1 Autochanger =3D YES Drive Index =3D 0 Offline On Unmount =3D no Hardware End of Medium =3D yes BSF at EOM =3D yes Backward Space Record =3D no Fast Forward Space File =3D no TWO EOF =3D yes } FYI, http://www.freebsddiary.org/digital-tl891.php (from 2006) has a = btape test on this same model. Here's the test I ran tonight: [root@cuppy:/usr/home/dan] # btape -c = /usr/local/etc/bacula/bacula-sd.conf /dev/nsa1 = =20 Tape block granularity is 1024 bytes. btape: butil.c:287-0 Using device: "/dev/nsa1" for writing. btape: btape.c:469-0 open device "DLT" (/dev/nsa1): OK *test =3D=3D=3D Write, rewind, and re-read test =3D=3D=3D I'm going to write 10000 records and an EOF then write 10000 records and an EOF, then rewind, and re-read the data to verify that it is correct. This is an *essential* feature ... btape: btape.c:1152-0 Wrote 10000 blocks of 64412 bytes. btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) btape: btape.c:1168-0 Wrote 10000 blocks of 64412 bytes. btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) btape: btape.c:1210-0 Rewind OK. 10000 blocks re-read correctly. Got EOF on tape. 10000 blocks re-read correctly. =3D=3D=3D Test Succeeded. End Write, rewind, and re-read test =3D=3D=3D btape: btape.c:1277-0 Block position test btape: btape.c:1289-0 Rewind OK. Reposition to file:block 0:4 Block 5 re-read correctly. Reposition to file:block 0:200 Block 201 re-read correctly. Reposition to file:block 0:9999 Block 10000 re-read correctly. Reposition to file:block 1:0 Block 10001 re-read correctly. Reposition to file:block 1:600 Block 10601 re-read correctly. Reposition to file:block 1:9999 Block 20000 re-read correctly. =3D=3D=3D Test Succeeded. End Write, rewind, and re-read test =3D=3D=3D =3D=3D=3D Append files test =3D=3D=3D This test is essential to Bacula. I'm going to write one record in file 0, two records in file 1, and three records in file 2 btape: btape.c:574-0 Rewound "DLT" (/dev/nsa1) btape: btape.c:1907-0 Wrote one record of 64412 bytes. btape: btape.c:1909-0 Wrote block to device. btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) btape: btape.c:1907-0 Wrote one record of 64412 bytes. btape: btape.c:1909-0 Wrote block to device. btape: btape.c:1907-0 Wrote one record of 64412 bytes. btape: btape.c:1909-0 Wrote block to device. btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) btape: btape.c:1907-0 Wrote one record of 64412 bytes. btape: btape.c:1909-0 Wrote block to device. btape: btape.c:1907-0 Wrote one record of 64412 bytes. btape: btape.c:1909-0 Wrote block to device. btape: btape.c:1907-0 Wrote one record of 64412 bytes. btape: btape.c:1909-0 Wrote block to device. btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) btape: btape.c:469-0 open device "DLT" (/dev/nsa1): OK btape: btape.c:574-0 Rewound "DLT" (/dev/nsa1) btape: btape.c:1420-0 Now moving to end of medium. btape: btape.c:622-0 tape_dev.c:345 ioctl MTIOCGET error on "DLT" = (/dev/nsa1). ERR=3DNo error: 0. We should be in file 3. I am at file 0. This is NOT correct!!!! Append test failed. Attempting again. Setting "Hardware End of Medium =3D no and "Fast Forward Space File =3D no and retrying append test. =3D=3D=3D Append files test =3D=3D=3D This test is essential to Bacula. I'm going to write one record in file 0, two records in file 1, and three records in file 2 btape: btape.c:574-0 Rewound "DLT" (/dev/nsa1) btape: btape.c:1907-0 Wrote one record of 64412 bytes. btape: btape.c:1909-0 Wrote block to device. btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) btape: btape.c:1907-0 Wrote one record of 64412 bytes. btape: btape.c:1909-0 Wrote block to device. btape: btape.c:1907-0 Wrote one record of 64412 bytes. btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) btape: btape.c:1907-0 Wrote one record of 64412 bytes. btape: btape.c:1909-0 Wrote block to device. btape: btape.c:1907-0 Wrote one record of 64412 bytes. btape: btape.c:1909-0 Wrote block to device. btape: btape.c:1907-0 Wrote one record of 64412 bytes. btape: btape.c:1909-0 Wrote block to device. btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) btape: btape.c:469-0 open device "DLT" (/dev/nsa1): OK btape: btape.c:574-0 Rewound "DLT" (/dev/nsa1) btape: btape.c:1420-0 Now moving to end of medium. btape: btape.c:625-0 Moved to end of medium. We should be in file 3. I am at file 3. This is correct! Now the important part, I am going to attempt to append to the tape. btape: btape.c:1907-0 Wrote one record of 64412 bytes. btape: btape.c:1909-0 Wrote block to device. btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) btape: btape.c:574-0 Rewound "DLT" (/dev/nsa1) Done appending, there should be no I/O errors Doing Bacula scan of blocks: 1 block of 64448 bytes in file 1 End of File mark. 2 blocks of 64448 bytes in file 2 End of File mark. 3 blocks of 64448 bytes in file 3 End of File mark. 1 block of 64448 bytes in file 4 End of File mark. Total files=3D4, blocks=3D7, bytes =3D 451,136 End scanning the tape. We should be in file 4. I am at file 4. This is correct! It looks like the test worked this time, please add: Hardware End of Medium =3D No Fast Forward Space File =3D No to your Device resource in the Storage conf file. The above Bacula scan should have output identical to what follows. Please double check it ... =3D=3D=3D Sample correct output =3D=3D=3D 1 block of 64448 bytes in file 1 End of File mark. 2 blocks of 64448 bytes in file 2 End of File mark. 3 blocks of 64448 bytes in file 3 End of File mark. 1 block of 64448 bytes in file 4 End of File mark. Total files=3D4, blocks=3D7, bytes =3D 451,136 =3D=3D=3D End sample correct output =3D=3D=3D If the above scan output is not identical to the sample output, you MUST correct the problem or Bacula will not be able to write multiple Jobs to=20 the tape. Skipping read backwards test because BSR turned off. =3D=3D=3D Forward space files test =3D=3D=3D This test is essential to Bacula. I'm going to write five files then test forward spacing btape: btape.c:574-0 Rewound "DLT" (/dev/nsa1) btape: btape.c:1907-0 Wrote one record of 64412 bytes. btape: btape.c:1909-0 Wrote block to device. btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) btape: btape.c:1907-0 Wrote one record of 64412 bytes. btape: btape.c:1909-0 Wrote block to device. btape: btape.c:1907-0 Wrote one record of 64412 bytes. btape: btape.c:1909-0 Wrote block to device. btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) btape: btape.c:1907-0 Wrote one record of 64412 bytes. btape: btape.c:1909-0 Wrote block to device. btape: btape.c:1907-0 Wrote one record of 64412 bytes. btape: btape.c:1909-0 Wrote block to device. btape: btape.c:1907-0 Wrote one record of 64412 bytes. btape: btape.c:1909-0 Wrote block to device. btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) btape: btape.c:1907-0 Wrote one record of 64412 bytes. btape: btape.c:1909-0 Wrote block to device. btape: btape.c:1907-0 Wrote one record of 64412 bytes. btape: btape.c:1909-0 Wrote block to device. btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) btape: btape.c:1907-0 Wrote one record of 64412 bytes. btape: btape.c:1909-0 Wrote block to device. btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) btape: btape.c:574-0 Rewound "DLT" (/dev/nsa1) btape: btape.c:1634-0 Now forward spacing 1 file. We should be in file 1. I am at file 1. This is correct! btape: btape.c:1646-0 Now forward spacing 2 files. We should be in file 3. I am at file 3. This is correct! btape: btape.c:574-0 Rewound "DLT" (/dev/nsa1) btape: btape.c:1659-0 Now forward spacing 4 files. We should be in file 4. I am at file 4. This is correct! btape: btape.c:1677-0 Now forward spacing 1 more file. We should be in file 5. I am at file 5. This is correct! =3D=3D=3D End Forward space files test =3D=3D=3D Ah, I see you have an autochanger configured. To test the autochanger you must have a blank tape that I can write on in Slot 1. Do you wish to continue with the Autochanger test? (y/n): y =3D=3D=3D Autochanger test =3D=3D=3D 3301 Issuing autochanger "loaded" command. Nothing loaded in the drive. OK. 3303 Issuing autochanger "load 1 0" command. 3303 Autochanger "load 1 0" status is OK. btape: btape.c:469-0 open device "DLT" (/dev/nsa1): OK btape: btape.c:1564-0 Rewound "DLT" (/dev/nsa1) btape: btape.c:1571-0 Wrote EOF to "DLT" (/dev/nsa1) The test autochanger worked!! * >=20 >> I'll let the other Bacula devs know about this. They deal with the = hardware. I work on PostgreSQL. >>=20 >=20 > Thanks! If there are additional features they would like out of the = tape > driver, I'm happy to talk about it. (Or help if they'd like to use = the new > status reporting ioctl, MTIOCEXTGET or any of the other new ioctls.) Errors are interesting to me. Especially corrected errors. They are a = good indicator of tape quality. =E2=80=94=20 Dan Langille http://langille.org/ From owner-freebsd-current@FreeBSD.ORG Mon Mar 2 00:18:44 2015 Return-Path: Delivered-To: current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 0A9A2F89; Mon, 2 Mar 2015 00:18:44 +0000 (UTC) Received: from mithlond.kdm.org (mithlond.kdm.org [70.56.43.85]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "A1-33714", Issuer "A1-33714" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id A1D0FA32; Mon, 2 Mar 2015 00:18:43 +0000 (UTC) Received: from mithlond.kdm.org (localhost [127.0.0.1]) by mithlond.kdm.org (8.14.9/8.14.9) with ESMTP id t220IXO3071820 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Sun, 1 Mar 2015 17:18:33 -0700 (MST) (envelope-from ken@mithlond.kdm.org) Received: (from ken@localhost) by mithlond.kdm.org (8.14.9/8.14.9/Submit) id t220IXK4071819; Sun, 1 Mar 2015 17:18:33 -0700 (MST) (envelope-from ken) Date: Sun, 1 Mar 2015 17:18:33 -0700 From: "Kenneth D. Merry" To: Dan Langille Subject: Re: sa(4) driver changes available for test Message-ID: <20150302001833.GA71528@mithlond.kdm.org> References: <20150214003232.GA63990@mithlond.kdm.org> <4A478C5C-7965-498E-9F0F-80192265E310@langille.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4A478C5C-7965-498E-9F0F-80192265E310@langille.org> User-Agent: Mutt/1.5.23 (2014-03-12) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (mithlond.kdm.org [127.0.0.1]); Sun, 01 Mar 2015 17:18:33 -0700 (MST) X-Spam-Status: No, score=-2.9 required=5.0 tests=ALL_TRUSTED,BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on mithlond.kdm.org Cc: current@freebsd.org, scsi@freebsd.org X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Mar 2015 00:18:44 -0000 On Sun, Mar 01, 2015 at 17:06:24 -0500, Dan Langille wrote: > > > On Feb 13, 2015, at 7:32 PM, Kenneth D. Merry wrote: > > > > > > I have a fairly large set of changes to the sa(4) driver and mt(1) driver > > that I'm planning to commit in the near future. > > > > A description of the changes is here and below in this message. > > > > If you have tape hardware and the inclination, I'd appreciate testing and > > feedback. > > > > ============ > > Rough draft commit message: > > > > http://people.freebsd.org/~ken/sa_changes_commitmsg.20150213.3.txt > > > > The patches against FreeBSD/head as of SVN revision 278706: > > > > http://people.freebsd.org/~ken/sa_changes.20150213.3.txt > > > > And (untested) patches against FreeBSD stable/10 as of SVN revision 278721. > > > > http://people.freebsd.org/~ken/sa_changes.stable_10.20150213.3.txt > > ============ > > > > The intent is to get the tape infrastructure more up to date, so we can > > support LTFS and more modern tape drives: > > > > http://www.ibm.com/systems/storage/tape/ltfs/ > > > > I have ported IBM's LTFS Single Drive Edition to FreeBSD. The port depends > > on the patches linked above. It isn't fully cleaned up and ready for > > redistribution. If you're interested, though, let me know and I'll tell > > you when it is ready to go out. You need an IBM LTO-5, LTO-6, TS1140 or > > TS1150 tape drive. HP drives aren't supported by IBM's LTFS, and older > > drives don't have the necessary features to support LTFS. > > > > The commit message below outlines most of the changes. > > > > A few comments: > > > > 1. I'm planning to commit the XPT_DEV_ADVINFO changes separately. > > > > 2. The XML output is similar to what GEOM and CTL do. It would be nice to > > figure out how to put a standard schema on it so that standard tools > > could read it. I don't know how feasible that is, since I haven't > > time to dig into it. If anyone has suggestions on whether that is > > feasible or advisable, I'd appreciate feedback. > > > > 3. I have tested with a reasonable amount of tape hardware (see below for a > > list), but more testing and feedback would be good. > > > > 4. Standard 'mt status' output looks like this: > > > > # mt -f /dev/nsa3 status -v > > Drive: sa3: Serial Number: 101500520A > > --------------------------------- > > Mode Density Blocksize bpi Compression > > Current: 0x5a:LTO-6 variable 384607 enabled (0xff) > > --------------------------------- > > Current Driver State: at rest. > > --------------------------------- > > Partition: 0 Calc File Number: 0 Calc Record Number: 0 > > Residual: 0 Reported File Number: 0 Reported Record Number: 0 > > Flags: BOP > > > > 5. 'mt status -v' looks like this: > > > > # mt -f /dev/nsa3 status -v > > Drive: sa3: Serial Number: 101500520A > > --------------------------------- > > Mode Density Blocksize bpi Compression > > Current: 0x5a:LTO-6 variable 384607 enabled (0xff) > > --------------------------------- > > Current Driver State: at rest. > > --------------------------------- > > Partition: 0 Calc File Number: 0 Calc Record Number: 0 > > Residual: 0 Reported File Number: 0 Reported Record Number: 0 > > Flags: BOP > > --------------------------------- > > Tape I/O parameters: > > Maximum I/O size allowed by driver and controller (maxio): 1081344 bytes > > Maximum I/O size reported by controller (cpi_maxio): 5197824 bytes > > Maximum block size supported by tape drive and media (max_blk): 8388608 bytes > > Minimum block size supported by tape drive and media (min_blk): 1 bytes > > Block granularity supported by tape drive and media (blk_gran): 0 bytes > > Maximum possible I/O size (max_effective_iosize): 1081344 bytes > > > # mtx -f /dev/pass0 status > Storage Changer /dev/pass0:2 Drives, 10 Slots ( 0 Import/Export ) > Data Transfer Element 0:Empty > Data Transfer Element 1:Empty > Storage Element 1:Empty > Storage Element 2:Empty > Storage Element 3:Empty > Storage Element 4:Full :VolumeTag=FAI260 > Storage Element 5:Full :VolumeTag=FAI261 > Storage Element 6:Full :VolumeTag=FAI262 > Storage Element 7:Full :VolumeTag=FAI263 > Storage Element 8:Empty > Storage Element 9:Empty > Storage Element 10:Empty > > > It was at this point I spent the next 90 minute trying to get the tape > drive out of the tape library to free a stuck tape. Some of this was spent > attempting, and failing, to undo a stripped screw. I stopped the attempt when > I noticed the screw did need to be removed. :/ Thanks for all of the effort! Looks like it is paying off! :) > When I do this command, I hear the drive move a bit, to read the tape: > > # mt -f /dev/nsa1 status > Drive: sa1: Serial Number: CXA09S1340 > --------------------------------- > Mode Density Blocksize bpi Compression > Current: 0x1b:DLTapeIV(35GB) variable 85937 enabled (IDRC) > --------------------------------- > Current Driver State: at rest. > --------------------------------- > Partition: 0 Calc File Number: 0 Calc Record Number: 0 > Residual: 0 Reported File Number: -1 Reported Record Number: -1 > Flags: None Looks like the drive isn't reporting position information. It will still be useful to try it with Bacula, though. > # mt -f /dev/nsa1 ostatus > Mode Density Blocksize bpi Compression > Current: 0x1b:DLTapeIV(35GB) variable 85937 IDRC > ---------available modes--------- > 0: 0x1b:DLTapeIV(35GB) variable 85937 IDRC > 1: 0x1b:DLTapeIV(35GB) variable 85937 IDRC > 2: 0x1b:DLTapeIV(35GB) variable 85937 IDRC > 3: 0x1b:DLTapeIV(35GB) variable 85937 IDRC > --------------------------------- > Current Driver State: at rest. > --------------------------------- > File Number: 0 Record Number: 0 Residual Count 0 > > > After doing a very small tar -c and tar -x, I have: > > # mt -f /dev/nsa1 /dev/nsa1 ostatus > Mode Density Blocksize bpi Compression > Current: 0x1b:DLTapeIV(35GB) variable 85937 IDRC > ---------available modes--------- > 0: 0x1b:DLTapeIV(35GB) variable 85937 IDRC > 1: 0x1b:DLTapeIV(35GB) variable 85937 IDRC > 2: 0x1b:DLTapeIV(35GB) variable 85937 IDRC > 3: 0x1b:DLTapeIV(35GB) variable 85937 IDRC > --------------------------------- > Current Driver State: at rest. > --------------------------------- > File Number: 0 Record Number: 7 Residual Count 0 Woohoo! It works. > # mt -f /dev/nsa1 status -v > Drive: sa1: Serial Number: CXA09S1340 > --------------------------------- > Mode Density Blocksize bpi Compression > Current: 0x1b:DLTapeIV(35GB) variable 85937 enabled (IDRC) > --------------------------------- > Current Driver State: at rest. > --------------------------------- > Partition: 0 Calc File Number: 0 Calc Record Number: 7 > Residual: 0 Reported File Number: -1 Reported Record Number: -1 > Flags: None > --------------------------------- > Tape I/O parameters: > Maximum I/O size allowed by driver and controller (maxio): 65536 bytes > Maximum I/O size reported by controller (cpi_maxio): 0 bytes > Maximum block size supported by tape drive and media (max_blk): 16777214 bytes > Minimum block size supported by tape drive and media (min_blk): 2 bytes > Block granularity supported by tape drive and media (blk_gran): 0 bytes > Maximum possible I/O size (max_effective_iosize): 65536 bytes > > I may not get to testing Bacula today. > > Based on the above, is there any commands you'd like me to try? Aside from making sure things work okay with Bacula, that is probably sufficient. These drives won't support density reports or position information. > Read below regarding two tape drives > > > > > 6. Existing applications should work without changes. If not, please let > > me know. Hopefully they will move over time to the new interfaces. > > > > 7. There are lots of additional features that could be added later. > > Append-only support, encryption, more log pages, etc. > > > > 8. I have SCSI READ ATTRIBUTE changes for camcontrol(8) that will go in > > separately. These changes allow displaying the contents of the MAM > > (Medium Auxiliary Memory) chips on LTO, TS and other modern tape drives. > > These are good, and a future possible direction is adding attributes > > to the status XML from the sa(4) driver. > > > > ============ > > Significant upgrades to sa(4) and mt(1). > > > > The primary focus of these changes is to modernize FreeBSD's > > tape infrastructure so that we can take advantage of some of the > > features of modern tape drives and allow support for LTFS. > > > > Significant changes and new features include: > > > > o sa(4) driver status and parameter information is now exported via an > > XML structure. This will allow for changes and improvements later > > on that will not break userland applications. The old MTIOCGET > > status ioctl remains, so applications using the existing interface > > will not break. > > > > o 'mt status' now reports drive-reported tape position information > > as well as the previously available calculated tape position > > information. These numbers will be different at times, because > > the drive-reported block numbers are relative to BOP (Beginning > > of Partition), but the block numbers calculated previously via > > sa(4) (and still provided) are relative to the last filemark. > > Both numbers are now provided. 'mt status' now also shows the > > drive INQUIRY information, serial number and any position flags > > (BOP, EOT, etc.) provided with the tape position information. > > 'mt status -v' adds information on the maximum possible I/O size, > > and the underlying values used to calculate it. > > > > o The extra sa(4) /dev entries (/dev/saN.[0-3]) have been removed. > > How does this affect a tape library with more than one tape drive? > > [root@cuppy:~] # camcontrol amcontrol devlist > at scbus0 target 0 lun 0 (pass0,ch0) > at scbus0 target 2 lun 0 (sa1,pass2) > at scbus1 target 0 lun 0 (pass3,ada0) > at scbus2 target 0 lun 0 (pass4,ada1) > at scbus3 target 0 lun 0 (pass5,ses0) > > This system has two tapes drives and I can access them through the front panel but: > > # ls -l /dev/*sa* > crw-rw---- 1 root operator 0x65 Feb 28 22:04 /dev/esa1 > crw-rw---- 1 root operator 0x64 Mar 1 22:43 /dev/nsa1 > crw-rw---- 1 root operator 0x63 Feb 28 22:04 /dev/sa1 > crw-rw---- 1 root operator 0x62 Feb 28 22:04 /dev/sa1.ctl > > ... only one tape drives shows up. Hmm. The tape drive is listed as sa1, which implies that there may be an sa0 that was there previously or is in the process of probing. What does dmesg show? How about 'camcontrol devlist -v'? I would look at cabling and termination. Is this your library? http://manx.classiccmp.org/collections/mds-199909/cd3/tape/tl891uga.pdf If it is close enough, there are 6 connectors on the back. You would want to have something plugged into all 6, either a cable or a terminator. In the manual above, the SCSI IDs are set via the front panel. If the other drive is on the same bus as the drive above and the library device, it should be at a separate SCSI ID. > > The extra devices were originally added as place holders for > > density-specific device nodes. Some OSes (NetBSD, NetApp's OnTap > > and Solaris) have had device nodes that, when you write to them, > > will automatically select a given density for particular tape drives. > > > > This is a convenient way of switching densities, but it was never > > implemented in FreeBSD. Only the device nodes were there, and that > > sometimes confused users. > > > > For modern tape devices, the density is generally not selectable > > (e.g. with LTO) or defaults to the highest availble density when > > the tape is rewritten from BOT (e.g. TS11X0). So, for most users, > > density selection won't be necessary. If they do need to select > > the density, it is easy enough to use 'mt density' to change it. > > > > o Protection information is now supported. This is either a > > Reed-Solomon CRC or CRC32 that is included at the end of each block > > read and written. On write, the tape drive verifies the CRC, and > > on read, the tape drive provides a CRC for the userland application > > to verify. > > > > o New, extensible tape driver parameter get/set interface. > > > > o Density reporting information. For drives that support it, > > 'mt getdensity' will show detailed information on what formats the > > tape drive supports, and what formats the tape drive supports. > > > > o Some mt(1) functionality moved into a new mt(3) library so that > > external applications can reuse the code. > > > > o The new mt(3) library includes helper routines to aid in parsing > > the XML output of the sa(4) driver, and build a tree of driver > > metadata. > > > > o Support for the MTLOAD (load a tape in the drive) and MTWEOFI > > (write filemark immediate) ioctls needed by IBM's LTFS > > implementation. > > > > o Improve device departure behavior for the sa(4) driver. The previous > > implementation led to hangs when the device was open. > > > > o This has been tested on the following types of drives: > > IBM TS1150 > > IBM TS1140 > > IBM LTO-6 > > IBM LTO-5 > > HP LTO-2 > > Seagate DDS-4 > > Quantum DLT-4000 > > Exabyte 8505 > > Sony DDS-2 > > > > contrib/groff/tmac/doc-syms, > > share/mk/bsd.libnames.mk, > > lib/Makefile, > > Add libmt. > > > > lib/libmt/Makefile, > > lib/libmt/mt.3, > > lib/libmt/mtlib.c, > > lib/libmt/mtlib.h, > > New mt(3) library that contains functions moved from mt(1) and > > new functions needed to interact with the updated sa(4) driver. > > > > This includes XML parser helper functions that application writers > > can use when writing code to query tape parameters. > > > > rescue/rescue/Makefile: > > Add -lmt to CRUNCH_LIBS. > > > > sys/cam/cam_ccb.h > > Add a new flag value for the XPT_DEV_ADVINFO CCB, CDAI_FLAG_NONE. > > > > sbin/camcontrol/camcontrol.c, > > sys/cam/scsi/scsi_da.c, > > sys/cam/scsi/scsi_enc_ses.c, > > sys/dev/mps/mps_sas.c: > > Make sure the flags for the XPT_DEV_ADVINFO CCB are set correctly. > > This prevents unintended attempts to set advanced information > > values when XPT_DEV_ADVINFO CCBs are not pre-zeroed. > > > > src/share/man/man4/mtio.4 > > Clarify this man page a bit, and since it contains what is > > essentially the mtio.h header file, add new ioctls and structure > > definitions from mtio.h. > > > > src/share/man/man4/sa.4 > > Update BUGS and maintainer section. > > > > sys/cam/scsi/scsi_all.c, > > sys/cam/scsi/scsi_all.h: > > Add SCSI SECURITY PROTOCOL IN/OUT CDB definitions and CDB building > > functions. > > > > sys/cam/scsi/scsi_sa.c > > sys/cam/scsi/scsi_sa.h > > Many tape driver changes, largely outlined above. > > > > Increase the sa(4) driver read/write timeout from 4 to 32 > > minutes. This is based on the recommended values for IBM LTO > > 5/6 drives. This may also avoid timeouts for other tape > > hardware that can take a long time to do retries and error > > recovery. Longer term, a better way to handle this is to ask > > the drive for recommended timeout values using the REPORT > > SUPPORTED OPCODES command. Modern IBM and Oracle tape drives > > at least support that command, and it would allow for more > > accurate timeout values. > > > > Add XML status generation. This is done with a series of > > macros to eliminate as much duplicate code as possible. The > > new XML-based status values are reported through the new > > MTIOCEXTGET ioctl. > > > > Add XML driver parameter reporting, using the new MTIOCPARAMGET > > ioctl. > > > > Add a new driver parameter setting interface, using the new > > MTIOCPARAMSET and MTIOCSETLIST ioctls. > > > > Add a new MTIOCRBLIM ioctl to get block limits information. > > > > Add CCB/CDB building routines scsi_locate_16, scsi_locate_10, > > and scsi_read_position_10(). > > > > scsi_locate_10 implements the LOCATE command, as does the > > existing scsi_set_position() command. It just supports > > additional arguments and features. If/when we figure out a > > good way to provide backward compatibility for older > > applications using the old function API, we can just revamp > > scsi_set_position(). The same goes for > > scsi_read_position_10() and the existing scsi_read_position() > > function. > > > > Revamp sasetpos() to take the new mtlocate structure as an > > argument. It now will use either scsi_locate_10() or > > scsi_locate_16(), depending upon the arguments the user > > supplies. As before, once we change position we don't have a > > clear idea of what the current logical position of the tape > > drive is. > > > > For tape drives that support long form position data, we > > read the current position and store that for later reporting > > after changing the position. This should help applications > > like Bacula speed tape access under FreeBSD once they are > > modified to support the new ioctls. > > > > Add a new quirk, SA_QUIRK_NO_LONG_POS, that is set for all > > drives that report SCSI-2 or older, as well as drives that > > report an Illegal Request type error for READ POSITION with > > the long format. So we should automatically detect drives > > that don't support the long form and stop asking for it after > > an initial try. > > > > Add a partition number to the sa(4) softc. > > > > Improve device departure handling. The previous implementation > > led to hangs when the device was open. > > > > If an application had the sa(4) driver open, and attempted to > > close it after it went away, the cam_periph_release() call in > > saclose() would cause the periph to get destroyed because that > > was the last reference to it. Because destroy_dev() was > > called from the sa(4) driver's cleanup routine (sacleanup()), > > and would block waiting for the close to happen, a deadlock > > would result. > > > > So instead of calling destroy_dev() from the cleanup routine, > > call destroy_dev_sched_cb() from saoninvalidate() and wait for > > the callback. > > > > Acquire a reference for devfs in saregister(), and release it > > in the new sadevgonecb() routine when all devfs devices for > > the particular sa(4) driver instance are gone. > > > > Add a new function, sasetupdev(), to centralize setting > > per-instance devfs device parameters instead of repeating the > > code in saregister(). > > > > Add an open count to the softc, so we know how many > > peripheral driver references are a result of open > > sessions. > > > > Add the D_TRACKCLOSE flag to the cdevsw flags so > > that we get a 1:1 mapping of open to close calls > > instead of a N:1 mapping. > > > > This should be a no-op for everything except the > > control device, since we don't allow more than one > > open on non-control devices. > > > > However, since we do allow multiple opens on the > > control device, the combination of the open count > > and the D_TRACKCLOSE flag should result in an > > accurate peripheral driver reference count, and an > > accurate open count. > > > > The accurate open count allows us to release all > > peripheral driver references that are the result > > of open contexts once we get the callback from devfs. > > > > sys/sys/mtio.h: > > Add a number of new mt(4) ioctls and the requisite data > > structures. None of the existing interfaces been removed > > or changed. > > > > This includes definitions for the following new ioctls: > > > > MTIOCRBLIM /* get block limits */ > > MTIOCEXTLOCATE /* seek to position */ > > MTIOCEXTGET /* get tape status */ > > MTIOCPARAMGET /* get tape params */ > > MTIOCPARAMSET /* set tape params */ > > MTIOCSETLIST /* set N params */ > > > > usr.bin/mt/Makefile: > > mt(1) now depends on libmt, libsbuf and libbsdxml. > > > > usr.bin/mt/mt.1: > > Document new mt(1) features and subcommands. > > > > usr.bin/mt/mt.c: > > Implement support for mt(1) subcommands that need to > > use getopt(3) for their arguments. > > > > Implement a new 'mt status' command to replace the old > > 'mt status' command. The old status command has been > > renamed 'ostatus'. > > > > The new status function uses the MTIOCEXTGET ioctl, and > > therefore parses the XML data to determine drive status. > > The -x argument to 'mt status' allows the user to dump out > > the raw XML reported by the kernel. > > > > The new status display is mostly the same as the old status > > display, except that it doesn't print the redundant density > > mode information, and it does print the current partition > > number and position flags. > > > > Add a new command, 'mt locate', that will supersede the > > old 'mt setspos' and 'mt sethpos' commands. 'mt locate' > > implements all of the functionality of the MTIOCEXTLOCATE > > ioctl, and allows the user to change the logical position > > of the tape drive in a number of ways. (Partition, > > block number, file number, set mark number, end of data.) > > The immediate bit and the explicit address bits are > > implemented, but not documented in the man page. > > > > Add a new 'mt weofi' command to use the new MTWEOFI ioctl. > > This allows the user to ask the drive to write a filemark > > without waiting around for the operation to complete. > > > > Add a new 'mt getdensity' command that gets the XML-based > > tape drive density report from the sa(4) driver and displays > > it. This uses the SCSI REPORT DENSITY SUPPORT command > > to get comprehensive information from the tape drive about > > what formats it is able to read and write. > > > > Add a new 'mt protect' command that allows getting and setting > > tape drive protection information. The protection information > > is a CRC tacked on to the end of every read/write from and to > > the tape drive. > > > > Sponsored by: Spectra Logic > > MFC after: 1 month > > > > Thanks, > > > > Ken > > -- > > Kenneth Merry > > ken@FreeBSD.ORG > > _______________________________________________ > > freebsd-scsi@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-scsi > > To unsubscribe, send any mail to "freebsd-scsi-unsubscribe@freebsd.org" > > ? > Dan Langille > http://langille.org/ > > > > > -- Kenneth Merry ken@FreeBSD.ORG From owner-freebsd-current@FreeBSD.ORG Mon Mar 2 00:21:09 2015 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 99F19162; Mon, 2 Mar 2015 00:21:09 +0000 (UTC) Received: from mail-pd0-x230.google.com (mail-pd0-x230.google.com [IPv6:2607:f8b0:400e:c02::230]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 72C55A54; Mon, 2 Mar 2015 00:21:09 +0000 (UTC) Received: by pdbnh10 with SMTP id nh10so9280881pdb.4; Sun, 01 Mar 2015 16:21:09 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; bh=vCFZMZGNcKClrYkdvJAzz4Bk0FOuizOdCHTkIBBeGLQ=; b=FgwldTMqX6OI+Uh3EIwjnPs8MOkTeXtBMcd6UzS3GQynlAT5PZ37+aKx7q/BEqXYJ6 Ul4BRImgNIwqqprqcZvEgKE3emn5H51zKpeADZS6m4MkR/FBxqVqMvj28IM/RFvRwtcF OS4wMDccW8gzncG/uv2vhbaLKr7PeCi0cf8eGKc2Ym+FZ2nPjyypb20+P2eQpy+7LuhT MqaFxtjbHskedKB0OBwDCIixj0NSHw6Kt1MzBnh9FQXZ+QF1s4saBdsQpucxzLmCt5AH gGJzdWxfFGCVIwgmutDdeKvUa82mEFL05BqQLxZ14LmVMJDK6BkU1/5PWpJZ20TXYWI2 FSZg== X-Received: by 10.66.63.72 with SMTP id e8mr41628832pas.3.1425255668962; Sun, 01 Mar 2015 16:21:08 -0800 (PST) MIME-Version: 1.0 Received: by 10.66.237.39 with HTTP; Sun, 1 Mar 2015 16:20:48 -0800 (PST) In-Reply-To: <54F36431.30506@freebsd.org> References: <54F31510.7050607@hot.ee> <54F34B6E.2040809@astrodoggroup.com> <54F35F29.4000603@astrodoggroup.com> <54F36431.30506@freebsd.org> From: Arseny Nasokin Date: Mon, 2 Mar 2015 04:20:48 +0400 Message-ID: Subject: Re: Massive libxo-zation that breaks everything To: Allan Jude Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 Cc: freebsd-current X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Mar 2015 00:21:09 -0000 On 1 March 2015 at 22:10, Allan Jude wrote: > On 2015-03-01 13:49, Harrison Grundy wrote: > > Thanks! > > > > That does seem useful, but I'm not sure I see the reasoning behind > > putting into base, over a port or package, since processing XML in base > > is a pain, and it can't serve up JSON or HTML without additional > > utilities anyway. > > > > (If I'm reviving a long-settled thing, let me know and I'll drop it. I'm > > trying to understand the use case for this.) > > > > --- Harrison > > > > On 03/01/15 10:31, Craig Rodrigues wrote: > >> On Sun, Mar 1, 2015 at 9:25 AM, Harrison Grundy < > >> harrison.grundy@astrodoggroup.com> wrote: > >> > >>> > >>> > >>> If someone could summarize what this is, I'd greatly appreciate it. > >>> > >> > >> https://lists.freebsd.org/pipermail/freebsd-arch/2014-July/015633.html > >> _______________________________________________ > >> freebsd-current@freebsd.org mailing list > >> http://lists.freebsd.org/mailman/listinfo/freebsd-current > >> To unsubscribe, send any mail to " > freebsd-current-unsubscribe@freebsd.org" > >> > > _______________________________________________ > > freebsd-current@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-current > > To unsubscribe, send any mail to " > freebsd-current-unsubscribe@freebsd.org" > > > > I think you're missing the important bit here. > > This isn't about adding a parser for anything, this is about making the > tools in base, like netstat, wc, uptime, etc, output in JSON or XML, so > you can use the data programmatically. > > Your scripts no longer have to rely on awk/sed/grep magic to get a > specific bit of information out of the uptime command, the command can > just output the data in a structured machine readable format. > > I am not sure how you can put netstat into the ports tree. > > > -- > Allan Jude > > Hi, Do we have command-line tools in base which work with XML/JSON from stdin or file? -- Eir Nym From owner-freebsd-current@FreeBSD.ORG Mon Mar 2 00:28:40 2015 Return-Path: Delivered-To: current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 6F1EC2BB; Mon, 2 Mar 2015 00:28:40 +0000 (UTC) Received: from clavin1.langille.org (clavin.langille.org [162.208.116.86]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "clavin.langille.org", Issuer "StartCom Class 2 Primary Intermediate Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 19F7DB58; Mon, 2 Mar 2015 00:28:39 +0000 (UTC) Received: from (clavin1.int.langille.org (clavin1.int.unixathome.org [10.4.7.7]) (Authenticated sender: hidden) with ESMTPSA id ABDBAF64 ; Mon, 2 Mar 2015 00:28:38 +0000 (UTC) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 8.2 \(2070.6\)) Subject: Re: sa(4) driver changes available for test From: Dan Langille In-Reply-To: <20150302001833.GA71528@mithlond.kdm.org> Date: Sun, 1 Mar 2015 19:28:37 -0500 Content-Transfer-Encoding: quoted-printable Message-Id: <6C82281F-649A-4DA8-8ACF-17E81C04F730@langille.org> References: <20150214003232.GA63990@mithlond.kdm.org> <4A478C5C-7965-498E-9F0F-80192265E310@langille.org> <20150302001833.GA71528@mithlond.kdm.org> To: "Kenneth D. Merry" X-Mailer: Apple Mail (2.2070.6) Cc: current@freebsd.org, scsi@freebsd.org X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Mar 2015 00:28:40 -0000 > On Mar 1, 2015, at 7:18 PM, Kenneth D. Merry wrote: >=20 > On Sun, Mar 01, 2015 at 17:06:24 -0500, Dan Langille wrote: >>=20 >>> On Feb 13, 2015, at 7:32 PM, Kenneth D. Merry = wrote: >>>=20 >>>=20 >>> I have a fairly large set of changes to the sa(4) driver and mt(1) = driver >>> that I'm planning to commit in the near future. >>>=20 >>> A description of the changes is here and below in this message. >>>=20 >>> If you have tape hardware and the inclination, I'd appreciate = testing and >>> feedback. >>>=20 >>> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D >>> Rough draft commit message: >>>=20 >>> http://people.freebsd.org/~ken/sa_changes_commitmsg.20150213.3.txt >>>=20 >>> The patches against FreeBSD/head as of SVN revision 278706: >>>=20 >>> http://people.freebsd.org/~ken/sa_changes.20150213.3.txt >>>=20 >>> And (untested) patches against FreeBSD stable/10 as of SVN revision = 278721. >>>=20 >>> http://people.freebsd.org/~ken/sa_changes.stable_10.20150213.3.txt >>> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D >>>=20 >>> The intent is to get the tape infrastructure more up to date, so we = can >>> support LTFS and more modern tape drives: >>>=20 >>> http://www.ibm.com/systems/storage/tape/ltfs/ >>>=20 >>> I have ported IBM's LTFS Single Drive Edition to FreeBSD. The port = depends >>> on the patches linked above. It isn't fully cleaned up and ready = for >>> redistribution. If you're interested, though, let me know and I'll = tell >>> you when it is ready to go out. You need an IBM LTO-5, LTO-6, = TS1140 or >>> TS1150 tape drive. HP drives aren't supported by IBM's LTFS, and = older >>> drives don't have the necessary features to support LTFS. >>>=20 >>> The commit message below outlines most of the changes. >>>=20 >>> A few comments: >>>=20 >>> 1. I'm planning to commit the XPT_DEV_ADVINFO changes separately. >>>=20 >>> 2. The XML output is similar to what GEOM and CTL do. It would be = nice to >>> figure out how to put a standard schema on it so that standard = tools >>> could read it. I don't know how feasible that is, since I haven't >>> time to dig into it. If anyone has suggestions on whether that is >>> feasible or advisable, I'd appreciate feedback. >>>=20 >>> 3. I have tested with a reasonable amount of tape hardware (see = below for a >>> list), but more testing and feedback would be good. >>>=20 >>> 4. Standard 'mt status' output looks like this: >>>=20 >>> # mt -f /dev/nsa3 status -v >>> Drive: sa3: Serial Number: 101500520A >>> --------------------------------- >>> Mode Density Blocksize bpi Compression >>> Current: 0x5a:LTO-6 variable 384607 enabled = (0xff) >>> --------------------------------- >>> Current Driver State: at rest. >>> --------------------------------- >>> Partition: 0 Calc File Number: 0 Calc Record Number: 0 >>> Residual: 0 Reported File Number: 0 Reported Record Number: 0 >>> Flags: BOP >>>=20 >>> 5. 'mt status -v' looks like this: >>>=20 >>> # mt -f /dev/nsa3 status -v >>> Drive: sa3: Serial Number: 101500520A >>> --------------------------------- >>> Mode Density Blocksize bpi Compression >>> Current: 0x5a:LTO-6 variable 384607 enabled = (0xff) >>> --------------------------------- >>> Current Driver State: at rest. >>> --------------------------------- >>> Partition: 0 Calc File Number: 0 Calc Record Number: 0 >>> Residual: 0 Reported File Number: 0 Reported Record Number: 0 >>> Flags: BOP >>> --------------------------------- >>> Tape I/O parameters: >>> Maximum I/O size allowed by driver and controller (maxio): 1081344 = bytes >>> Maximum I/O size reported by controller (cpi_maxio): 5197824 bytes >>> Maximum block size supported by tape drive and media (max_blk): = 8388608 bytes >>> Minimum block size supported by tape drive and media (min_blk): 1 = bytes >>> Block granularity supported by tape drive and media (blk_gran): 0 = bytes >>> Maximum possible I/O size (max_effective_iosize): 1081344 bytes >>=20 >>=20 >> # mtx -f /dev/pass0 status >> Storage Changer /dev/pass0:2 Drives, 10 Slots ( 0 Import/Export ) >> Data Transfer Element 0:Empty >> Data Transfer Element 1:Empty >> Storage Element 1:Empty >> Storage Element 2:Empty >> Storage Element 3:Empty >> Storage Element 4:Full :VolumeTag=3DFAI260 = =20 >> Storage Element 5:Full :VolumeTag=3DFAI261 = =20 >> Storage Element 6:Full :VolumeTag=3DFAI262 = =20 >> Storage Element 7:Full :VolumeTag=3DFAI263 = =20 >> Storage Element 8:Empty >> Storage Element 9:Empty >> Storage Element 10:Empty >>=20 >>=20 >> It was at this point I spent the next 90 minute trying to get the = tape=20 >> drive out of the tape library to free a stuck tape. Some of this was = spent >> attempting, and failing, to undo a stripped screw. I stopped the = attempt when >> I noticed the screw did need to be removed. :/ >=20 > Thanks for all of the effort! Looks like it is paying off! :) >=20 >> When I do this command, I hear the drive move a bit, to read the = tape: >>=20 >> # mt -f /dev/nsa1 status >> Drive: sa1: Serial Number: CXA09S1340 >> --------------------------------- >> Mode Density Blocksize bpi Compression >> Current: 0x1b:DLTapeIV(35GB) variable 85937 enabled = (IDRC) >> --------------------------------- >> Current Driver State: at rest. >> --------------------------------- >> Partition: 0 Calc File Number: 0 Calc Record Number: 0 >> Residual: 0 Reported File Number: -1 Reported Record Number: -1 >> Flags: None >=20 > Looks like the drive isn't reporting position information. It will = still > be useful to try it with Bacula, though. >=20 >> # mt -f /dev/nsa1 ostatus =20 >> Mode Density Blocksize bpi Compression >> Current: 0x1b:DLTapeIV(35GB) variable 85937 IDRC >> ---------available modes--------- >> 0: 0x1b:DLTapeIV(35GB) variable 85937 IDRC >> 1: 0x1b:DLTapeIV(35GB) variable 85937 IDRC >> 2: 0x1b:DLTapeIV(35GB) variable 85937 IDRC >> 3: 0x1b:DLTapeIV(35GB) variable 85937 IDRC >> --------------------------------- >> Current Driver State: at rest. >> --------------------------------- >> File Number: 0 Record Number: 0 Residual Count 0 >>=20 >>=20 >> After doing a very small tar -c and tar -x, I have: >>=20 >> # mt -f /dev/nsa1 /dev/nsa1 ostatus >> Mode Density Blocksize bpi Compression >> Current: 0x1b:DLTapeIV(35GB) variable 85937 IDRC >> ---------available modes--------- >> 0: 0x1b:DLTapeIV(35GB) variable 85937 IDRC >> 1: 0x1b:DLTapeIV(35GB) variable 85937 IDRC >> 2: 0x1b:DLTapeIV(35GB) variable 85937 IDRC >> 3: 0x1b:DLTapeIV(35GB) variable 85937 IDRC >> --------------------------------- >> Current Driver State: at rest. >> --------------------------------- >> File Number: 0 Record Number: 7 Residual Count 0 >=20 > Woohoo! It works. >=20 >> # mt -f /dev/nsa1 status -v >> Drive: sa1: Serial Number: CXA09S1340 >> --------------------------------- >> Mode Density Blocksize bpi Compression >> Current: 0x1b:DLTapeIV(35GB) variable 85937 enabled = (IDRC) >> --------------------------------- >> Current Driver State: at rest. >> --------------------------------- >> Partition: 0 Calc File Number: 0 Calc Record Number: 7 >> Residual: 0 Reported File Number: -1 Reported Record Number: -1 >> Flags: None >> --------------------------------- >> Tape I/O parameters: >> Maximum I/O size allowed by driver and controller (maxio): 65536 = bytes >> Maximum I/O size reported by controller (cpi_maxio): 0 bytes >> Maximum block size supported by tape drive and media (max_blk): = 16777214 bytes >> Minimum block size supported by tape drive and media (min_blk): 2 = bytes >> Block granularity supported by tape drive and media (blk_gran): 0 = bytes >> Maximum possible I/O size (max_effective_iosize): 65536 bytes >>=20 >> I may not get to testing Bacula today. =20 >>=20 >> Based on the above, is there any commands you'd like me to try? >=20 > Aside from making sure things work okay with Bacula, that is probably > sufficient. These drives won't support density reports or position > information. >=20 >> Read below regarding two tape drives >>=20 >>>=20 >>> 6. Existing applications should work without changes. If not, = please let >>> me know. Hopefully they will move over time to the new interfaces. >>>=20 >>> 7. There are lots of additional features that could be added later. >>> Append-only support, encryption, more log pages, etc. >>>=20 >>> 8. I have SCSI READ ATTRIBUTE changes for camcontrol(8) that will go = in >>> separately. These changes allow displaying the contents of the MAM >>> (Medium Auxiliary Memory) chips on LTO, TS and other modern tape = drives. >>> These are good, and a future possible direction is adding = attributes=20 >>> to the status XML from the sa(4) driver. >>>=20 >>> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D >>> Significant upgrades to sa(4) and mt(1). >>>=20 >>> The primary focus of these changes is to modernize FreeBSD's >>> tape infrastructure so that we can take advantage of some of the >>> features of modern tape drives and allow support for LTFS. >>>=20 >>> Significant changes and new features include: >>>=20 >>> o sa(4) driver status and parameter information is now exported via = an >>> XML structure. This will allow for changes and improvements later >>> on that will not break userland applications. The old MTIOCGET >>> status ioctl remains, so applications using the existing interface >>> will not break. >>>=20 >>> o 'mt status' now reports drive-reported tape position information >>> as well as the previously available calculated tape position >>> information. These numbers will be different at times, because >>> the drive-reported block numbers are relative to BOP (Beginning >>> of Partition), but the block numbers calculated previously via >>> sa(4) (and still provided) are relative to the last filemark. >>> Both numbers are now provided. 'mt status' now also shows the >>> drive INQUIRY information, serial number and any position flags >>> (BOP, EOT, etc.) provided with the tape position information. >>> 'mt status -v' adds information on the maximum possible I/O size, >>> and the underlying values used to calculate it. >>>=20 >>> o The extra sa(4) /dev entries (/dev/saN.[0-3]) have been removed. >>=20 >> How does this affect a tape library with more than one tape drive? >>=20 >> [root@cuppy:~] # camcontrol amcontrol devlist >> at scbus0 target 0 lun 0 = (pass0,ch0) >> at scbus0 target 2 lun 0 = (sa1,pass2) >> at scbus1 target 0 lun 0 = (pass3,ada0) >> at scbus2 target 0 lun 0 = (pass4,ada1) >> at scbus3 target 0 lun 0 = (pass5,ses0) >>=20 >> This system has two tapes drives and I can access them through the = front panel but: >>=20 >> # ls -l /dev/*sa* >> crw-rw---- 1 root operator 0x65 Feb 28 22:04 /dev/esa1 >> crw-rw---- 1 root operator 0x64 Mar 1 22:43 /dev/nsa1 >> crw-rw---- 1 root operator 0x63 Feb 28 22:04 /dev/sa1 >> crw-rw---- 1 root operator 0x62 Feb 28 22:04 /dev/sa1.ctl >>=20 >> ... only one tape drives shows up. >=20 >=20 > Hmm. The tape drive is listed as sa1, which implies that there may be = an > sa0 that was there previously or is in the process of probing. What = does > dmesg show? How about 'camcontrol devlist -v'? # camcontrol devlist -v scbus0 on ahc0 bus 0: at scbus0 target 0 lun 0 (pass0,ch0) at scbus0 target 2 lun 0 (sa1,pass2) <> at scbus0 target -1 lun ffffffff () scbus1 on ahcich2 bus 0: at scbus1 target 0 lun 0 (pass3,ada0) <> at scbus1 target -1 lun ffffffff () scbus2 on ahcich4 bus 0: at scbus2 target 0 lun 0 (pass4,ada1) <> at scbus2 target -1 lun ffffffff () scbus3 on ahciem0 bus 0: at scbus3 target 0 lun 0 (pass5,ses0) <> at scbus3 target -1 lun ffffffff () scbus-1 on xpt0 bus 0: <> at scbus-1 target -1 lun ffffffff = (xpt0) BUT! # grep sa /var/run/dmesg.boot=20 VT-x: (disabled in BIOS) PAT,HLT,MTF,PAUSE,EPT,UG,VPID module_register_init: MOD_LOAD (vesa, 0xffffffff80de3720, 0) error 19 alc0: Using 1 MSIX message(s). isab0: at device 31.0 on pci0 isa0: on isab0 orm0: at iomem 0xce800-0xcefff on isa0 atkbdc0: at port 0x60,0x64 on isa0 sa0 at ahc0 bus 0 scbus0 target 1 lun 0 sa0: Removable Sequential Access SCSI-2 = device=20 sa0: Serial Number CXA22S2338 sa0: 10.000MB/s transfers (10.000MHz, offset 15) sa0: quirks=3D0x100 sa1 at ahc0 bus 0 scbus0 target 2 lun 0 sa1: Removable Sequential Access SCSI-2 = device=20 sa1: Serial Number CXA09S1340 sa1: 10.000MB/s transfers (10.000MHz, offset 15) sa1: quirks=3D0x100 >=20 > I would look at cabling and termination. Is this your library? Yes, it is. =20 >=20 > = http://manx.classiccmp.org/collections/mds-199909/cd3/tape/tl891uga.pdf >=20 > If it is close enough, there are 6 connectors on the back. You would = want > to have something plugged into all 6, either a cable or a terminator. Yes, that's mine, and yes, there's two short cables, a terminator, and = the cable to the SCSI card in my computer. >=20 > In the manual above, the SCSI IDs are set via the front panel. If the > other drive is on the same bus as the drive above and the library = device, > it should be at a separate SCSI ID. I did have the entire thing torn apart today, to extract a tape which = would not budge. >=20 >>> The extra devices were originally added as place holders for >>> density-specific device nodes. Some OSes (NetBSD, NetApp's OnTap >>> and Solaris) have had device nodes that, when you write to them, >>> will automatically select a given density for particular tape = drives. >>>=20 >>> This is a convenient way of switching densities, but it was never >>> implemented in FreeBSD. Only the device nodes were there, and that >>> sometimes confused users. >>>=20 >>> For modern tape devices, the density is generally not selectable >>> (e.g. with LTO) or defaults to the highest availble density when >>> the tape is rewritten from BOT (e.g. TS11X0). So, for most users, >>> density selection won't be necessary. If they do need to select >>> the density, it is easy enough to use 'mt density' to change it. >>>=20 >>> o Protection information is now supported. This is either a >>> Reed-Solomon CRC or CRC32 that is included at the end of each block >>> read and written. On write, the tape drive verifies the CRC, and >>> on read, the tape drive provides a CRC for the userland application >>> to verify. >>>=20 >>> o New, extensible tape driver parameter get/set interface. >>>=20 >>> o Density reporting information. For drives that support it, >>> 'mt getdensity' will show detailed information on what formats the >>> tape drive supports, and what formats the tape drive supports. >>>=20 >>> o Some mt(1) functionality moved into a new mt(3) library so that >>> external applications can reuse the code. >>>=20 >>> o The new mt(3) library includes helper routines to aid in parsing >>> the XML output of the sa(4) driver, and build a tree of driver >>> metadata. >>>=20 >>> o Support for the MTLOAD (load a tape in the drive) and MTWEOFI >>> (write filemark immediate) ioctls needed by IBM's LTFS >>> implementation. >>>=20 >>> o Improve device departure behavior for the sa(4) driver. The = previous >>> implementation led to hangs when the device was open. >>>=20 >>> o This has been tested on the following types of drives: >>> IBM TS1150 >>> IBM TS1140 >>> IBM LTO-6 >>> IBM LTO-5 >>> HP LTO-2 >>> Seagate DDS-4 >>> Quantum DLT-4000 >>> Exabyte 8505 >>> Sony DDS-2 >>>=20 >>> contrib/groff/tmac/doc-syms, >>> share/mk/bsd.libnames.mk, >>> lib/Makefile, >>> Add libmt. >>>=20 >>> lib/libmt/Makefile, >>> lib/libmt/mt.3, >>> lib/libmt/mtlib.c, >>> lib/libmt/mtlib.h, >>> New mt(3) library that contains functions moved from mt(1) and >>> new functions needed to interact with the updated sa(4) driver. >>>=20 >>> This includes XML parser helper functions that application = writers >>> can use when writing code to query tape parameters. >>>=20 >>> rescue/rescue/Makefile: >>> Add -lmt to CRUNCH_LIBS. >>>=20 >>> sys/cam/cam_ccb.h >>> Add a new flag value for the XPT_DEV_ADVINFO CCB, = CDAI_FLAG_NONE. >>>=20 >>> sbin/camcontrol/camcontrol.c, >>> sys/cam/scsi/scsi_da.c, >>> sys/cam/scsi/scsi_enc_ses.c, >>> sys/dev/mps/mps_sas.c: >>> Make sure the flags for the XPT_DEV_ADVINFO CCB are set = correctly. >>> This prevents unintended attempts to set advanced information >>> values when XPT_DEV_ADVINFO CCBs are not pre-zeroed. >>>=20 >>> src/share/man/man4/mtio.4 >>> Clarify this man page a bit, and since it contains what is >>> essentially the mtio.h header file, add new ioctls and structure >>> definitions from mtio.h. >>>=20 >>> src/share/man/man4/sa.4 >>> Update BUGS and maintainer section. >>>=20 >>> sys/cam/scsi/scsi_all.c, >>> sys/cam/scsi/scsi_all.h: >>> Add SCSI SECURITY PROTOCOL IN/OUT CDB definitions and CDB = building >>> functions. >>>=20 >>> sys/cam/scsi/scsi_sa.c >>> sys/cam/scsi/scsi_sa.h >>> Many tape driver changes, largely outlined above. >>>=20 >>> Increase the sa(4) driver read/write timeout from 4 to 32 >>> minutes. This is based on the recommended values for IBM LTO >>> 5/6 drives. This may also avoid timeouts for other tape >>> hardware that can take a long time to do retries and error >>> recovery. Longer term, a better way to handle this is to ask >>> the drive for recommended timeout values using the REPORT >>> SUPPORTED OPCODES command. Modern IBM and Oracle tape drives >>> at least support that command, and it would allow for more >>> accurate timeout values. >>>=20 >>> Add XML status generation. This is done with a series of >>> macros to eliminate as much duplicate code as possible. The >>> new XML-based status values are reported through the new >>> MTIOCEXTGET ioctl. >>>=20 >>> Add XML driver parameter reporting, using the new MTIOCPARAMGET >>> ioctl. >>>=20 >>> Add a new driver parameter setting interface, using the new >>> MTIOCPARAMSET and MTIOCSETLIST ioctls. >>>=20 >>> Add a new MTIOCRBLIM ioctl to get block limits information. >>>=20 >>> Add CCB/CDB building routines scsi_locate_16, scsi_locate_10, >>> and scsi_read_position_10(). >>>=20 >>> scsi_locate_10 implements the LOCATE command, as does the >>> existing scsi_set_position() command. It just supports >>> additional arguments and features. If/when we figure out a >>> good way to provide backward compatibility for older >>> applications using the old function API, we can just revamp >>> scsi_set_position(). The same goes for >>> scsi_read_position_10() and the existing scsi_read_position() >>> function. >>>=20 >>> Revamp sasetpos() to take the new mtlocate structure as an >>> argument. It now will use either scsi_locate_10() or >>> scsi_locate_16(), depending upon the arguments the user >>> supplies. As before, once we change position we don't have a >>> clear idea of what the current logical position of the tape >>> drive is. >>>=20 >>> For tape drives that support long form position data, we >>> read the current position and store that for later reporting >>> after changing the position. This should help applications >>> like Bacula speed tape access under FreeBSD once they are >>> modified to support the new ioctls. >>>=20 >>> Add a new quirk, SA_QUIRK_NO_LONG_POS, that is set for all >>> drives that report SCSI-2 or older, as well as drives that >>> report an Illegal Request type error for READ POSITION with >>> the long format. So we should automatically detect drives >>> that don't support the long form and stop asking for it after >>> an initial try. >>>=20 >>> Add a partition number to the sa(4) softc. >>>=20 >>> Improve device departure handling. The previous implementation >>> led to hangs when the device was open. >>>=20 >>> If an application had the sa(4) driver open, and attempted to >>> close it after it went away, the cam_periph_release() call in >>> saclose() would cause the periph to get destroyed because that >>> was the last reference to it. Because destroy_dev() was >>> called from the sa(4) driver's cleanup routine (sacleanup()), >>> and would block waiting for the close to happen, a deadlock >>> would result. >>>=20 >>> So instead of calling destroy_dev() from the cleanup routine, >>> call destroy_dev_sched_cb() from saoninvalidate() and wait for >>> the callback. >>>=20 >>> Acquire a reference for devfs in saregister(), and release it >>> in the new sadevgonecb() routine when all devfs devices for=09 >>> the particular sa(4) driver instance are gone. >>>=20 >>> Add a new function, sasetupdev(), to centralize setting >>> per-instance devfs device parameters instead of repeating the >>> code in saregister(). >>>=20 >>> Add an open count to the softc, so we know how many >>> peripheral driver references are a result of open >>> sessions. >>>=20 >>> Add the D_TRACKCLOSE flag to the cdevsw flags so >>> that we get a 1:1 mapping of open to close calls >>> instead of a N:1 mapping. >>>=20 >>> This should be a no-op for everything except the >>> control device, since we don't allow more than one >>> open on non-control devices. >>>=20 >>> However, since we do allow multiple opens on the >>> control device, the combination of the open count >>> and the D_TRACKCLOSE flag should result in an >>> accurate peripheral driver reference count, and an >>> accurate open count. >>>=20 >>> The accurate open count allows us to release all >>> peripheral driver references that are the result >>> of open contexts once we get the callback from devfs. >>>=20 >>> sys/sys/mtio.h: >>> Add a number of new mt(4) ioctls and the requisite data >>> structures. None of the existing interfaces been removed >>> or changed. >>>=20 >>> This includes definitions for the following new ioctls: >>>=20 >>> MTIOCRBLIM /* get block limits */ >>> MTIOCEXTLOCATE /* seek to position */ >>> MTIOCEXTGET /* get tape status */ >>> MTIOCPARAMGET /* get tape params */ >>> MTIOCPARAMSET /* set tape params */ >>> MTIOCSETLIST /* set N params */ >>>=20 >>> usr.bin/mt/Makefile: >>> mt(1) now depends on libmt, libsbuf and libbsdxml. >>>=20 >>> usr.bin/mt/mt.1: >>> Document new mt(1) features and subcommands. >>>=20 >>> usr.bin/mt/mt.c: >>> Implement support for mt(1) subcommands that need to >>> use getopt(3) for their arguments. >>>=20 >>> Implement a new 'mt status' command to replace the old >>> 'mt status' command. The old status command has been >>> renamed 'ostatus'. >>>=20 >>> The new status function uses the MTIOCEXTGET ioctl, and >>> therefore parses the XML data to determine drive status. >>> The -x argument to 'mt status' allows the user to dump out >>> the raw XML reported by the kernel. >>>=20 >>> The new status display is mostly the same as the old status >>> display, except that it doesn't print the redundant density >>> mode information, and it does print the current partition >>> number and position flags. >>>=20 >>> Add a new command, 'mt locate', that will supersede the >>> old 'mt setspos' and 'mt sethpos' commands. 'mt locate' >>> implements all of the functionality of the MTIOCEXTLOCATE >>> ioctl, and allows the user to change the logical position >>> of the tape drive in a number of ways. (Partition, >>> block number, file number, set mark number, end of data.) >>> The immediate bit and the explicit address bits are >>> implemented, but not documented in the man page. >>>=20 >>> Add a new 'mt weofi' command to use the new MTWEOFI ioctl. >>> This allows the user to ask the drive to write a filemark >>> without waiting around for the operation to complete. >>>=20 >>> Add a new 'mt getdensity' command that gets the XML-based >>> tape drive density report from the sa(4) driver and displays >>> it. This uses the SCSI REPORT DENSITY SUPPORT command >>> to get comprehensive information from the tape drive about >>> what formats it is able to read and write. >>>=20 >>> Add a new 'mt protect' command that allows getting and setting >>> tape drive protection information. The protection information >>> is a CRC tacked on to the end of every read/write from and to >>> the tape drive. >>>=20 >>> Sponsored by: Spectra Logic >>> MFC after: 1 month >>>=20 >>> Thanks, >>>=20 >>> Ken >>> --=20 >>> Kenneth Merry >>> ken@FreeBSD.ORG >>> _______________________________________________ >>> freebsd-scsi@freebsd.org mailing list >>> http://lists.freebsd.org/mailman/listinfo/freebsd-scsi >>> To unsubscribe, send any mail to = "freebsd-scsi-unsubscribe@freebsd.org" >>=20 >> ?=20 >> Dan Langille >> http://langille.org/ >>=20 >>=20 >>=20 >>=20 >>=20 >=20 > --=20 > Kenneth Merry > ken@FreeBSD.ORG =E2=80=94=20 Dan Langille http://langille.org/ From owner-freebsd-current@FreeBSD.ORG Mon Mar 2 00:31:53 2015 Return-Path: Delivered-To: current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id CAF2248F; Mon, 2 Mar 2015 00:31:53 +0000 (UTC) Received: from mithlond.kdm.org (mithlond.kdm.org [70.56.43.85]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "A1-33714", Issuer "A1-33714" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 771B7C0C; Mon, 2 Mar 2015 00:31:53 +0000 (UTC) Received: from mithlond.kdm.org (localhost [127.0.0.1]) by mithlond.kdm.org (8.14.9/8.14.9) with ESMTP id t220Vo4q072240 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Sun, 1 Mar 2015 17:31:50 -0700 (MST) (envelope-from ken@mithlond.kdm.org) Received: (from ken@localhost) by mithlond.kdm.org (8.14.9/8.14.9/Submit) id t220VoNw072239; Sun, 1 Mar 2015 17:31:50 -0700 (MST) (envelope-from ken) Date: Sun, 1 Mar 2015 17:31:50 -0700 From: "Kenneth D. Merry" To: Dan Langille Subject: Re: sa(4) driver changes available for test Message-ID: <20150302003150.GB71528@mithlond.kdm.org> References: <20150214003232.GA63990@mithlond.kdm.org> <7CA52DF3-E073-4F50-BE4E-01C51CCDF2C7@langille.org> <20150217183645.GA30947@mithlond.kdm.org> <303C48AC-0DA8-447C-8DE7-0A1DD5DD7351@langille.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <303C48AC-0DA8-447C-8DE7-0A1DD5DD7351@langille.org> User-Agent: Mutt/1.5.23 (2014-03-12) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (mithlond.kdm.org [127.0.0.1]); Sun, 01 Mar 2015 17:31:50 -0700 (MST) X-Spam-Status: No, score=-2.9 required=5.0 tests=ALL_TRUSTED,BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on mithlond.kdm.org Cc: current@freebsd.org, scsi@freebsd.org X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Mar 2015 00:31:53 -0000 On Sun, Mar 01, 2015 at 19:15:05 -0500, Dan Langille wrote: > > > On Feb 17, 2015, at 1:36 PM, Kenneth D. Merry wrote: > > > > On Sat, Feb 14, 2015 at 18:22:43 -0500, Dan Langille wrote: > >> > >>> On Feb 13, 2015, at 7:32 PM, Kenneth D. Merry wrote: > >>> > >>> > >>> I have a fairly large set of changes to the sa(4) driver and mt(1) driver > >>> that I'm planning to commit in the near future. > >>> > >>> A description of the changes is here and below in this message. > >>> > >>> If you have tape hardware and the inclination, I'd appreciate testing and > >>> feedback. > >> > >> I have a DLT 8000 and an SDLT 220. > >> > >> I don't have anything running current, but I have a spare machine which I could use for testing. > >> > >> Do you see any value is tests with that hardware? I'd be testing it via Bacula. > >> > >> disclosure: I'm the sysutils/bacula-* maintainer and a Bacula committer. > >> > > > > Actually, yes. Bacula is a bit tricky to configure, so your trying it out > > would be helpful if you have the time. > > > > In looking at the manuals for both the SDLT 220 and the DLT 8000, they both > > claim to support long position information for the SCSI READ POSITION > > command. > > > > You can see what I'm talking about by doing: > > > > mt eod > > mt status > > > > On my DDS-4 tape drive, this shows: > > > > # mt -f /dev/nsa3 status > > Drive: sa3: Serial Number: HJ00YWY > > --------------------------------- > > Mode Density Blocksize bpi Compression > > Current: 0x26:DDS-4 1024 bytes 97000 enabled (DCLZ) > > --------------------------------- > > Current Driver State: at rest. > > --------------------------------- > > Partition: 0 Calc File Number: -1 Calc Record Number: -1 > > Residual: 0 Reported File Number: -1 Reported Record Number: -1 > > Flags: None > > > > But on an LTO-5, which will give long position information, I get: > > > > [root@doc ~]# mt status > > Drive: sa0: > > --------------------------------- > > Mode Density Blocksize bpi Compression > > Current: 0x58:LTO-5 variable 384607 enabled (0x1) > > --------------------------------- > > Current Driver State: at rest. > > --------------------------------- > > Partition: 0 Calc File Number: 2 Calc Record Number: -1 > > Residual: 0 Reported File Number: 2 Reported Record Number: 32373 > > Flags: None > > > > That, in combination with the changes I made to the position information > > code in the driver, mean that even the old MTIOCGET ioctl should return an > > accurate file number at end of data. e.g., on the LTO-5: > > > > [root@doc ~]# mt ostatus > > Mode Density Blocksize bpi Compression > > Current: 0x58:LTO-5 variable 384607 0x1 > > ---------available modes--------- > > 0: 0x58:LTO-5 variable 384607 0x1 > > 1: 0x58:LTO-5 variable 384607 0x1 > > 2: 0x58:LTO-5 variable 384607 0x1 > > 3: 0x58:LTO-5 variable 384607 0x1 > > --------------------------------- > > Current Driver State: at rest. > > --------------------------------- > > File Number: 2 Record Number: -1 Residual Count -1 > > > > So the thing to try, in addition to just making sure that Bacula continues > > to work properly, is to try setting this for the tape drive in > > bacula-sd.conf: > > > > Hardware End of Medium = yes > > > > It looks like the Bacula tape program (btape) has a test mode, and it would > > be good to run through the tests on one of the tape drives and see whether > > they work, and whether the results are different before and after the > > changes. I'm not sure how to enable the test mode. > > I have this in /usr/local/etc/bacula/bacula-sd.conf > > Device { > Name = DLT > Description = "QUANTUM DLT7000 1624" > Media Type = DLT > Archive Device = /dev/nsa1 > > Autochanger = YES > Drive Index = 0 > > Offline On Unmount = no > Hardware End of Medium = yes > BSF at EOM = yes > Backward Space Record = no > Fast Forward Space File = no > TWO EOF = yes > } > > FYI, http://www.freebsddiary.org/digital-tl891.php (from 2006) has a btape test on this same model. > > Here's the test I ran tonight: > > [root@cuppy:/usr/home/dan] # btape -c /usr/local/etc/bacula/bacula-sd.conf /dev/nsa1 > Tape block granularity is 1024 bytes. > btape: butil.c:287-0 Using device: "/dev/nsa1" for writing. > btape: btape.c:469-0 open device "DLT" (/dev/nsa1): OK > *test > > === Write, rewind, and re-read test === > > I'm going to write 10000 records and an EOF > then write 10000 records and an EOF, then rewind, > and re-read the data to verify that it is correct. > > This is an *essential* feature ... > > btape: btape.c:1152-0 Wrote 10000 blocks of 64412 bytes. > btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) > btape: btape.c:1168-0 Wrote 10000 blocks of 64412 bytes. > btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) > btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) > btape: btape.c:1210-0 Rewind OK. > 10000 blocks re-read correctly. > Got EOF on tape. > 10000 blocks re-read correctly. > === Test Succeeded. End Write, rewind, and re-read test === > > btape: btape.c:1277-0 Block position test > btape: btape.c:1289-0 Rewind OK. > Reposition to file:block 0:4 > Block 5 re-read correctly. > Reposition to file:block 0:200 > Block 201 re-read correctly. > Reposition to file:block 0:9999 > Block 10000 re-read correctly. > Reposition to file:block 1:0 > Block 10001 re-read correctly. > Reposition to file:block 1:600 > Block 10601 re-read correctly. > Reposition to file:block 1:9999 > Block 20000 re-read correctly. > === Test Succeeded. End Write, rewind, and re-read test === > > > > === Append files test === > > This test is essential to Bacula. > > I'm going to write one record in file 0, > two records in file 1, > and three records in file 2 > > btape: btape.c:574-0 Rewound "DLT" (/dev/nsa1) > btape: btape.c:1907-0 Wrote one record of 64412 bytes. > btape: btape.c:1909-0 Wrote block to device. > btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) > btape: btape.c:1907-0 Wrote one record of 64412 bytes. > btape: btape.c:1909-0 Wrote block to device. > btape: btape.c:1907-0 Wrote one record of 64412 bytes. > btape: btape.c:1909-0 Wrote block to device. > btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) > btape: btape.c:1907-0 Wrote one record of 64412 bytes. > btape: btape.c:1909-0 Wrote block to device. > btape: btape.c:1907-0 Wrote one record of 64412 bytes. > btape: btape.c:1909-0 Wrote block to device. > btape: btape.c:1907-0 Wrote one record of 64412 bytes. > btape: btape.c:1909-0 Wrote block to device. > btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) > btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) > btape: btape.c:469-0 open device "DLT" (/dev/nsa1): OK > btape: btape.c:574-0 Rewound "DLT" (/dev/nsa1) > btape: btape.c:1420-0 Now moving to end of medium. This is the critical piece. The test moves the tape to the end of the medium. With hardware position information, you can tell what filemark you're on. Without it, you can't. > btape: btape.c:622-0 tape_dev.c:345 ioctl MTIOCGET error on "DLT" (/dev/nsa1). ERR=No error: 0. > We should be in file 3. I am at file 0. This is NOT correct!!!! > > Append test failed. Attempting again. > Setting "Hardware End of Medium = no > and "Fast Forward Space File = no > and retrying append test. This is not surprsing, given that the drive doesn't support long read position data. (It's a SCSI-2 device.) So that means that Bacula will need to do it manually. > === Append files test === > > This test is essential to Bacula. > > I'm going to write one record in file 0, > two records in file 1, > and three records in file 2 > > btape: btape.c:574-0 Rewound "DLT" (/dev/nsa1) > btape: btape.c:1907-0 Wrote one record of 64412 bytes. > btape: btape.c:1909-0 Wrote block to device. > btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) > btape: btape.c:1907-0 Wrote one record of 64412 bytes. > btape: btape.c:1909-0 Wrote block to device. > btape: btape.c:1907-0 Wrote one record of 64412 bytes. > btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) > btape: btape.c:1907-0 Wrote one record of 64412 bytes. > btape: btape.c:1909-0 Wrote block to device. > btape: btape.c:1907-0 Wrote one record of 64412 bytes. > btape: btape.c:1909-0 Wrote block to device. > btape: btape.c:1907-0 Wrote one record of 64412 bytes. > btape: btape.c:1909-0 Wrote block to device. > btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) > btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) > btape: btape.c:469-0 open device "DLT" (/dev/nsa1): OK > btape: btape.c:574-0 Rewound "DLT" (/dev/nsa1) > btape: btape.c:1420-0 Now moving to end of medium. > btape: btape.c:625-0 Moved to end of medium. > We should be in file 3. I am at file 3. This is correct! > > Now the important part, I am going to attempt to append to the tape. > > btape: btape.c:1907-0 Wrote one record of 64412 bytes. > btape: btape.c:1909-0 Wrote block to device. > btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) > btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) > btape: btape.c:574-0 Rewound "DLT" (/dev/nsa1) > Done appending, there should be no I/O errors > > Doing Bacula scan of blocks: > 1 block of 64448 bytes in file 1 > End of File mark. > 2 blocks of 64448 bytes in file 2 > End of File mark. > 3 blocks of 64448 bytes in file 3 > End of File mark. > 1 block of 64448 bytes in file 4 > End of File mark. > Total files=4, blocks=7, bytes = 451,136 > End scanning the tape. > We should be in file 4. I am at file 4. This is correct! > > > It looks like the test worked this time, please add: > > Hardware End of Medium = No > > Fast Forward Space File = No > to your Device resource in the Storage conf file. > > The above Bacula scan should have output identical to what follows. > Please double check it ... > === Sample correct output === > 1 block of 64448 bytes in file 1 > End of File mark. > 2 blocks of 64448 bytes in file 2 > End of File mark. > 3 blocks of 64448 bytes in file 3 > End of File mark. > 1 block of 64448 bytes in file 4 > End of File mark. > Total files=4, blocks=7, bytes = 451,136 > === End sample correct output === > > If the above scan output is not identical to the > sample output, you MUST correct the problem > or Bacula will not be able to write multiple Jobs to > the tape. > > Skipping read backwards test because BSR turned off. > > > === Forward space files test === > > This test is essential to Bacula. > > I'm going to write five files then test forward spacing > > btape: btape.c:574-0 Rewound "DLT" (/dev/nsa1) > btape: btape.c:1907-0 Wrote one record of 64412 bytes. > btape: btape.c:1909-0 Wrote block to device. > btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) > btape: btape.c:1907-0 Wrote one record of 64412 bytes. > btape: btape.c:1909-0 Wrote block to device. > btape: btape.c:1907-0 Wrote one record of 64412 bytes. > btape: btape.c:1909-0 Wrote block to device. > btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) > btape: btape.c:1907-0 Wrote one record of 64412 bytes. > btape: btape.c:1909-0 Wrote block to device. > btape: btape.c:1907-0 Wrote one record of 64412 bytes. > btape: btape.c:1909-0 Wrote block to device. > btape: btape.c:1907-0 Wrote one record of 64412 bytes. > btape: btape.c:1909-0 Wrote block to device. > btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) > btape: btape.c:1907-0 Wrote one record of 64412 bytes. > btape: btape.c:1909-0 Wrote block to device. > btape: btape.c:1907-0 Wrote one record of 64412 bytes. > btape: btape.c:1909-0 Wrote block to device. > btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) > btape: btape.c:1907-0 Wrote one record of 64412 bytes. > btape: btape.c:1909-0 Wrote block to device. > btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) > btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) > btape: btape.c:574-0 Rewound "DLT" (/dev/nsa1) > btape: btape.c:1634-0 Now forward spacing 1 file. > We should be in file 1. I am at file 1. This is correct! > btape: btape.c:1646-0 Now forward spacing 2 files. > We should be in file 3. I am at file 3. This is correct! > btape: btape.c:574-0 Rewound "DLT" (/dev/nsa1) > btape: btape.c:1659-0 Now forward spacing 4 files. > We should be in file 4. I am at file 4. This is correct! > > btape: btape.c:1677-0 Now forward spacing 1 more file. > We should be in file 5. I am at file 5. This is correct! > > === End Forward space files test === > > > Ah, I see you have an autochanger configured. > To test the autochanger you must have a blank tape > that I can write on in Slot 1. > > Do you wish to continue with the Autochanger test? (y/n): y > > > === Autochanger test === > > 3301 Issuing autochanger "loaded" command. > Nothing loaded in the drive. OK. > 3303 Issuing autochanger "load 1 0" command. > 3303 Autochanger "load 1 0" status is OK. > btape: btape.c:469-0 open device "DLT" (/dev/nsa1): OK > btape: btape.c:1564-0 Rewound "DLT" (/dev/nsa1) > btape: btape.c:1571-0 Wrote EOF to "DLT" (/dev/nsa1) > > The test autochanger worked!! Great, thanks for running the test! Looks like things are working as well as they were before. > > > >> I'll let the other Bacula devs know about this. They deal with the hardware. I work on PostgreSQL. > >> > > > > Thanks! If there are additional features they would like out of the tape > > driver, I'm happy to talk about it. (Or help if they'd like to use the new > > status reporting ioctl, MTIOCEXTGET or any of the other new ioctls.) > > Errors are interesting to me. Especially corrected errors. They are a good indicator of tape quality. > Yes. At least on modern drives, there is a good bit available in the log pages. You can try seeing what logs your drive supports by installing the sg3_utils package and running 'sg_logs sa1 --all'. Given the large amount of data available on some drives, and the difficulty distilling it down to a clear good/bad, I probably won't stick that in the 'mt status' output. But you can certainly take a look at it and have an idea of whether your particular tape/drive are experiencing issues. Ken -- Kenneth Merry ken@FreeBSD.ORG From owner-freebsd-current@FreeBSD.ORG Mon Mar 2 00:37:03 2015 Return-Path: Delivered-To: current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 497E35EA; Mon, 2 Mar 2015 00:37:03 +0000 (UTC) Received: from mithlond.kdm.org (mithlond.kdm.org [70.56.43.85]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "A1-33714", Issuer "A1-33714" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id BD3F4C37; Mon, 2 Mar 2015 00:37:02 +0000 (UTC) Received: from mithlond.kdm.org (localhost [127.0.0.1]) by mithlond.kdm.org (8.14.9/8.14.9) with ESMTP id t220aw4k072311 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Sun, 1 Mar 2015 17:36:58 -0700 (MST) (envelope-from ken@mithlond.kdm.org) Received: (from ken@localhost) by mithlond.kdm.org (8.14.9/8.14.9/Submit) id t220aw8f072310; Sun, 1 Mar 2015 17:36:58 -0700 (MST) (envelope-from ken) Date: Sun, 1 Mar 2015 17:36:58 -0700 From: "Kenneth D. Merry" To: Dan Langille Subject: Re: sa(4) driver changes available for test Message-ID: <20150302003658.GA72258@mithlond.kdm.org> References: <20150214003232.GA63990@mithlond.kdm.org> <4A478C5C-7965-498E-9F0F-80192265E310@langille.org> <20150302001833.GA71528@mithlond.kdm.org> <6C82281F-649A-4DA8-8ACF-17E81C04F730@langille.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <6C82281F-649A-4DA8-8ACF-17E81C04F730@langille.org> User-Agent: Mutt/1.5.23 (2014-03-12) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (mithlond.kdm.org [127.0.0.1]); Sun, 01 Mar 2015 17:36:58 -0700 (MST) X-Spam-Status: No, score=-2.9 required=5.0 tests=ALL_TRUSTED,BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on mithlond.kdm.org Cc: current@freebsd.org, scsi@freebsd.org X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Mar 2015 00:37:03 -0000 On Sun, Mar 01, 2015 at 19:28:37 -0500, Dan Langille wrote: > > > On Mar 1, 2015, at 7:18 PM, Kenneth D. Merry wrote: > > > > On Sun, Mar 01, 2015 at 17:06:24 -0500, Dan Langille wrote: > >> > >>> On Feb 13, 2015, at 7:32 PM, Kenneth D. Merry wrote: > >>> > >>> > >>> I have a fairly large set of changes to the sa(4) driver and mt(1) driver > >>> that I'm planning to commit in the near future. > >>> > >>> A description of the changes is here and below in this message. > >>> > >>> If you have tape hardware and the inclination, I'd appreciate testing and > >>> feedback. > >>> > >>> ============ > >>> Rough draft commit message: > >>> > >>> http://people.freebsd.org/~ken/sa_changes_commitmsg.20150213.3.txt > >>> > >>> The patches against FreeBSD/head as of SVN revision 278706: > >>> > >>> http://people.freebsd.org/~ken/sa_changes.20150213.3.txt > >>> > >>> And (untested) patches against FreeBSD stable/10 as of SVN revision 278721. > >>> > >>> http://people.freebsd.org/~ken/sa_changes.stable_10.20150213.3.txt > >>> ============ > >>> > >>> The intent is to get the tape infrastructure more up to date, so we can > >>> support LTFS and more modern tape drives: > >>> > >>> http://www.ibm.com/systems/storage/tape/ltfs/ > >>> > >>> I have ported IBM's LTFS Single Drive Edition to FreeBSD. The port depends > >>> on the patches linked above. It isn't fully cleaned up and ready for > >>> redistribution. If you're interested, though, let me know and I'll tell > >>> you when it is ready to go out. You need an IBM LTO-5, LTO-6, TS1140 or > >>> TS1150 tape drive. HP drives aren't supported by IBM's LTFS, and older > >>> drives don't have the necessary features to support LTFS. > >>> > >>> The commit message below outlines most of the changes. > >>> > >>> A few comments: > >>> > >>> 1. I'm planning to commit the XPT_DEV_ADVINFO changes separately. > >>> > >>> 2. The XML output is similar to what GEOM and CTL do. It would be nice to > >>> figure out how to put a standard schema on it so that standard tools > >>> could read it. I don't know how feasible that is, since I haven't > >>> time to dig into it. If anyone has suggestions on whether that is > >>> feasible or advisable, I'd appreciate feedback. > >>> > >>> 3. I have tested with a reasonable amount of tape hardware (see below for a > >>> list), but more testing and feedback would be good. > >>> > >>> 4. Standard 'mt status' output looks like this: > >>> > >>> # mt -f /dev/nsa3 status -v > >>> Drive: sa3: Serial Number: 101500520A > >>> --------------------------------- > >>> Mode Density Blocksize bpi Compression > >>> Current: 0x5a:LTO-6 variable 384607 enabled (0xff) > >>> --------------------------------- > >>> Current Driver State: at rest. > >>> --------------------------------- > >>> Partition: 0 Calc File Number: 0 Calc Record Number: 0 > >>> Residual: 0 Reported File Number: 0 Reported Record Number: 0 > >>> Flags: BOP > >>> > >>> 5. 'mt status -v' looks like this: > >>> > >>> # mt -f /dev/nsa3 status -v > >>> Drive: sa3: Serial Number: 101500520A > >>> --------------------------------- > >>> Mode Density Blocksize bpi Compression > >>> Current: 0x5a:LTO-6 variable 384607 enabled (0xff) > >>> --------------------------------- > >>> Current Driver State: at rest. > >>> --------------------------------- > >>> Partition: 0 Calc File Number: 0 Calc Record Number: 0 > >>> Residual: 0 Reported File Number: 0 Reported Record Number: 0 > >>> Flags: BOP > >>> --------------------------------- > >>> Tape I/O parameters: > >>> Maximum I/O size allowed by driver and controller (maxio): 1081344 bytes > >>> Maximum I/O size reported by controller (cpi_maxio): 5197824 bytes > >>> Maximum block size supported by tape drive and media (max_blk): 8388608 bytes > >>> Minimum block size supported by tape drive and media (min_blk): 1 bytes > >>> Block granularity supported by tape drive and media (blk_gran): 0 bytes > >>> Maximum possible I/O size (max_effective_iosize): 1081344 bytes > >> > >> > >> # mtx -f /dev/pass0 status > >> Storage Changer /dev/pass0:2 Drives, 10 Slots ( 0 Import/Export ) > >> Data Transfer Element 0:Empty > >> Data Transfer Element 1:Empty > >> Storage Element 1:Empty > >> Storage Element 2:Empty > >> Storage Element 3:Empty > >> Storage Element 4:Full :VolumeTag=FAI260 > >> Storage Element 5:Full :VolumeTag=FAI261 > >> Storage Element 6:Full :VolumeTag=FAI262 > >> Storage Element 7:Full :VolumeTag=FAI263 > >> Storage Element 8:Empty > >> Storage Element 9:Empty > >> Storage Element 10:Empty > >> > >> > >> It was at this point I spent the next 90 minute trying to get the tape > >> drive out of the tape library to free a stuck tape. Some of this was spent > >> attempting, and failing, to undo a stripped screw. I stopped the attempt when > >> I noticed the screw did need to be removed. :/ > > > > Thanks for all of the effort! Looks like it is paying off! :) > > > >> When I do this command, I hear the drive move a bit, to read the tape: > >> > >> # mt -f /dev/nsa1 status > >> Drive: sa1: Serial Number: CXA09S1340 > >> --------------------------------- > >> Mode Density Blocksize bpi Compression > >> Current: 0x1b:DLTapeIV(35GB) variable 85937 enabled (IDRC) > >> --------------------------------- > >> Current Driver State: at rest. > >> --------------------------------- > >> Partition: 0 Calc File Number: 0 Calc Record Number: 0 > >> Residual: 0 Reported File Number: -1 Reported Record Number: -1 > >> Flags: None > > > > Looks like the drive isn't reporting position information. It will still > > be useful to try it with Bacula, though. > > > >> # mt -f /dev/nsa1 ostatus > >> Mode Density Blocksize bpi Compression > >> Current: 0x1b:DLTapeIV(35GB) variable 85937 IDRC > >> ---------available modes--------- > >> 0: 0x1b:DLTapeIV(35GB) variable 85937 IDRC > >> 1: 0x1b:DLTapeIV(35GB) variable 85937 IDRC > >> 2: 0x1b:DLTapeIV(35GB) variable 85937 IDRC > >> 3: 0x1b:DLTapeIV(35GB) variable 85937 IDRC > >> --------------------------------- > >> Current Driver State: at rest. > >> --------------------------------- > >> File Number: 0 Record Number: 0 Residual Count 0 > >> > >> > >> After doing a very small tar -c and tar -x, I have: > >> > >> # mt -f /dev/nsa1 /dev/nsa1 ostatus > >> Mode Density Blocksize bpi Compression > >> Current: 0x1b:DLTapeIV(35GB) variable 85937 IDRC > >> ---------available modes--------- > >> 0: 0x1b:DLTapeIV(35GB) variable 85937 IDRC > >> 1: 0x1b:DLTapeIV(35GB) variable 85937 IDRC > >> 2: 0x1b:DLTapeIV(35GB) variable 85937 IDRC > >> 3: 0x1b:DLTapeIV(35GB) variable 85937 IDRC > >> --------------------------------- > >> Current Driver State: at rest. > >> --------------------------------- > >> File Number: 0 Record Number: 7 Residual Count 0 > > > > Woohoo! It works. > > > >> # mt -f /dev/nsa1 status -v > >> Drive: sa1: Serial Number: CXA09S1340 > >> --------------------------------- > >> Mode Density Blocksize bpi Compression > >> Current: 0x1b:DLTapeIV(35GB) variable 85937 enabled (IDRC) > >> --------------------------------- > >> Current Driver State: at rest. > >> --------------------------------- > >> Partition: 0 Calc File Number: 0 Calc Record Number: 7 > >> Residual: 0 Reported File Number: -1 Reported Record Number: -1 > >> Flags: None > >> --------------------------------- > >> Tape I/O parameters: > >> Maximum I/O size allowed by driver and controller (maxio): 65536 bytes > >> Maximum I/O size reported by controller (cpi_maxio): 0 bytes > >> Maximum block size supported by tape drive and media (max_blk): 16777214 bytes > >> Minimum block size supported by tape drive and media (min_blk): 2 bytes > >> Block granularity supported by tape drive and media (blk_gran): 0 bytes > >> Maximum possible I/O size (max_effective_iosize): 65536 bytes > >> > >> I may not get to testing Bacula today. > >> > >> Based on the above, is there any commands you'd like me to try? > > > > Aside from making sure things work okay with Bacula, that is probably > > sufficient. These drives won't support density reports or position > > information. > > > >> Read below regarding two tape drives > >> > >>> > >>> 6. Existing applications should work without changes. If not, please let > >>> me know. Hopefully they will move over time to the new interfaces. > >>> > >>> 7. There are lots of additional features that could be added later. > >>> Append-only support, encryption, more log pages, etc. > >>> > >>> 8. I have SCSI READ ATTRIBUTE changes for camcontrol(8) that will go in > >>> separately. These changes allow displaying the contents of the MAM > >>> (Medium Auxiliary Memory) chips on LTO, TS and other modern tape drives. > >>> These are good, and a future possible direction is adding attributes > >>> to the status XML from the sa(4) driver. > >>> > >>> ============ > >>> Significant upgrades to sa(4) and mt(1). > >>> > >>> The primary focus of these changes is to modernize FreeBSD's > >>> tape infrastructure so that we can take advantage of some of the > >>> features of modern tape drives and allow support for LTFS. > >>> > >>> Significant changes and new features include: > >>> > >>> o sa(4) driver status and parameter information is now exported via an > >>> XML structure. This will allow for changes and improvements later > >>> on that will not break userland applications. The old MTIOCGET > >>> status ioctl remains, so applications using the existing interface > >>> will not break. > >>> > >>> o 'mt status' now reports drive-reported tape position information > >>> as well as the previously available calculated tape position > >>> information. These numbers will be different at times, because > >>> the drive-reported block numbers are relative to BOP (Beginning > >>> of Partition), but the block numbers calculated previously via > >>> sa(4) (and still provided) are relative to the last filemark. > >>> Both numbers are now provided. 'mt status' now also shows the > >>> drive INQUIRY information, serial number and any position flags > >>> (BOP, EOT, etc.) provided with the tape position information. > >>> 'mt status -v' adds information on the maximum possible I/O size, > >>> and the underlying values used to calculate it. > >>> > >>> o The extra sa(4) /dev entries (/dev/saN.[0-3]) have been removed. > >> > >> How does this affect a tape library with more than one tape drive? > >> > >> [root@cuppy:~] # camcontrol amcontrol devlist > >> at scbus0 target 0 lun 0 (pass0,ch0) > >> at scbus0 target 2 lun 0 (sa1,pass2) > >> at scbus1 target 0 lun 0 (pass3,ada0) > >> at scbus2 target 0 lun 0 (pass4,ada1) > >> at scbus3 target 0 lun 0 (pass5,ses0) > >> > >> This system has two tapes drives and I can access them through the front panel but: > >> > >> # ls -l /dev/*sa* > >> crw-rw---- 1 root operator 0x65 Feb 28 22:04 /dev/esa1 > >> crw-rw---- 1 root operator 0x64 Mar 1 22:43 /dev/nsa1 > >> crw-rw---- 1 root operator 0x63 Feb 28 22:04 /dev/sa1 > >> crw-rw---- 1 root operator 0x62 Feb 28 22:04 /dev/sa1.ctl > >> > >> ... only one tape drives shows up. > > > > > > Hmm. The tape drive is listed as sa1, which implies that there may be an > > sa0 that was there previously or is in the process of probing. What does > > dmesg show? How about 'camcontrol devlist -v'? > > # camcontrol devlist -v > scbus0 on ahc0 bus 0: > at scbus0 target 0 lun 0 (pass0,ch0) > at scbus0 target 2 lun 0 (sa1,pass2) > <> at scbus0 target -1 lun ffffffff () > scbus1 on ahcich2 bus 0: > at scbus1 target 0 lun 0 (pass3,ada0) > <> at scbus1 target -1 lun ffffffff () > scbus2 on ahcich4 bus 0: > at scbus2 target 0 lun 0 (pass4,ada1) > <> at scbus2 target -1 lun ffffffff () > scbus3 on ahciem0 bus 0: > at scbus3 target 0 lun 0 (pass5,ses0) > <> at scbus3 target -1 lun ffffffff () > scbus-1 on xpt0 bus 0: > <> at scbus-1 target -1 lun ffffffff (xpt0) > > > BUT! > > # grep sa /var/run/dmesg.boot > VT-x: (disabled in BIOS) PAT,HLT,MTF,PAUSE,EPT,UG,VPID > module_register_init: MOD_LOAD (vesa, 0xffffffff80de3720, 0) error 19 > alc0: Using 1 MSIX message(s). > isab0: at device 31.0 on pci0 > isa0: on isab0 > orm0: at iomem 0xce800-0xcefff on isa0 > atkbdc0: at port 0x60,0x64 on isa0 > sa0 at ahc0 bus 0 scbus0 target 1 lun 0 > sa0: Removable Sequential Access SCSI-2 device > sa0: Serial Number CXA22S2338 > sa0: 10.000MB/s transfers (10.000MHz, offset 15) > sa0: quirks=0x100 > sa1 at ahc0 bus 0 scbus0 target 2 lun 0 > sa1: Removable Sequential Access SCSI-2 device > sa1: Serial Number CXA09S1340 > sa1: 10.000MB/s transfers (10.000MHz, offset 15) > sa1: quirks=0x100 If you run 'dmesg', you should have seen a message when it went away. Perhaps there will be something preceding it that will give us a clue about the problem. (Generally a selection timeout.) At least this does show that sa0 is at target 1, and so should not conflict with the library or sa1. > > > > I would look at cabling and termination. Is this your library? > > Yes, it is. > > > > > http://manx.classiccmp.org/collections/mds-199909/cd3/tape/tl891uga.pdf > > > > If it is close enough, there are 6 connectors on the back. You would want > > to have something plugged into all 6, either a cable or a terminator. > > Yes, that's mine, and yes, there's two short cables, a terminator, and the cable to the SCSI card in my computer. That sounds correct for a configuration with everything on one bus. > > > > In the manual above, the SCSI IDs are set via the front panel. If the > > other drive is on the same bus as the drive above and the library device, > > it should be at a separate SCSI ID. > > I did have the entire thing torn apart today, to extract a tape which would not budge. Ahh. The SCSI IDs are right, so that doesn't appear to be the issue. > > > >>> The extra devices were originally added as place holders for > >>> density-specific device nodes. Some OSes (NetBSD, NetApp's OnTap > >>> and Solaris) have had device nodes that, when you write to them, > >>> will automatically select a given density for particular tape drives. > >>> > >>> This is a convenient way of switching densities, but it was never > >>> implemented in FreeBSD. Only the device nodes were there, and that > >>> sometimes confused users. > >>> > >>> For modern tape devices, the density is generally not selectable > >>> (e.g. with LTO) or defaults to the highest availble density when > >>> the tape is rewritten from BOT (e.g. TS11X0). So, for most users, > >>> density selection won't be necessary. If they do need to select > >>> the density, it is easy enough to use 'mt density' to change it. > >>> > >>> o Protection information is now supported. This is either a > >>> Reed-Solomon CRC or CRC32 that is included at the end of each block > >>> read and written. On write, the tape drive verifies the CRC, and > >>> on read, the tape drive provides a CRC for the userland application > >>> to verify. > >>> > >>> o New, extensible tape driver parameter get/set interface. > >>> > >>> o Density reporting information. For drives that support it, > >>> 'mt getdensity' will show detailed information on what formats the > >>> tape drive supports, and what formats the tape drive supports. > >>> > >>> o Some mt(1) functionality moved into a new mt(3) library so that > >>> external applications can reuse the code. > >>> > >>> o The new mt(3) library includes helper routines to aid in parsing > >>> the XML output of the sa(4) driver, and build a tree of driver > >>> metadata. > >>> > >>> o Support for the MTLOAD (load a tape in the drive) and MTWEOFI > >>> (write filemark immediate) ioctls needed by IBM's LTFS > >>> implementation. > >>> > >>> o Improve device departure behavior for the sa(4) driver. The previous > >>> implementation led to hangs when the device was open. > >>> > >>> o This has been tested on the following types of drives: > >>> IBM TS1150 > >>> IBM TS1140 > >>> IBM LTO-6 > >>> IBM LTO-5 > >>> HP LTO-2 > >>> Seagate DDS-4 > >>> Quantum DLT-4000 > >>> Exabyte 8505 > >>> Sony DDS-2 > >>> > >>> contrib/groff/tmac/doc-syms, > >>> share/mk/bsd.libnames.mk, > >>> lib/Makefile, > >>> Add libmt. > >>> > >>> lib/libmt/Makefile, > >>> lib/libmt/mt.3, > >>> lib/libmt/mtlib.c, > >>> lib/libmt/mtlib.h, > >>> New mt(3) library that contains functions moved from mt(1) and > >>> new functions needed to interact with the updated sa(4) driver. > >>> > >>> This includes XML parser helper functions that application writers > >>> can use when writing code to query tape parameters. > >>> > >>> rescue/rescue/Makefile: > >>> Add -lmt to CRUNCH_LIBS. > >>> > >>> sys/cam/cam_ccb.h > >>> Add a new flag value for the XPT_DEV_ADVINFO CCB, CDAI_FLAG_NONE. > >>> > >>> sbin/camcontrol/camcontrol.c, > >>> sys/cam/scsi/scsi_da.c, > >>> sys/cam/scsi/scsi_enc_ses.c, > >>> sys/dev/mps/mps_sas.c: > >>> Make sure the flags for the XPT_DEV_ADVINFO CCB are set correctly. > >>> This prevents unintended attempts to set advanced information > >>> values when XPT_DEV_ADVINFO CCBs are not pre-zeroed. > >>> > >>> src/share/man/man4/mtio.4 > >>> Clarify this man page a bit, and since it contains what is > >>> essentially the mtio.h header file, add new ioctls and structure > >>> definitions from mtio.h. > >>> > >>> src/share/man/man4/sa.4 > >>> Update BUGS and maintainer section. > >>> > >>> sys/cam/scsi/scsi_all.c, > >>> sys/cam/scsi/scsi_all.h: > >>> Add SCSI SECURITY PROTOCOL IN/OUT CDB definitions and CDB building > >>> functions. > >>> > >>> sys/cam/scsi/scsi_sa.c > >>> sys/cam/scsi/scsi_sa.h > >>> Many tape driver changes, largely outlined above. > >>> > >>> Increase the sa(4) driver read/write timeout from 4 to 32 > >>> minutes. This is based on the recommended values for IBM LTO > >>> 5/6 drives. This may also avoid timeouts for other tape > >>> hardware that can take a long time to do retries and error > >>> recovery. Longer term, a better way to handle this is to ask > >>> the drive for recommended timeout values using the REPORT > >>> SUPPORTED OPCODES command. Modern IBM and Oracle tape drives > >>> at least support that command, and it would allow for more > >>> accurate timeout values. > >>> > >>> Add XML status generation. This is done with a series of > >>> macros to eliminate as much duplicate code as possible. The > >>> new XML-based status values are reported through the new > >>> MTIOCEXTGET ioctl. > >>> > >>> Add XML driver parameter reporting, using the new MTIOCPARAMGET > >>> ioctl. > >>> > >>> Add a new driver parameter setting interface, using the new > >>> MTIOCPARAMSET and MTIOCSETLIST ioctls. > >>> > >>> Add a new MTIOCRBLIM ioctl to get block limits information. > >>> > >>> Add CCB/CDB building routines scsi_locate_16, scsi_locate_10, > >>> and scsi_read_position_10(). > >>> > >>> scsi_locate_10 implements the LOCATE command, as does the > >>> existing scsi_set_position() command. It just supports > >>> additional arguments and features. If/when we figure out a > >>> good way to provide backward compatibility for older > >>> applications using the old function API, we can just revamp > >>> scsi_set_position(). The same goes for > >>> scsi_read_position_10() and the existing scsi_read_position() > >>> function. > >>> > >>> Revamp sasetpos() to take the new mtlocate structure as an > >>> argument. It now will use either scsi_locate_10() or > >>> scsi_locate_16(), depending upon the arguments the user > >>> supplies. As before, once we change position we don't have a > >>> clear idea of what the current logical position of the tape > >>> drive is. > >>> > >>> For tape drives that support long form position data, we > >>> read the current position and store that for later reporting > >>> after changing the position. This should help applications > >>> like Bacula speed tape access under FreeBSD once they are > >>> modified to support the new ioctls. > >>> > >>> Add a new quirk, SA_QUIRK_NO_LONG_POS, that is set for all > >>> drives that report SCSI-2 or older, as well as drives that > >>> report an Illegal Request type error for READ POSITION with > >>> the long format. So we should automatically detect drives > >>> that don't support the long form and stop asking for it after > >>> an initial try. > >>> > >>> Add a partition number to the sa(4) softc. > >>> > >>> Improve device departure handling. The previous implementation > >>> led to hangs when the device was open. > >>> > >>> If an application had the sa(4) driver open, and attempted to > >>> close it after it went away, the cam_periph_release() call in > >>> saclose() would cause the periph to get destroyed because that > >>> was the last reference to it. Because destroy_dev() was > >>> called from the sa(4) driver's cleanup routine (sacleanup()), > >>> and would block waiting for the close to happen, a deadlock > >>> would result. > >>> > >>> So instead of calling destroy_dev() from the cleanup routine, > >>> call destroy_dev_sched_cb() from saoninvalidate() and wait for > >>> the callback. > >>> > >>> Acquire a reference for devfs in saregister(), and release it > >>> in the new sadevgonecb() routine when all devfs devices for > >>> the particular sa(4) driver instance are gone. > >>> > >>> Add a new function, sasetupdev(), to centralize setting > >>> per-instance devfs device parameters instead of repeating the > >>> code in saregister(). > >>> > >>> Add an open count to the softc, so we know how many > >>> peripheral driver references are a result of open > >>> sessions. > >>> > >>> Add the D_TRACKCLOSE flag to the cdevsw flags so > >>> that we get a 1:1 mapping of open to close calls > >>> instead of a N:1 mapping. > >>> > >>> This should be a no-op for everything except the > >>> control device, since we don't allow more than one > >>> open on non-control devices. > >>> > >>> However, since we do allow multiple opens on the > >>> control device, the combination of the open count > >>> and the D_TRACKCLOSE flag should result in an > >>> accurate peripheral driver reference count, and an > >>> accurate open count. > >>> > >>> The accurate open count allows us to release all > >>> peripheral driver references that are the result > >>> of open contexts once we get the callback from devfs. > >>> > >>> sys/sys/mtio.h: > >>> Add a number of new mt(4) ioctls and the requisite data > >>> structures. None of the existing interfaces been removed > >>> or changed. > >>> > >>> This includes definitions for the following new ioctls: > >>> > >>> MTIOCRBLIM /* get block limits */ > >>> MTIOCEXTLOCATE /* seek to position */ > >>> MTIOCEXTGET /* get tape status */ > >>> MTIOCPARAMGET /* get tape params */ > >>> MTIOCPARAMSET /* set tape params */ > >>> MTIOCSETLIST /* set N params */ > >>> > >>> usr.bin/mt/Makefile: > >>> mt(1) now depends on libmt, libsbuf and libbsdxml. > >>> > >>> usr.bin/mt/mt.1: > >>> Document new mt(1) features and subcommands. > >>> > >>> usr.bin/mt/mt.c: > >>> Implement support for mt(1) subcommands that need to > >>> use getopt(3) for their arguments. > >>> > >>> Implement a new 'mt status' command to replace the old > >>> 'mt status' command. The old status command has been > >>> renamed 'ostatus'. > >>> > >>> The new status function uses the MTIOCEXTGET ioctl, and > >>> therefore parses the XML data to determine drive status. > >>> The -x argument to 'mt status' allows the user to dump out > >>> the raw XML reported by the kernel. > >>> > >>> The new status display is mostly the same as the old status > >>> display, except that it doesn't print the redundant density > >>> mode information, and it does print the current partition > >>> number and position flags. > >>> > >>> Add a new command, 'mt locate', that will supersede the > >>> old 'mt setspos' and 'mt sethpos' commands. 'mt locate' > >>> implements all of the functionality of the MTIOCEXTLOCATE > >>> ioctl, and allows the user to change the logical position > >>> of the tape drive in a number of ways. (Partition, > >>> block number, file number, set mark number, end of data.) > >>> The immediate bit and the explicit address bits are > >>> implemented, but not documented in the man page. > >>> > >>> Add a new 'mt weofi' command to use the new MTWEOFI ioctl. > >>> This allows the user to ask the drive to write a filemark > >>> without waiting around for the operation to complete. > >>> > >>> Add a new 'mt getdensity' command that gets the XML-based > >>> tape drive density report from the sa(4) driver and displays > >>> it. This uses the SCSI REPORT DENSITY SUPPORT command > >>> to get comprehensive information from the tape drive about > >>> what formats it is able to read and write. > >>> > >>> Add a new 'mt protect' command that allows getting and setting > >>> tape drive protection information. The protection information > >>> is a CRC tacked on to the end of every read/write from and to > >>> the tape drive. > >>> > >>> Sponsored by: Spectra Logic > >>> MFC after: 1 month > >>> > >>> Thanks, > >>> > >>> Ken > >>> -- > >>> Kenneth Merry > >>> ken@FreeBSD.ORG > >>> _______________________________________________ > >>> freebsd-scsi@freebsd.org mailing list > >>> http://lists.freebsd.org/mailman/listinfo/freebsd-scsi > >>> To unsubscribe, send any mail to "freebsd-scsi-unsubscribe@freebsd.org" > >> > >> ? > >> Dan Langille > >> http://langille.org/ > >> > >> > >> > >> > >> > > > > -- > > Kenneth Merry > > ken@FreeBSD.ORG > > ? > Dan Langille > http://langille.org/ > > > > > Ken -- Kenneth Merry ken@FreeBSD.ORG From owner-freebsd-current@FreeBSD.ORG Mon Mar 2 00:40:43 2015 Return-Path: Delivered-To: current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 45959E1F; Mon, 2 Mar 2015 00:40:43 +0000 (UTC) Received: from clavin1.langille.org (clavin.langille.org [162.208.116.86]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "clavin.langille.org", Issuer "StartCom Class 2 Primary Intermediate Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 19AFFD04; Mon, 2 Mar 2015 00:40:42 +0000 (UTC) Received: from (clavin1.int.langille.org (clavin1.int.unixathome.org [10.4.7.7]) (Authenticated sender: hidden) with ESMTPSA id BA8C0F6E ; Mon, 2 Mar 2015 00:40:41 +0000 (UTC) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 8.2 \(2070.6\)) Subject: Re: sa(4) driver changes available for test From: Dan Langille In-Reply-To: <20150302003658.GA72258@mithlond.kdm.org> Date: Sun, 1 Mar 2015 19:40:40 -0500 Content-Transfer-Encoding: quoted-printable Message-Id: References: <20150214003232.GA63990@mithlond.kdm.org> <4A478C5C-7965-498E-9F0F-80192265E310@langille.org> <20150302001833.GA71528@mithlond.kdm.org> <6C82281F-649A-4DA8-8ACF-17E81C04F730@langille.org> <20150302003658.GA72258@mithlond.kdm.org> To: "Kenneth D. Merry" X-Mailer: Apple Mail (2.2070.6) Cc: current@freebsd.org, scsi@freebsd.org X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Mar 2015 00:40:43 -0000 > On Mar 1, 2015, at 7:36 PM, Kenneth D. Merry wrote: >=20 > On Sun, Mar 01, 2015 at 19:28:37 -0500, Dan Langille wrote: >>=20 >>> On Mar 1, 2015, at 7:18 PM, Kenneth D. Merry = wrote: >>>=20 >>> On Sun, Mar 01, 2015 at 17:06:24 -0500, Dan Langille wrote: >>>>=20 >>>>> On Feb 13, 2015, at 7:32 PM, Kenneth D. Merry = wrote: >>>>>=20 >>>>>=20 >>>>> I have a fairly large set of changes to the sa(4) driver and mt(1) = driver >>>>> that I'm planning to commit in the near future. >>>>>=20 >>>>> A description of the changes is here and below in this message. >>>>>=20 >>>>> If you have tape hardware and the inclination, I'd appreciate = testing and >>>>> feedback. >>>>>=20 >>>>> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D >>>>> Rough draft commit message: >>>>>=20 >>>>> http://people.freebsd.org/~ken/sa_changes_commitmsg.20150213.3.txt >>>>>=20 >>>>> The patches against FreeBSD/head as of SVN revision 278706: >>>>>=20 >>>>> http://people.freebsd.org/~ken/sa_changes.20150213.3.txt >>>>>=20 >>>>> And (untested) patches against FreeBSD stable/10 as of SVN = revision 278721. >>>>>=20 >>>>> http://people.freebsd.org/~ken/sa_changes.stable_10.20150213.3.txt >>>>> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D >>>>>=20 >>>>> The intent is to get the tape infrastructure more up to date, so = we can >>>>> support LTFS and more modern tape drives: >>>>>=20 >>>>> http://www.ibm.com/systems/storage/tape/ltfs/ >>>>>=20 >>>>> I have ported IBM's LTFS Single Drive Edition to FreeBSD. The = port depends >>>>> on the patches linked above. It isn't fully cleaned up and ready = for >>>>> redistribution. If you're interested, though, let me know and = I'll tell >>>>> you when it is ready to go out. You need an IBM LTO-5, LTO-6, = TS1140 or >>>>> TS1150 tape drive. HP drives aren't supported by IBM's LTFS, and = older >>>>> drives don't have the necessary features to support LTFS. >>>>>=20 >>>>> The commit message below outlines most of the changes. >>>>>=20 >>>>> A few comments: >>>>>=20 >>>>> 1. I'm planning to commit the XPT_DEV_ADVINFO changes separately. >>>>>=20 >>>>> 2. The XML output is similar to what GEOM and CTL do. It would be = nice to >>>>> figure out how to put a standard schema on it so that standard = tools >>>>> could read it. I don't know how feasible that is, since I haven't >>>>> time to dig into it. If anyone has suggestions on whether that is >>>>> feasible or advisable, I'd appreciate feedback. >>>>>=20 >>>>> 3. I have tested with a reasonable amount of tape hardware (see = below for a >>>>> list), but more testing and feedback would be good. >>>>>=20 >>>>> 4. Standard 'mt status' output looks like this: >>>>>=20 >>>>> # mt -f /dev/nsa3 status -v >>>>> Drive: sa3: Serial Number: 101500520A >>>>> --------------------------------- >>>>> Mode Density Blocksize bpi Compression >>>>> Current: 0x5a:LTO-6 variable 384607 enabled = (0xff) >>>>> --------------------------------- >>>>> Current Driver State: at rest. >>>>> --------------------------------- >>>>> Partition: 0 Calc File Number: 0 Calc Record Number: = 0 >>>>> Residual: 0 Reported File Number: 0 Reported Record Number: = 0 >>>>> Flags: BOP >>>>>=20 >>>>> 5. 'mt status -v' looks like this: >>>>>=20 >>>>> # mt -f /dev/nsa3 status -v >>>>> Drive: sa3: Serial Number: 101500520A >>>>> --------------------------------- >>>>> Mode Density Blocksize bpi Compression >>>>> Current: 0x5a:LTO-6 variable 384607 enabled = (0xff) >>>>> --------------------------------- >>>>> Current Driver State: at rest. >>>>> --------------------------------- >>>>> Partition: 0 Calc File Number: 0 Calc Record Number: = 0 >>>>> Residual: 0 Reported File Number: 0 Reported Record Number: = 0 >>>>> Flags: BOP >>>>> --------------------------------- >>>>> Tape I/O parameters: >>>>> Maximum I/O size allowed by driver and controller (maxio): 1081344 = bytes >>>>> Maximum I/O size reported by controller (cpi_maxio): 5197824 bytes >>>>> Maximum block size supported by tape drive and media (max_blk): = 8388608 bytes >>>>> Minimum block size supported by tape drive and media (min_blk): 1 = bytes >>>>> Block granularity supported by tape drive and media (blk_gran): 0 = bytes >>>>> Maximum possible I/O size (max_effective_iosize): 1081344 bytes >>>>=20 >>>>=20 >>>> # mtx -f /dev/pass0 status >>>> Storage Changer /dev/pass0:2 Drives, 10 Slots ( 0 Import/Export ) >>>> Data Transfer Element 0:Empty >>>> Data Transfer Element 1:Empty >>>> Storage Element 1:Empty >>>> Storage Element 2:Empty >>>> Storage Element 3:Empty >>>> Storage Element 4:Full :VolumeTag=3DFAI260 = =20 >>>> Storage Element 5:Full :VolumeTag=3DFAI261 = =20 >>>> Storage Element 6:Full :VolumeTag=3DFAI262 = =20 >>>> Storage Element 7:Full :VolumeTag=3DFAI263 = =20 >>>> Storage Element 8:Empty >>>> Storage Element 9:Empty >>>> Storage Element 10:Empty >>>>=20 >>>>=20 >>>> It was at this point I spent the next 90 minute trying to get the = tape=20 >>>> drive out of the tape library to free a stuck tape. Some of this = was spent >>>> attempting, and failing, to undo a stripped screw. I stopped the = attempt when >>>> I noticed the screw did need to be removed. :/ >>>=20 >>> Thanks for all of the effort! Looks like it is paying off! :) >>>=20 >>>> When I do this command, I hear the drive move a bit, to read the = tape: >>>>=20 >>>> # mt -f /dev/nsa1 status >>>> Drive: sa1: Serial Number: CXA09S1340 >>>> --------------------------------- >>>> Mode Density Blocksize bpi = Compression >>>> Current: 0x1b:DLTapeIV(35GB) variable 85937 enabled = (IDRC) >>>> --------------------------------- >>>> Current Driver State: at rest. >>>> --------------------------------- >>>> Partition: 0 Calc File Number: 0 Calc Record Number: 0 >>>> Residual: 0 Reported File Number: -1 Reported Record Number: = -1 >>>> Flags: None >>>=20 >>> Looks like the drive isn't reporting position information. It will = still >>> be useful to try it with Bacula, though. >>>=20 >>>> # mt -f /dev/nsa1 ostatus =20 >>>> Mode Density Blocksize bpi Compression >>>> Current: 0x1b:DLTapeIV(35GB) variable 85937 IDRC >>>> ---------available modes--------- >>>> 0: 0x1b:DLTapeIV(35GB) variable 85937 IDRC >>>> 1: 0x1b:DLTapeIV(35GB) variable 85937 IDRC >>>> 2: 0x1b:DLTapeIV(35GB) variable 85937 IDRC >>>> 3: 0x1b:DLTapeIV(35GB) variable 85937 IDRC >>>> --------------------------------- >>>> Current Driver State: at rest. >>>> --------------------------------- >>>> File Number: 0 Record Number: 0 Residual Count 0 >>>>=20 >>>>=20 >>>> After doing a very small tar -c and tar -x, I have: >>>>=20 >>>> # mt -f /dev/nsa1 /dev/nsa1 ostatus >>>> Mode Density Blocksize bpi Compression >>>> Current: 0x1b:DLTapeIV(35GB) variable 85937 IDRC >>>> ---------available modes--------- >>>> 0: 0x1b:DLTapeIV(35GB) variable 85937 IDRC >>>> 1: 0x1b:DLTapeIV(35GB) variable 85937 IDRC >>>> 2: 0x1b:DLTapeIV(35GB) variable 85937 IDRC >>>> 3: 0x1b:DLTapeIV(35GB) variable 85937 IDRC >>>> --------------------------------- >>>> Current Driver State: at rest. >>>> --------------------------------- >>>> File Number: 0 Record Number: 7 Residual Count 0 >>>=20 >>> Woohoo! It works. >>>=20 >>>> # mt -f /dev/nsa1 status -v >>>> Drive: sa1: Serial Number: CXA09S1340 >>>> --------------------------------- >>>> Mode Density Blocksize bpi = Compression >>>> Current: 0x1b:DLTapeIV(35GB) variable 85937 enabled = (IDRC) >>>> --------------------------------- >>>> Current Driver State: at rest. >>>> --------------------------------- >>>> Partition: 0 Calc File Number: 0 Calc Record Number: 7 >>>> Residual: 0 Reported File Number: -1 Reported Record Number: = -1 >>>> Flags: None >>>> --------------------------------- >>>> Tape I/O parameters: >>>> Maximum I/O size allowed by driver and controller (maxio): 65536 = bytes >>>> Maximum I/O size reported by controller (cpi_maxio): 0 bytes >>>> Maximum block size supported by tape drive and media (max_blk): = 16777214 bytes >>>> Minimum block size supported by tape drive and media (min_blk): 2 = bytes >>>> Block granularity supported by tape drive and media (blk_gran): 0 = bytes >>>> Maximum possible I/O size (max_effective_iosize): 65536 bytes >>>>=20 >>>> I may not get to testing Bacula today. =20 >>>>=20 >>>> Based on the above, is there any commands you'd like me to try? >>>=20 >>> Aside from making sure things work okay with Bacula, that is = probably >>> sufficient. These drives won't support density reports or position >>> information. >>>=20 >>>> Read below regarding two tape drives >>>>=20 >>>>>=20 >>>>> 6. Existing applications should work without changes. If not, = please let >>>>> me know. Hopefully they will move over time to the new = interfaces. >>>>>=20 >>>>> 7. There are lots of additional features that could be added = later. >>>>> Append-only support, encryption, more log pages, etc. >>>>>=20 >>>>> 8. I have SCSI READ ATTRIBUTE changes for camcontrol(8) that will = go in >>>>> separately. These changes allow displaying the contents of the = MAM >>>>> (Medium Auxiliary Memory) chips on LTO, TS and other modern tape = drives. >>>>> These are good, and a future possible direction is adding = attributes=20 >>>>> to the status XML from the sa(4) driver. >>>>>=20 >>>>> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D >>>>> Significant upgrades to sa(4) and mt(1). >>>>>=20 >>>>> The primary focus of these changes is to modernize FreeBSD's >>>>> tape infrastructure so that we can take advantage of some of the >>>>> features of modern tape drives and allow support for LTFS. >>>>>=20 >>>>> Significant changes and new features include: >>>>>=20 >>>>> o sa(4) driver status and parameter information is now exported = via an >>>>> XML structure. This will allow for changes and improvements later >>>>> on that will not break userland applications. The old MTIOCGET >>>>> status ioctl remains, so applications using the existing interface >>>>> will not break. >>>>>=20 >>>>> o 'mt status' now reports drive-reported tape position information >>>>> as well as the previously available calculated tape position >>>>> information. These numbers will be different at times, because >>>>> the drive-reported block numbers are relative to BOP (Beginning >>>>> of Partition), but the block numbers calculated previously via >>>>> sa(4) (and still provided) are relative to the last filemark. >>>>> Both numbers are now provided. 'mt status' now also shows the >>>>> drive INQUIRY information, serial number and any position flags >>>>> (BOP, EOT, etc.) provided with the tape position information. >>>>> 'mt status -v' adds information on the maximum possible I/O size, >>>>> and the underlying values used to calculate it. >>>>>=20 >>>>> o The extra sa(4) /dev entries (/dev/saN.[0-3]) have been removed. >>>>=20 >>>> How does this affect a tape library with more than one tape drive? >>>>=20 >>>> [root@cuppy:~] # camcontrol amcontrol devlist >>>> at scbus0 target 0 lun 0 = (pass0,ch0) >>>> at scbus0 target 2 lun 0 = (sa1,pass2) >>>> at scbus1 target 0 lun 0 = (pass3,ada0) >>>> at scbus2 target 0 lun 0 = (pass4,ada1) >>>> at scbus3 target 0 lun 0 = (pass5,ses0) >>>>=20 >>>> This system has two tapes drives and I can access them through the = front panel but: >>>>=20 >>>> # ls -l /dev/*sa* >>>> crw-rw---- 1 root operator 0x65 Feb 28 22:04 /dev/esa1 >>>> crw-rw---- 1 root operator 0x64 Mar 1 22:43 /dev/nsa1 >>>> crw-rw---- 1 root operator 0x63 Feb 28 22:04 /dev/sa1 >>>> crw-rw---- 1 root operator 0x62 Feb 28 22:04 /dev/sa1.ctl >>>>=20 >>>> ... only one tape drives shows up. >>>=20 >>>=20 >>> Hmm. The tape drive is listed as sa1, which implies that there may = be an >>> sa0 that was there previously or is in the process of probing. What = does >>> dmesg show? How about 'camcontrol devlist -v'? >>=20 >> # camcontrol devlist -v >> scbus0 on ahc0 bus 0: >> at scbus0 target 0 lun 0 = (pass0,ch0) >> at scbus0 target 2 lun 0 = (sa1,pass2) >> <> at scbus0 target -1 lun ffffffff = () >> scbus1 on ahcich2 bus 0: >> at scbus1 target 0 lun 0 = (pass3,ada0) >> <> at scbus1 target -1 lun ffffffff = () >> scbus2 on ahcich4 bus 0: >> at scbus2 target 0 lun 0 = (pass4,ada1) >> <> at scbus2 target -1 lun ffffffff = () >> scbus3 on ahciem0 bus 0: >> at scbus3 target 0 lun 0 = (pass5,ses0) >> <> at scbus3 target -1 lun ffffffff = () >> scbus-1 on xpt0 bus 0: >> <> at scbus-1 target -1 lun ffffffff = (xpt0) >>=20 >>=20 >> BUT! >>=20 >> # grep sa /var/run/dmesg.boot=20 >> VT-x: (disabled in BIOS) PAT,HLT,MTF,PAUSE,EPT,UG,VPID >> module_register_init: MOD_LOAD (vesa, 0xffffffff80de3720, 0) error 19 >> alc0: Using 1 MSIX message(s). >> isab0: at device 31.0 on pci0 >> isa0: on isab0 >> orm0: at iomem 0xce800-0xcefff on isa0 >> atkbdc0: at port 0x60,0x64 on isa0 >> sa0 at ahc0 bus 0 scbus0 target 1 lun 0 >> sa0: Removable Sequential Access SCSI-2 = device=20 >> sa0: Serial Number CXA22S2338 >> sa0: 10.000MB/s transfers (10.000MHz, offset 15) >> sa0: quirks=3D0x100 >> sa1 at ahc0 bus 0 scbus0 target 2 lun 0 >> sa1: Removable Sequential Access SCSI-2 = device=20 >> sa1: Serial Number CXA09S1340 >> sa1: 10.000MB/s transfers (10.000MHz, offset 15) >> sa1: quirks=3D0x100 >=20 > If you run 'dmesg', you should have seen a message when it went away. = Perhaps > there will be something preceding it that will give us a clue about = the > problem. (Generally a selection timeout.) At least this does show = that > sa0 is at target 1, and so should not conflict with the library or = sa1. Ahh: Trying to mount root from zfs:system/bootenv/FreeBSDHEad []... sa0 at ahc0 bus 0 scbus0 target 1 lun 0 sa0: s/n CXA22S2338 detached (sa0:ahc0:0:1:0): Periph destroyed arp: 10.55.0.60 moved from e4:ce:8f:46:f1:98 to 78:ca:39:fe:d6:b3 on em0 arp: 10.55.0.60 moved from e4:ce:8f:46:f1:98 to 78:ca:39:fe:d6:b3 on em0 arp: 10.55.0.60 moved from 78:ca:39:fe:d6:b3 to e4:ce:8f:46:f1:98 on em0 (sa1:ahc0:0:2:0): 64512-byte tape record bigger than supplied buffer (sa1:ahc0:0:2:0): 10240-byte tape record bigger than supplied buffer >=20 >>>=20 >>> I would look at cabling and termination. Is this your library? >>=20 >> Yes, it is. =20 >>=20 >>>=20 >>> = http://manx.classiccmp.org/collections/mds-199909/cd3/tape/tl891uga.pdf >>>=20 >>> If it is close enough, there are 6 connectors on the back. You = would want >>> to have something plugged into all 6, either a cable or a = terminator. >>=20 >> Yes, that's mine, and yes, there's two short cables, a terminator, = and the cable to the SCSI card in my computer. >=20 > That sounds correct for a configuration with everything on one bus. >=20 >>>=20 >>> In the manual above, the SCSI IDs are set via the front panel. If = the >>> other drive is on the same bus as the drive above and the library = device, >>> it should be at a separate SCSI ID. >>=20 >> I did have the entire thing torn apart today, to extract a tape which = would not budge. >=20 > Ahh. The SCSI IDs are right, so that doesn't appear to be the issue. >=20 >>>=20 >>>>> The extra devices were originally added as place holders for >>>>> density-specific device nodes. Some OSes (NetBSD, NetApp's OnTap >>>>> and Solaris) have had device nodes that, when you write to them, >>>>> will automatically select a given density for particular tape = drives. >>>>>=20 >>>>> This is a convenient way of switching densities, but it was never >>>>> implemented in FreeBSD. Only the device nodes were there, and = that >>>>> sometimes confused users. >>>>>=20 >>>>> For modern tape devices, the density is generally not selectable >>>>> (e.g. with LTO) or defaults to the highest availble density when >>>>> the tape is rewritten from BOT (e.g. TS11X0). So, for most users, >>>>> density selection won't be necessary. If they do need to select >>>>> the density, it is easy enough to use 'mt density' to change it. >>>>>=20 >>>>> o Protection information is now supported. This is either a >>>>> Reed-Solomon CRC or CRC32 that is included at the end of each = block >>>>> read and written. On write, the tape drive verifies the CRC, and >>>>> on read, the tape drive provides a CRC for the userland = application >>>>> to verify. >>>>>=20 >>>>> o New, extensible tape driver parameter get/set interface. >>>>>=20 >>>>> o Density reporting information. For drives that support it, >>>>> 'mt getdensity' will show detailed information on what formats the >>>>> tape drive supports, and what formats the tape drive supports. >>>>>=20 >>>>> o Some mt(1) functionality moved into a new mt(3) library so that >>>>> external applications can reuse the code. >>>>>=20 >>>>> o The new mt(3) library includes helper routines to aid in parsing >>>>> the XML output of the sa(4) driver, and build a tree of driver >>>>> metadata. >>>>>=20 >>>>> o Support for the MTLOAD (load a tape in the drive) and MTWEOFI >>>>> (write filemark immediate) ioctls needed by IBM's LTFS >>>>> implementation. >>>>>=20 >>>>> o Improve device departure behavior for the sa(4) driver. The = previous >>>>> implementation led to hangs when the device was open. >>>>>=20 >>>>> o This has been tested on the following types of drives: >>>>> IBM TS1150 >>>>> IBM TS1140 >>>>> IBM LTO-6 >>>>> IBM LTO-5 >>>>> HP LTO-2 >>>>> Seagate DDS-4 >>>>> Quantum DLT-4000 >>>>> Exabyte 8505 >>>>> Sony DDS-2 >>>>>=20 >>>>> contrib/groff/tmac/doc-syms, >>>>> share/mk/bsd.libnames.mk, >>>>> lib/Makefile, >>>>> Add libmt. >>>>>=20 >>>>> lib/libmt/Makefile, >>>>> lib/libmt/mt.3, >>>>> lib/libmt/mtlib.c, >>>>> lib/libmt/mtlib.h, >>>>> New mt(3) library that contains functions moved from mt(1) and >>>>> new functions needed to interact with the updated sa(4) driver. >>>>>=20 >>>>> This includes XML parser helper functions that application = writers >>>>> can use when writing code to query tape parameters. >>>>>=20 >>>>> rescue/rescue/Makefile: >>>>> Add -lmt to CRUNCH_LIBS. >>>>>=20 >>>>> sys/cam/cam_ccb.h >>>>> Add a new flag value for the XPT_DEV_ADVINFO CCB, = CDAI_FLAG_NONE. >>>>>=20 >>>>> sbin/camcontrol/camcontrol.c, >>>>> sys/cam/scsi/scsi_da.c, >>>>> sys/cam/scsi/scsi_enc_ses.c, >>>>> sys/dev/mps/mps_sas.c: >>>>> Make sure the flags for the XPT_DEV_ADVINFO CCB are set = correctly. >>>>> This prevents unintended attempts to set advanced information >>>>> values when XPT_DEV_ADVINFO CCBs are not pre-zeroed. >>>>>=20 >>>>> src/share/man/man4/mtio.4 >>>>> Clarify this man page a bit, and since it contains what is >>>>> essentially the mtio.h header file, add new ioctls and structure >>>>> definitions from mtio.h. >>>>>=20 >>>>> src/share/man/man4/sa.4 >>>>> Update BUGS and maintainer section. >>>>>=20 >>>>> sys/cam/scsi/scsi_all.c, >>>>> sys/cam/scsi/scsi_all.h: >>>>> Add SCSI SECURITY PROTOCOL IN/OUT CDB definitions and CDB = building >>>>> functions. >>>>>=20 >>>>> sys/cam/scsi/scsi_sa.c >>>>> sys/cam/scsi/scsi_sa.h >>>>> Many tape driver changes, largely outlined above. >>>>>=20 >>>>> Increase the sa(4) driver read/write timeout from 4 to 32 >>>>> minutes. This is based on the recommended values for IBM LTO >>>>> 5/6 drives. This may also avoid timeouts for other tape >>>>> hardware that can take a long time to do retries and error >>>>> recovery. Longer term, a better way to handle this is to ask >>>>> the drive for recommended timeout values using the REPORT >>>>> SUPPORTED OPCODES command. Modern IBM and Oracle tape drives >>>>> at least support that command, and it would allow for more >>>>> accurate timeout values. >>>>>=20 >>>>> Add XML status generation. This is done with a series of >>>>> macros to eliminate as much duplicate code as possible. The >>>>> new XML-based status values are reported through the new >>>>> MTIOCEXTGET ioctl. >>>>>=20 >>>>> Add XML driver parameter reporting, using the new MTIOCPARAMGET >>>>> ioctl. >>>>>=20 >>>>> Add a new driver parameter setting interface, using the new >>>>> MTIOCPARAMSET and MTIOCSETLIST ioctls. >>>>>=20 >>>>> Add a new MTIOCRBLIM ioctl to get block limits information. >>>>>=20 >>>>> Add CCB/CDB building routines scsi_locate_16, scsi_locate_10, >>>>> and scsi_read_position_10(). >>>>>=20 >>>>> scsi_locate_10 implements the LOCATE command, as does the >>>>> existing scsi_set_position() command. It just supports >>>>> additional arguments and features. If/when we figure out a >>>>> good way to provide backward compatibility for older >>>>> applications using the old function API, we can just revamp >>>>> scsi_set_position(). The same goes for >>>>> scsi_read_position_10() and the existing scsi_read_position() >>>>> function. >>>>>=20 >>>>> Revamp sasetpos() to take the new mtlocate structure as an >>>>> argument. It now will use either scsi_locate_10() or >>>>> scsi_locate_16(), depending upon the arguments the user >>>>> supplies. As before, once we change position we don't have a >>>>> clear idea of what the current logical position of the tape >>>>> drive is. >>>>>=20 >>>>> For tape drives that support long form position data, we >>>>> read the current position and store that for later reporting >>>>> after changing the position. This should help applications >>>>> like Bacula speed tape access under FreeBSD once they are >>>>> modified to support the new ioctls. >>>>>=20 >>>>> Add a new quirk, SA_QUIRK_NO_LONG_POS, that is set for all >>>>> drives that report SCSI-2 or older, as well as drives that >>>>> report an Illegal Request type error for READ POSITION with >>>>> the long format. So we should automatically detect drives >>>>> that don't support the long form and stop asking for it after >>>>> an initial try. >>>>>=20 >>>>> Add a partition number to the sa(4) softc. >>>>>=20 >>>>> Improve device departure handling. The previous implementation >>>>> led to hangs when the device was open. >>>>>=20 >>>>> If an application had the sa(4) driver open, and attempted to >>>>> close it after it went away, the cam_periph_release() call in >>>>> saclose() would cause the periph to get destroyed because that >>>>> was the last reference to it. Because destroy_dev() was >>>>> called from the sa(4) driver's cleanup routine (sacleanup()), >>>>> and would block waiting for the close to happen, a deadlock >>>>> would result. >>>>>=20 >>>>> So instead of calling destroy_dev() from the cleanup routine, >>>>> call destroy_dev_sched_cb() from saoninvalidate() and wait for >>>>> the callback. >>>>>=20 >>>>> Acquire a reference for devfs in saregister(), and release it >>>>> in the new sadevgonecb() routine when all devfs devices for=09 >>>>> the particular sa(4) driver instance are gone. >>>>>=20 >>>>> Add a new function, sasetupdev(), to centralize setting >>>>> per-instance devfs device parameters instead of repeating the >>>>> code in saregister(). >>>>>=20 >>>>> Add an open count to the softc, so we know how many >>>>> peripheral driver references are a result of open >>>>> sessions. >>>>>=20 >>>>> Add the D_TRACKCLOSE flag to the cdevsw flags so >>>>> that we get a 1:1 mapping of open to close calls >>>>> instead of a N:1 mapping. >>>>>=20 >>>>> This should be a no-op for everything except the >>>>> control device, since we don't allow more than one >>>>> open on non-control devices. >>>>>=20 >>>>> However, since we do allow multiple opens on the >>>>> control device, the combination of the open count >>>>> and the D_TRACKCLOSE flag should result in an >>>>> accurate peripheral driver reference count, and an >>>>> accurate open count. >>>>>=20 >>>>> The accurate open count allows us to release all >>>>> peripheral driver references that are the result >>>>> of open contexts once we get the callback from devfs. >>>>>=20 >>>>> sys/sys/mtio.h: >>>>> Add a number of new mt(4) ioctls and the requisite data >>>>> structures. None of the existing interfaces been removed >>>>> or changed. >>>>>=20 >>>>> This includes definitions for the following new ioctls: >>>>>=20 >>>>> MTIOCRBLIM /* get block limits */ >>>>> MTIOCEXTLOCATE /* seek to position */ >>>>> MTIOCEXTGET /* get tape status */ >>>>> MTIOCPARAMGET /* get tape params */ >>>>> MTIOCPARAMSET /* set tape params */ >>>>> MTIOCSETLIST /* set N params */ >>>>>=20 >>>>> usr.bin/mt/Makefile: >>>>> mt(1) now depends on libmt, libsbuf and libbsdxml. >>>>>=20 >>>>> usr.bin/mt/mt.1: >>>>> Document new mt(1) features and subcommands. >>>>>=20 >>>>> usr.bin/mt/mt.c: >>>>> Implement support for mt(1) subcommands that need to >>>>> use getopt(3) for their arguments. >>>>>=20 >>>>> Implement a new 'mt status' command to replace the old >>>>> 'mt status' command. The old status command has been >>>>> renamed 'ostatus'. >>>>>=20 >>>>> The new status function uses the MTIOCEXTGET ioctl, and >>>>> therefore parses the XML data to determine drive status. >>>>> The -x argument to 'mt status' allows the user to dump out >>>>> the raw XML reported by the kernel. >>>>>=20 >>>>> The new status display is mostly the same as the old status >>>>> display, except that it doesn't print the redundant density >>>>> mode information, and it does print the current partition >>>>> number and position flags. >>>>>=20 >>>>> Add a new command, 'mt locate', that will supersede the >>>>> old 'mt setspos' and 'mt sethpos' commands. 'mt locate' >>>>> implements all of the functionality of the MTIOCEXTLOCATE >>>>> ioctl, and allows the user to change the logical position >>>>> of the tape drive in a number of ways. (Partition, >>>>> block number, file number, set mark number, end of data.) >>>>> The immediate bit and the explicit address bits are >>>>> implemented, but not documented in the man page. >>>>>=20 >>>>> Add a new 'mt weofi' command to use the new MTWEOFI ioctl. >>>>> This allows the user to ask the drive to write a filemark >>>>> without waiting around for the operation to complete. >>>>>=20 >>>>> Add a new 'mt getdensity' command that gets the XML-based >>>>> tape drive density report from the sa(4) driver and displays >>>>> it. This uses the SCSI REPORT DENSITY SUPPORT command >>>>> to get comprehensive information from the tape drive about >>>>> what formats it is able to read and write. >>>>>=20 >>>>> Add a new 'mt protect' command that allows getting and setting >>>>> tape drive protection information. The protection information >>>>> is a CRC tacked on to the end of every read/write from and to >>>>> the tape drive. >>>>>=20 >>>>> Sponsored by: Spectra Logic >>>>> MFC after: 1 month >>>>>=20 >>>>> Thanks, >>>>>=20 >>>>> Ken >>>>> --=20 >>>>> Kenneth Merry >>>>> ken@FreeBSD.ORG >>>>> _______________________________________________ >>>>> freebsd-scsi@freebsd.org mailing list >>>>> http://lists.freebsd.org/mailman/listinfo/freebsd-scsi >>>>> To unsubscribe, send any mail to = "freebsd-scsi-unsubscribe@freebsd.org" >>>>=20 >>>> ?=20 >>>> Dan Langille >>>> http://langille.org/ >>>>=20 >>>>=20 >>>>=20 >>>>=20 >>>>=20 >>>=20 >>> --=20 >>> Kenneth Merry >>> ken@FreeBSD.ORG >>=20 >> ?=20 >> Dan Langille >> http://langille.org/ >>=20 >>=20 >>=20 >>=20 >>=20 >=20 > Ken > --=20 > Kenneth Merry > ken@FreeBSD.ORG =E2=80=94=20 Dan Langille http://langille.org/ From owner-freebsd-current@FreeBSD.ORG Mon Mar 2 00:41:09 2015 Return-Path: Delivered-To: current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C728BB2; Mon, 2 Mar 2015 00:41:09 +0000 (UTC) Received: from clavin1.langille.org (clavin.langille.org [162.208.116.86]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "clavin.langille.org", Issuer "StartCom Class 2 Primary Intermediate Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 9C8A9D17; Mon, 2 Mar 2015 00:41:09 +0000 (UTC) Received: from (clavin1.int.langille.org (clavin1.int.unixathome.org [10.4.7.7]) (Authenticated sender: hidden) with ESMTPSA id 8B385F6F ; Mon, 2 Mar 2015 00:41:08 +0000 (UTC) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 8.2 \(2070.6\)) Subject: Re: sa(4) driver changes available for test From: Dan Langille In-Reply-To: <20150302003150.GB71528@mithlond.kdm.org> Date: Sun, 1 Mar 2015 19:41:07 -0500 Content-Transfer-Encoding: quoted-printable Message-Id: References: <20150214003232.GA63990@mithlond.kdm.org> <7CA52DF3-E073-4F50-BE4E-01C51CCDF2C7@langille.org> <20150217183645.GA30947@mithlond.kdm.org> <303C48AC-0DA8-447C-8DE7-0A1DD5DD7351@langille.org> <20150302003150.GB71528@mithlond.kdm.org> To: "Kenneth D. Merry" X-Mailer: Apple Mail (2.2070.6) Cc: current@freebsd.org, scsi@freebsd.org X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Mar 2015 00:41:09 -0000 > On Mar 1, 2015, at 7:31 PM, Kenneth D. Merry wrote: >=20 > On Sun, Mar 01, 2015 at 19:15:05 -0500, Dan Langille wrote: >>=20 >>> On Feb 17, 2015, at 1:36 PM, Kenneth D. Merry = wrote: >>>=20 >>> On Sat, Feb 14, 2015 at 18:22:43 -0500, Dan Langille wrote: >>>>=20 >>>>> On Feb 13, 2015, at 7:32 PM, Kenneth D. Merry = wrote: >>>>>=20 >>>>>=20 >>>>> I have a fairly large set of changes to the sa(4) driver and mt(1) = driver >>>>> that I'm planning to commit in the near future. >>>>>=20 >>>>> A description of the changes is here and below in this message. >>>>>=20 >>>>> If you have tape hardware and the inclination, I'd appreciate = testing and >>>>> feedback. >>>>=20 >>>> I have a DLT 8000 and an SDLT 220. >>>>=20 >>>> I don't have anything running current, but I have a spare machine = which I could use for testing. >>>>=20 >>>> Do you see any value is tests with that hardware? I'd be testing it = via Bacula. >>>>=20 >>>> disclosure: I'm the sysutils/bacula-* maintainer and a Bacula = committer. >>>>=20 >>>=20 >>> Actually, yes. Bacula is a bit tricky to configure, so your trying = it out >>> would be helpful if you have the time. >>>=20 >>> In looking at the manuals for both the SDLT 220 and the DLT 8000, = they both >>> claim to support long position information for the SCSI READ = POSITION >>> command. >>>=20 >>> You can see what I'm talking about by doing: >>>=20 >>> mt eod >>> mt status >>>=20 >>> On my DDS-4 tape drive, this shows: >>>=20 >>> # mt -f /dev/nsa3 status >>> Drive: sa3: Serial Number: HJ00YWY >>> --------------------------------- >>> Mode Density Blocksize bpi Compression >>> Current: 0x26:DDS-4 1024 bytes 97000 enabled = (DCLZ) >>> --------------------------------- >>> Current Driver State: at rest. >>> --------------------------------- >>> Partition: 0 Calc File Number: -1 Calc Record Number: -1 >>> Residual: 0 Reported File Number: -1 Reported Record Number: -1 >>> Flags: None >>>=20 >>> But on an LTO-5, which will give long position information, I get: >>>=20 >>> [root@doc ~]# mt status >>> Drive: sa0: >>> --------------------------------- >>> Mode Density Blocksize bpi Compression >>> Current: 0x58:LTO-5 variable 384607 enabled (0x1) >>> --------------------------------- >>> Current Driver State: at rest. >>> --------------------------------- >>> Partition: 0 Calc File Number: 2 Calc Record Number: -1 >>> Residual: 0 Reported File Number: 2 Reported Record Number: = 32373 >>> Flags: None >>>=20 >>> That, in combination with the changes I made to the position = information >>> code in the driver, mean that even the old MTIOCGET ioctl should = return an >>> accurate file number at end of data. e.g., on the LTO-5: >>>=20 >>> [root@doc ~]# mt ostatus >>> Mode Density Blocksize bpi Compression >>> Current: 0x58:LTO-5 variable 384607 0x1 >>> ---------available modes--------- >>> 0: 0x58:LTO-5 variable 384607 0x1 >>> 1: 0x58:LTO-5 variable 384607 0x1 >>> 2: 0x58:LTO-5 variable 384607 0x1 >>> 3: 0x58:LTO-5 variable 384607 0x1 >>> --------------------------------- >>> Current Driver State: at rest. >>> --------------------------------- >>> File Number: 2 Record Number: -1 Residual Count -1 >>>=20 >>> So the thing to try, in addition to just making sure that Bacula = continues >>> to work properly, is to try setting this for the tape drive in >>> bacula-sd.conf: >>>=20 >>> Hardware End of Medium =3D yes >>>=20 >>> It looks like the Bacula tape program (btape) has a test mode, and = it would >>> be good to run through the tests on one of the tape drives and see = whether >>> they work, and whether the results are different before and after = the >>> changes. I'm not sure how to enable the test mode. >>=20 >> I have this in /usr/local/etc/bacula/bacula-sd.conf >>=20 >> Device { >> Name =3D DLT >> Description =3D "QUANTUM DLT7000 1624" >> Media Type =3D DLT >> Archive Device =3D /dev/nsa1 >>=20 >> Autochanger =3D YES >> Drive Index =3D 0 >>=20 >> Offline On Unmount =3D no >> Hardware End of Medium =3D yes >> BSF at EOM =3D yes >> Backward Space Record =3D no >> Fast Forward Space File =3D no >> TWO EOF =3D yes >> } >>=20 >> FYI, http://www.freebsddiary.org/digital-tl891.php (from 2006) has a = btape test on this same model. >>=20 >> Here's the test I ran tonight: >>=20 >> [root@cuppy:/usr/home/dan] # btape -c = /usr/local/etc/bacula/bacula-sd.conf /dev/nsa1 = =20 >> Tape block granularity is 1024 bytes. >> btape: butil.c:287-0 Using device: "/dev/nsa1" for writing. >> btape: btape.c:469-0 open device "DLT" (/dev/nsa1): OK >> *test >>=20 >> =3D=3D=3D Write, rewind, and re-read test =3D=3D=3D >>=20 >> I'm going to write 10000 records and an EOF >> then write 10000 records and an EOF, then rewind, >> and re-read the data to verify that it is correct. >>=20 >> This is an *essential* feature ... >>=20 >> btape: btape.c:1152-0 Wrote 10000 blocks of 64412 bytes. >> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) >> btape: btape.c:1168-0 Wrote 10000 blocks of 64412 bytes. >> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) >> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) >> btape: btape.c:1210-0 Rewind OK. >> 10000 blocks re-read correctly. >> Got EOF on tape. >> 10000 blocks re-read correctly. >> =3D=3D=3D Test Succeeded. End Write, rewind, and re-read test =3D=3D=3D= >>=20 >> btape: btape.c:1277-0 Block position test >> btape: btape.c:1289-0 Rewind OK. >> Reposition to file:block 0:4 >> Block 5 re-read correctly. >> Reposition to file:block 0:200 >> Block 201 re-read correctly. >> Reposition to file:block 0:9999 >> Block 10000 re-read correctly. >> Reposition to file:block 1:0 >> Block 10001 re-read correctly. >> Reposition to file:block 1:600 >> Block 10601 re-read correctly. >> Reposition to file:block 1:9999 >> Block 20000 re-read correctly. >> =3D=3D=3D Test Succeeded. End Write, rewind, and re-read test =3D=3D=3D= >>=20 >>=20 >>=20 >> =3D=3D=3D Append files test =3D=3D=3D >>=20 >> This test is essential to Bacula. >>=20 >> I'm going to write one record in file 0, >> two records in file 1, >> and three records in file 2 >>=20 >> btape: btape.c:574-0 Rewound "DLT" (/dev/nsa1) >> btape: btape.c:1907-0 Wrote one record of 64412 bytes. >> btape: btape.c:1909-0 Wrote block to device. >> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) >> btape: btape.c:1907-0 Wrote one record of 64412 bytes. >> btape: btape.c:1909-0 Wrote block to device. >> btape: btape.c:1907-0 Wrote one record of 64412 bytes. >> btape: btape.c:1909-0 Wrote block to device. >> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) >> btape: btape.c:1907-0 Wrote one record of 64412 bytes. >> btape: btape.c:1909-0 Wrote block to device. >> btape: btape.c:1907-0 Wrote one record of 64412 bytes. >> btape: btape.c:1909-0 Wrote block to device. >> btape: btape.c:1907-0 Wrote one record of 64412 bytes. >> btape: btape.c:1909-0 Wrote block to device. >> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) >> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) >> btape: btape.c:469-0 open device "DLT" (/dev/nsa1): OK >> btape: btape.c:574-0 Rewound "DLT" (/dev/nsa1) >> btape: btape.c:1420-0 Now moving to end of medium. >=20 > This is the critical piece. The test moves the tape to the end of the > medium. With hardware position information, you can tell what = filemark > you're on. Without it, you can't. >=20 >> btape: btape.c:622-0 tape_dev.c:345 ioctl MTIOCGET error on "DLT" = (/dev/nsa1). ERR=3DNo error: 0. >> We should be in file 3. I am at file 0. This is NOT correct!!!! >>=20 >> Append test failed. Attempting again. >> Setting "Hardware End of Medium =3D no >> and "Fast Forward Space File =3D no >> and retrying append test. >=20 > This is not surprsing, given that the drive doesn't support long read > position data. (It's a SCSI-2 device.) So that means that Bacula = will > need to do it manually. Yes, I have nothing newer than SCSI-2. Even my SDLT is SCSI-2 but that tape library is hooked up to a different computer and was doing backups = today. > =3D=3D=3D Append files test =3D=3D=3D >>=20 >> This test is essential to Bacula. >>=20 >> I'm going to write one record in file 0, >> two records in file 1, >> and three records in file 2 >>=20 >> btape: btape.c:574-0 Rewound "DLT" (/dev/nsa1) >> btape: btape.c:1907-0 Wrote one record of 64412 bytes. >> btape: btape.c:1909-0 Wrote block to device. >> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) >> btape: btape.c:1907-0 Wrote one record of 64412 bytes. >> btape: btape.c:1909-0 Wrote block to device. >> btape: btape.c:1907-0 Wrote one record of 64412 bytes. >> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) >> btape: btape.c:1907-0 Wrote one record of 64412 bytes. >> btape: btape.c:1909-0 Wrote block to device. >> btape: btape.c:1907-0 Wrote one record of 64412 bytes. >> btape: btape.c:1909-0 Wrote block to device. >> btape: btape.c:1907-0 Wrote one record of 64412 bytes. >> btape: btape.c:1909-0 Wrote block to device. >> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) >> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) >> btape: btape.c:469-0 open device "DLT" (/dev/nsa1): OK >> btape: btape.c:574-0 Rewound "DLT" (/dev/nsa1) >> btape: btape.c:1420-0 Now moving to end of medium. >> btape: btape.c:625-0 Moved to end of medium. >> We should be in file 3. I am at file 3. This is correct! >>=20 >> Now the important part, I am going to attempt to append to the tape. >>=20 >> btape: btape.c:1907-0 Wrote one record of 64412 bytes. >> btape: btape.c:1909-0 Wrote block to device. >> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) >> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) >> btape: btape.c:574-0 Rewound "DLT" (/dev/nsa1) >> Done appending, there should be no I/O errors >>=20 >> Doing Bacula scan of blocks: >> 1 block of 64448 bytes in file 1 >> End of File mark. >> 2 blocks of 64448 bytes in file 2 >> End of File mark. >> 3 blocks of 64448 bytes in file 3 >> End of File mark. >> 1 block of 64448 bytes in file 4 >> End of File mark. >> Total files=3D4, blocks=3D7, bytes =3D 451,136 >> End scanning the tape. >> We should be in file 4. I am at file 4. This is correct! >>=20 >>=20 >> It looks like the test worked this time, please add: >>=20 >> Hardware End of Medium =3D No >>=20 >> Fast Forward Space File =3D No >> to your Device resource in the Storage conf file. >>=20 >> The above Bacula scan should have output identical to what follows. >> Please double check it ... >> =3D=3D=3D Sample correct output =3D=3D=3D >> 1 block of 64448 bytes in file 1 >> End of File mark. >> 2 blocks of 64448 bytes in file 2 >> End of File mark. >> 3 blocks of 64448 bytes in file 3 >> End of File mark. >> 1 block of 64448 bytes in file 4 >> End of File mark. >> Total files=3D4, blocks=3D7, bytes =3D 451,136 >> =3D=3D=3D End sample correct output =3D=3D=3D >>=20 >> If the above scan output is not identical to the >> sample output, you MUST correct the problem >> or Bacula will not be able to write multiple Jobs to=20 >> the tape. >>=20 >> Skipping read backwards test because BSR turned off. >>=20 >>=20 >> =3D=3D=3D Forward space files test =3D=3D=3D >>=20 >> This test is essential to Bacula. >>=20 >> I'm going to write five files then test forward spacing >>=20 >> btape: btape.c:574-0 Rewound "DLT" (/dev/nsa1) >> btape: btape.c:1907-0 Wrote one record of 64412 bytes. >> btape: btape.c:1909-0 Wrote block to device. >> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) >> btape: btape.c:1907-0 Wrote one record of 64412 bytes. >> btape: btape.c:1909-0 Wrote block to device. >> btape: btape.c:1907-0 Wrote one record of 64412 bytes. >> btape: btape.c:1909-0 Wrote block to device. >> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) >> btape: btape.c:1907-0 Wrote one record of 64412 bytes. >> btape: btape.c:1909-0 Wrote block to device. >> btape: btape.c:1907-0 Wrote one record of 64412 bytes. >> btape: btape.c:1909-0 Wrote block to device. >> btape: btape.c:1907-0 Wrote one record of 64412 bytes. >> btape: btape.c:1909-0 Wrote block to device. >> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) >> btape: btape.c:1907-0 Wrote one record of 64412 bytes. >> btape: btape.c:1909-0 Wrote block to device. >> btape: btape.c:1907-0 Wrote one record of 64412 bytes. >> btape: btape.c:1909-0 Wrote block to device. >> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) >> btape: btape.c:1907-0 Wrote one record of 64412 bytes. >> btape: btape.c:1909-0 Wrote block to device. >> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) >> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) >> btape: btape.c:574-0 Rewound "DLT" (/dev/nsa1) >> btape: btape.c:1634-0 Now forward spacing 1 file. >> We should be in file 1. I am at file 1. This is correct! >> btape: btape.c:1646-0 Now forward spacing 2 files. >> We should be in file 3. I am at file 3. This is correct! >> btape: btape.c:574-0 Rewound "DLT" (/dev/nsa1) >> btape: btape.c:1659-0 Now forward spacing 4 files. >> We should be in file 4. I am at file 4. This is correct! >>=20 >> btape: btape.c:1677-0 Now forward spacing 1 more file. >> We should be in file 5. I am at file 5. This is correct! >>=20 >> =3D=3D=3D End Forward space files test =3D=3D=3D >>=20 >>=20 >> Ah, I see you have an autochanger configured. >> To test the autochanger you must have a blank tape >> that I can write on in Slot 1. >>=20 >> Do you wish to continue with the Autochanger test? (y/n): y >>=20 >>=20 >> =3D=3D=3D Autochanger test =3D=3D=3D >>=20 >> 3301 Issuing autochanger "loaded" command. >> Nothing loaded in the drive. OK. >> 3303 Issuing autochanger "load 1 0" command. >> 3303 Autochanger "load 1 0" status is OK. >> btape: btape.c:469-0 open device "DLT" (/dev/nsa1): OK >> btape: btape.c:1564-0 Rewound "DLT" (/dev/nsa1) >> btape: btape.c:1571-0 Wrote EOF to "DLT" (/dev/nsa1) >>=20 >> The test autochanger worked!! >=20 > Great, thanks for running the test! Looks like things are working as = well > as they were before. >=20 >>>=20 >>>> I'll let the other Bacula devs know about this. They deal with the = hardware. I work on PostgreSQL. >>>>=20 >>>=20 >>> Thanks! If there are additional features they would like out of the = tape >>> driver, I'm happy to talk about it. (Or help if they'd like to use = the new >>> status reporting ioctl, MTIOCEXTGET or any of the other new ioctls.) >>=20 >> Errors are interesting to me. Especially corrected errors. They are = a good indicator of tape quality. >>=20 >=20 > Yes. At least on modern drives, there is a good bit available in the = log > pages. You can try seeing what logs your drive supports by installing = the > sg3_utils package and running 'sg_logs sa1 --all'. >=20 > Given the large amount of data available on some drives, and the = difficulty > distilling it down to a clear good/bad, I probably won't stick that in = the > 'mt status' output. >=20 > But you can certainly take a look at it and have an idea of whether = your > particular tape/drive are experiencing issues. That's a lot of output: = https://gist.github.com/dlangille/0e15a7fbf7acab56fd32 I will run some Bacula jobs soon. I'm still setting up config files. =E2=80=94=20 Dan Langille http://langille.org/ From owner-freebsd-current@FreeBSD.ORG Mon Mar 2 01:47:35 2015 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 0DEE199C for ; Mon, 2 Mar 2015 01:47:35 +0000 (UTC) Received: from mx1.scaleengine.net (beauharnois2.bhs1.scaleengine.net [142.4.218.15]) by mx1.freebsd.org (Postfix) with ESMTP id C12645E8 for ; Mon, 2 Mar 2015 01:47:34 +0000 (UTC) Received: from [192.168.1.2] (Seawolf.HML3.ScaleEngine.net [209.51.186.28]) (Authenticated sender: allanjude.freebsd@scaleengine.com) by mx1.scaleengine.net (Postfix) with ESMTPSA id 966EC93DC6 for ; Mon, 2 Mar 2015 01:47:32 +0000 (UTC) Message-ID: <54F3C134.5080207@freebsd.org> Date: Sun, 01 Mar 2015 20:47:32 -0500 From: Allan Jude User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:31.0) Gecko/20100101 Thunderbird/31.5.0 MIME-Version: 1.0 To: freebsd-current@freebsd.org Subject: Re: Massive libxo-zation that breaks everything References: <54F31510.7050607@hot.ee> <54F34B6E.2040809@astrodoggroup.com> <54F35F29.4000603@astrodoggroup.com> <54F36431.30506@freebsd.org> In-Reply-To: Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="UmMbL2qheo0sCWXh5BUhKMsIVwUi1Fogt" X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Mar 2015 01:47:35 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --UmMbL2qheo0sCWXh5BUhKMsIVwUi1Fogt Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable On 2015-03-01 19:20, Arseny Nasokin wrote: > On 1 March 2015 at 22:10, Allan Jude wrote: >=20 >> On 2015-03-01 13:49, Harrison Grundy wrote: >>> Thanks! >>> >>> That does seem useful, but I'm not sure I see the reasoning behind >>> putting into base, over a port or package, since processing XML in ba= se >>> is a pain, and it can't serve up JSON or HTML without additional >>> utilities anyway. >>> >>> (If I'm reviving a long-settled thing, let me know and I'll drop it. = I'm >>> trying to understand the use case for this.) >>> >>> --- Harrison >>> >>> On 03/01/15 10:31, Craig Rodrigues wrote: >>>> On Sun, Mar 1, 2015 at 9:25 AM, Harrison Grundy < >>>> harrison.grundy@astrodoggroup.com> wrote: >>>> >>>>> >>>>> >>>>> If someone could summarize what this is, I'd greatly appreciate it.= >>>>> >>>> >>>> https://lists.freebsd.org/pipermail/freebsd-arch/2014-July/015633.ht= ml >>>> _______________________________________________ >>>> freebsd-current@freebsd.org mailing list >>>> http://lists.freebsd.org/mailman/listinfo/freebsd-current >>>> To unsubscribe, send any mail to " >> freebsd-current-unsubscribe@freebsd.org" >>>> >>> _______________________________________________ >>> freebsd-current@freebsd.org mailing list >>> http://lists.freebsd.org/mailman/listinfo/freebsd-current >>> To unsubscribe, send any mail to " >> freebsd-current-unsubscribe@freebsd.org" >>> >> >> I think you're missing the important bit here. >> >> This isn't about adding a parser for anything, this is about making th= e >> tools in base, like netstat, wc, uptime, etc, output in JSON or XML, s= o >> you can use the data programmatically. >> >> Your scripts no longer have to rely on awk/sed/grep magic to get a >> specific bit of information out of the uptime command, the command can= >> just output the data in a structured machine readable format. >> >> I am not sure how you can put netstat into the ports tree. >> >> >> -- >> Allan Jude >> >> > Hi, >=20 > Do we have command-line tools in base which work with XML/JSON from std= in > or file? >=20 > -- Eir Nym > _______________________________________________ > freebsd-current@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-current > To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.o= rg" >=20 Not really, although there are libraries for such (bsdxml, libucl which can read JSON) in base. I am working on a tool that can do some of this: https://github.com/allanjude/uclcmd/ --=20 Allan Jude --UmMbL2qheo0sCWXh5BUhKMsIVwUi1Fogt Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (MingW32) iQIcBAEBAgAGBQJU88E4AAoJEJrBFpNRJZKfhtIQALkqM3ZFNF8IdVjIbO57a/M9 NIpm6Uf6FH9aX0gtw87vrtIsTmBYJne8hGB2xWamIJ4ZtwrCf8GfqGF+c0sONOi8 T0VJrreyfvOcF3tymFg1RH8OJFzNRzbafMpcs/I1P7cEd41nQJMEdVJa18iHqyJd NDVRLmsoonLViwu00ZzviDh2yhHPEHWjUUYeyl0PAFxMDzHBRyIz8M5nc8P3U7Aa fWRXN2ET8bZB7veZL6j/iX2EfTccGgkkfKHAhCcujvAigoFqR19YmiftqRFIjzrj /pvptYe/zBYot3AX2KSbc5ojbenAo5QE3DFShWFje/aKo5Ok5RKPEFbW2FK5AiJ+ cdBaFX3muMi03vSTGabn27qF7dt8KoBAHtGTgbmgChh6K2/E4QliXCC5WyuhQuB+ XWzLnaP6VIZRxhOZKsHNGg/6iiizAzFIW+Wm6soEkNdKyFKxFZ3HPoDDO9A975mn JPSbJFWzUg3I0eAKlidTYjOmE/UD6O+rTy6e/kg/U/CJLRVGiH8rsSsSk6A3qg6k 2FatEVciYzOSYrrdNY4r9yhJrZ5wdqxJUrVeKNLpQDxlgaSWMM7vIn5i/XQ54IKo OP1f7BEJSprtwVS+90gwhpezHTqmixpacudKmFWSo3m8u7VfxS0XLr/y9/TdnR5a 2RGqOLogJC8YDiNkAdTm =M26X -----END PGP SIGNATURE----- --UmMbL2qheo0sCWXh5BUhKMsIVwUi1Fogt-- From owner-freebsd-current@FreeBSD.ORG Mon Mar 2 02:06:13 2015 Return-Path: Delivered-To: current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E8976DD1; Mon, 2 Mar 2015 02:06:13 +0000 (UTC) Received: from mithlond.kdm.org (mithlond.kdm.org [70.56.43.85]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "A1-33714", Issuer "A1-33714" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 83D387FE; Mon, 2 Mar 2015 02:06:13 +0000 (UTC) Received: from mithlond.kdm.org (localhost [127.0.0.1]) by mithlond.kdm.org (8.14.9/8.14.9) with ESMTP id t22269Sv073476 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Sun, 1 Mar 2015 19:06:09 -0700 (MST) (envelope-from ken@mithlond.kdm.org) Received: (from ken@localhost) by mithlond.kdm.org (8.14.9/8.14.9/Submit) id t22268O8073475; Sun, 1 Mar 2015 19:06:08 -0700 (MST) (envelope-from ken) Date: Sun, 1 Mar 2015 19:06:08 -0700 From: "Kenneth D. Merry" To: Dan Langille Subject: Re: sa(4) driver changes available for test Message-ID: <20150302020608.GA73433@mithlond.kdm.org> References: <20150214003232.GA63990@mithlond.kdm.org> <4A478C5C-7965-498E-9F0F-80192265E310@langille.org> <20150302001833.GA71528@mithlond.kdm.org> <6C82281F-649A-4DA8-8ACF-17E81C04F730@langille.org> <20150302003658.GA72258@mithlond.kdm.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.23 (2014-03-12) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (mithlond.kdm.org [127.0.0.1]); Sun, 01 Mar 2015 19:06:10 -0700 (MST) X-Spam-Status: No, score=-2.9 required=5.0 tests=ALL_TRUSTED,BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS autolearn=ham autolearn_force=no version=3.4.0 X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on mithlond.kdm.org Cc: current@freebsd.org, scsi@freebsd.org X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Mar 2015 02:06:14 -0000 On Sun, Mar 01, 2015 at 19:40:40 -0500, Dan Langille wrote: > > > On Mar 1, 2015, at 7:36 PM, Kenneth D. Merry wrote: > > > > On Sun, Mar 01, 2015 at 19:28:37 -0500, Dan Langille wrote: > >> > >>> On Mar 1, 2015, at 7:18 PM, Kenneth D. Merry wrote: > >>> > >>> On Sun, Mar 01, 2015 at 17:06:24 -0500, Dan Langille wrote: > >>>> > >>>>> On Feb 13, 2015, at 7:32 PM, Kenneth D. Merry wrote: > >>>>> > >>>>> > >>>>> I have a fairly large set of changes to the sa(4) driver and mt(1) driver > >>>>> that I'm planning to commit in the near future. > >>>>> > >>>>> A description of the changes is here and below in this message. > >>>>> > >>>>> If you have tape hardware and the inclination, I'd appreciate testing and > >>>>> feedback. > >>>>> > >>>>> ============ > >>>>> Rough draft commit message: > >>>>> > >>>>> http://people.freebsd.org/~ken/sa_changes_commitmsg.20150213.3.txt > >>>>> > >>>>> The patches against FreeBSD/head as of SVN revision 278706: > >>>>> > >>>>> http://people.freebsd.org/~ken/sa_changes.20150213.3.txt > >>>>> > >>>>> And (untested) patches against FreeBSD stable/10 as of SVN revision 278721. > >>>>> > >>>>> http://people.freebsd.org/~ken/sa_changes.stable_10.20150213.3.txt > >>>>> ============ > >>>>> > >>>>> The intent is to get the tape infrastructure more up to date, so we can > >>>>> support LTFS and more modern tape drives: > >>>>> > >>>>> http://www.ibm.com/systems/storage/tape/ltfs/ > >>>>> > >>>>> I have ported IBM's LTFS Single Drive Edition to FreeBSD. The port depends > >>>>> on the patches linked above. It isn't fully cleaned up and ready for > >>>>> redistribution. If you're interested, though, let me know and I'll tell > >>>>> you when it is ready to go out. You need an IBM LTO-5, LTO-6, TS1140 or > >>>>> TS1150 tape drive. HP drives aren't supported by IBM's LTFS, and older > >>>>> drives don't have the necessary features to support LTFS. > >>>>> > >>>>> The commit message below outlines most of the changes. > >>>>> > >>>>> A few comments: > >>>>> > >>>>> 1. I'm planning to commit the XPT_DEV_ADVINFO changes separately. > >>>>> > >>>>> 2. The XML output is similar to what GEOM and CTL do. It would be nice to > >>>>> figure out how to put a standard schema on it so that standard tools > >>>>> could read it. I don't know how feasible that is, since I haven't > >>>>> time to dig into it. If anyone has suggestions on whether that is > >>>>> feasible or advisable, I'd appreciate feedback. > >>>>> > >>>>> 3. I have tested with a reasonable amount of tape hardware (see below for a > >>>>> list), but more testing and feedback would be good. > >>>>> > >>>>> 4. Standard 'mt status' output looks like this: > >>>>> > >>>>> # mt -f /dev/nsa3 status -v > >>>>> Drive: sa3: Serial Number: 101500520A > >>>>> --------------------------------- > >>>>> Mode Density Blocksize bpi Compression > >>>>> Current: 0x5a:LTO-6 variable 384607 enabled (0xff) > >>>>> --------------------------------- > >>>>> Current Driver State: at rest. > >>>>> --------------------------------- > >>>>> Partition: 0 Calc File Number: 0 Calc Record Number: 0 > >>>>> Residual: 0 Reported File Number: 0 Reported Record Number: 0 > >>>>> Flags: BOP > >>>>> > >>>>> 5. 'mt status -v' looks like this: > >>>>> > >>>>> # mt -f /dev/nsa3 status -v > >>>>> Drive: sa3: Serial Number: 101500520A > >>>>> --------------------------------- > >>>>> Mode Density Blocksize bpi Compression > >>>>> Current: 0x5a:LTO-6 variable 384607 enabled (0xff) > >>>>> --------------------------------- > >>>>> Current Driver State: at rest. > >>>>> --------------------------------- > >>>>> Partition: 0 Calc File Number: 0 Calc Record Number: 0 > >>>>> Residual: 0 Reported File Number: 0 Reported Record Number: 0 > >>>>> Flags: BOP > >>>>> --------------------------------- > >>>>> Tape I/O parameters: > >>>>> Maximum I/O size allowed by driver and controller (maxio): 1081344 bytes > >>>>> Maximum I/O size reported by controller (cpi_maxio): 5197824 bytes > >>>>> Maximum block size supported by tape drive and media (max_blk): 8388608 bytes > >>>>> Minimum block size supported by tape drive and media (min_blk): 1 bytes > >>>>> Block granularity supported by tape drive and media (blk_gran): 0 bytes > >>>>> Maximum possible I/O size (max_effective_iosize): 1081344 bytes > >>>> > >>>> > >>>> # mtx -f /dev/pass0 status > >>>> Storage Changer /dev/pass0:2 Drives, 10 Slots ( 0 Import/Export ) > >>>> Data Transfer Element 0:Empty > >>>> Data Transfer Element 1:Empty > >>>> Storage Element 1:Empty > >>>> Storage Element 2:Empty > >>>> Storage Element 3:Empty > >>>> Storage Element 4:Full :VolumeTag=FAI260 > >>>> Storage Element 5:Full :VolumeTag=FAI261 > >>>> Storage Element 6:Full :VolumeTag=FAI262 > >>>> Storage Element 7:Full :VolumeTag=FAI263 > >>>> Storage Element 8:Empty > >>>> Storage Element 9:Empty > >>>> Storage Element 10:Empty > >>>> > >>>> > >>>> It was at this point I spent the next 90 minute trying to get the tape > >>>> drive out of the tape library to free a stuck tape. Some of this was spent > >>>> attempting, and failing, to undo a stripped screw. I stopped the attempt when > >>>> I noticed the screw did need to be removed. :/ > >>> > >>> Thanks for all of the effort! Looks like it is paying off! :) > >>> > >>>> When I do this command, I hear the drive move a bit, to read the tape: > >>>> > >>>> # mt -f /dev/nsa1 status > >>>> Drive: sa1: Serial Number: CXA09S1340 > >>>> --------------------------------- > >>>> Mode Density Blocksize bpi Compression > >>>> Current: 0x1b:DLTapeIV(35GB) variable 85937 enabled (IDRC) > >>>> --------------------------------- > >>>> Current Driver State: at rest. > >>>> --------------------------------- > >>>> Partition: 0 Calc File Number: 0 Calc Record Number: 0 > >>>> Residual: 0 Reported File Number: -1 Reported Record Number: -1 > >>>> Flags: None > >>> > >>> Looks like the drive isn't reporting position information. It will still > >>> be useful to try it with Bacula, though. > >>> > >>>> # mt -f /dev/nsa1 ostatus > >>>> Mode Density Blocksize bpi Compression > >>>> Current: 0x1b:DLTapeIV(35GB) variable 85937 IDRC > >>>> ---------available modes--------- > >>>> 0: 0x1b:DLTapeIV(35GB) variable 85937 IDRC > >>>> 1: 0x1b:DLTapeIV(35GB) variable 85937 IDRC > >>>> 2: 0x1b:DLTapeIV(35GB) variable 85937 IDRC > >>>> 3: 0x1b:DLTapeIV(35GB) variable 85937 IDRC > >>>> --------------------------------- > >>>> Current Driver State: at rest. > >>>> --------------------------------- > >>>> File Number: 0 Record Number: 0 Residual Count 0 > >>>> > >>>> > >>>> After doing a very small tar -c and tar -x, I have: > >>>> > >>>> # mt -f /dev/nsa1 /dev/nsa1 ostatus > >>>> Mode Density Blocksize bpi Compression > >>>> Current: 0x1b:DLTapeIV(35GB) variable 85937 IDRC > >>>> ---------available modes--------- > >>>> 0: 0x1b:DLTapeIV(35GB) variable 85937 IDRC > >>>> 1: 0x1b:DLTapeIV(35GB) variable 85937 IDRC > >>>> 2: 0x1b:DLTapeIV(35GB) variable 85937 IDRC > >>>> 3: 0x1b:DLTapeIV(35GB) variable 85937 IDRC > >>>> --------------------------------- > >>>> Current Driver State: at rest. > >>>> --------------------------------- > >>>> File Number: 0 Record Number: 7 Residual Count 0 > >>> > >>> Woohoo! It works. > >>> > >>>> # mt -f /dev/nsa1 status -v > >>>> Drive: sa1: Serial Number: CXA09S1340 > >>>> --------------------------------- > >>>> Mode Density Blocksize bpi Compression > >>>> Current: 0x1b:DLTapeIV(35GB) variable 85937 enabled (IDRC) > >>>> --------------------------------- > >>>> Current Driver State: at rest. > >>>> --------------------------------- > >>>> Partition: 0 Calc File Number: 0 Calc Record Number: 7 > >>>> Residual: 0 Reported File Number: -1 Reported Record Number: -1 > >>>> Flags: None > >>>> --------------------------------- > >>>> Tape I/O parameters: > >>>> Maximum I/O size allowed by driver and controller (maxio): 65536 bytes > >>>> Maximum I/O size reported by controller (cpi_maxio): 0 bytes > >>>> Maximum block size supported by tape drive and media (max_blk): 16777214 bytes > >>>> Minimum block size supported by tape drive and media (min_blk): 2 bytes > >>>> Block granularity supported by tape drive and media (blk_gran): 0 bytes > >>>> Maximum possible I/O size (max_effective_iosize): 65536 bytes > >>>> > >>>> I may not get to testing Bacula today. > >>>> > >>>> Based on the above, is there any commands you'd like me to try? > >>> > >>> Aside from making sure things work okay with Bacula, that is probably > >>> sufficient. These drives won't support density reports or position > >>> information. > >>> > >>>> Read below regarding two tape drives > >>>> > >>>>> > >>>>> 6. Existing applications should work without changes. If not, please let > >>>>> me know. Hopefully they will move over time to the new interfaces. > >>>>> > >>>>> 7. There are lots of additional features that could be added later. > >>>>> Append-only support, encryption, more log pages, etc. > >>>>> > >>>>> 8. I have SCSI READ ATTRIBUTE changes for camcontrol(8) that will go in > >>>>> separately. These changes allow displaying the contents of the MAM > >>>>> (Medium Auxiliary Memory) chips on LTO, TS and other modern tape drives. > >>>>> These are good, and a future possible direction is adding attributes > >>>>> to the status XML from the sa(4) driver. > >>>>> > >>>>> ============ > >>>>> Significant upgrades to sa(4) and mt(1). > >>>>> > >>>>> The primary focus of these changes is to modernize FreeBSD's > >>>>> tape infrastructure so that we can take advantage of some of the > >>>>> features of modern tape drives and allow support for LTFS. > >>>>> > >>>>> Significant changes and new features include: > >>>>> > >>>>> o sa(4) driver status and parameter information is now exported via an > >>>>> XML structure. This will allow for changes and improvements later > >>>>> on that will not break userland applications. The old MTIOCGET > >>>>> status ioctl remains, so applications using the existing interface > >>>>> will not break. > >>>>> > >>>>> o 'mt status' now reports drive-reported tape position information > >>>>> as well as the previously available calculated tape position > >>>>> information. These numbers will be different at times, because > >>>>> the drive-reported block numbers are relative to BOP (Beginning > >>>>> of Partition), but the block numbers calculated previously via > >>>>> sa(4) (and still provided) are relative to the last filemark. > >>>>> Both numbers are now provided. 'mt status' now also shows the > >>>>> drive INQUIRY information, serial number and any position flags > >>>>> (BOP, EOT, etc.) provided with the tape position information. > >>>>> 'mt status -v' adds information on the maximum possible I/O size, > >>>>> and the underlying values used to calculate it. > >>>>> > >>>>> o The extra sa(4) /dev entries (/dev/saN.[0-3]) have been removed. > >>>> > >>>> How does this affect a tape library with more than one tape drive? > >>>> > >>>> [root@cuppy:~] # camcontrol amcontrol devlist > >>>> at scbus0 target 0 lun 0 (pass0,ch0) > >>>> at scbus0 target 2 lun 0 (sa1,pass2) > >>>> at scbus1 target 0 lun 0 (pass3,ada0) > >>>> at scbus2 target 0 lun 0 (pass4,ada1) > >>>> at scbus3 target 0 lun 0 (pass5,ses0) > >>>> > >>>> This system has two tapes drives and I can access them through the front panel but: > >>>> > >>>> # ls -l /dev/*sa* > >>>> crw-rw---- 1 root operator 0x65 Feb 28 22:04 /dev/esa1 > >>>> crw-rw---- 1 root operator 0x64 Mar 1 22:43 /dev/nsa1 > >>>> crw-rw---- 1 root operator 0x63 Feb 28 22:04 /dev/sa1 > >>>> crw-rw---- 1 root operator 0x62 Feb 28 22:04 /dev/sa1.ctl > >>>> > >>>> ... only one tape drives shows up. > >>> > >>> > >>> Hmm. The tape drive is listed as sa1, which implies that there may be an > >>> sa0 that was there previously or is in the process of probing. What does > >>> dmesg show? How about 'camcontrol devlist -v'? > >> > >> # camcontrol devlist -v > >> scbus0 on ahc0 bus 0: > >> at scbus0 target 0 lun 0 (pass0,ch0) > >> at scbus0 target 2 lun 0 (sa1,pass2) > >> <> at scbus0 target -1 lun ffffffff () > >> scbus1 on ahcich2 bus 0: > >> at scbus1 target 0 lun 0 (pass3,ada0) > >> <> at scbus1 target -1 lun ffffffff () > >> scbus2 on ahcich4 bus 0: > >> at scbus2 target 0 lun 0 (pass4,ada1) > >> <> at scbus2 target -1 lun ffffffff () > >> scbus3 on ahciem0 bus 0: > >> at scbus3 target 0 lun 0 (pass5,ses0) > >> <> at scbus3 target -1 lun ffffffff () > >> scbus-1 on xpt0 bus 0: > >> <> at scbus-1 target -1 lun ffffffff (xpt0) > >> > >> > >> BUT! > >> > >> # grep sa /var/run/dmesg.boot > >> VT-x: (disabled in BIOS) PAT,HLT,MTF,PAUSE,EPT,UG,VPID > >> module_register_init: MOD_LOAD (vesa, 0xffffffff80de3720, 0) error 19 > >> alc0: Using 1 MSIX message(s). > >> isab0: at device 31.0 on pci0 > >> isa0: on isab0 > >> orm0: at iomem 0xce800-0xcefff on isa0 > >> atkbdc0: at port 0x60,0x64 on isa0 > >> sa0 at ahc0 bus 0 scbus0 target 1 lun 0 > >> sa0: Removable Sequential Access SCSI-2 device > >> sa0: Serial Number CXA22S2338 > >> sa0: 10.000MB/s transfers (10.000MHz, offset 15) > >> sa0: quirks=0x100 > >> sa1 at ahc0 bus 0 scbus0 target 2 lun 0 > >> sa1: Removable Sequential Access SCSI-2 device > >> sa1: Serial Number CXA09S1340 > >> sa1: 10.000MB/s transfers (10.000MHz, offset 15) > >> sa1: quirks=0x100 > > > > If you run 'dmesg', you should have seen a message when it went away. Perhaps > > there will be something preceding it that will give us a clue about the > > problem. (Generally a selection timeout.) At least this does show that > > sa0 is at target 1, and so should not conflict with the library or sa1. > > Ahh: > > Trying to mount root from zfs:system/bootenv/FreeBSDHEad []... > sa0 at ahc0 bus 0 scbus0 target 1 lun 0 > sa0: s/n CXA22S2338 detached > (sa0:ahc0:0:1:0): Periph destroyed > arp: 10.55.0.60 moved from e4:ce:8f:46:f1:98 to 78:ca:39:fe:d6:b3 on em0 > arp: 10.55.0.60 moved from e4:ce:8f:46:f1:98 to 78:ca:39:fe:d6:b3 on em0 > arp: 10.55.0.60 moved from 78:ca:39:fe:d6:b3 to e4:ce:8f:46:f1:98 on em0 > (sa1:ahc0:0:2:0): 64512-byte tape record bigger than supplied buffer > (sa1:ahc0:0:2:0): 10240-byte tape record bigger than supplied buffer Okay. Well, no indication of what happened. Perhaps boot -v will show it, perhaps not. Ken -- Kenneth Merry ken@FreeBSD.ORG From owner-freebsd-current@FreeBSD.ORG Mon Mar 2 02:30:01 2015 Return-Path: Delivered-To: current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 9658754C; Mon, 2 Mar 2015 02:30:01 +0000 (UTC) Received: from mithlond.kdm.org (mithlond.kdm.org [70.56.43.85]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "A1-33714", Issuer "A1-33714" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 419149A8; Mon, 2 Mar 2015 02:30:00 +0000 (UTC) Received: from mithlond.kdm.org (localhost [127.0.0.1]) by mithlond.kdm.org (8.14.9/8.14.9) with ESMTP id t222Tw6M073775 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Sun, 1 Mar 2015 19:29:58 -0700 (MST) (envelope-from ken@mithlond.kdm.org) Received: (from ken@localhost) by mithlond.kdm.org (8.14.9/8.14.9/Submit) id t222Tvde073774; Sun, 1 Mar 2015 19:29:57 -0700 (MST) (envelope-from ken) Date: Sun, 1 Mar 2015 19:29:57 -0700 From: "Kenneth D. Merry" To: Dan Langille Subject: Re: sa(4) driver changes available for test Message-ID: <20150302022957.GB73433@mithlond.kdm.org> References: <20150214003232.GA63990@mithlond.kdm.org> <7CA52DF3-E073-4F50-BE4E-01C51CCDF2C7@langille.org> <20150217183645.GA30947@mithlond.kdm.org> <303C48AC-0DA8-447C-8DE7-0A1DD5DD7351@langille.org> <20150302003150.GB71528@mithlond.kdm.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.23 (2014-03-12) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (mithlond.kdm.org [127.0.0.1]); Sun, 01 Mar 2015 19:29:58 -0700 (MST) X-Spam-Status: No, score=-2.9 required=5.0 tests=ALL_TRUSTED,BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on mithlond.kdm.org Cc: current@freebsd.org, scsi@freebsd.org X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Mar 2015 02:30:01 -0000 On Sun, Mar 01, 2015 at 19:41:07 -0500, Dan Langille wrote: > > > On Mar 1, 2015, at 7:31 PM, Kenneth D. Merry wrote: > > > > On Sun, Mar 01, 2015 at 19:15:05 -0500, Dan Langille wrote: > >> > >>> On Feb 17, 2015, at 1:36 PM, Kenneth D. Merry wrote: > >>> > >>> On Sat, Feb 14, 2015 at 18:22:43 -0500, Dan Langille wrote: > >>>> > >>>>> On Feb 13, 2015, at 7:32 PM, Kenneth D. Merry wrote: > >>>>> > >>>>> > >>>>> I have a fairly large set of changes to the sa(4) driver and mt(1) driver > >>>>> that I'm planning to commit in the near future. > >>>>> > >>>>> A description of the changes is here and below in this message. > >>>>> > >>>>> If you have tape hardware and the inclination, I'd appreciate testing and > >>>>> feedback. > >>>> > >>>> I have a DLT 8000 and an SDLT 220. > >>>> > >>>> I don't have anything running current, but I have a spare machine which I could use for testing. > >>>> > >>>> Do you see any value is tests with that hardware? I'd be testing it via Bacula. > >>>> > >>>> disclosure: I'm the sysutils/bacula-* maintainer and a Bacula committer. > >>>> > >>> > >>> Actually, yes. Bacula is a bit tricky to configure, so your trying it out > >>> would be helpful if you have the time. > >>> > >>> In looking at the manuals for both the SDLT 220 and the DLT 8000, they both > >>> claim to support long position information for the SCSI READ POSITION > >>> command. > >>> > >>> You can see what I'm talking about by doing: > >>> > >>> mt eod > >>> mt status > >>> > >>> On my DDS-4 tape drive, this shows: > >>> > >>> # mt -f /dev/nsa3 status > >>> Drive: sa3: Serial Number: HJ00YWY > >>> --------------------------------- > >>> Mode Density Blocksize bpi Compression > >>> Current: 0x26:DDS-4 1024 bytes 97000 enabled (DCLZ) > >>> --------------------------------- > >>> Current Driver State: at rest. > >>> --------------------------------- > >>> Partition: 0 Calc File Number: -1 Calc Record Number: -1 > >>> Residual: 0 Reported File Number: -1 Reported Record Number: -1 > >>> Flags: None > >>> > >>> But on an LTO-5, which will give long position information, I get: > >>> > >>> [root@doc ~]# mt status > >>> Drive: sa0: > >>> --------------------------------- > >>> Mode Density Blocksize bpi Compression > >>> Current: 0x58:LTO-5 variable 384607 enabled (0x1) > >>> --------------------------------- > >>> Current Driver State: at rest. > >>> --------------------------------- > >>> Partition: 0 Calc File Number: 2 Calc Record Number: -1 > >>> Residual: 0 Reported File Number: 2 Reported Record Number: 32373 > >>> Flags: None > >>> > >>> That, in combination with the changes I made to the position information > >>> code in the driver, mean that even the old MTIOCGET ioctl should return an > >>> accurate file number at end of data. e.g., on the LTO-5: > >>> > >>> [root@doc ~]# mt ostatus > >>> Mode Density Blocksize bpi Compression > >>> Current: 0x58:LTO-5 variable 384607 0x1 > >>> ---------available modes--------- > >>> 0: 0x58:LTO-5 variable 384607 0x1 > >>> 1: 0x58:LTO-5 variable 384607 0x1 > >>> 2: 0x58:LTO-5 variable 384607 0x1 > >>> 3: 0x58:LTO-5 variable 384607 0x1 > >>> --------------------------------- > >>> Current Driver State: at rest. > >>> --------------------------------- > >>> File Number: 2 Record Number: -1 Residual Count -1 > >>> > >>> So the thing to try, in addition to just making sure that Bacula continues > >>> to work properly, is to try setting this for the tape drive in > >>> bacula-sd.conf: > >>> > >>> Hardware End of Medium = yes > >>> > >>> It looks like the Bacula tape program (btape) has a test mode, and it would > >>> be good to run through the tests on one of the tape drives and see whether > >>> they work, and whether the results are different before and after the > >>> changes. I'm not sure how to enable the test mode. > >> > >> I have this in /usr/local/etc/bacula/bacula-sd.conf > >> > >> Device { > >> Name = DLT > >> Description = "QUANTUM DLT7000 1624" > >> Media Type = DLT > >> Archive Device = /dev/nsa1 > >> > >> Autochanger = YES > >> Drive Index = 0 > >> > >> Offline On Unmount = no > >> Hardware End of Medium = yes > >> BSF at EOM = yes > >> Backward Space Record = no > >> Fast Forward Space File = no > >> TWO EOF = yes > >> } > >> > >> FYI, http://www.freebsddiary.org/digital-tl891.php (from 2006) has a btape test on this same model. > >> > >> Here's the test I ran tonight: > >> > >> [root@cuppy:/usr/home/dan] # btape -c /usr/local/etc/bacula/bacula-sd.conf /dev/nsa1 > >> Tape block granularity is 1024 bytes. > >> btape: butil.c:287-0 Using device: "/dev/nsa1" for writing. > >> btape: btape.c:469-0 open device "DLT" (/dev/nsa1): OK > >> *test > >> > >> === Write, rewind, and re-read test === > >> > >> I'm going to write 10000 records and an EOF > >> then write 10000 records and an EOF, then rewind, > >> and re-read the data to verify that it is correct. > >> > >> This is an *essential* feature ... > >> > >> btape: btape.c:1152-0 Wrote 10000 blocks of 64412 bytes. > >> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) > >> btape: btape.c:1168-0 Wrote 10000 blocks of 64412 bytes. > >> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) > >> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) > >> btape: btape.c:1210-0 Rewind OK. > >> 10000 blocks re-read correctly. > >> Got EOF on tape. > >> 10000 blocks re-read correctly. > >> === Test Succeeded. End Write, rewind, and re-read test === > >> > >> btape: btape.c:1277-0 Block position test > >> btape: btape.c:1289-0 Rewind OK. > >> Reposition to file:block 0:4 > >> Block 5 re-read correctly. > >> Reposition to file:block 0:200 > >> Block 201 re-read correctly. > >> Reposition to file:block 0:9999 > >> Block 10000 re-read correctly. > >> Reposition to file:block 1:0 > >> Block 10001 re-read correctly. > >> Reposition to file:block 1:600 > >> Block 10601 re-read correctly. > >> Reposition to file:block 1:9999 > >> Block 20000 re-read correctly. > >> === Test Succeeded. End Write, rewind, and re-read test === > >> > >> > >> > >> === Append files test === > >> > >> This test is essential to Bacula. > >> > >> I'm going to write one record in file 0, > >> two records in file 1, > >> and three records in file 2 > >> > >> btape: btape.c:574-0 Rewound "DLT" (/dev/nsa1) > >> btape: btape.c:1907-0 Wrote one record of 64412 bytes. > >> btape: btape.c:1909-0 Wrote block to device. > >> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) > >> btape: btape.c:1907-0 Wrote one record of 64412 bytes. > >> btape: btape.c:1909-0 Wrote block to device. > >> btape: btape.c:1907-0 Wrote one record of 64412 bytes. > >> btape: btape.c:1909-0 Wrote block to device. > >> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) > >> btape: btape.c:1907-0 Wrote one record of 64412 bytes. > >> btape: btape.c:1909-0 Wrote block to device. > >> btape: btape.c:1907-0 Wrote one record of 64412 bytes. > >> btape: btape.c:1909-0 Wrote block to device. > >> btape: btape.c:1907-0 Wrote one record of 64412 bytes. > >> btape: btape.c:1909-0 Wrote block to device. > >> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) > >> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) > >> btape: btape.c:469-0 open device "DLT" (/dev/nsa1): OK > >> btape: btape.c:574-0 Rewound "DLT" (/dev/nsa1) > >> btape: btape.c:1420-0 Now moving to end of medium. > > > > This is the critical piece. The test moves the tape to the end of the > > medium. With hardware position information, you can tell what filemark > > you're on. Without it, you can't. > > > >> btape: btape.c:622-0 tape_dev.c:345 ioctl MTIOCGET error on "DLT" (/dev/nsa1). ERR=No error: 0. > >> We should be in file 3. I am at file 0. This is NOT correct!!!! > >> > >> Append test failed. Attempting again. > >> Setting "Hardware End of Medium = no > >> and "Fast Forward Space File = no > >> and retrying append test. > > > > This is not surprsing, given that the drive doesn't support long read > > position data. (It's a SCSI-2 device.) So that means that Bacula will > > need to do it manually. > > Yes, I have nothing newer than SCSI-2. Even my SDLT is SCSI-2 but that > tape library is hooked up to a different computer and was doing backups today. So, here is one thing that we can try to see whether these drives support long position information, even though they only claim to be SCSI-2. If they do, we can potentially add a quirk (or autodetection) to enable it. The code currently doesn't bother asking drives that claim to be SCSI-2 for long position information. (Because that feature was added in the SSC spec, which came after SCSI-2.) Issue a READ POSITION with the short form specified: camcontrol cmd sa1 -v -c "34 0 0 0 0 0 0 0 0 0" -i 20 - |hd Issue a READ POSITION with the vendor-specific block numbers: camcontrol cmd sa1 -v -c "34 1 0 0 0 0 0 0 0 0" -i 20 - |hd Issue a READ POSITION with the long form data: camcontrol cmd sa1 -v -c "34 6 0 0 0 0 0 0 0 0" -i 32 - |hd If it supports the last one, then I can put a quirk (or autodetection) in the driver and Bacula will get the hardware filemarks. You should try this on your SDLT as well. It may well support it. Google didn't quickly produce a SCSI manual for the DEC drive, but the Quantum SDLT manual indicates that it supports long position data, despite identifying itself as a SCSI-2 drive. > > === Append files test === > >> > >> This test is essential to Bacula. > >> > >> I'm going to write one record in file 0, > >> two records in file 1, > >> and three records in file 2 > >> > >> btape: btape.c:574-0 Rewound "DLT" (/dev/nsa1) > >> btape: btape.c:1907-0 Wrote one record of 64412 bytes. > >> btape: btape.c:1909-0 Wrote block to device. > >> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) > >> btape: btape.c:1907-0 Wrote one record of 64412 bytes. > >> btape: btape.c:1909-0 Wrote block to device. > >> btape: btape.c:1907-0 Wrote one record of 64412 bytes. > >> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) > >> btape: btape.c:1907-0 Wrote one record of 64412 bytes. > >> btape: btape.c:1909-0 Wrote block to device. > >> btape: btape.c:1907-0 Wrote one record of 64412 bytes. > >> btape: btape.c:1909-0 Wrote block to device. > >> btape: btape.c:1907-0 Wrote one record of 64412 bytes. > >> btape: btape.c:1909-0 Wrote block to device. > >> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) > >> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) > >> btape: btape.c:469-0 open device "DLT" (/dev/nsa1): OK > >> btape: btape.c:574-0 Rewound "DLT" (/dev/nsa1) > >> btape: btape.c:1420-0 Now moving to end of medium. > >> btape: btape.c:625-0 Moved to end of medium. > >> We should be in file 3. I am at file 3. This is correct! > >> > >> Now the important part, I am going to attempt to append to the tape. > >> > >> btape: btape.c:1907-0 Wrote one record of 64412 bytes. > >> btape: btape.c:1909-0 Wrote block to device. > >> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) > >> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) > >> btape: btape.c:574-0 Rewound "DLT" (/dev/nsa1) > >> Done appending, there should be no I/O errors > >> > >> Doing Bacula scan of blocks: > >> 1 block of 64448 bytes in file 1 > >> End of File mark. > >> 2 blocks of 64448 bytes in file 2 > >> End of File mark. > >> 3 blocks of 64448 bytes in file 3 > >> End of File mark. > >> 1 block of 64448 bytes in file 4 > >> End of File mark. > >> Total files=4, blocks=7, bytes = 451,136 > >> End scanning the tape. > >> We should be in file 4. I am at file 4. This is correct! > >> > >> > >> It looks like the test worked this time, please add: > >> > >> Hardware End of Medium = No > >> > >> Fast Forward Space File = No > >> to your Device resource in the Storage conf file. > >> > >> The above Bacula scan should have output identical to what follows. > >> Please double check it ... > >> === Sample correct output === > >> 1 block of 64448 bytes in file 1 > >> End of File mark. > >> 2 blocks of 64448 bytes in file 2 > >> End of File mark. > >> 3 blocks of 64448 bytes in file 3 > >> End of File mark. > >> 1 block of 64448 bytes in file 4 > >> End of File mark. > >> Total files=4, blocks=7, bytes = 451,136 > >> === End sample correct output === > >> > >> If the above scan output is not identical to the > >> sample output, you MUST correct the problem > >> or Bacula will not be able to write multiple Jobs to > >> the tape. > >> > >> Skipping read backwards test because BSR turned off. > >> > >> > >> === Forward space files test === > >> > >> This test is essential to Bacula. > >> > >> I'm going to write five files then test forward spacing > >> > >> btape: btape.c:574-0 Rewound "DLT" (/dev/nsa1) > >> btape: btape.c:1907-0 Wrote one record of 64412 bytes. > >> btape: btape.c:1909-0 Wrote block to device. > >> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) > >> btape: btape.c:1907-0 Wrote one record of 64412 bytes. > >> btape: btape.c:1909-0 Wrote block to device. > >> btape: btape.c:1907-0 Wrote one record of 64412 bytes. > >> btape: btape.c:1909-0 Wrote block to device. > >> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) > >> btape: btape.c:1907-0 Wrote one record of 64412 bytes. > >> btape: btape.c:1909-0 Wrote block to device. > >> btape: btape.c:1907-0 Wrote one record of 64412 bytes. > >> btape: btape.c:1909-0 Wrote block to device. > >> btape: btape.c:1907-0 Wrote one record of 64412 bytes. > >> btape: btape.c:1909-0 Wrote block to device. > >> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) > >> btape: btape.c:1907-0 Wrote one record of 64412 bytes. > >> btape: btape.c:1909-0 Wrote block to device. > >> btape: btape.c:1907-0 Wrote one record of 64412 bytes. > >> btape: btape.c:1909-0 Wrote block to device. > >> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) > >> btape: btape.c:1907-0 Wrote one record of 64412 bytes. > >> btape: btape.c:1909-0 Wrote block to device. > >> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) > >> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) > >> btape: btape.c:574-0 Rewound "DLT" (/dev/nsa1) > >> btape: btape.c:1634-0 Now forward spacing 1 file. > >> We should be in file 1. I am at file 1. This is correct! > >> btape: btape.c:1646-0 Now forward spacing 2 files. > >> We should be in file 3. I am at file 3. This is correct! > >> btape: btape.c:574-0 Rewound "DLT" (/dev/nsa1) > >> btape: btape.c:1659-0 Now forward spacing 4 files. > >> We should be in file 4. I am at file 4. This is correct! > >> > >> btape: btape.c:1677-0 Now forward spacing 1 more file. > >> We should be in file 5. I am at file 5. This is correct! > >> > >> === End Forward space files test === > >> > >> > >> Ah, I see you have an autochanger configured. > >> To test the autochanger you must have a blank tape > >> that I can write on in Slot 1. > >> > >> Do you wish to continue with the Autochanger test? (y/n): y > >> > >> > >> === Autochanger test === > >> > >> 3301 Issuing autochanger "loaded" command. > >> Nothing loaded in the drive. OK. > >> 3303 Issuing autochanger "load 1 0" command. > >> 3303 Autochanger "load 1 0" status is OK. > >> btape: btape.c:469-0 open device "DLT" (/dev/nsa1): OK > >> btape: btape.c:1564-0 Rewound "DLT" (/dev/nsa1) > >> btape: btape.c:1571-0 Wrote EOF to "DLT" (/dev/nsa1) > >> > >> The test autochanger worked!! > > > > Great, thanks for running the test! Looks like things are working as well > > as they were before. > > > >>> > >>>> I'll let the other Bacula devs know about this. They deal with the hardware. I work on PostgreSQL. > >>>> > >>> > >>> Thanks! If there are additional features they would like out of the tape > >>> driver, I'm happy to talk about it. (Or help if they'd like to use the new > >>> status reporting ioctl, MTIOCEXTGET or any of the other new ioctls.) > >> > >> Errors are interesting to me. Especially corrected errors. They are a good indicator of tape quality. > >> > > > > Yes. At least on modern drives, there is a good bit available in the log > > pages. You can try seeing what logs your drive supports by installing the > > sg3_utils package and running 'sg_logs sa1 --all'. > > > > Given the large amount of data available on some drives, and the difficulty > > distilling it down to a clear good/bad, I probably won't stick that in the > > 'mt status' output. > > > > But you can certainly take a look at it and have an idea of whether your > > particular tape/drive are experiencing issues. > > That's a lot of output: https://gist.github.com/dlangille/0e15a7fbf7acab56fd32 That is a lot. > I will run some Bacula jobs soon. I'm still setting up config files. Thanks for all the testing, I really appreciate it! Ken -- Kenneth Merry ken@FreeBSD.ORG From owner-freebsd-current@FreeBSD.ORG Mon Mar 2 03:37:00 2015 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id CD86FC7D; Mon, 2 Mar 2015 03:37:00 +0000 (UTC) Received: from mail-wg0-x22c.google.com (mail-wg0-x22c.google.com [IPv6:2a00:1450:400c:c00::22c]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 59D42DE; Mon, 2 Mar 2015 03:37:00 +0000 (UTC) Received: by wghl18 with SMTP id l18so30828784wgh.7; Sun, 01 Mar 2015 19:36:58 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:content-type:content-transfer-encoding; bh=XdNy4jDv0dKu2pz4Pq/46AR8fF0XTjr9Gczr1GTNKNM=; b=kfh0OnGLnsmo0wVC2TgMvUw7aEVeOomL6lsCqNS9sGoV75JY2AMNOGaJHDAYSfefM7 sownw/qCLl2iY8QsWF7LqOSkK7tMs2LOTsWJY9lNQEG9yeLVwabbssfiMkbwPZkldG3O QCqj46TT0K37MdFJOvh6H0ranRI4uw5UodsJnhWFPZJo/8rfKbkhJxLDDyhnYXc9S6wE 1dvYmbYcwmIWmuOcFAXvR4tQrxM8L/UGsmfdNjmNr/8MBe7xhfXeUheuzTuO4OhfMgDQ ZWPGg5xzE4ukC+ByDhuQC+sHwea5gVSDiBY5n8D5orjvAiCZFTI8aQPvLIbmQ+uD4urh QqrQ== X-Received: by 10.180.96.232 with SMTP id dv8mr31587200wib.31.1425267418784; Sun, 01 Mar 2015 19:36:58 -0800 (PST) Received: from ketas-laptop.mydomain (65-38-190-90.dyn.estpak.ee. [90.190.38.65]) by mx.google.com with ESMTPSA id a13sm17288643wjx.30.2015.03.01.19.36.57 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Sun, 01 Mar 2015 19:36:57 -0800 (PST) Sender: Sulev-Madis Silber Message-ID: <54F3DA8C.9010602@hot.ee> Date: Mon, 02 Mar 2015 05:35:40 +0200 From: "Sulev-Madis Silber (ketas)" User-Agent: Mozilla/5.0 (X11; FreeBSD i386; rv:15.0) Gecko/20120912 Thunderbird/15.0.1 MIME-Version: 1.0 To: Arseny Nasokin Subject: Re: Massive libxo-zation that breaks everything References: <54F31510.7050607@hot.ee> <54F34B6E.2040809@astrodoggroup.com> <54F35F29.4000603@astrodoggroup.com> <54F36431.30506@freebsd.org> In-Reply-To: X-TagToolbar-Keys: D20150302053539998 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Cc: freebsd-current , Allan Jude X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Mar 2015 03:37:01 -0000 How about we allow JSON input on those utils too... Then we get into full-blown hell faster. Hmm... I would like to talk with system using JSON. JSON would be in utils that are or at least function similarly to rm, mv, ls, find, mount, zpool, zfs, geom, mdconfig, tar, df, netstat, ifconfig... (or maybe even talk JSON directly to the kernel?!). How to have all that goodness, while at same time not having extra library dependency? Without that library the system wouldn't work at all (and there is no fallback mode for "manual operation"). Fragile library that requires you to have worse-than-bad-Perl-looking (funnily I write lot of Perl myself) parts everywhere in your code. And requires huge set of tests to verify correct operation. Hope that you never see things like this: > df Shared object "libxo.so.0" not found, required by "df" Or this: > netstat -nhbdWi Name Mtu Network Address Ipkts Ierrs Idrop Ibytes Opkts Oerrs Obytes Coll Drop smc0 %6s 1.5K 52:54:00:12:34:56 %8s 4.4M %5s 0 %5s 0 %10s 290M %8s 3.3M %5s 0 %10s 756M %5s 0 %5s 11 smc0 - fe80::5054:ff fe80::5054:ff:fe1 %8s 8.7K - - %10s 592K %8s 17K - %10s 1.0M - - smc0 - 2001:ad0:91f: 2001:ad0:91f:0:50 %8s 4.4M - - %10s 225M %8s 3.3M - %10s 709M - - smc0 - 10.0.0.0/24 10.0.0.48 %8s 4.0K - - %10s 504K %8s 3.2K - %10s 233K - - lo0 %6s 16K %8s 272 %5s 0 %5s 0 %10s 24K %8s 272 %5s 0 %10s 24K %5s 0 %5s 0 lo0 - ::1/128 ::1 %8s 136 - - %10s 16K %8s 136 - %10s 16K - - lo0 - fe80::1%lo0/6 fe80::1%lo0 %8s 0 - - %10s 0 %8s 7 - %10s 963 - - lo0 - 127.0.0.0/8 127.0.0.1 %8s 136 - - %10s 8.6K %8s 136 - %10s 8.6K - - Or, at least you only see it (occasionally) in CURRENT. ( And, all current libxo issues seem to be fixed, for now... ) From owner-freebsd-current@FreeBSD.ORG Mon Mar 2 04:43:23 2015 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id BF58E363; Mon, 2 Mar 2015 04:43:23 +0000 (UTC) Received: from elvis.mu.org (elvis.mu.org [IPv6:2001:470:1f05:b76::196]) by mx1.freebsd.org (Postfix) with ESMTP id A88DD8E0; Mon, 2 Mar 2015 04:43:23 +0000 (UTC) Received: from AlfredMacbookAir.local (hudsonhotel209.h.subnet.rcn.com [207.237.151.136]) by elvis.mu.org (Postfix) with ESMTPSA id 3242C341F8B2; Sun, 1 Mar 2015 20:43:21 -0800 (PST) Message-ID: <54F3EB48.1030807@mu.org> Date: Sun, 01 Mar 2015 23:47:04 -0500 From: Alfred Perlstein User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:31.0) Gecko/20100101 Thunderbird/31.5.0 MIME-Version: 1.0 To: Rui Paulo , David Chisnall Subject: Re: Massive libxo-zation that breaks everything References: <54F31510.7050607@hot.ee> <54F34B6E.2040809@astrodoggroup.com> <54F35F29.4000603@astrodoggroup.com> <75C49F53-C675-4712-A446-370025EED037@me.com> In-Reply-To: <75C49F53-C675-4712-A446-370025EED037@me.com> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit Cc: Harrison Grundy , freebsd-current@freebsd.org X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Mar 2015 04:43:23 -0000 On 3/1/15 4:29 PM, Rui Paulo wrote: > On Mar 1, 2015, at 11:11, David Chisnall wrote: >> How would it be in a port? It involves modifying core utilities (some of which, like ifconfig, rely on kernel APIs that change between releases) to emit structured output. Maintaining two copies of each utility, one in the base system with plain-text output only and another in ports with XML/JSON output would be very painful. > It would work fine if we had *libraries* for ifconfig/netstat/route/etc. Obviously that's not the case and no one has stepped up to implement them. I've also seen FreeBSD committers expressing their distaste for libraries for "trivial" command line utilities, which implies they are unaware of another world beyond the CLI. :-) +1. Next step is making those utils into libraries. Should be easy, just make all globals into TLS and don't munge stdin/stdout in dumb ways and we're there. Glad you are a big thinker Rui. :) -Alfred From owner-freebsd-current@FreeBSD.ORG Mon Mar 2 04:48:10 2015 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D7CE149F for ; Mon, 2 Mar 2015 04:48:10 +0000 (UTC) Received: from elvis.mu.org (elvis.mu.org [IPv6:2001:470:1f05:b76::196]) by mx1.freebsd.org (Postfix) with ESMTP id C1F0E909 for ; Mon, 2 Mar 2015 04:48:10 +0000 (UTC) Received: from AlfredMacbookAir.local (hudsonhotel209.h.subnet.rcn.com [207.237.151.136]) by elvis.mu.org (Postfix) with ESMTPSA id A0EA8341F90E; Sun, 1 Mar 2015 20:48:09 -0800 (PST) Message-ID: <54F3EC68.3020405@mu.org> Date: Sun, 01 Mar 2015 23:51:52 -0500 From: Alfred Perlstein User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:31.0) Gecko/20100101 Thunderbird/31.5.0 MIME-Version: 1.0 To: Harrison Grundy , freebsd-current@freebsd.org Subject: Re: Massive libxo-zation that breaks everything References: <54F31510.7050607@hot.ee> <54F34B6E.2040809@astrodoggroup.com> <54F35F29.4000603@astrodoggroup.com> <54F38B52.6090209@astrodoggroup.com> In-Reply-To: <54F38B52.6090209@astrodoggroup.com> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Mar 2015 04:48:10 -0000 On 3/1/15 4:57 PM, Harrison Grundy wrote: > I like the idea behind this... where I'm running into difficulty is why > these bits of functionality need to be combined. What someone does with > ifconfig on the command line, versus what someone wants to know about > their network interfaces in an XML dump may be very distinct bits of > information. (For instance, an XML dump of network information may want > to incorporate data from netstat, route, and ifconfig, while ifconfig is > really only designed to deal with the interfaces themselves.) Part of this is putting the burden of machine readable output into the utilities themselves. Meaning that people have complained against libxo because "well you can get the same information by combining data points X, Y and Z from sysctl" whereas ifconfig just does that for you. So if we split the code then those interfaces would wither and die from neglect because honestly very few core freebsd devs care about XML and JSON. So in order to prevent bitotting and make this a "feature" of FreeBSD, we decided to put it into the base utils so that we would be sure to carry the burden and keep it working. > I know it involves some notable reworking, but would there be any > interest in seeing what "libifying" a handful of binaries and splitting > the libxo and CLI functionality looks like? It seems like this may give > FreeBSD the best of both worlds, so the XML output isn't logically > limited by how the CLI needs to work, and the CLI won't have to deal > with the complexity XML output can require. > > I'd be happy to get this set up for a few binaries so people can see how > it might work and what it could look like if there's any interest in > going that route. Please do. That would be great. Please make sure they are thread safe. :) -Alfred From owner-freebsd-current@FreeBSD.ORG Mon Mar 2 08:37:09 2015 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 2A7DBB87; Mon, 2 Mar 2015 08:37:09 +0000 (UTC) Received: from vps1.elischer.org (vps1.elischer.org [204.109.63.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "vps1.elischer.org", Issuer "CA Cert Signing Authority" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id EF7C5F5F; Mon, 2 Mar 2015 08:37:08 +0000 (UTC) Received: from Julian-MBP3.local (50-196-156-133-static.hfc.comcastbusiness.net [50.196.156.133]) (authenticated bits=0) by vps1.elischer.org (8.14.9/8.14.9) with ESMTP id t228b7GM003043 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO); Mon, 2 Mar 2015 00:37:07 -0800 (PST) (envelope-from julian@freebsd.org) Message-ID: <54F4212E.9030702@freebsd.org> Date: Mon, 02 Mar 2015 00:37:02 -0800 From: Julian Elischer User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:31.0) Gecko/20100101 Thunderbird/31.5.0 MIME-Version: 1.0 To: Benjamin Kaduk Subject: Re: FreeBSD FUSE calls truncate() on read-only files References: <6676D082-5C06-4E59-B22C-5C00D1FD229F@netapp.com> <54F018DD.30800@freebsd.org> In-Reply-To: Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-current@freebsd.org X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Mar 2015 08:37:09 -0000 On 2/27/15 7:32 AM, Benjamin Kaduk wrote: > On Fri, 27 Feb 2015, Julian Elischer wrote: > >> for example it caches information when it shouldn't, even from 'dynamic' file >> systems >> We had to change the code to disable it as our data is synthetic and might >> change between reads. >> fstat info is also cached and confused our apps mightily. > You are of course planning to file bug reports about these issues, I > presume? > > -Ben > > First I hope to actually understand the problems. From owner-freebsd-current@FreeBSD.ORG Mon Mar 2 08:53:31 2015 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 49535E15; Mon, 2 Mar 2015 08:53:31 +0000 (UTC) Received: from mail-pa0-x22c.google.com (mail-pa0-x22c.google.com [IPv6:2607:f8b0:400e:c03::22c]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 14908198; Mon, 2 Mar 2015 08:53:31 +0000 (UTC) Received: by padfb1 with SMTP id fb1so3332361pad.7; Mon, 02 Mar 2015 00:53:30 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; bh=muWHXxLBAfbPEnDiqxHLw1UI8q6dCOtrQeGgiqFdcno=; b=rIkLibTiVKCa+pbZls46RQ/uR/fOJLXSR6cpS4QMrBQlC846Kw7RKMt6wqSDfyOanX YXvnWP4XYe6TjLF9O9d+DD5PrztDNy5HNj3hjFLkzJL6ghni7rb8RzSNEutCg9WQbBEs Wv9MBw7I//8LQF4XlzuRHwOmKYI9tMRK/ceAwmUQ1oofDECu0DBzjHGrkkykJh7yjYbR Azr7a+CiqKrZMFMJ+3R2r0AZMoXuai3d3M7osF3wdY8Z3pTdvWYaUyAhD8IF20LC4xyy WDVRgqC73tjoC7PAnu7BgW7SJIOWsOehtqGqXbKViTgbtUJpgWU9IILnEPY/5hgGKovb fv4Q== X-Received: by 10.70.128.15 with SMTP id nk15mr44552452pdb.121.1425286410639; Mon, 02 Mar 2015 00:53:30 -0800 (PST) MIME-Version: 1.0 Received: by 10.66.237.39 with HTTP; Mon, 2 Mar 2015 00:53:10 -0800 (PST) In-Reply-To: <54F3C134.5080207@freebsd.org> References: <54F31510.7050607@hot.ee> <54F34B6E.2040809@astrodoggroup.com> <54F35F29.4000603@astrodoggroup.com> <54F36431.30506@freebsd.org> <54F3C134.5080207@freebsd.org> From: Arseny Nasokin Date: Mon, 2 Mar 2015 12:53:10 +0400 Message-ID: Subject: Re: Massive libxo-zation that breaks everything To: Allan Jude Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 Cc: freebsd-current X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Mar 2015 08:53:31 -0000 On 2 March 2015 at 04:47, Allan Jude wrote: > On 2015-03-01 19:20, Arseny Nasokin wrote: > > On 1 March 2015 at 22:10, Allan Jude wrote: > > > >> On 2015-03-01 13:49, Harrison Grundy wrote: > >>> Thanks! > >>> > >>> That does seem useful, but I'm not sure I see the reasoning behind > >>> putting into base, over a port or package, since processing XML in base > >>> is a pain, and it can't serve up JSON or HTML without additional > >>> utilities anyway. > >>> > >>> (If I'm reviving a long-settled thing, let me know and I'll drop it. > I'm > >>> trying to understand the use case for this.) > >>> > >>> --- Harrison > >>> > >>> On 03/01/15 10:31, Craig Rodrigues wrote: > >>>> On Sun, Mar 1, 2015 at 9:25 AM, Harrison Grundy < > >>>> harrison.grundy@astrodoggroup.com> wrote: > >>>> > >>>>> > >>>>> > >>>>> If someone could summarize what this is, I'd greatly appreciate it. > >>>>> > >>>> > >>>> > https://lists.freebsd.org/pipermail/freebsd-arch/2014-July/015633.html > >>>> _______________________________________________ > >>>> freebsd-current@freebsd.org mailing list > >>>> http://lists.freebsd.org/mailman/listinfo/freebsd-current > >>>> To unsubscribe, send any mail to " > >> freebsd-current-unsubscribe@freebsd.org" > >>>> > >>> _______________________________________________ > >>> freebsd-current@freebsd.org mailing list > >>> http://lists.freebsd.org/mailman/listinfo/freebsd-current > >>> To unsubscribe, send any mail to " > >> freebsd-current-unsubscribe@freebsd.org" > >>> > >> > >> I think you're missing the important bit here. > >> > >> This isn't about adding a parser for anything, this is about making the > >> tools in base, like netstat, wc, uptime, etc, output in JSON or XML, so > >> you can use the data programmatically. > >> > >> Your scripts no longer have to rely on awk/sed/grep magic to get a > >> specific bit of information out of the uptime command, the command can > >> just output the data in a structured machine readable format. > >> > >> I am not sure how you can put netstat into the ports tree. > >> > >> > >> -- > >> Allan Jude > >> > >> > > Hi, > > > > Do we have command-line tools in base which work with XML/JSON from stdin > > or file? > > > > -- Eir Nym > > _______________________________________________ > > freebsd-current@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-current > > To unsubscribe, send any mail to " > freebsd-current-unsubscribe@freebsd.org" > > > > Not really, although there are libraries for such (bsdxml, libucl which > can read JSON) in base. > > I am working on a tool that can do some of this: > > https://github.com/allanjude/uclcmd/ > > -- > Allan Jude > > Allan, What do you think about tools like plutil(1) or defaults(1) from Mac OS X. Plists are dict-like structures (XML and JSON too). -- Eir Nym From owner-freebsd-current@FreeBSD.ORG Mon Mar 2 09:14:32 2015 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A08E24BF for ; Mon, 2 Mar 2015 09:14:32 +0000 (UTC) Received: from vps1.elischer.org (vps1.elischer.org [204.109.63.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "vps1.elischer.org", Issuer "CA Cert Signing Authority" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 709DF3E3 for ; Mon, 2 Mar 2015 09:14:32 +0000 (UTC) Received: from Julian-MBP3.local (50-196-156-133-static.hfc.comcastbusiness.net [50.196.156.133]) (authenticated bits=0) by vps1.elischer.org (8.14.9/8.14.9) with ESMTP id t229ETkJ003286 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO); Mon, 2 Mar 2015 01:14:30 -0800 (PST) (envelope-from julian@freebsd.org) Message-ID: <54F429EF.5050400@freebsd.org> Date: Mon, 02 Mar 2015 01:14:23 -0800 From: Julian Elischer User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:31.0) Gecko/20100101 Thunderbird/31.5.0 MIME-Version: 1.0 To: Harrison Grundy , freebsd-current@freebsd.org Subject: Re: Massive libxo-zation that breaks everything References: <54F31510.7050607@hot.ee> <54F34B6E.2040809@astrodoggroup.com> <54F35F29.4000603@astrodoggroup.com> In-Reply-To: <54F35F29.4000603@astrodoggroup.com> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Mar 2015 09:14:32 -0000 On 3/1/15 10:49 AM, Harrison Grundy wrote: > Thanks! > > That does seem useful, but I'm not sure I see the reasoning behind > putting into base, over a port or package, since processing XML in base > is a pain, and it can't serve up JSON or HTML without additional > utilities anyway. > > (If I'm reviving a long-settled thing, let me know and I'll drop it. I'm > trying to understand the use case for this.) To me it would almost seem more useful to have a programmable filter for which you could produce parse grammars to parse the output of various programs.. thus ifconfig -a | xmlize -g ifconfig | your-favourite-xml-parser with a set of grammars in /usr/share/xmlize/ then we could use it for out-of-tree programs as well if we wrote grammars for them.. The sentiment of machine-readable output is nice, but I think it's slightly off target. we shouldn't have to change all out utilities, and it isn't going to help at all with 3rd party apps, e.g. samba stuff. A generally easy to program output grammar parser would be truely useful. and not just for FreeBSD. I've been watching with an uncomfortable feeling, but it's taken me a while to put my finger on what it was.. > > --- Harrison > > > From owner-freebsd-current@FreeBSD.ORG Mon Mar 2 09:16:57 2015 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 4072B7CC; Mon, 2 Mar 2015 09:16:57 +0000 (UTC) Received: from vps1.elischer.org (vps1.elischer.org [204.109.63.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "vps1.elischer.org", Issuer "CA Cert Signing Authority" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id E967F5EE; Mon, 2 Mar 2015 09:16:56 +0000 (UTC) Received: from Julian-MBP3.local (50-196-156-133-static.hfc.comcastbusiness.net [50.196.156.133]) (authenticated bits=0) by vps1.elischer.org (8.14.9/8.14.9) with ESMTP id t229Gt0G003317 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO); Mon, 2 Mar 2015 01:16:55 -0800 (PST) (envelope-from julian@freebsd.org) Message-ID: <54F42A82.1020308@freebsd.org> Date: Mon, 02 Mar 2015 01:16:50 -0800 From: Julian Elischer User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:31.0) Gecko/20100101 Thunderbird/31.5.0 MIME-Version: 1.0 To: Allan Jude , freebsd-current@freebsd.org Subject: Re: Massive libxo-zation that breaks everything References: <54F31510.7050607@hot.ee> <54F34B6E.2040809@astrodoggroup.com> <54F35F29.4000603@astrodoggroup.com> <54F36431.30506@freebsd.org> In-Reply-To: <54F36431.30506@freebsd.org> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Mar 2015 09:16:57 -0000 On 3/1/15 11:10 AM, Allan Jude wrote: > On 2015-03-01 13:49, Harrison Grundy wrote: >> Thanks! >> >> That does seem useful, but I'm not sure I see the reasoning behind >> putting into base, over a port or package, since processing XML in base >> is a pain, and it can't serve up JSON or HTML without additional >> utilities anyway. >> >> (If I'm reviving a long-settled thing, let me know and I'll drop it. I'm >> trying to understand the use case for this.) >> >> --- Harrison >> >> On 03/01/15 10:31, Craig Rodrigues wrote: >>> On Sun, Mar 1, 2015 at 9:25 AM, Harrison Grundy < >>> harrison.grundy@astrodoggroup.com> wrote: >>> >>>> >>>> If someone could summarize what this is, I'd greatly appreciate it. >>>> >>> https://lists.freebsd.org/pipermail/freebsd-arch/2014-July/015633.html >>> _______________________________________________ >>> freebsd-current@freebsd.org mailing list >>> http://lists.freebsd.org/mailman/listinfo/freebsd-current >>> To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org" >>> >> _______________________________________________ >> freebsd-current@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-current >> To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org" >> > I think you're missing the important bit here. > > This isn't about adding a parser for anything, this is about making the > tools in base, like netstat, wc, uptime, etc, output in JSON or XML, so > you can use the data programmatically. exactly. I think that's the wrong path to take. we have to change EVERY PROGRAM IN THE WORLD. if we develop a suitable post processor with pluggable grammars, we save a lot of work. given enough examples you could almost have automatically generated grammars. > > Your scripts no longer have to rely on awk/sed/grep magic to get a > specific bit of information out of the uptime command, the command can > just output the data in a structured machine readable format. > > I am not sure how you can put netstat into the ports tree. > > From owner-freebsd-current@FreeBSD.ORG Mon Mar 2 09:24:08 2015 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 38379AAC for ; Mon, 2 Mar 2015 09:24:08 +0000 (UTC) Received: from theravensnest.org (theraven.freebsd.your.org [216.14.102.27]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "cloud.theravensnest.org", Issuer "StartCom Class 1 Primary Intermediate Server CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id D34E46DA for ; Mon, 2 Mar 2015 09:24:07 +0000 (UTC) Received: from [192.168.0.7] (cpc16-cmbg15-2-0-cust60.5-4.cable.virginm.net [86.5.162.61]) (authenticated bits=0) by theravensnest.org (8.15.1/8.14.9) with ESMTPSA id t229O0qo024376 (version=TLSv1 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 2 Mar 2015 09:24:02 GMT (envelope-from theraven@FreeBSD.org) X-Authentication-Warning: theravensnest.org: Host cpc16-cmbg15-2-0-cust60.5-4.cable.virginm.net [86.5.162.61] claimed to be [192.168.0.7] Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 8.2 \(2070.6\)) Subject: Re: Massive libxo-zation that breaks everything From: David Chisnall In-Reply-To: <75C49F53-C675-4712-A446-370025EED037@me.com> Date: Mon, 2 Mar 2015 09:23:55 +0000 Content-Transfer-Encoding: quoted-printable Message-Id: References: <54F31510.7050607@hot.ee> <54F34B6E.2040809@astrodoggroup.com> <54F35F29.4000603@astrodoggroup.com> <75C49F53-C675-4712-A446-370025EED037@me.com> To: Rui Paulo X-Mailer: Apple Mail (2.2070.6) Cc: Harrison Grundy , freebsd-current@freebsd.org X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Mar 2015 09:24:08 -0000 On 1 Mar 2015, at 21:29, Rui Paulo wrote: >=20 > On Mar 1, 2015, at 11:11, David Chisnall wrote: >> How would it be in a port? It involves modifying core utilities = (some of which, like ifconfig, rely on kernel APIs that change between = releases) to emit structured output. Maintaining two copies of each = utility, one in the base system with plain-text output only and another = in ports with XML/JSON output would be very painful. >=20 > It would work fine if we had *libraries* for = ifconfig/netstat/route/etc. Obviously that's not the case and no one = has stepped up to implement them. I've also seen FreeBSD committers = expressing their distaste for libraries for "trivial" command line = utilities, which implies they are unaware of another world beyond the = CLI. :-) I am completely in favour of libraries for the underlying functionality = of these commands and would love to see all of the system management = commands become thin wrappers around a library, though it's a lot of = engineering work. In particular, these libraries will need to have = stable APIs that we can support across multiple major releases, and = getting those right is difficult. We really don't want to be stuck in = 10 years maintaining a hastily designed API for a library. I see one use of the libxo output as helping to design those APIs. = People are going to wrap various tools in libraries for their favourite = scripting languages and this will give us a corpus for experimenting. It's also worth noting that often invoking a tool and consuming its = output is the easiest way to get a stable API and ABI where performance = is not a primary concern (i.e. most management interfaces). As to a world beyond the CLI, I saw a nice demo a few years ago of a = terminal emulator that used WebKit and came with a hacked-up set of = parsers for common tools. I'd love to have something simpler (no need = for a full WebKit - simple outline and table views would be enough and = could be done with curses for ssh) for FreeBSD where I could type ls in = the CLI and get a table view that I could then sort and filter by = selecting column headings. Those of us that have used Lisp and = Smalltalk environments know that a CLI doesn't have to be a teletype = emulator. David From owner-freebsd-current@FreeBSD.ORG Mon Mar 2 09:26:02 2015 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7C662BC8; Mon, 2 Mar 2015 09:26:02 +0000 (UTC) Received: from theravensnest.org (theraven.freebsd.your.org [216.14.102.27]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "cloud.theravensnest.org", Issuer "StartCom Class 1 Primary Intermediate Server CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 41553750; Mon, 2 Mar 2015 09:26:01 +0000 (UTC) Received: from [192.168.0.7] (cpc16-cmbg15-2-0-cust60.5-4.cable.virginm.net [86.5.162.61]) (authenticated bits=0) by theravensnest.org (8.15.1/8.14.9) with ESMTPSA id t229Pwl1024388 (version=TLSv1 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 2 Mar 2015 09:26:00 GMT (envelope-from theraven@FreeBSD.org) X-Authentication-Warning: theravensnest.org: Host cpc16-cmbg15-2-0-cust60.5-4.cable.virginm.net [86.5.162.61] claimed to be [192.168.0.7] Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 8.2 \(2070.6\)) Subject: Re: Massive libxo-zation that breaks everything From: David Chisnall In-Reply-To: <54F42A82.1020308@freebsd.org> Date: Mon, 2 Mar 2015 09:25:53 +0000 Content-Transfer-Encoding: quoted-printable Message-Id: References: <54F31510.7050607@hot.ee> <54F34B6E.2040809@astrodoggroup.com> <54F35F29.4000603@astrodoggroup.com> <54F36431.30506@freebsd.org> <54F42A82.1020308@freebsd.org> To: Julian Elischer X-Mailer: Apple Mail (2.2070.6) Cc: freebsd-current@freebsd.org, Allan Jude X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Mar 2015 09:26:02 -0000 On 2 Mar 2015, at 09:16, Julian Elischer wrote: >=20 > if we develop a suitable post processor with pluggable grammars, we = save a lot of work. > given enough examples you could almost have automatically generated = grammars. This decoupled approach is problematic. A large part of the point of = libxo is to allow changing the human-readable output without breaking = tools that consume the output. Now I need to keep the tool that = consumes it and the tool that produces it in sync, so that's an extra = set of moving parts. When you throw jails with multiple versions of = world into the mix, it becomes a recipe for disaster. David From owner-freebsd-current@FreeBSD.ORG Mon Mar 2 10:53:08 2015 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C1F6EABB for ; Mon, 2 Mar 2015 10:53:08 +0000 (UTC) Received: from outpost1.zedat.fu-berlin.de (outpost1.zedat.fu-berlin.de [130.133.4.66]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 7DB18F4 for ; Mon, 2 Mar 2015 10:53:08 +0000 (UTC) Received: from inpost2.zedat.fu-berlin.de ([130.133.4.69]) by outpost.zedat.fu-berlin.de (Exim 4.85) for freebsd-current@freebsd.org with esmtp (envelope-from ) id <1YSNy8-003f4F-At>; Mon, 02 Mar 2015 11:53:00 +0100 Received: from p578a69f9.dip0.t-ipconnect.de ([87.138.105.249] helo=prometheus) by inpost2.zedat.fu-berlin.de (Exim 4.85) for freebsd-current@freebsd.org with esmtpsa (envelope-from ) id <1YSNy8-000Oo7-6m>; Mon, 02 Mar 2015 11:53:00 +0100 Date: Mon, 2 Mar 2015 11:50:57 +0100 From: "O. Hartmann" To: freebsd-current Subject: r279514: buildworld failure: /usr/src/lib/libnv/tests/dnv_tests.cc:453:2: error: use of overloaded operator '<<' is ambiguous Message-ID: <20150302115057.62d2c74c@prometheus> Organization: FU Berlin X-Mailer: Claws Mail 3.11.1 (GTK+ 2.24.25; amd64-portbld-freebsd11.0) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: base64 X-Originating-IP: 87.138.105.249 X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Mar 2015 10:53:08 -0000 LS0tLS1CRUdJTiBQR1AgU0lHTkVEIE1FU1NBR0UtLS0tLQ0KSGFzaDogU0hBMjU2DQoNCkJ1aWxk d29ybGQgZmFpbHMgaW4gQ1VSUkVOVCByMjc5NTE0IHdpdGggdGhlIGZvbGxvd2luZyBlcnJvcjoN Cg0KWy4uLl0NCg0KLSAtLS0gc2Jpbi5hbGxfX0QgLS0tDQotIC0tLSBjYW1jb250cm9sLm8gLS0t DQpjYyAgLU8yIC1waXBlIC1PMyAtTzMgLXBpcGUgLW1hcmNoPW5hdGl2ZSAgLXN0ZD1nbnU5OSAt ZnN0YWNrLXByb3RlY3Rvcg0KLSAtV3N5c3RlbS1oZWFkZXJzIC1XZXJyb3IgLVdhbGwgLVduby1m b3JtYXQteTJrIC1XIC1Xbm8tdW51c2VkLXBhcmFtZXRlcg0KLSAtV3N0cmljdC1wcm90b3R5cGVz IC1XbWlzc2luZy1wcm90b3R5cGVzIC1XcG9pbnRlci1hcml0aCAtV3JldHVybi10eXBlDQotIC1X Y2FzdC1xdWFsIC1Xd3JpdGUtc3RyaW5ncyAtV3N3aXRjaCAtV3NoYWRvdyAtV3VudXNlZC1wYXJh bWV0ZXIgLVdjYXN0LWFsaWduDQotIC1XY2hhci1zdWJzY3JpcHRzIC1XaW5saW5lIC1XbmVzdGVk LWV4dGVybnMgLVdyZWR1bmRhbnQtZGVjbHMNCi0gLVdvbGQtc3R5bGUtZGVmaW5pdGlvbiAtV25v LXBvaW50ZXItc2lnbiAtV21pc3NpbmctdmFyaWFibGUtZGVjbGFyYXRpb25zDQotIC1XdGhyZWFk LXNhZmV0eSAtV25vLWVtcHR5LWJvZHkgLVduby1zdHJpbmctcGx1cy1pbnQgLVduby11bnVzZWQt Y29uc3QtdmFyaWFibGUNCi0gLVF1bnVzZWQtYXJndW1lbnRzIC1jIC91c3Ivc3JjL3NiaW4vY2Ft Y29udHJvbC9jYW1jb250cm9sLmMgLS0tIGdudS5hbGxfX0QNCi0gLS0tIC91c3Ivc3JjL2dudS91 c3IuYmluL3Jjcy9saWIvcmNzcmV2LmM6NzUwOjg6IHdhcm5pbmc6IGFkZCBleHBsaWNpdCBicmFj ZXMNCnRvIGF2b2lkIGRhbmdsaW5nIGVsc2UgWy1XZGFuZ2xpbmctZWxzZV0gZWxzZSB7IF4gLS0t IGxpYi5hbGxfX0QNCi0gLS0tIC91c3Ivc3JjL2xpYi9saWJudi90ZXN0cy9kbnZfdGVzdHMuY2M6 NDUzOjI6IGVycm9yOiB1c2Ugb2Ygb3ZlcmxvYWRlZA0Kb3BlcmF0b3IgJzw8JyBpcyBhbWJpZ3Vv dXMgKHdpdGggb3BlcmFuZCB0eXBlcyAnYmFzaWNfb3N0cmVhbTxjaGFyLA0Kc3RkOjpfXzE6OmNo YXJfdHJhaXRzPGNoYXI+ID4nIGFuZCAnbnVsbHB0cl90JykgQVRGX1JFUVVJUkVfRVEoYWN0dWFs X3ZhbCwNCk5VTEwpOyAtLS0ga2VyYmVyb3M1LmFsbF9fRA0KLSAtLS0gL3Vzci9zcmMva2VyYmVy b3M1L3Vzci5iaW4va3N1Ly4uLy4uLy4uL2NyeXB0by9oZWltZGFsL2FwcGwvc3Uvc3UuYzo0Mzc6 NDI6DQp3YXJuaW5nOiBkYXRhIGFyZ3VtZW50IG5vdCB1c2VkIGJ5IGZvcm1hdCBzdHJpbmcgWy1X Zm9ybWF0LWV4dHJhLWFyZ3NdIC0tLQ0KbGliLmFsbF9fRCAtLS0NCl5+fn5+fn5+fn5+fn5+fn5+ fn5+fn5+fn5+fn5+fn5+IC91c3Ivb2JqL3Vzci9zcmMvdG1wL3Vzci9pbmNsdWRlL2F0Zi1jKysv bWFjcm9zLmhwcDoxMTQ6NTM6DQpub3RlOiBleHBhbmRlZCBmcm9tIG1hY3JvICdBVEZfUkVRVUlS RV9FUScgPDwgIiAoIiA8PCAoZXhwZWN0ZWQpIDw8ICIgIT0gIiA8PA0KKGFjdHVhbCkgPDwgIiki OyBcIH5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn4gXiAgfn5+fn5+fn4NCi0tLS0tQkVH SU4gUEdQIFNJR05BVFVSRS0tLS0tDQpWZXJzaW9uOiBHbnVQRyB2Mg0KDQppUUVjQkFFQkNBQUdC UUpVOUVDUkFBb0pFT2dCY0Q3QS81TjgyNklILzNpOWw4ejVFUjZqcW8zNGVjbzRNaDlNDQpGd3RM cjR2S01NTkRJREU0RFJGKzMrNzFLK1BwWDBwT2kxZVZ1Z0RwUjcxWTdvVEs2SmE4dktMMnQ3R3pw QTFKDQp5emtRTnl6TWd6ckVEUjhCNGZxUkNxQmxzc2crWmtaR1ZDUUhVeHIwaVRKeVR1M2xjZjRz Y1UxZ2huMXJ0OTU0DQpLaytQU05vd3lSQWFrQ0ExbDNtOVExNUlEZFFUTDZMQkdYNzZPelFBSklF Q21Db3dLbktGbk93NnZrMTRmZE9HDQpFaURlOVgwejc0ajc5Q1VmQUtXWmdmT3Q0RnFxVlhLbys0 U1pteUY2UjlveGNBcXRxbHdpeFY1bms4aUZZNDdjDQpsdjVXQlBKWHBOWVo1KzBsajI4QkxvamVZ NGRWMzJscThCWmVwYnZTWkcwMTNWdlpEVFpMSXdsV0I5d1pFRGM9DQo9NSs4Zg0KLS0tLS1FTkQg UEdQIFNJR05BVFVSRS0tLS0tDQo= From owner-freebsd-current@FreeBSD.ORG Mon Mar 2 09:28:05 2015 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 2BE57CEF for ; Mon, 2 Mar 2015 09:28:05 +0000 (UTC) Received: from mailout.easymail.ca (mailout.easymail.ca [64.68.201.169]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id D3C5F772 for ; Mon, 2 Mar 2015 09:28:04 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by mailout.easymail.ca (Postfix) with ESMTP id D72C6E43C for ; Mon, 2 Mar 2015 04:28:02 -0500 (EST) X-Virus-Scanned: Debian amavisd-new at mailout.easymail.ca X-Spam-Flag: NO X-Spam-Score: -3.85 X-Spam-Level: X-Spam-Status: No, score=-3.85 required=5 tests=[ALL_TRUSTED=-1.8, AWL=-0.143, BAYES_00=-2.599, DNS_FROM_AHBL_RHSBL=0.692] Received: from mailout.easymail.ca ([127.0.0.1]) by localhost (easymail-mailout.easydns.vpn [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Ijo8H24aAKnz for ; Mon, 2 Mar 2015 04:28:02 -0500 (EST) Received: from bsddt1241.lv01.astrodoggroup.com (unknown [40.141.24.126]) (using TLSv1 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by mailout.easymail.ca (Postfix) with ESMTPSA id 82C67E435 for ; Mon, 2 Mar 2015 04:28:02 -0500 (EST) Message-ID: <54F42C6A.1000309@astrodoggroup.com> Date: Mon, 02 Mar 2015 01:24:58 -0800 From: Harrison Grundy User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:31.0) Gecko/20100101 Thunderbird/31.3.0 MIME-Version: 1.0 To: freebsd-current@freebsd.org Subject: Re: Massive libxo-zation that breaks everything References: <54F31510.7050607@hot.ee> <54F34B6E.2040809@astrodoggroup.com> <54F35F29.4000603@astrodoggroup.com> <75C49F53-C675-4712-A446-370025EED037@me.com> In-Reply-To: Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-Mailman-Approved-At: Mon, 02 Mar 2015 12:33:11 +0000 X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Mar 2015 09:28:05 -0000 On 03/02/15 01:23, David Chisnall wrote: > On 1 Mar 2015, at 21:29, Rui Paulo wrote: >> >> On Mar 1, 2015, at 11:11, David Chisnall >> wrote: >>> How would it be in a port? It involves modifying core >>> utilities (some of which, like ifconfig, rely on kernel APIs >>> that change between releases) to emit structured output. >>> Maintaining two copies of each utility, one in the base system >>> with plain-text output only and another in ports with XML/JSON >>> output would be very painful. >> >> It would work fine if we had *libraries* for >> ifconfig/netstat/route/etc. Obviously that's not the case and no >> one has stepped up to implement them. I've also seen FreeBSD >> committers expressing their distaste for libraries for "trivial" >> command line utilities, which implies they are unaware of another >> world beyond the CLI. :-) > > I am completely in favour of libraries for the underlying > functionality of these commands and would love to see all of the > system management commands become thin wrappers around a library, > though it's a lot of engineering work. In particular, these > libraries will need to have stable APIs that we can support across > multiple major releases, and getting those right is difficult. We > really don't want to be stuck in 10 years maintaining a hastily > designed API for a library. > > I see one use of the libxo output as helping to design those APIs. > People are going to wrap various tools in libraries for their > favourite scripting languages and this will give us a corpus for > experimenting. > > It's also worth noting that often invoking a tool and consuming its > output is the easiest way to get a stable API and ABI where > performance is not a primary concern (i.e. most management > interfaces). > > As to a world beyond the CLI, I saw a nice demo a few years ago of > a terminal emulator that used WebKit and came with a hacked-up set > of parsers for common tools. I'd love to have something simpler > (no need for a full WebKit - simple outline and table views would > be enough and could be done with curses for ssh) for FreeBSD where > I could type ls in the CLI and get a table view that I could then > sort and filter by selecting column headings. Those of us that > have used Lisp and Smalltalk environments know that a CLI doesn't > have to be a teletype emulator. > > David It would seem like the libxo stuff runs the risk of becoming this same API. --- Harrison From owner-freebsd-current@FreeBSD.ORG Mon Mar 2 12:36:05 2015 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 82AD0972 for ; Mon, 2 Mar 2015 12:36:05 +0000 (UTC) Received: from theravensnest.org (theraven.freebsd.your.org [216.14.102.27]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "cloud.theravensnest.org", Issuer "StartCom Class 1 Primary Intermediate Server CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4A451EF1 for ; Mon, 2 Mar 2015 12:36:04 +0000 (UTC) Received: from dhcp-172-17-158-227.eduroam.wireless.private.cam.ac.uk (global-1-26.nat.csx.cam.ac.uk [131.111.184.26]) (authenticated bits=0) by theravensnest.org (8.15.1/8.14.9) with ESMTPSA id t22CZsxt025338 (version=TLSv1 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 2 Mar 2015 12:35:56 GMT (envelope-from theraven@FreeBSD.org) X-Authentication-Warning: theravensnest.org: Host global-1-26.nat.csx.cam.ac.uk [131.111.184.26] claimed to be dhcp-172-17-158-227.eduroam.wireless.private.cam.ac.uk Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 8.2 \(2070.6\)) Subject: Re: Massive libxo-zation that breaks everything From: David Chisnall In-Reply-To: <54F42C6A.1000309@astrodoggroup.com> Date: Mon, 2 Mar 2015 12:35:48 +0000 Content-Transfer-Encoding: quoted-printable Message-Id: References: <54F31510.7050607@hot.ee> <54F34B6E.2040809@astrodoggroup.com> <54F35F29.4000603@astrodoggroup.com> <75C49F53-C675-4712-A446-370025EED037@me.com> <54F42C6A.1000309@astrodoggroup.com> To: Harrison Grundy X-Mailer: Apple Mail (2.2070.6) Cc: freebsd-current@freebsd.org X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Mar 2015 12:36:05 -0000 On 2 Mar 2015, at 09:24, Harrison Grundy = wrote: >=20 > It would seem like the libxo stuff runs the risk of becoming this same > API. Why? The 'API' in the case of an libxo-ised program is a stream on = stdout that is then consumed by a JSON or XML parser. XML and JSON are = intrinsically extensible formats. This is *the entire point* of libxo: = that we can extend the output from these tools without breaking things = that wish to consume them and which currently rely on fragile parsers. David From owner-freebsd-current@FreeBSD.ORG Mon Mar 2 13:22:01 2015 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C85A5101 for ; Mon, 2 Mar 2015 13:22:01 +0000 (UTC) Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by mx1.freebsd.org (Postfix) with ESMTP id B2C0769E for ; Mon, 2 Mar 2015 13:22:01 +0000 (UTC) Received: from AlfredMacbookAir.local (hudsonhotel209.h.subnet.rcn.com [207.237.151.136]) by elvis.mu.org (Postfix) with ESMTPSA id 731A7341F90D for ; Mon, 2 Mar 2015 05:21:55 -0800 (PST) Message-ID: <54F464D1.6060603@mu.org> Date: Mon, 02 Mar 2015 08:25:37 -0500 From: Alfred Perlstein User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:31.0) Gecko/20100101 Thunderbird/31.5.0 MIME-Version: 1.0 To: freebsd-current@freebsd.org Subject: Re: Massive libxo-zation that breaks everything References: <54F31510.7050607@hot.ee> <54F34B6E.2040809@astrodoggroup.com> <54F35F29.4000603@astrodoggroup.com> <54F36431.30506@freebsd.org> <54F42A82.1020308@freebsd.org> In-Reply-To: Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Mar 2015 13:22:01 -0000 On 3/2/15 4:25 AM, David Chisnall wrote: > On 2 Mar 2015, at 09:16, Julian Elischer wrote: >> if we develop a suitable post processor with pluggable grammars, we save a lot of work. >> given enough examples you could almost have automatically generated grammars. > This decoupled approach is problematic. A large part of the point of libxo is to allow changing the human-readable output without breaking tools that consume the output. Now I need to keep the tool that consumes it and the tool that produces it in sync, so that's an extra set of moving parts. When you throw jails with multiple versions of world into the mix, it becomes a recipe for disaster. > +1 From owner-freebsd-current@FreeBSD.ORG Mon Mar 2 13:23:35 2015 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 207512F4; Mon, 2 Mar 2015 13:23:35 +0000 (UTC) Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by mx1.freebsd.org (Postfix) with ESMTP id 0BC6C6BA; Mon, 2 Mar 2015 13:23:34 +0000 (UTC) Received: from AlfredMacbookAir.local (hudsonhotel209.h.subnet.rcn.com [207.237.151.136]) by elvis.mu.org (Postfix) with ESMTPSA id E42FD341F90D; Mon, 2 Mar 2015 05:23:33 -0800 (PST) Message-ID: <54F46536.8040607@mu.org> Date: Mon, 02 Mar 2015 08:27:18 -0500 From: Alfred Perlstein User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:31.0) Gecko/20100101 Thunderbird/31.5.0 MIME-Version: 1.0 To: Julian Elischer , Harrison Grundy , freebsd-current@freebsd.org Subject: Re: Massive libxo-zation that breaks everything References: <54F31510.7050607@hot.ee> <54F34B6E.2040809@astrodoggroup.com> <54F35F29.4000603@astrodoggroup.com> <54F429EF.5050400@freebsd.org> In-Reply-To: <54F429EF.5050400@freebsd.org> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Mar 2015 13:23:35 -0000 On 3/2/15 4:14 AM, Julian Elischer wrote: > On 3/1/15 10:49 AM, Harrison Grundy wrote: >> Thanks! >> >> That does seem useful, but I'm not sure I see the reasoning behind >> putting into base, over a port or package, since processing XML in base >> is a pain, and it can't serve up JSON or HTML without additional >> utilities anyway. >> >> (If I'm reviving a long-settled thing, let me know and I'll drop it. I'm >> trying to understand the use case for this.) > > To me it would almost seem more useful to have a programmable filter > for which you could produce > parse grammars to parse the output of various programs.. > thus > > ifconfig -a | xmlize -g ifconfig | your-favourite-xml-parser > with a set of grammars in /usr/share/xmlize/ > then we could use it for out-of-tree programs as well if we wrote > grammars for them.. > > The sentiment of machine-readable output is nice, but I think it's > slightly off target. > we shouldn't have to change all out utilities, and it isn't going to > help at all with 3rd party apps, > e.g. samba stuff. A generally easy to program output grammar parser > would be truely useful. > and not just for FreeBSD. > > I've been watching with an uncomfortable feeling, but it's taken me a > while to put my > finger on what it was.. > > Are you sure it's not the hairs on the back of your neck standing up due to NIH? Juniper has been doing this for years and it's very useful for them. -Alfred From owner-freebsd-current@FreeBSD.ORG Mon Mar 2 13:58:06 2015 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3CEA4F3A for ; Mon, 2 Mar 2015 13:58:06 +0000 (UTC) Received: from mail-ie0-x234.google.com (mail-ie0-x234.google.com [IPv6:2607:f8b0:4001:c03::234]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 03386A66 for ; Mon, 2 Mar 2015 13:58:06 +0000 (UTC) Received: by iecrl12 with SMTP id rl12so47833945iec.2 for ; Mon, 02 Mar 2015 05:58:05 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=kYABdcAIxKwf9N7uCJeZCZ4jRl9ZDhvgUj37ezAXgJE=; b=BTLDn5mo/qHS/JfoF8lafQoxqoEUbcswRcKWUPp2TwHlK98nsSI60l4jIhTgIAOhb3 bxCwz0KxmuiHJAOtQ500KyQvic7kCAsQNx7ECBnrFexkSsZYxIJUmmpLFwK33qJoLKnt 3Jp0KYKgMZ7+0Zm5+XUEvVf9gK6FVTRsWiI7+8wsbN+IcjRuIN5To7UiQ+o9lsvQ/w02 a+LmX463hgchpCfF1H9iqIJerbrailHK2PKw3ClECzfsHVSrIQrNMEakfco+lFpD55Sc 1HotFPLy2ofJ8kk2jTvsvNg6wl0Ri5tQbXM39QQ+9chx3iXuvmM4WK6B6wO/aModF71l kJsg== MIME-Version: 1.0 X-Received: by 10.42.210.20 with SMTP id gi20mr30961474icb.34.1425304685388; Mon, 02 Mar 2015 05:58:05 -0800 (PST) Received: by 10.107.156.75 with HTTP; Mon, 2 Mar 2015 05:58:05 -0800 (PST) In-Reply-To: <20150302115057.62d2c74c@prometheus> References: <20150302115057.62d2c74c@prometheus> Date: Mon, 2 Mar 2015 08:58:05 -0500 Message-ID: Subject: Re: r279514: buildworld failure: /usr/src/lib/libnv/tests/dnv_tests.cc:453:2: error: use of overloaded operator '<<' is ambiguous From: Ryan Stone To: "O. Hartmann" Content-Type: text/plain; charset=UTF-8 Cc: freebsd-current X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Mar 2015 13:58:06 -0000 Can you post the contents of your make.conf and src.conf? I didn't see this in any of my "make tinderbox" runs From owner-freebsd-current@FreeBSD.ORG Mon Mar 2 15:08:41 2015 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1BC08BCF for ; Mon, 2 Mar 2015 15:08:41 +0000 (UTC) Received: from udns.ultimatedns.net (unknown [IPv6:2602:d1:b4d6:e600:4261:86ff:fef6:aa2a]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id DCCFB380 for ; Mon, 2 Mar 2015 15:08:40 +0000 (UTC) Received: from ultimatedns.net (localhost [127.0.0.1]) by udns.ultimatedns.net (8.14.9/8.14.9) with ESMTP id t22F9dUA047085 for ; Mon, 2 Mar 2015 07:09:39 -0800 (PST) (envelope-from bsd-lists@bsdforge.com) To: In-Reply-To: References: <54F31510.7050607@hot.ee> <54F34B6E.2040809@astrodoggroup.com> <54F35F29.4000603@astrodoggroup.com> <75C49F53-C675-4712-A446-370025EED037@me.com>, From: "Chris H" Subject: Re: Massive libxo-zation that breaks everything Date: Mon, 02 Mar 2015 07:09:39 -0800 Content-Type: text/plain; charset=UTF-8; format=fixed MIME-Version: 1.0 Message-id: Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Mar 2015 15:08:41 -0000 On Mon, 2 Mar 2015 09:23:55 +0000 David Chisnall wrote > On 1 Mar 2015, at 21:29, Rui Paulo wrote: > > > > On Mar 1, 2015, at 11:11, David Chisnall wrote: > >> How would it be in a port? It involves modifying core utilities (some of > >> which, like ifconfig, rely on kernel APIs that change between releases) to > >> emit structured output. Maintaining two copies of each utility, one in the > >> base system with plain-text output only and another in ports with XML/JSON > >> output would be very painful. > > > It would work fine if we had *libraries* for ifconfig/netstat/route/etc. > > Obviously that's not the case and no one has stepped up to implement them. > > I've also seen FreeBSD committers expressing their distaste for libraries for > > "trivial" command line utilities, which implies they are unaware of another > > world beyond the CLI. :-) > > I am completely in favour of libraries for the underlying functionality of > these commands and would love to see all of the system management commands > become thin wrappers around a library, though it's a lot of engineering work. > In particular, these libraries will need to have stable APIs that we can > support across multiple major releases, and getting those right is difficult. > We really don't want to be stuck in 10 years maintaining a hastily designed > API for a library. > > I see one use of the libxo output as helping to design those APIs. People > are going to wrap various tools in libraries for their favourite scripting > languages and this will give us a corpus for experimenting. > > It's also worth noting that often invoking a tool and consuming its output is > the easiest way to get a stable API and ABI where performance is not a > primary concern (i.e. most management interfaces). > > As to a world beyond the CLI, I saw a nice demo a few years ago of a terminal > emulator that used WebKit and came with a hacked-up set of parsers for common > tools. I'd love to have something simpler (no need for a full WebKit - > simple outline and table views would be enough and could be done with curses > for ssh) for FreeBSD where I could type ls in the CLI and get a table view > that I could then sort and filter by selecting column headings. Those of us > that have used Lisp and Smalltalk environments know that a CLI doesn't have > to be a teletype emulator. Apologies in advance, if I'm way off base, or simply reiterating here; But what about something like kernshell(1)? Every database has it's own "shell", the system has it's own contributed shells. Couldn't a shell be adapted with a language/dialect to make use of already existing parsers/languages/3rd party utilities? Then it would simply be one item to maintain, and the language could be modified/adapted, as needed. Again, apologies, if I haven't thought this through enough. But in an effort to simplify everything, for all concerned. This seemed like a possible direction. So thought I'd mention it. --Chris > > David > > _______________________________________________ > freebsd-current@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-current > To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org" From owner-freebsd-current@FreeBSD.ORG Mon Mar 2 16:10:00 2015 Return-Path: Delivered-To: current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1E742516; Mon, 2 Mar 2015 16:10:00 +0000 (UTC) Received: from clavin1.langille.org (clavin.langille.org [162.208.116.86]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "clavin.langille.org", Issuer "StartCom Class 2 Primary Intermediate Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id BC9D1C22; Mon, 2 Mar 2015 16:09:59 +0000 (UTC) Received: from (clavin1.int.langille.org (clavin1.int.unixathome.org [10.4.7.7]) (Authenticated sender: hidden) with ESMTPSA id 2AC1F6D7E ; Mon, 2 Mar 2015 16:09:58 +0000 (UTC) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 8.2 \(2070.6\)) Subject: Re: sa(4) driver changes available for test From: Dan Langille In-Reply-To: <20150302022957.GB73433@mithlond.kdm.org> Date: Mon, 2 Mar 2015 11:09:57 -0500 Content-Transfer-Encoding: quoted-printable Message-Id: <0F17CDDB-4CE9-4881-B948-6B9BEDEB6455@langille.org> References: <20150214003232.GA63990@mithlond.kdm.org> <7CA52DF3-E073-4F50-BE4E-01C51CCDF2C7@langille.org> <20150217183645.GA30947@mithlond.kdm.org> <303C48AC-0DA8-447C-8DE7-0A1DD5DD7351@langille.org> <20150302003150.GB71528@mithlond.kdm.org> <20150302022957.GB73433@mithlond.kdm.org> To: "Kenneth D. Merry" X-Mailer: Apple Mail (2.2070.6) Cc: current@freebsd.org, scsi@freebsd.org X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Mar 2015 16:10:00 -0000 > On Mar 1, 2015, at 9:29 PM, Kenneth D. Merry wrote: >=20 > On Sun, Mar 01, 2015 at 19:41:07 -0500, Dan Langille wrote: >>=20 >>> On Mar 1, 2015, at 7:31 PM, Kenneth D. Merry = wrote: >>>=20 >>> On Sun, Mar 01, 2015 at 19:15:05 -0500, Dan Langille wrote: >>>>=20 >>>>> On Feb 17, 2015, at 1:36 PM, Kenneth D. Merry = wrote: >>>>>=20 >>>>> On Sat, Feb 14, 2015 at 18:22:43 -0500, Dan Langille wrote: >>>>>>=20 >>>>>>> On Feb 13, 2015, at 7:32 PM, Kenneth D. Merry = wrote: >>>>>>>=20 >>>>>>>=20 >>>>>>> I have a fairly large set of changes to the sa(4) driver and = mt(1) driver >>>>>>> that I'm planning to commit in the near future. >>>>>>>=20 >>>>>>> A description of the changes is here and below in this message. >>>>>>>=20 >>>>>>> If you have tape hardware and the inclination, I'd appreciate = testing and >>>>>>> feedback. >>>>>>=20 >>>>>> I have a DLT 8000 and an SDLT 220. >>>>>>=20 >>>>>> I don't have anything running current, but I have a spare machine = which I could use for testing. >>>>>>=20 >>>>>> Do you see any value is tests with that hardware? I'd be testing = it via Bacula. >>>>>>=20 >>>>>> disclosure: I'm the sysutils/bacula-* maintainer and a Bacula = committer. >>>>>>=20 >>>>>=20 >>>>> Actually, yes. Bacula is a bit tricky to configure, so your = trying it out >>>>> would be helpful if you have the time. >>>>>=20 >>>>> In looking at the manuals for both the SDLT 220 and the DLT 8000, = they both >>>>> claim to support long position information for the SCSI READ = POSITION >>>>> command. >>>>>=20 >>>>> You can see what I'm talking about by doing: >>>>>=20 >>>>> mt eod >>>>> mt status >>>>>=20 >>>>> On my DDS-4 tape drive, this shows: >>>>>=20 >>>>> # mt -f /dev/nsa3 status >>>>> Drive: sa3: Serial Number: HJ00YWY >>>>> --------------------------------- >>>>> Mode Density Blocksize bpi Compression >>>>> Current: 0x26:DDS-4 1024 bytes 97000 enabled = (DCLZ) >>>>> --------------------------------- >>>>> Current Driver State: at rest. >>>>> --------------------------------- >>>>> Partition: 0 Calc File Number: -1 Calc Record Number: = -1 >>>>> Residual: 0 Reported File Number: -1 Reported Record Number: = -1 >>>>> Flags: None >>>>>=20 >>>>> But on an LTO-5, which will give long position information, I get: >>>>>=20 >>>>> [root@doc ~]# mt status >>>>> Drive: sa0: >>>>> --------------------------------- >>>>> Mode Density Blocksize bpi Compression >>>>> Current: 0x58:LTO-5 variable 384607 enabled = (0x1) >>>>> --------------------------------- >>>>> Current Driver State: at rest. >>>>> --------------------------------- >>>>> Partition: 0 Calc File Number: 2 Calc Record Number: = -1 >>>>> Residual: 0 Reported File Number: 2 Reported Record Number: = 32373 >>>>> Flags: None >>>>>=20 >>>>> That, in combination with the changes I made to the position = information >>>>> code in the driver, mean that even the old MTIOCGET ioctl should = return an >>>>> accurate file number at end of data. e.g., on the LTO-5: >>>>>=20 >>>>> [root@doc ~]# mt ostatus >>>>> Mode Density Blocksize bpi Compression >>>>> Current: 0x58:LTO-5 variable 384607 0x1 >>>>> ---------available modes--------- >>>>> 0: 0x58:LTO-5 variable 384607 0x1 >>>>> 1: 0x58:LTO-5 variable 384607 0x1 >>>>> 2: 0x58:LTO-5 variable 384607 0x1 >>>>> 3: 0x58:LTO-5 variable 384607 0x1 >>>>> --------------------------------- >>>>> Current Driver State: at rest. >>>>> --------------------------------- >>>>> File Number: 2 Record Number: -1 Residual Count -1 >>>>>=20 >>>>> So the thing to try, in addition to just making sure that Bacula = continues >>>>> to work properly, is to try setting this for the tape drive in >>>>> bacula-sd.conf: >>>>>=20 >>>>> Hardware End of Medium =3D yes >>>>>=20 >>>>> It looks like the Bacula tape program (btape) has a test mode, and = it would >>>>> be good to run through the tests on one of the tape drives and see = whether >>>>> they work, and whether the results are different before and after = the >>>>> changes. I'm not sure how to enable the test mode. >>>>=20 >>>> I have this in /usr/local/etc/bacula/bacula-sd.conf >>>>=20 >>>> Device { >>>> Name =3D DLT >>>> Description =3D "QUANTUM DLT7000 1624" >>>> Media Type =3D DLT >>>> Archive Device =3D /dev/nsa1 >>>>=20 >>>> Autochanger =3D YES >>>> Drive Index =3D 0 >>>>=20 >>>> Offline On Unmount =3D no >>>> Hardware End of Medium =3D yes >>>> BSF at EOM =3D yes >>>> Backward Space Record =3D no >>>> Fast Forward Space File =3D no >>>> TWO EOF =3D yes >>>> } >>>>=20 >>>> FYI, http://www.freebsddiary.org/digital-tl891.php (from 2006) has = a btape test on this same model. >>>>=20 >>>> Here's the test I ran tonight: >>>>=20 >>>> [root@cuppy:/usr/home/dan] # btape -c = /usr/local/etc/bacula/bacula-sd.conf /dev/nsa1 = =20 >>>> Tape block granularity is 1024 bytes. >>>> btape: butil.c:287-0 Using device: "/dev/nsa1" for writing. >>>> btape: btape.c:469-0 open device "DLT" (/dev/nsa1): OK >>>> *test >>>>=20 >>>> =3D=3D=3D Write, rewind, and re-read test =3D=3D=3D >>>>=20 >>>> I'm going to write 10000 records and an EOF >>>> then write 10000 records and an EOF, then rewind, >>>> and re-read the data to verify that it is correct. >>>>=20 >>>> This is an *essential* feature ... >>>>=20 >>>> btape: btape.c:1152-0 Wrote 10000 blocks of 64412 bytes. >>>> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) >>>> btape: btape.c:1168-0 Wrote 10000 blocks of 64412 bytes. >>>> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) >>>> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) >>>> btape: btape.c:1210-0 Rewind OK. >>>> 10000 blocks re-read correctly. >>>> Got EOF on tape. >>>> 10000 blocks re-read correctly. >>>> =3D=3D=3D Test Succeeded. End Write, rewind, and re-read test =3D=3D=3D= >>>>=20 >>>> btape: btape.c:1277-0 Block position test >>>> btape: btape.c:1289-0 Rewind OK. >>>> Reposition to file:block 0:4 >>>> Block 5 re-read correctly. >>>> Reposition to file:block 0:200 >>>> Block 201 re-read correctly. >>>> Reposition to file:block 0:9999 >>>> Block 10000 re-read correctly. >>>> Reposition to file:block 1:0 >>>> Block 10001 re-read correctly. >>>> Reposition to file:block 1:600 >>>> Block 10601 re-read correctly. >>>> Reposition to file:block 1:9999 >>>> Block 20000 re-read correctly. >>>> =3D=3D=3D Test Succeeded. End Write, rewind, and re-read test =3D=3D=3D= >>>>=20 >>>>=20 >>>>=20 >>>> =3D=3D=3D Append files test =3D=3D=3D >>>>=20 >>>> This test is essential to Bacula. >>>>=20 >>>> I'm going to write one record in file 0, >>>> two records in file 1, >>>> and three records in file 2 >>>>=20 >>>> btape: btape.c:574-0 Rewound "DLT" (/dev/nsa1) >>>> btape: btape.c:1907-0 Wrote one record of 64412 bytes. >>>> btape: btape.c:1909-0 Wrote block to device. >>>> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) >>>> btape: btape.c:1907-0 Wrote one record of 64412 bytes. >>>> btape: btape.c:1909-0 Wrote block to device. >>>> btape: btape.c:1907-0 Wrote one record of 64412 bytes. >>>> btape: btape.c:1909-0 Wrote block to device. >>>> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) >>>> btape: btape.c:1907-0 Wrote one record of 64412 bytes. >>>> btape: btape.c:1909-0 Wrote block to device. >>>> btape: btape.c:1907-0 Wrote one record of 64412 bytes. >>>> btape: btape.c:1909-0 Wrote block to device. >>>> btape: btape.c:1907-0 Wrote one record of 64412 bytes. >>>> btape: btape.c:1909-0 Wrote block to device. >>>> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) >>>> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) >>>> btape: btape.c:469-0 open device "DLT" (/dev/nsa1): OK >>>> btape: btape.c:574-0 Rewound "DLT" (/dev/nsa1) >>>> btape: btape.c:1420-0 Now moving to end of medium. >>>=20 >>> This is the critical piece. The test moves the tape to the end of = the >>> medium. With hardware position information, you can tell what = filemark >>> you're on. Without it, you can't. >>>=20 >>>> btape: btape.c:622-0 tape_dev.c:345 ioctl MTIOCGET error on "DLT" = (/dev/nsa1). ERR=3DNo error: 0. >>>> We should be in file 3. I am at file 0. This is NOT correct!!!! >>>>=20 >>>> Append test failed. Attempting again. >>>> Setting "Hardware End of Medium =3D no >>>> and "Fast Forward Space File =3D no >>>> and retrying append test. >>>=20 >>> This is not surprsing, given that the drive doesn't support long = read >>> position data. (It's a SCSI-2 device.) So that means that Bacula = will >>> need to do it manually. >>=20 >> Yes, I have nothing newer than SCSI-2. Even my SDLT is SCSI-2 but = that >> tape library is hooked up to a different computer and was doing = backups today. >=20 > So, here is one thing that we can try to see whether these drives = support > long position information, even though they only claim to be SCSI-2. = If > they do, we can potentially add a quirk (or autodetection) to enable = it. > The code currently doesn't bother asking drives that claim to be = SCSI-2 > for long position information. (Because that feature was added in the > SSC spec, which came after SCSI-2.) >=20 > Issue a READ POSITION with the short form specified: >=20 > camcontrol cmd sa1 -v -c "34 0 0 0 0 0 0 0 0 0" -i 20 - |hd >=20 > Issue a READ POSITION with the vendor-specific block numbers: >=20 > camcontrol cmd sa1 -v -c "34 1 0 0 0 0 0 0 0 0" -i 20 - |hd >=20 > Issue a READ POSITION with the long form data: >=20 > camcontrol cmd sa1 -v -c "34 6 0 0 0 0 0 0 0 0" -i 32 - |hd >=20 > If it supports the last one, then I can put a quirk (or autodetection) = in > the driver and Bacula will get the hardware filemarks. You should try = this > on your SDLT as well. It may well support it. Sadly, no: [root@cuppy:~] # camcontrol cmd sa1 -v -c "34 0 0 0 0 0 0 0 0 0" -i 20 - = |hd 00000000 80 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 = |................| 00000010 00 00 00 00 |....| 00000014 [root@cuppy:~] # camcontrol cmd sa1 -v -c "34 1 0 0 0 0 0 0 0 0" -i 20 - = |hd 00000000 80 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 = |................| 00000010 00 00 00 00 |....| 00000014 [root@cuppy:~] # camcontrol cmd sa1 -v -c "34 6 0 0 0 0 0 0 0 0" -i 32 - = |hd camcontrol: error sending command (pass2:ahc0:0:2:0): READ POSITION. CDB: 34 06 00 00 00 00 00 00 00 00=20 (pass2:ahc0:0:2:0): CAM status: SCSI Status Error (pass2:ahc0:0:2:0): SCSI status: Check Condition (pass2:ahc0:0:2:0): SCSI sense: ILLEGAL REQUEST asc:24,0 (Invalid field = in CDB) (pass2:ahc0:0:2:0): Command byte 1 bit 2 is invalid [root@cuppy:~] #=20 The SDLT server is on 9.3 though: [root@knew:/usr/home/dan] # camcontrol cmd sa1 -v -c "34 0 0 0 0 0 0 0 0 = 0" -i 20 - |hd camcontrol: cam_lookup_pass: CAMGETPASSTHRU ioctl failed cam_lookup_pass: No such file or directory cam_lookup_pass: either the pass driver isn't in your kernel cam_lookup_pass: or sa1 doesn't exist [root@knew:/usr/home/dan] # uname -a FreeBSD knew.unixathome.org 9.3-RELEASE-p10 FreeBSD 9.3-RELEASE-p10 #0: = Tue Feb 24 21:28:03 UTC 2015 = root@amd64-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC amd64 [root@knew:/usr/home/dan] #=20 It took me a while to figure that out... there is no sa1 on *this* = system. But, my SDLT: [root@knew:/usr/home/dan] # camcontrol cmd sa0 -v -c "34 0 0 0 0 0 0 0 0 = 0" -i 20 - |hd 00000000 80 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 = |................| 00000010 00 00 00 00 |....| 00000014 [root@knew:/usr/home/dan] # camcontrol cmd sa0 -v -c "34 1 0 0 0 0 0 0 0 = 0" -i 20 - |hd 00000000 80 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 = |................| 00000010 00 00 00 00 |....| 00000014 [root@knew:/usr/home/dan] # camcontrol cmd sa0 -v -c "34 6 0 0 0 0 0 0 0 = 0" -i 32 - |hd 00000000 80 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 = |................| 00000010 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 = |................| 00000020 [root@knew:/usr/home/dan] #=20 >=20 > Google didn't quickly produce a SCSI manual for the DEC drive, but the > Quantum SDLT manual indicates that it supports long position data, = despite > identifying itself as a SCSI-2 drive. >=20 >>> =3D=3D=3D Append files test =3D=3D=3D >>>>=20 >>>> This test is essential to Bacula. >>>>=20 >>>> I'm going to write one record in file 0, >>>> two records in file 1, >>>> and three records in file 2 >>>>=20 >>>> btape: btape.c:574-0 Rewound "DLT" (/dev/nsa1) >>>> btape: btape.c:1907-0 Wrote one record of 64412 bytes. >>>> btape: btape.c:1909-0 Wrote block to device. >>>> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) >>>> btape: btape.c:1907-0 Wrote one record of 64412 bytes. >>>> btape: btape.c:1909-0 Wrote block to device. >>>> btape: btape.c:1907-0 Wrote one record of 64412 bytes. >>>> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) >>>> btape: btape.c:1907-0 Wrote one record of 64412 bytes. >>>> btape: btape.c:1909-0 Wrote block to device. >>>> btape: btape.c:1907-0 Wrote one record of 64412 bytes. >>>> btape: btape.c:1909-0 Wrote block to device. >>>> btape: btape.c:1907-0 Wrote one record of 64412 bytes. >>>> btape: btape.c:1909-0 Wrote block to device. >>>> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) >>>> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) >>>> btape: btape.c:469-0 open device "DLT" (/dev/nsa1): OK >>>> btape: btape.c:574-0 Rewound "DLT" (/dev/nsa1) >>>> btape: btape.c:1420-0 Now moving to end of medium. >>>> btape: btape.c:625-0 Moved to end of medium. >>>> We should be in file 3. I am at file 3. This is correct! >>>>=20 >>>> Now the important part, I am going to attempt to append to the = tape. >>>>=20 >>>> btape: btape.c:1907-0 Wrote one record of 64412 bytes. >>>> btape: btape.c:1909-0 Wrote block to device. >>>> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) >>>> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) >>>> btape: btape.c:574-0 Rewound "DLT" (/dev/nsa1) >>>> Done appending, there should be no I/O errors >>>>=20 >>>> Doing Bacula scan of blocks: >>>> 1 block of 64448 bytes in file 1 >>>> End of File mark. >>>> 2 blocks of 64448 bytes in file 2 >>>> End of File mark. >>>> 3 blocks of 64448 bytes in file 3 >>>> End of File mark. >>>> 1 block of 64448 bytes in file 4 >>>> End of File mark. >>>> Total files=3D4, blocks=3D7, bytes =3D 451,136 >>>> End scanning the tape. >>>> We should be in file 4. I am at file 4. This is correct! >>>>=20 >>>>=20 >>>> It looks like the test worked this time, please add: >>>>=20 >>>> Hardware End of Medium =3D No >>>>=20 >>>> Fast Forward Space File =3D No >>>> to your Device resource in the Storage conf file. >>>>=20 >>>> The above Bacula scan should have output identical to what follows. >>>> Please double check it ... >>>> =3D=3D=3D Sample correct output =3D=3D=3D >>>> 1 block of 64448 bytes in file 1 >>>> End of File mark. >>>> 2 blocks of 64448 bytes in file 2 >>>> End of File mark. >>>> 3 blocks of 64448 bytes in file 3 >>>> End of File mark. >>>> 1 block of 64448 bytes in file 4 >>>> End of File mark. >>>> Total files=3D4, blocks=3D7, bytes =3D 451,136 >>>> =3D=3D=3D End sample correct output =3D=3D=3D >>>>=20 >>>> If the above scan output is not identical to the >>>> sample output, you MUST correct the problem >>>> or Bacula will not be able to write multiple Jobs to=20 >>>> the tape. >>>>=20 >>>> Skipping read backwards test because BSR turned off. >>>>=20 >>>>=20 >>>> =3D=3D=3D Forward space files test =3D=3D=3D >>>>=20 >>>> This test is essential to Bacula. >>>>=20 >>>> I'm going to write five files then test forward spacing >>>>=20 >>>> btape: btape.c:574-0 Rewound "DLT" (/dev/nsa1) >>>> btape: btape.c:1907-0 Wrote one record of 64412 bytes. >>>> btape: btape.c:1909-0 Wrote block to device. >>>> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) >>>> btape: btape.c:1907-0 Wrote one record of 64412 bytes. >>>> btape: btape.c:1909-0 Wrote block to device. >>>> btape: btape.c:1907-0 Wrote one record of 64412 bytes. >>>> btape: btape.c:1909-0 Wrote block to device. >>>> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) >>>> btape: btape.c:1907-0 Wrote one record of 64412 bytes. >>>> btape: btape.c:1909-0 Wrote block to device. >>>> btape: btape.c:1907-0 Wrote one record of 64412 bytes. >>>> btape: btape.c:1909-0 Wrote block to device. >>>> btape: btape.c:1907-0 Wrote one record of 64412 bytes. >>>> btape: btape.c:1909-0 Wrote block to device. >>>> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) >>>> btape: btape.c:1907-0 Wrote one record of 64412 bytes. >>>> btape: btape.c:1909-0 Wrote block to device. >>>> btape: btape.c:1907-0 Wrote one record of 64412 bytes. >>>> btape: btape.c:1909-0 Wrote block to device. >>>> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) >>>> btape: btape.c:1907-0 Wrote one record of 64412 bytes. >>>> btape: btape.c:1909-0 Wrote block to device. >>>> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) >>>> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) >>>> btape: btape.c:574-0 Rewound "DLT" (/dev/nsa1) >>>> btape: btape.c:1634-0 Now forward spacing 1 file. >>>> We should be in file 1. I am at file 1. This is correct! >>>> btape: btape.c:1646-0 Now forward spacing 2 files. >>>> We should be in file 3. I am at file 3. This is correct! >>>> btape: btape.c:574-0 Rewound "DLT" (/dev/nsa1) >>>> btape: btape.c:1659-0 Now forward spacing 4 files. >>>> We should be in file 4. I am at file 4. This is correct! >>>>=20 >>>> btape: btape.c:1677-0 Now forward spacing 1 more file. >>>> We should be in file 5. I am at file 5. This is correct! >>>>=20 >>>> =3D=3D=3D End Forward space files test =3D=3D=3D >>>>=20 >>>>=20 >>>> Ah, I see you have an autochanger configured. >>>> To test the autochanger you must have a blank tape >>>> that I can write on in Slot 1. >>>>=20 >>>> Do you wish to continue with the Autochanger test? (y/n): y >>>>=20 >>>>=20 >>>> =3D=3D=3D Autochanger test =3D=3D=3D >>>>=20 >>>> 3301 Issuing autochanger "loaded" command. >>>> Nothing loaded in the drive. OK. >>>> 3303 Issuing autochanger "load 1 0" command. >>>> 3303 Autochanger "load 1 0" status is OK. >>>> btape: btape.c:469-0 open device "DLT" (/dev/nsa1): OK >>>> btape: btape.c:1564-0 Rewound "DLT" (/dev/nsa1) >>>> btape: btape.c:1571-0 Wrote EOF to "DLT" (/dev/nsa1) >>>>=20 >>>> The test autochanger worked!! >>>=20 >>> Great, thanks for running the test! Looks like things are working = as well >>> as they were before. >>>=20 >>>>>=20 >>>>>> I'll let the other Bacula devs know about this. They deal with = the hardware. I work on PostgreSQL. >>>>>>=20 >>>>>=20 >>>>> Thanks! If there are additional features they would like out of = the tape >>>>> driver, I'm happy to talk about it. (Or help if they'd like to = use the new >>>>> status reporting ioctl, MTIOCEXTGET or any of the other new = ioctls.) >>>>=20 >>>> Errors are interesting to me. Especially corrected errors. They = are a good indicator of tape quality. >>>>=20 >>>=20 >>> Yes. At least on modern drives, there is a good bit available in = the log >>> pages. You can try seeing what logs your drive supports by = installing the >>> sg3_utils package and running 'sg_logs sa1 --all'. >>>=20 >>> Given the large amount of data available on some drives, and the = difficulty >>> distilling it down to a clear good/bad, I probably won't stick that = in the >>> 'mt status' output. >>>=20 >>> But you can certainly take a look at it and have an idea of whether = your >>> particular tape/drive are experiencing issues. >>=20 >> That's a lot of output: = https://gist.github.com/dlangille/0e15a7fbf7acab56fd32 >=20 > That is a lot. >=20 >> I will run some Bacula jobs soon. I'm still setting up config files. >=20 > Thanks for all the testing, I really appreciate it! >=20 > Ken > --=20 > Kenneth Merry > ken@FreeBSD.ORG =E2=80=94=20 Dan Langille http://langille.org/ From owner-freebsd-current@FreeBSD.ORG Mon Mar 2 16:31:52 2015 Return-Path: Delivered-To: current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 75ADD8B8; Mon, 2 Mar 2015 16:31:52 +0000 (UTC) Received: from mithlond.kdm.org (mithlond.kdm.org [70.56.43.85]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "A1-33714", Issuer "A1-33714" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 01E28ED4; Mon, 2 Mar 2015 16:31:51 +0000 (UTC) Received: from mithlond.kdm.org (localhost [127.0.0.1]) by mithlond.kdm.org (8.14.9/8.14.9) with ESMTP id t22GVl4f085578 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Mon, 2 Mar 2015 09:31:47 -0700 (MST) (envelope-from ken@mithlond.kdm.org) Received: (from ken@localhost) by mithlond.kdm.org (8.14.9/8.14.9/Submit) id t22GVkpb085577; Mon, 2 Mar 2015 09:31:46 -0700 (MST) (envelope-from ken) Date: Mon, 2 Mar 2015 09:31:46 -0700 From: "Kenneth D. Merry" To: Dan Langille Subject: Re: sa(4) driver changes available for test Message-ID: <20150302163146.GA85515@mithlond.kdm.org> References: <20150214003232.GA63990@mithlond.kdm.org> <7CA52DF3-E073-4F50-BE4E-01C51CCDF2C7@langille.org> <20150217183645.GA30947@mithlond.kdm.org> <303C48AC-0DA8-447C-8DE7-0A1DD5DD7351@langille.org> <20150302003150.GB71528@mithlond.kdm.org> <20150302022957.GB73433@mithlond.kdm.org> <0F17CDDB-4CE9-4881-B948-6B9BEDEB6455@langille.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <0F17CDDB-4CE9-4881-B948-6B9BEDEB6455@langille.org> User-Agent: Mutt/1.5.23 (2014-03-12) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (mithlond.kdm.org [127.0.0.1]); Mon, 02 Mar 2015 09:31:47 -0700 (MST) X-Spam-Status: No, score=-2.9 required=5.0 tests=ALL_TRUSTED,BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on mithlond.kdm.org Cc: current@freebsd.org, scsi@freebsd.org X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Mar 2015 16:31:52 -0000 On Mon, Mar 02, 2015 at 11:09:57 -0500, Dan Langille wrote: > > > On Mar 1, 2015, at 9:29 PM, Kenneth D. Merry wrote: > > > > On Sun, Mar 01, 2015 at 19:41:07 -0500, Dan Langille wrote: > >> > >>> On Mar 1, 2015, at 7:31 PM, Kenneth D. Merry wrote: > >>> > >>> On Sun, Mar 01, 2015 at 19:15:05 -0500, Dan Langille wrote: > >>>> > >>>>> On Feb 17, 2015, at 1:36 PM, Kenneth D. Merry wrote: > >>>>> > >>>>> On Sat, Feb 14, 2015 at 18:22:43 -0500, Dan Langille wrote: > >>>>>> > >>>>>>> On Feb 13, 2015, at 7:32 PM, Kenneth D. Merry wrote: > >>>>>>> > >>>>>>> > >>>>>>> I have a fairly large set of changes to the sa(4) driver and mt(1) driver > >>>>>>> that I'm planning to commit in the near future. > >>>>>>> > >>>>>>> A description of the changes is here and below in this message. > >>>>>>> > >>>>>>> If you have tape hardware and the inclination, I'd appreciate testing and > >>>>>>> feedback. > >>>>>> > >>>>>> I have a DLT 8000 and an SDLT 220. > >>>>>> > >>>>>> I don't have anything running current, but I have a spare machine which I could use for testing. > >>>>>> > >>>>>> Do you see any value is tests with that hardware? I'd be testing it via Bacula. > >>>>>> > >>>>>> disclosure: I'm the sysutils/bacula-* maintainer and a Bacula committer. > >>>>>> > >>>>> > >>>>> Actually, yes. Bacula is a bit tricky to configure, so your trying it out > >>>>> would be helpful if you have the time. > >>>>> > >>>>> In looking at the manuals for both the SDLT 220 and the DLT 8000, they both > >>>>> claim to support long position information for the SCSI READ POSITION > >>>>> command. > >>>>> > >>>>> You can see what I'm talking about by doing: > >>>>> > >>>>> mt eod > >>>>> mt status > >>>>> > >>>>> On my DDS-4 tape drive, this shows: > >>>>> > >>>>> # mt -f /dev/nsa3 status > >>>>> Drive: sa3: Serial Number: HJ00YWY > >>>>> --------------------------------- > >>>>> Mode Density Blocksize bpi Compression > >>>>> Current: 0x26:DDS-4 1024 bytes 97000 enabled (DCLZ) > >>>>> --------------------------------- > >>>>> Current Driver State: at rest. > >>>>> --------------------------------- > >>>>> Partition: 0 Calc File Number: -1 Calc Record Number: -1 > >>>>> Residual: 0 Reported File Number: -1 Reported Record Number: -1 > >>>>> Flags: None > >>>>> > >>>>> But on an LTO-5, which will give long position information, I get: > >>>>> > >>>>> [root@doc ~]# mt status > >>>>> Drive: sa0: > >>>>> --------------------------------- > >>>>> Mode Density Blocksize bpi Compression > >>>>> Current: 0x58:LTO-5 variable 384607 enabled (0x1) > >>>>> --------------------------------- > >>>>> Current Driver State: at rest. > >>>>> --------------------------------- > >>>>> Partition: 0 Calc File Number: 2 Calc Record Number: -1 > >>>>> Residual: 0 Reported File Number: 2 Reported Record Number: 32373 > >>>>> Flags: None > >>>>> > >>>>> That, in combination with the changes I made to the position information > >>>>> code in the driver, mean that even the old MTIOCGET ioctl should return an > >>>>> accurate file number at end of data. e.g., on the LTO-5: > >>>>> > >>>>> [root@doc ~]# mt ostatus > >>>>> Mode Density Blocksize bpi Compression > >>>>> Current: 0x58:LTO-5 variable 384607 0x1 > >>>>> ---------available modes--------- > >>>>> 0: 0x58:LTO-5 variable 384607 0x1 > >>>>> 1: 0x58:LTO-5 variable 384607 0x1 > >>>>> 2: 0x58:LTO-5 variable 384607 0x1 > >>>>> 3: 0x58:LTO-5 variable 384607 0x1 > >>>>> --------------------------------- > >>>>> Current Driver State: at rest. > >>>>> --------------------------------- > >>>>> File Number: 2 Record Number: -1 Residual Count -1 > >>>>> > >>>>> So the thing to try, in addition to just making sure that Bacula continues > >>>>> to work properly, is to try setting this for the tape drive in > >>>>> bacula-sd.conf: > >>>>> > >>>>> Hardware End of Medium = yes > >>>>> > >>>>> It looks like the Bacula tape program (btape) has a test mode, and it would > >>>>> be good to run through the tests on one of the tape drives and see whether > >>>>> they work, and whether the results are different before and after the > >>>>> changes. I'm not sure how to enable the test mode. > >>>> > >>>> I have this in /usr/local/etc/bacula/bacula-sd.conf > >>>> > >>>> Device { > >>>> Name = DLT > >>>> Description = "QUANTUM DLT7000 1624" > >>>> Media Type = DLT > >>>> Archive Device = /dev/nsa1 > >>>> > >>>> Autochanger = YES > >>>> Drive Index = 0 > >>>> > >>>> Offline On Unmount = no > >>>> Hardware End of Medium = yes > >>>> BSF at EOM = yes > >>>> Backward Space Record = no > >>>> Fast Forward Space File = no > >>>> TWO EOF = yes > >>>> } > >>>> > >>>> FYI, http://www.freebsddiary.org/digital-tl891.php (from 2006) has a btape test on this same model. > >>>> > >>>> Here's the test I ran tonight: > >>>> > >>>> [root@cuppy:/usr/home/dan] # btape -c /usr/local/etc/bacula/bacula-sd.conf /dev/nsa1 > >>>> Tape block granularity is 1024 bytes. > >>>> btape: butil.c:287-0 Using device: "/dev/nsa1" for writing. > >>>> btape: btape.c:469-0 open device "DLT" (/dev/nsa1): OK > >>>> *test > >>>> > >>>> === Write, rewind, and re-read test === > >>>> > >>>> I'm going to write 10000 records and an EOF > >>>> then write 10000 records and an EOF, then rewind, > >>>> and re-read the data to verify that it is correct. > >>>> > >>>> This is an *essential* feature ... > >>>> > >>>> btape: btape.c:1152-0 Wrote 10000 blocks of 64412 bytes. > >>>> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) > >>>> btape: btape.c:1168-0 Wrote 10000 blocks of 64412 bytes. > >>>> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) > >>>> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) > >>>> btape: btape.c:1210-0 Rewind OK. > >>>> 10000 blocks re-read correctly. > >>>> Got EOF on tape. > >>>> 10000 blocks re-read correctly. > >>>> === Test Succeeded. End Write, rewind, and re-read test === > >>>> > >>>> btape: btape.c:1277-0 Block position test > >>>> btape: btape.c:1289-0 Rewind OK. > >>>> Reposition to file:block 0:4 > >>>> Block 5 re-read correctly. > >>>> Reposition to file:block 0:200 > >>>> Block 201 re-read correctly. > >>>> Reposition to file:block 0:9999 > >>>> Block 10000 re-read correctly. > >>>> Reposition to file:block 1:0 > >>>> Block 10001 re-read correctly. > >>>> Reposition to file:block 1:600 > >>>> Block 10601 re-read correctly. > >>>> Reposition to file:block 1:9999 > >>>> Block 20000 re-read correctly. > >>>> === Test Succeeded. End Write, rewind, and re-read test === > >>>> > >>>> > >>>> > >>>> === Append files test === > >>>> > >>>> This test is essential to Bacula. > >>>> > >>>> I'm going to write one record in file 0, > >>>> two records in file 1, > >>>> and three records in file 2 > >>>> > >>>> btape: btape.c:574-0 Rewound "DLT" (/dev/nsa1) > >>>> btape: btape.c:1907-0 Wrote one record of 64412 bytes. > >>>> btape: btape.c:1909-0 Wrote block to device. > >>>> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) > >>>> btape: btape.c:1907-0 Wrote one record of 64412 bytes. > >>>> btape: btape.c:1909-0 Wrote block to device. > >>>> btape: btape.c:1907-0 Wrote one record of 64412 bytes. > >>>> btape: btape.c:1909-0 Wrote block to device. > >>>> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) > >>>> btape: btape.c:1907-0 Wrote one record of 64412 bytes. > >>>> btape: btape.c:1909-0 Wrote block to device. > >>>> btape: btape.c:1907-0 Wrote one record of 64412 bytes. > >>>> btape: btape.c:1909-0 Wrote block to device. > >>>> btape: btape.c:1907-0 Wrote one record of 64412 bytes. > >>>> btape: btape.c:1909-0 Wrote block to device. > >>>> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) > >>>> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) > >>>> btape: btape.c:469-0 open device "DLT" (/dev/nsa1): OK > >>>> btape: btape.c:574-0 Rewound "DLT" (/dev/nsa1) > >>>> btape: btape.c:1420-0 Now moving to end of medium. > >>> > >>> This is the critical piece. The test moves the tape to the end of the > >>> medium. With hardware position information, you can tell what filemark > >>> you're on. Without it, you can't. > >>> > >>>> btape: btape.c:622-0 tape_dev.c:345 ioctl MTIOCGET error on "DLT" (/dev/nsa1). ERR=No error: 0. > >>>> We should be in file 3. I am at file 0. This is NOT correct!!!! > >>>> > >>>> Append test failed. Attempting again. > >>>> Setting "Hardware End of Medium = no > >>>> and "Fast Forward Space File = no > >>>> and retrying append test. > >>> > >>> This is not surprsing, given that the drive doesn't support long read > >>> position data. (It's a SCSI-2 device.) So that means that Bacula will > >>> need to do it manually. > >> > >> Yes, I have nothing newer than SCSI-2. Even my SDLT is SCSI-2 but that > >> tape library is hooked up to a different computer and was doing backups today. > > > > So, here is one thing that we can try to see whether these drives support > > long position information, even though they only claim to be SCSI-2. If > > they do, we can potentially add a quirk (or autodetection) to enable it. > > The code currently doesn't bother asking drives that claim to be SCSI-2 > > for long position information. (Because that feature was added in the > > SSC spec, which came after SCSI-2.) > > > > Issue a READ POSITION with the short form specified: > > > > camcontrol cmd sa1 -v -c "34 0 0 0 0 0 0 0 0 0" -i 20 - |hd > > > > Issue a READ POSITION with the vendor-specific block numbers: > > > > camcontrol cmd sa1 -v -c "34 1 0 0 0 0 0 0 0 0" -i 20 - |hd > > > > Issue a READ POSITION with the long form data: > > > > camcontrol cmd sa1 -v -c "34 6 0 0 0 0 0 0 0 0" -i 32 - |hd > > > > If it supports the last one, then I can put a quirk (or autodetection) in > > the driver and Bacula will get the hardware filemarks. You should try this > > on your SDLT as well. It may well support it. > > Sadly, no: > > [root@cuppy:~] # camcontrol cmd sa1 -v -c "34 0 0 0 0 0 0 0 0 0" -i 20 - |hd > 00000000 80 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| > 00000010 00 00 00 00 |....| > 00000014 > [root@cuppy:~] # camcontrol cmd sa1 -v -c "34 1 0 0 0 0 0 0 0 0" -i 20 - |hd > 00000000 80 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| > 00000010 00 00 00 00 |....| > 00000014 > [root@cuppy:~] # camcontrol cmd sa1 -v -c "34 6 0 0 0 0 0 0 0 0" -i 32 - |hd > camcontrol: error sending command > (pass2:ahc0:0:2:0): READ POSITION. CDB: 34 06 00 00 00 00 00 00 00 00 > (pass2:ahc0:0:2:0): CAM status: SCSI Status Error > (pass2:ahc0:0:2:0): SCSI status: Check Condition > (pass2:ahc0:0:2:0): SCSI sense: ILLEGAL REQUEST asc:24,0 (Invalid field in CDB) > (pass2:ahc0:0:2:0): Command byte 1 bit 2 is invalid > [root@cuppy:~] # Okay. Not too surprising I suppose. > The SDLT server is on 9.3 though: > > [root@knew:/usr/home/dan] # camcontrol cmd sa1 -v -c "34 0 0 0 0 0 0 0 0 0" -i 20 - |hd > camcontrol: cam_lookup_pass: CAMGETPASSTHRU ioctl failed > cam_lookup_pass: No such file or directory > cam_lookup_pass: either the pass driver isn't in your kernel > cam_lookup_pass: or sa1 doesn't exist > [root@knew:/usr/home/dan] # uname -a > FreeBSD knew.unixathome.org 9.3-RELEASE-p10 FreeBSD 9.3-RELEASE-p10 #0: Tue Feb 24 21:28:03 UTC 2015 root@amd64-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC amd64 > [root@knew:/usr/home/dan] # > > > It took me a while to figure that out... there is no sa1 on *this* system. > > But, my SDLT: > > [root@knew:/usr/home/dan] # camcontrol cmd sa0 -v -c "34 0 0 0 0 0 0 0 0 0" -i 20 - |hd > 00000000 80 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| > 00000010 00 00 00 00 |....| > 00000014 > [root@knew:/usr/home/dan] # camcontrol cmd sa0 -v -c "34 1 0 0 0 0 0 0 0 0" -i 20 - |hd > 00000000 80 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| > 00000010 00 00 00 00 |....| > 00000014 > [root@knew:/usr/home/dan] # camcontrol cmd sa0 -v -c "34 6 0 0 0 0 0 0 0 0" -i 32 - |hd > 00000000 80 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| > 00000010 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| > 00000020 > [root@knew:/usr/home/dan] # Just to confirm, can you send the output of: camcontrol inquiry sa0 -v I want to make certain it reports that it is SCSI-2. If so, I'll change the check in the driver to try asking for long position information on SCSI-2 devices. If it fails, it'll fall back to the regular method. > > > > Google didn't quickly produce a SCSI manual for the DEC drive, but the > > Quantum SDLT manual indicates that it supports long position data, despite > > identifying itself as a SCSI-2 drive. > > > >>> === Append files test === > >>>> > >>>> This test is essential to Bacula. > >>>> > >>>> I'm going to write one record in file 0, > >>>> two records in file 1, > >>>> and three records in file 2 > >>>> > >>>> btape: btape.c:574-0 Rewound "DLT" (/dev/nsa1) > >>>> btape: btape.c:1907-0 Wrote one record of 64412 bytes. > >>>> btape: btape.c:1909-0 Wrote block to device. > >>>> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) > >>>> btape: btape.c:1907-0 Wrote one record of 64412 bytes. > >>>> btape: btape.c:1909-0 Wrote block to device. > >>>> btape: btape.c:1907-0 Wrote one record of 64412 bytes. > >>>> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) > >>>> btape: btape.c:1907-0 Wrote one record of 64412 bytes. > >>>> btape: btape.c:1909-0 Wrote block to device. > >>>> btape: btape.c:1907-0 Wrote one record of 64412 bytes. > >>>> btape: btape.c:1909-0 Wrote block to device. > >>>> btape: btape.c:1907-0 Wrote one record of 64412 bytes. > >>>> btape: btape.c:1909-0 Wrote block to device. > >>>> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) > >>>> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) > >>>> btape: btape.c:469-0 open device "DLT" (/dev/nsa1): OK > >>>> btape: btape.c:574-0 Rewound "DLT" (/dev/nsa1) > >>>> btape: btape.c:1420-0 Now moving to end of medium. > >>>> btape: btape.c:625-0 Moved to end of medium. > >>>> We should be in file 3. I am at file 3. This is correct! > >>>> > >>>> Now the important part, I am going to attempt to append to the tape. > >>>> > >>>> btape: btape.c:1907-0 Wrote one record of 64412 bytes. > >>>> btape: btape.c:1909-0 Wrote block to device. > >>>> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) > >>>> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) > >>>> btape: btape.c:574-0 Rewound "DLT" (/dev/nsa1) > >>>> Done appending, there should be no I/O errors > >>>> > >>>> Doing Bacula scan of blocks: > >>>> 1 block of 64448 bytes in file 1 > >>>> End of File mark. > >>>> 2 blocks of 64448 bytes in file 2 > >>>> End of File mark. > >>>> 3 blocks of 64448 bytes in file 3 > >>>> End of File mark. > >>>> 1 block of 64448 bytes in file 4 > >>>> End of File mark. > >>>> Total files=4, blocks=7, bytes = 451,136 > >>>> End scanning the tape. > >>>> We should be in file 4. I am at file 4. This is correct! > >>>> > >>>> > >>>> It looks like the test worked this time, please add: > >>>> > >>>> Hardware End of Medium = No > >>>> > >>>> Fast Forward Space File = No > >>>> to your Device resource in the Storage conf file. > >>>> > >>>> The above Bacula scan should have output identical to what follows. > >>>> Please double check it ... > >>>> === Sample correct output === > >>>> 1 block of 64448 bytes in file 1 > >>>> End of File mark. > >>>> 2 blocks of 64448 bytes in file 2 > >>>> End of File mark. > >>>> 3 blocks of 64448 bytes in file 3 > >>>> End of File mark. > >>>> 1 block of 64448 bytes in file 4 > >>>> End of File mark. > >>>> Total files=4, blocks=7, bytes = 451,136 > >>>> === End sample correct output === > >>>> > >>>> If the above scan output is not identical to the > >>>> sample output, you MUST correct the problem > >>>> or Bacula will not be able to write multiple Jobs to > >>>> the tape. > >>>> > >>>> Skipping read backwards test because BSR turned off. > >>>> > >>>> > >>>> === Forward space files test === > >>>> > >>>> This test is essential to Bacula. > >>>> > >>>> I'm going to write five files then test forward spacing > >>>> > >>>> btape: btape.c:574-0 Rewound "DLT" (/dev/nsa1) > >>>> btape: btape.c:1907-0 Wrote one record of 64412 bytes. > >>>> btape: btape.c:1909-0 Wrote block to device. > >>>> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) > >>>> btape: btape.c:1907-0 Wrote one record of 64412 bytes. > >>>> btape: btape.c:1909-0 Wrote block to device. > >>>> btape: btape.c:1907-0 Wrote one record of 64412 bytes. > >>>> btape: btape.c:1909-0 Wrote block to device. > >>>> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) > >>>> btape: btape.c:1907-0 Wrote one record of 64412 bytes. > >>>> btape: btape.c:1909-0 Wrote block to device. > >>>> btape: btape.c:1907-0 Wrote one record of 64412 bytes. > >>>> btape: btape.c:1909-0 Wrote block to device. > >>>> btape: btape.c:1907-0 Wrote one record of 64412 bytes. > >>>> btape: btape.c:1909-0 Wrote block to device. > >>>> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) > >>>> btape: btape.c:1907-0 Wrote one record of 64412 bytes. > >>>> btape: btape.c:1909-0 Wrote block to device. > >>>> btape: btape.c:1907-0 Wrote one record of 64412 bytes. > >>>> btape: btape.c:1909-0 Wrote block to device. > >>>> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) > >>>> btape: btape.c:1907-0 Wrote one record of 64412 bytes. > >>>> btape: btape.c:1909-0 Wrote block to device. > >>>> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) > >>>> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) > >>>> btape: btape.c:574-0 Rewound "DLT" (/dev/nsa1) > >>>> btape: btape.c:1634-0 Now forward spacing 1 file. > >>>> We should be in file 1. I am at file 1. This is correct! > >>>> btape: btape.c:1646-0 Now forward spacing 2 files. > >>>> We should be in file 3. I am at file 3. This is correct! > >>>> btape: btape.c:574-0 Rewound "DLT" (/dev/nsa1) > >>>> btape: btape.c:1659-0 Now forward spacing 4 files. > >>>> We should be in file 4. I am at file 4. This is correct! > >>>> > >>>> btape: btape.c:1677-0 Now forward spacing 1 more file. > >>>> We should be in file 5. I am at file 5. This is correct! > >>>> > >>>> === End Forward space files test === > >>>> > >>>> > >>>> Ah, I see you have an autochanger configured. > >>>> To test the autochanger you must have a blank tape > >>>> that I can write on in Slot 1. > >>>> > >>>> Do you wish to continue with the Autochanger test? (y/n): y > >>>> > >>>> > >>>> === Autochanger test === > >>>> > >>>> 3301 Issuing autochanger "loaded" command. > >>>> Nothing loaded in the drive. OK. > >>>> 3303 Issuing autochanger "load 1 0" command. > >>>> 3303 Autochanger "load 1 0" status is OK. > >>>> btape: btape.c:469-0 open device "DLT" (/dev/nsa1): OK > >>>> btape: btape.c:1564-0 Rewound "DLT" (/dev/nsa1) > >>>> btape: btape.c:1571-0 Wrote EOF to "DLT" (/dev/nsa1) > >>>> > >>>> The test autochanger worked!! > >>> > >>> Great, thanks for running the test! Looks like things are working as well > >>> as they were before. > >>> > >>>>> > >>>>>> I'll let the other Bacula devs know about this. They deal with the hardware. I work on PostgreSQL. > >>>>>> > >>>>> > >>>>> Thanks! If there are additional features they would like out of the tape > >>>>> driver, I'm happy to talk about it. (Or help if they'd like to use the new > >>>>> status reporting ioctl, MTIOCEXTGET or any of the other new ioctls.) > >>>> > >>>> Errors are interesting to me. Especially corrected errors. They are a good indicator of tape quality. > >>>> > >>> > >>> Yes. At least on modern drives, there is a good bit available in the log > >>> pages. You can try seeing what logs your drive supports by installing the > >>> sg3_utils package and running 'sg_logs sa1 --all'. > >>> > >>> Given the large amount of data available on some drives, and the difficulty > >>> distilling it down to a clear good/bad, I probably won't stick that in the > >>> 'mt status' output. > >>> > >>> But you can certainly take a look at it and have an idea of whether your > >>> particular tape/drive are experiencing issues. > >> > >> That's a lot of output: https://gist.github.com/dlangille/0e15a7fbf7acab56fd32 > > > > That is a lot. > > > >> I will run some Bacula jobs soon. I'm still setting up config files. > > > > Thanks for all the testing, I really appreciate it! > > > > Ken > > -- > > Kenneth Merry > > ken@FreeBSD.ORG > > ? > Dan Langille > http://langille.org/ > > > > > Ken -- Kenneth Merry ken@FreeBSD.ORG From owner-freebsd-current@FreeBSD.ORG Mon Mar 2 16:43:18 2015 Return-Path: Delivered-To: current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 66E46B28; Mon, 2 Mar 2015 16:43:18 +0000 (UTC) Received: from clavin1.langille.org (clavin.langille.org [162.208.116.86]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "clavin.langille.org", Issuer "StartCom Class 2 Primary Intermediate Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 3A179FE4; Mon, 2 Mar 2015 16:43:17 +0000 (UTC) Received: from (clavin1.int.langille.org (clavin1.int.unixathome.org [10.4.7.7]) (Authenticated sender: hidden) with ESMTPSA id 903706D85 ; Mon, 2 Mar 2015 16:43:16 +0000 (UTC) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 8.2 \(2070.6\)) Subject: Re: sa(4) driver changes available for test From: Dan Langille In-Reply-To: <20150302020608.GA73433@mithlond.kdm.org> Date: Mon, 2 Mar 2015 11:43:15 -0500 Content-Transfer-Encoding: quoted-printable Message-Id: <30C55F00-2DE4-4596-96EA-2E3CC40B4DB6@langille.org> References: <20150214003232.GA63990@mithlond.kdm.org> <4A478C5C-7965-498E-9F0F-80192265E310@langille.org> <20150302001833.GA71528@mithlond.kdm.org> <6C82281F-649A-4DA8-8ACF-17E81C04F730@langille.org> <20150302003658.GA72258@mithlond.kdm.org> <20150302020608.GA73433@mithlond.kdm.org> To: "Kenneth D. Merry" X-Mailer: Apple Mail (2.2070.6) Cc: current@freebsd.org, scsi@freebsd.org X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Mar 2015 16:43:18 -0000 > On Mar 1, 2015, at 9:06 PM, Kenneth D. Merry wrote: >=20 > On Sun, Mar 01, 2015 at 19:40:40 -0500, Dan Langille wrote: >>=20 >>> On Mar 1, 2015, at 7:36 PM, Kenneth D. Merry = wrote: >>>=20 >>> On Sun, Mar 01, 2015 at 19:28:37 -0500, Dan Langille wrote: >>>>=20 >>>>> On Mar 1, 2015, at 7:18 PM, Kenneth D. Merry = wrote: >>>>>=20 >>>>> On Sun, Mar 01, 2015 at 17:06:24 -0500, Dan Langille wrote: >>>>>>=20 >>>>>>> On Feb 13, 2015, at 7:32 PM, Kenneth D. Merry = wrote: >>>>>>>=20 >>>>>>>=20 >>>>>>> I have a fairly large set of changes to the sa(4) driver and = mt(1) driver >>>>>>> that I'm planning to commit in the near future. >>>>>>>=20 >>>>>>> A description of the changes is here and below in this message. >>>>>>>=20 >>>>>>> If you have tape hardware and the inclination, I'd appreciate = testing and >>>>>>> feedback. >>>>>>>=20 >>>>>>> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D >>>>>>> Rough draft commit message: >>>>>>>=20 >>>>>>> = http://people.freebsd.org/~ken/sa_changes_commitmsg.20150213.3.txt >>>>>>>=20 >>>>>>> The patches against FreeBSD/head as of SVN revision 278706: >>>>>>>=20 >>>>>>> http://people.freebsd.org/~ken/sa_changes.20150213.3.txt >>>>>>>=20 >>>>>>> And (untested) patches against FreeBSD stable/10 as of SVN = revision 278721. >>>>>>>=20 >>>>>>> = http://people.freebsd.org/~ken/sa_changes.stable_10.20150213.3.txt >>>>>>> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D >>>>>>>=20 >>>>>>> The intent is to get the tape infrastructure more up to date, so = we can >>>>>>> support LTFS and more modern tape drives: >>>>>>>=20 >>>>>>> http://www.ibm.com/systems/storage/tape/ltfs/ >>>>>>>=20 >>>>>>> I have ported IBM's LTFS Single Drive Edition to FreeBSD. The = port depends >>>>>>> on the patches linked above. It isn't fully cleaned up and = ready for >>>>>>> redistribution. If you're interested, though, let me know and = I'll tell >>>>>>> you when it is ready to go out. You need an IBM LTO-5, LTO-6, = TS1140 or >>>>>>> TS1150 tape drive. HP drives aren't supported by IBM's LTFS, = and older >>>>>>> drives don't have the necessary features to support LTFS. >>>>>>>=20 >>>>>>> The commit message below outlines most of the changes. >>>>>>>=20 >>>>>>> A few comments: >>>>>>>=20 >>>>>>> 1. I'm planning to commit the XPT_DEV_ADVINFO changes = separately. >>>>>>>=20 >>>>>>> 2. The XML output is similar to what GEOM and CTL do. It would = be nice to >>>>>>> figure out how to put a standard schema on it so that standard = tools >>>>>>> could read it. I don't know how feasible that is, since I = haven't >>>>>>> time to dig into it. If anyone has suggestions on whether that = is >>>>>>> feasible or advisable, I'd appreciate feedback. >>>>>>>=20 >>>>>>> 3. I have tested with a reasonable amount of tape hardware (see = below for a >>>>>>> list), but more testing and feedback would be good. >>>>>>>=20 >>>>>>> 4. Standard 'mt status' output looks like this: >>>>>>>=20 >>>>>>> # mt -f /dev/nsa3 status -v >>>>>>> Drive: sa3: Serial Number: 101500520A >>>>>>> --------------------------------- >>>>>>> Mode Density Blocksize bpi = Compression >>>>>>> Current: 0x5a:LTO-6 variable 384607 enabled = (0xff) >>>>>>> --------------------------------- >>>>>>> Current Driver State: at rest. >>>>>>> --------------------------------- >>>>>>> Partition: 0 Calc File Number: 0 Calc Record = Number: 0 >>>>>>> Residual: 0 Reported File Number: 0 Reported Record = Number: 0 >>>>>>> Flags: BOP >>>>>>>=20 >>>>>>> 5. 'mt status -v' looks like this: >>>>>>>=20 >>>>>>> # mt -f /dev/nsa3 status -v >>>>>>> Drive: sa3: Serial Number: 101500520A >>>>>>> --------------------------------- >>>>>>> Mode Density Blocksize bpi = Compression >>>>>>> Current: 0x5a:LTO-6 variable 384607 enabled = (0xff) >>>>>>> --------------------------------- >>>>>>> Current Driver State: at rest. >>>>>>> --------------------------------- >>>>>>> Partition: 0 Calc File Number: 0 Calc Record = Number: 0 >>>>>>> Residual: 0 Reported File Number: 0 Reported Record = Number: 0 >>>>>>> Flags: BOP >>>>>>> --------------------------------- >>>>>>> Tape I/O parameters: >>>>>>> Maximum I/O size allowed by driver and controller (maxio): = 1081344 bytes >>>>>>> Maximum I/O size reported by controller (cpi_maxio): 5197824 = bytes >>>>>>> Maximum block size supported by tape drive and media (max_blk): = 8388608 bytes >>>>>>> Minimum block size supported by tape drive and media (min_blk): = 1 bytes >>>>>>> Block granularity supported by tape drive and media (blk_gran): = 0 bytes >>>>>>> Maximum possible I/O size (max_effective_iosize): 1081344 bytes >>>>>>=20 >>>>>>=20 >>>>>> # mtx -f /dev/pass0 status >>>>>> Storage Changer /dev/pass0:2 Drives, 10 Slots ( 0 Import/Export ) >>>>>> Data Transfer Element 0:Empty >>>>>> Data Transfer Element 1:Empty >>>>>> Storage Element 1:Empty >>>>>> Storage Element 2:Empty >>>>>> Storage Element 3:Empty >>>>>> Storage Element 4:Full :VolumeTag=3DFAI260 = =20 >>>>>> Storage Element 5:Full :VolumeTag=3DFAI261 = =20 >>>>>> Storage Element 6:Full :VolumeTag=3DFAI262 = =20 >>>>>> Storage Element 7:Full :VolumeTag=3DFAI263 = =20 >>>>>> Storage Element 8:Empty >>>>>> Storage Element 9:Empty >>>>>> Storage Element 10:Empty >>>>>>=20 >>>>>>=20 >>>>>> It was at this point I spent the next 90 minute trying to get the = tape=20 >>>>>> drive out of the tape library to free a stuck tape. Some of this = was spent >>>>>> attempting, and failing, to undo a stripped screw. I stopped the = attempt when >>>>>> I noticed the screw did need to be removed. :/ >>>>>=20 >>>>> Thanks for all of the effort! Looks like it is paying off! :) >>>>>=20 >>>>>> When I do this command, I hear the drive move a bit, to read the = tape: >>>>>>=20 >>>>>> # mt -f /dev/nsa1 status >>>>>> Drive: sa1: Serial Number: CXA09S1340 >>>>>> --------------------------------- >>>>>> Mode Density Blocksize bpi = Compression >>>>>> Current: 0x1b:DLTapeIV(35GB) variable 85937 enabled = (IDRC) >>>>>> --------------------------------- >>>>>> Current Driver State: at rest. >>>>>> --------------------------------- >>>>>> Partition: 0 Calc File Number: 0 Calc Record Number: = 0 >>>>>> Residual: 0 Reported File Number: -1 Reported Record Number: = -1 >>>>>> Flags: None >>>>>=20 >>>>> Looks like the drive isn't reporting position information. It = will still >>>>> be useful to try it with Bacula, though. >>>>>=20 >>>>>> # mt -f /dev/nsa1 ostatus =20 >>>>>> Mode Density Blocksize bpi = Compression >>>>>> Current: 0x1b:DLTapeIV(35GB) variable 85937 IDRC >>>>>> ---------available modes--------- >>>>>> 0: 0x1b:DLTapeIV(35GB) variable 85937 IDRC >>>>>> 1: 0x1b:DLTapeIV(35GB) variable 85937 IDRC >>>>>> 2: 0x1b:DLTapeIV(35GB) variable 85937 IDRC >>>>>> 3: 0x1b:DLTapeIV(35GB) variable 85937 IDRC >>>>>> --------------------------------- >>>>>> Current Driver State: at rest. >>>>>> --------------------------------- >>>>>> File Number: 0 Record Number: 0 Residual Count 0 >>>>>>=20 >>>>>>=20 >>>>>> After doing a very small tar -c and tar -x, I have: >>>>>>=20 >>>>>> # mt -f /dev/nsa1 /dev/nsa1 ostatus >>>>>> Mode Density Blocksize bpi = Compression >>>>>> Current: 0x1b:DLTapeIV(35GB) variable 85937 IDRC >>>>>> ---------available modes--------- >>>>>> 0: 0x1b:DLTapeIV(35GB) variable 85937 IDRC >>>>>> 1: 0x1b:DLTapeIV(35GB) variable 85937 IDRC >>>>>> 2: 0x1b:DLTapeIV(35GB) variable 85937 IDRC >>>>>> 3: 0x1b:DLTapeIV(35GB) variable 85937 IDRC >>>>>> --------------------------------- >>>>>> Current Driver State: at rest. >>>>>> --------------------------------- >>>>>> File Number: 0 Record Number: 7 Residual Count 0 >>>>>=20 >>>>> Woohoo! It works. >>>>>=20 >>>>>> # mt -f /dev/nsa1 status -v >>>>>> Drive: sa1: Serial Number: CXA09S1340 >>>>>> --------------------------------- >>>>>> Mode Density Blocksize bpi = Compression >>>>>> Current: 0x1b:DLTapeIV(35GB) variable 85937 enabled = (IDRC) >>>>>> --------------------------------- >>>>>> Current Driver State: at rest. >>>>>> --------------------------------- >>>>>> Partition: 0 Calc File Number: 0 Calc Record Number: = 7 >>>>>> Residual: 0 Reported File Number: -1 Reported Record Number: = -1 >>>>>> Flags: None >>>>>> --------------------------------- >>>>>> Tape I/O parameters: >>>>>> Maximum I/O size allowed by driver and controller (maxio): 65536 = bytes >>>>>> Maximum I/O size reported by controller (cpi_maxio): 0 bytes >>>>>> Maximum block size supported by tape drive and media (max_blk): = 16777214 bytes >>>>>> Minimum block size supported by tape drive and media (min_blk): 2 = bytes >>>>>> Block granularity supported by tape drive and media (blk_gran): 0 = bytes >>>>>> Maximum possible I/O size (max_effective_iosize): 65536 bytes >>>>>>=20 >>>>>> I may not get to testing Bacula today. =20 >>>>>>=20 >>>>>> Based on the above, is there any commands you'd like me to try? >>>>>=20 >>>>> Aside from making sure things work okay with Bacula, that is = probably >>>>> sufficient. These drives won't support density reports or = position >>>>> information. >>>>>=20 >>>>>> Read below regarding two tape drives >>>>>>=20 >>>>>>>=20 >>>>>>> 6. Existing applications should work without changes. If not, = please let >>>>>>> me know. Hopefully they will move over time to the new = interfaces. >>>>>>>=20 >>>>>>> 7. There are lots of additional features that could be added = later. >>>>>>> Append-only support, encryption, more log pages, etc. >>>>>>>=20 >>>>>>> 8. I have SCSI READ ATTRIBUTE changes for camcontrol(8) that = will go in >>>>>>> separately. These changes allow displaying the contents of the = MAM >>>>>>> (Medium Auxiliary Memory) chips on LTO, TS and other modern tape = drives. >>>>>>> These are good, and a future possible direction is adding = attributes=20 >>>>>>> to the status XML from the sa(4) driver. >>>>>>>=20 >>>>>>> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D >>>>>>> Significant upgrades to sa(4) and mt(1). >>>>>>>=20 >>>>>>> The primary focus of these changes is to modernize FreeBSD's >>>>>>> tape infrastructure so that we can take advantage of some of the >>>>>>> features of modern tape drives and allow support for LTFS. >>>>>>>=20 >>>>>>> Significant changes and new features include: >>>>>>>=20 >>>>>>> o sa(4) driver status and parameter information is now exported = via an >>>>>>> XML structure. This will allow for changes and improvements = later >>>>>>> on that will not break userland applications. The old MTIOCGET >>>>>>> status ioctl remains, so applications using the existing = interface >>>>>>> will not break. >>>>>>>=20 >>>>>>> o 'mt status' now reports drive-reported tape position = information >>>>>>> as well as the previously available calculated tape position >>>>>>> information. These numbers will be different at times, because >>>>>>> the drive-reported block numbers are relative to BOP (Beginning >>>>>>> of Partition), but the block numbers calculated previously via >>>>>>> sa(4) (and still provided) are relative to the last filemark. >>>>>>> Both numbers are now provided. 'mt status' now also shows the >>>>>>> drive INQUIRY information, serial number and any position flags >>>>>>> (BOP, EOT, etc.) provided with the tape position information. >>>>>>> 'mt status -v' adds information on the maximum possible I/O = size, >>>>>>> and the underlying values used to calculate it. >>>>>>>=20 >>>>>>> o The extra sa(4) /dev entries (/dev/saN.[0-3]) have been = removed. >>>>>>=20 >>>>>> How does this affect a tape library with more than one tape = drive? >>>>>>=20 >>>>>> [root@cuppy:~] # camcontrol amcontrol devlist >>>>>> at scbus0 target 0 lun 0 = (pass0,ch0) >>>>>> at scbus0 target 2 lun 0 = (sa1,pass2) >>>>>> at scbus1 target 0 lun 0 = (pass3,ada0) >>>>>> at scbus2 target 0 lun 0 = (pass4,ada1) >>>>>> at scbus3 target 0 lun 0 = (pass5,ses0) >>>>>>=20 >>>>>> This system has two tapes drives and I can access them through = the front panel but: >>>>>>=20 >>>>>> # ls -l /dev/*sa* >>>>>> crw-rw---- 1 root operator 0x65 Feb 28 22:04 /dev/esa1 >>>>>> crw-rw---- 1 root operator 0x64 Mar 1 22:43 /dev/nsa1 >>>>>> crw-rw---- 1 root operator 0x63 Feb 28 22:04 /dev/sa1 >>>>>> crw-rw---- 1 root operator 0x62 Feb 28 22:04 /dev/sa1.ctl >>>>>>=20 >>>>>> ... only one tape drives shows up. >>>>>=20 >>>>>=20 >>>>> Hmm. The tape drive is listed as sa1, which implies that there = may be an >>>>> sa0 that was there previously or is in the process of probing. = What does >>>>> dmesg show? How about 'camcontrol devlist -v'? >>>>=20 >>>> # camcontrol devlist -v >>>> scbus0 on ahc0 bus 0: >>>> at scbus0 target 0 lun 0 = (pass0,ch0) >>>> at scbus0 target 2 lun 0 = (sa1,pass2) >>>> <> at scbus0 target -1 lun ffffffff = () >>>> scbus1 on ahcich2 bus 0: >>>> at scbus1 target 0 lun 0 = (pass3,ada0) >>>> <> at scbus1 target -1 lun ffffffff = () >>>> scbus2 on ahcich4 bus 0: >>>> at scbus2 target 0 lun 0 = (pass4,ada1) >>>> <> at scbus2 target -1 lun ffffffff = () >>>> scbus3 on ahciem0 bus 0: >>>> at scbus3 target 0 lun 0 = (pass5,ses0) >>>> <> at scbus3 target -1 lun ffffffff = () >>>> scbus-1 on xpt0 bus 0: >>>> <> at scbus-1 target -1 lun = ffffffff (xpt0) >>>>=20 >>>>=20 >>>> BUT! >>>>=20 >>>> # grep sa /var/run/dmesg.boot=20 >>>> VT-x: (disabled in BIOS) PAT,HLT,MTF,PAUSE,EPT,UG,VPID >>>> module_register_init: MOD_LOAD (vesa, 0xffffffff80de3720, 0) error = 19 >>>> alc0: Using 1 MSIX message(s). >>>> isab0: at device 31.0 on pci0 >>>> isa0: on isab0 >>>> orm0: at iomem 0xce800-0xcefff on isa0 >>>> atkbdc0: at port 0x60,0x64 on isa0 >>>> sa0 at ahc0 bus 0 scbus0 target 1 lun 0 >>>> sa0: Removable Sequential Access SCSI-2 = device=20 >>>> sa0: Serial Number CXA22S2338 >>>> sa0: 10.000MB/s transfers (10.000MHz, offset 15) >>>> sa0: quirks=3D0x100 >>>> sa1 at ahc0 bus 0 scbus0 target 2 lun 0 >>>> sa1: Removable Sequential Access SCSI-2 = device=20 >>>> sa1: Serial Number CXA09S1340 >>>> sa1: 10.000MB/s transfers (10.000MHz, offset 15) >>>> sa1: quirks=3D0x100 >>>=20 >>> If you run 'dmesg', you should have seen a message when it went = away. Perhaps >>> there will be something preceding it that will give us a clue about = the >>> problem. (Generally a selection timeout.) At least this does show = that >>> sa0 is at target 1, and so should not conflict with the library or = sa1. >>=20 >> Ahh: >>=20 >> Trying to mount root from zfs:system/bootenv/FreeBSDHEad []... >> sa0 at ahc0 bus 0 scbus0 target 1 lun 0 >> sa0: s/n CXA22S2338 detached >> (sa0:ahc0:0:1:0): Periph destroyed >> arp: 10.55.0.60 moved from e4:ce:8f:46:f1:98 to 78:ca:39:fe:d6:b3 on = em0 >> arp: 10.55.0.60 moved from e4:ce:8f:46:f1:98 to 78:ca:39:fe:d6:b3 on = em0 >> arp: 10.55.0.60 moved from 78:ca:39:fe:d6:b3 to e4:ce:8f:46:f1:98 on = em0 >> (sa1:ahc0:0:2:0): 64512-byte tape record bigger than supplied buffer >> (sa1:ahc0:0:2:0): 10240-byte tape record bigger than supplied buffer >=20 > Okay. Well, no indication of what happened. Perhaps boot -v will = show it, > perhaps not. >=20 Good news. After a reboot, both tape drives are present: $ ls -l /dev/*sa* crw-rw---- 1 root operator 0x61 Mar 2 17:27 /dev/esa0 crw-rw---- 1 root operator 0x65 Mar 2 17:27 /dev/esa1 crw-rw---- 1 root operator 0x60 Mar 2 17:27 /dev/nsa0 crw-rw---- 1 root operator 0x64 Mar 2 17:27 /dev/nsa1 crw-rw---- 1 root operator 0x5f Mar 2 17:27 /dev/sa0 crw-rw---- 1 root operator 0x5e Mar 2 17:27 /dev/sa0.ctl crw-rw---- 1 root operator 0x63 Mar 2 17:27 /dev/sa1 crw-rw---- 1 root operator 0x62 Mar 2 17:27 /dev/sa1.ctl =E2=80=94=20 Dan Langille http://langille.org/ From owner-freebsd-current@FreeBSD.ORG Mon Mar 2 16:44:11 2015 Return-Path: Delivered-To: current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 94AD2C66; Mon, 2 Mar 2015 16:44:11 +0000 (UTC) Received: from clavin1.langille.org (clavin.langille.org [162.208.116.86]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "clavin.langille.org", Issuer "StartCom Class 2 Primary Intermediate Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5286CFF8; Mon, 2 Mar 2015 16:44:10 +0000 (UTC) Received: from (clavin1.int.langille.org (clavin1.int.unixathome.org [10.4.7.7]) (Authenticated sender: hidden) with ESMTPSA id D16C86D87 ; Mon, 2 Mar 2015 16:44:09 +0000 (UTC) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 8.2 \(2070.6\)) Subject: Re: sa(4) driver changes available for test From: Dan Langille In-Reply-To: <20150302163146.GA85515@mithlond.kdm.org> Date: Mon, 2 Mar 2015 11:44:09 -0500 Content-Transfer-Encoding: quoted-printable Message-Id: <85E2DDA0-4A2B-4316-A318-3706B3A05C11@langille.org> References: <20150214003232.GA63990@mithlond.kdm.org> <7CA52DF3-E073-4F50-BE4E-01C51CCDF2C7@langille.org> <20150217183645.GA30947@mithlond.kdm.org> <303C48AC-0DA8-447C-8DE7-0A1DD5DD7351@langille.org> <20150302003150.GB71528@mithlond.kdm.org> <20150302022957.GB73433@mithlond.kdm.org> <0F17CDDB-4CE9-4881-B948-6B9BEDEB6455@langille.org> <20150302163146.GA85515@mithlond.kdm.org> To: "Kenneth D. Merry" X-Mailer: Apple Mail (2.2070.6) Cc: current@freebsd.org, scsi@freebsd.org X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Mar 2015 16:44:11 -0000 > On Mar 2, 2015, at 11:31 AM, Kenneth D. Merry wrote: >=20 > On Mon, Mar 02, 2015 at 11:09:57 -0500, Dan Langille wrote: >>=20 >>> On Mar 1, 2015, at 9:29 PM, Kenneth D. Merry = wrote: >>>=20 >>> On Sun, Mar 01, 2015 at 19:41:07 -0500, Dan Langille wrote: >>>>=20 >>>>> On Mar 1, 2015, at 7:31 PM, Kenneth D. Merry = wrote: >>>>>=20 >>>>> On Sun, Mar 01, 2015 at 19:15:05 -0500, Dan Langille wrote: >>>>>>=20 >>>>>>> On Feb 17, 2015, at 1:36 PM, Kenneth D. Merry = wrote: >>>>>>>=20 >>>>>>> On Sat, Feb 14, 2015 at 18:22:43 -0500, Dan Langille wrote: >>>>>>>>=20 >>>>>>>>> On Feb 13, 2015, at 7:32 PM, Kenneth D. Merry = wrote: >>>>>>>>>=20 >>>>>>>>>=20 >>>>>>>>> I have a fairly large set of changes to the sa(4) driver and = mt(1) driver >>>>>>>>> that I'm planning to commit in the near future. >>>>>>>>>=20 >>>>>>>>> A description of the changes is here and below in this = message. >>>>>>>>>=20 >>>>>>>>> If you have tape hardware and the inclination, I'd appreciate = testing and >>>>>>>>> feedback. >>>>>>>>=20 >>>>>>>> I have a DLT 8000 and an SDLT 220. >>>>>>>>=20 >>>>>>>> I don't have anything running current, but I have a spare = machine which I could use for testing. >>>>>>>>=20 >>>>>>>> Do you see any value is tests with that hardware? I'd be = testing it via Bacula. >>>>>>>>=20 >>>>>>>> disclosure: I'm the sysutils/bacula-* maintainer and a Bacula = committer. >>>>>>>>=20 >>>>>>>=20 >>>>>>> Actually, yes. Bacula is a bit tricky to configure, so your = trying it out >>>>>>> would be helpful if you have the time. >>>>>>>=20 >>>>>>> In looking at the manuals for both the SDLT 220 and the DLT = 8000, they both >>>>>>> claim to support long position information for the SCSI READ = POSITION >>>>>>> command. >>>>>>>=20 >>>>>>> You can see what I'm talking about by doing: >>>>>>>=20 >>>>>>> mt eod >>>>>>> mt status >>>>>>>=20 >>>>>>> On my DDS-4 tape drive, this shows: >>>>>>>=20 >>>>>>> # mt -f /dev/nsa3 status >>>>>>> Drive: sa3: Serial Number: = HJ00YWY >>>>>>> --------------------------------- >>>>>>> Mode Density Blocksize bpi = Compression >>>>>>> Current: 0x26:DDS-4 1024 bytes 97000 enabled = (DCLZ) >>>>>>> --------------------------------- >>>>>>> Current Driver State: at rest. >>>>>>> --------------------------------- >>>>>>> Partition: 0 Calc File Number: -1 Calc Record = Number: -1 >>>>>>> Residual: 0 Reported File Number: -1 Reported Record = Number: -1 >>>>>>> Flags: None >>>>>>>=20 >>>>>>> But on an LTO-5, which will give long position information, I = get: >>>>>>>=20 >>>>>>> [root@doc ~]# mt status >>>>>>> Drive: sa0: >>>>>>> --------------------------------- >>>>>>> Mode Density Blocksize bpi = Compression >>>>>>> Current: 0x58:LTO-5 variable 384607 enabled = (0x1) >>>>>>> --------------------------------- >>>>>>> Current Driver State: at rest. >>>>>>> --------------------------------- >>>>>>> Partition: 0 Calc File Number: 2 Calc Record = Number: -1 >>>>>>> Residual: 0 Reported File Number: 2 Reported Record = Number: 32373 >>>>>>> Flags: None >>>>>>>=20 >>>>>>> That, in combination with the changes I made to the position = information >>>>>>> code in the driver, mean that even the old MTIOCGET ioctl should = return an >>>>>>> accurate file number at end of data. e.g., on the LTO-5: >>>>>>>=20 >>>>>>> [root@doc ~]# mt ostatus >>>>>>> Mode Density Blocksize bpi = Compression >>>>>>> Current: 0x58:LTO-5 variable 384607 0x1 >>>>>>> ---------available modes--------- >>>>>>> 0: 0x58:LTO-5 variable 384607 0x1 >>>>>>> 1: 0x58:LTO-5 variable 384607 0x1 >>>>>>> 2: 0x58:LTO-5 variable 384607 0x1 >>>>>>> 3: 0x58:LTO-5 variable 384607 0x1 >>>>>>> --------------------------------- >>>>>>> Current Driver State: at rest. >>>>>>> --------------------------------- >>>>>>> File Number: 2 Record Number: -1 Residual Count -1 >>>>>>>=20 >>>>>>> So the thing to try, in addition to just making sure that Bacula = continues >>>>>>> to work properly, is to try setting this for the tape drive in >>>>>>> bacula-sd.conf: >>>>>>>=20 >>>>>>> Hardware End of Medium =3D yes >>>>>>>=20 >>>>>>> It looks like the Bacula tape program (btape) has a test mode, = and it would >>>>>>> be good to run through the tests on one of the tape drives and = see whether >>>>>>> they work, and whether the results are different before and = after the >>>>>>> changes. I'm not sure how to enable the test mode. >>>>>>=20 >>>>>> I have this in /usr/local/etc/bacula/bacula-sd.conf >>>>>>=20 >>>>>> Device { >>>>>> Name =3D DLT >>>>>> Description =3D "QUANTUM DLT7000 1624" >>>>>> Media Type =3D DLT >>>>>> Archive Device =3D /dev/nsa1 >>>>>>=20 >>>>>> Autochanger =3D YES >>>>>> Drive Index =3D 0 >>>>>>=20 >>>>>> Offline On Unmount =3D no >>>>>> Hardware End of Medium =3D yes >>>>>> BSF at EOM =3D yes >>>>>> Backward Space Record =3D no >>>>>> Fast Forward Space File =3D no >>>>>> TWO EOF =3D yes >>>>>> } >>>>>>=20 >>>>>> FYI, http://www.freebsddiary.org/digital-tl891.php (from 2006) = has a btape test on this same model. >>>>>>=20 >>>>>> Here's the test I ran tonight: >>>>>>=20 >>>>>> [root@cuppy:/usr/home/dan] # btape -c = /usr/local/etc/bacula/bacula-sd.conf /dev/nsa1 = =20 >>>>>> Tape block granularity is 1024 bytes. >>>>>> btape: butil.c:287-0 Using device: "/dev/nsa1" for writing. >>>>>> btape: btape.c:469-0 open device "DLT" (/dev/nsa1): OK >>>>>> *test >>>>>>=20 >>>>>> =3D=3D=3D Write, rewind, and re-read test =3D=3D=3D >>>>>>=20 >>>>>> I'm going to write 10000 records and an EOF >>>>>> then write 10000 records and an EOF, then rewind, >>>>>> and re-read the data to verify that it is correct. >>>>>>=20 >>>>>> This is an *essential* feature ... >>>>>>=20 >>>>>> btape: btape.c:1152-0 Wrote 10000 blocks of 64412 bytes. >>>>>> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) >>>>>> btape: btape.c:1168-0 Wrote 10000 blocks of 64412 bytes. >>>>>> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) >>>>>> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) >>>>>> btape: btape.c:1210-0 Rewind OK. >>>>>> 10000 blocks re-read correctly. >>>>>> Got EOF on tape. >>>>>> 10000 blocks re-read correctly. >>>>>> =3D=3D=3D Test Succeeded. End Write, rewind, and re-read test =3D=3D= =3D >>>>>>=20 >>>>>> btape: btape.c:1277-0 Block position test >>>>>> btape: btape.c:1289-0 Rewind OK. >>>>>> Reposition to file:block 0:4 >>>>>> Block 5 re-read correctly. >>>>>> Reposition to file:block 0:200 >>>>>> Block 201 re-read correctly. >>>>>> Reposition to file:block 0:9999 >>>>>> Block 10000 re-read correctly. >>>>>> Reposition to file:block 1:0 >>>>>> Block 10001 re-read correctly. >>>>>> Reposition to file:block 1:600 >>>>>> Block 10601 re-read correctly. >>>>>> Reposition to file:block 1:9999 >>>>>> Block 20000 re-read correctly. >>>>>> =3D=3D=3D Test Succeeded. End Write, rewind, and re-read test =3D=3D= =3D >>>>>>=20 >>>>>>=20 >>>>>>=20 >>>>>> =3D=3D=3D Append files test =3D=3D=3D >>>>>>=20 >>>>>> This test is essential to Bacula. >>>>>>=20 >>>>>> I'm going to write one record in file 0, >>>>>> two records in file 1, >>>>>> and three records in file 2 >>>>>>=20 >>>>>> btape: btape.c:574-0 Rewound "DLT" (/dev/nsa1) >>>>>> btape: btape.c:1907-0 Wrote one record of 64412 bytes. >>>>>> btape: btape.c:1909-0 Wrote block to device. >>>>>> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) >>>>>> btape: btape.c:1907-0 Wrote one record of 64412 bytes. >>>>>> btape: btape.c:1909-0 Wrote block to device. >>>>>> btape: btape.c:1907-0 Wrote one record of 64412 bytes. >>>>>> btape: btape.c:1909-0 Wrote block to device. >>>>>> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) >>>>>> btape: btape.c:1907-0 Wrote one record of 64412 bytes. >>>>>> btape: btape.c:1909-0 Wrote block to device. >>>>>> btape: btape.c:1907-0 Wrote one record of 64412 bytes. >>>>>> btape: btape.c:1909-0 Wrote block to device. >>>>>> btape: btape.c:1907-0 Wrote one record of 64412 bytes. >>>>>> btape: btape.c:1909-0 Wrote block to device. >>>>>> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) >>>>>> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) >>>>>> btape: btape.c:469-0 open device "DLT" (/dev/nsa1): OK >>>>>> btape: btape.c:574-0 Rewound "DLT" (/dev/nsa1) >>>>>> btape: btape.c:1420-0 Now moving to end of medium. >>>>>=20 >>>>> This is the critical piece. The test moves the tape to the end of = the >>>>> medium. With hardware position information, you can tell what = filemark >>>>> you're on. Without it, you can't. >>>>>=20 >>>>>> btape: btape.c:622-0 tape_dev.c:345 ioctl MTIOCGET error on "DLT" = (/dev/nsa1). ERR=3DNo error: 0. >>>>>> We should be in file 3. I am at file 0. This is NOT correct!!!! >>>>>>=20 >>>>>> Append test failed. Attempting again. >>>>>> Setting "Hardware End of Medium =3D no >>>>>> and "Fast Forward Space File =3D no >>>>>> and retrying append test. >>>>>=20 >>>>> This is not surprsing, given that the drive doesn't support long = read >>>>> position data. (It's a SCSI-2 device.) So that means that Bacula = will >>>>> need to do it manually. >>>>=20 >>>> Yes, I have nothing newer than SCSI-2. Even my SDLT is SCSI-2 but = that >>>> tape library is hooked up to a different computer and was doing = backups today. >>>=20 >>> So, here is one thing that we can try to see whether these drives = support >>> long position information, even though they only claim to be SCSI-2. = If >>> they do, we can potentially add a quirk (or autodetection) to enable = it. >>> The code currently doesn't bother asking drives that claim to be = SCSI-2 >>> for long position information. (Because that feature was added in = the >>> SSC spec, which came after SCSI-2.) >>>=20 >>> Issue a READ POSITION with the short form specified: >>>=20 >>> camcontrol cmd sa1 -v -c "34 0 0 0 0 0 0 0 0 0" -i 20 - |hd >>>=20 >>> Issue a READ POSITION with the vendor-specific block numbers: >>>=20 >>> camcontrol cmd sa1 -v -c "34 1 0 0 0 0 0 0 0 0" -i 20 - |hd >>>=20 >>> Issue a READ POSITION with the long form data: >>>=20 >>> camcontrol cmd sa1 -v -c "34 6 0 0 0 0 0 0 0 0" -i 32 - |hd >>>=20 >>> If it supports the last one, then I can put a quirk (or = autodetection) in >>> the driver and Bacula will get the hardware filemarks. You should = try this >>> on your SDLT as well. It may well support it. >>=20 >> Sadly, no: >>=20 >> [root@cuppy:~] # camcontrol cmd sa1 -v -c "34 0 0 0 0 0 0 0 0 0" -i = 20 - |hd >> 00000000 80 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 = |................| >> 00000010 00 00 00 00 |....| >> 00000014 >> [root@cuppy:~] # camcontrol cmd sa1 -v -c "34 1 0 0 0 0 0 0 0 0" -i = 20 - |hd >> 00000000 80 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 = |................| >> 00000010 00 00 00 00 |....| >> 00000014 >> [root@cuppy:~] # camcontrol cmd sa1 -v -c "34 6 0 0 0 0 0 0 0 0" -i = 32 - |hd >> camcontrol: error sending command >> (pass2:ahc0:0:2:0): READ POSITION. CDB: 34 06 00 00 00 00 00 00 00 00=20= >> (pass2:ahc0:0:2:0): CAM status: SCSI Status Error >> (pass2:ahc0:0:2:0): SCSI status: Check Condition >> (pass2:ahc0:0:2:0): SCSI sense: ILLEGAL REQUEST asc:24,0 (Invalid = field in CDB) >> (pass2:ahc0:0:2:0): Command byte 1 bit 2 is invalid >> [root@cuppy:~] #=20 >=20 > Okay. Not too surprising I suppose. >=20 >> The SDLT server is on 9.3 though: >>=20 >> [root@knew:/usr/home/dan] # camcontrol cmd sa1 -v -c "34 0 0 0 0 0 0 = 0 0 0" -i 20 - |hd >> camcontrol: cam_lookup_pass: CAMGETPASSTHRU ioctl failed >> cam_lookup_pass: No such file or directory >> cam_lookup_pass: either the pass driver isn't in your kernel >> cam_lookup_pass: or sa1 doesn't exist >> [root@knew:/usr/home/dan] # uname -a >> FreeBSD knew.unixathome.org 9.3-RELEASE-p10 FreeBSD 9.3-RELEASE-p10 = #0: Tue Feb 24 21:28:03 UTC 2015 = root@amd64-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC amd64 >> [root@knew:/usr/home/dan] #=20 >>=20 >>=20 >> It took me a while to figure that out... there is no sa1 on *this* = system. >>=20 >> But, my SDLT: >>=20 >> [root@knew:/usr/home/dan] # camcontrol cmd sa0 -v -c "34 0 0 0 0 0 0 = 0 0 0" -i 20 - |hd >> 00000000 80 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 = |................| >> 00000010 00 00 00 00 |....| >> 00000014 >> [root@knew:/usr/home/dan] # camcontrol cmd sa0 -v -c "34 1 0 0 0 0 0 = 0 0 0" -i 20 - |hd >> 00000000 80 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 = |................| >> 00000010 00 00 00 00 |....| >> 00000014 >> [root@knew:/usr/home/dan] # camcontrol cmd sa0 -v -c "34 6 0 0 0 0 0 = 0 0 0" -i 32 - |hd >> 00000000 80 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 = |................| >> 00000010 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 = |................| >> 00000020 >> [root@knew:/usr/home/dan] #=20 >=20 > Just to confirm, can you send the output of: >=20 > camcontrol inquiry sa0 -v >=20 > I want to make certain it reports that it is SCSI-2. If so, I'll = change > the check in the driver to try asking for long position information on > SCSI-2 devices. If it fails, it'll fall back to the regular method. [dan@knew:~] $ sudo camcontrol inquiry sa0 -v pass10: Removable Sequential Access SCSI-2 = device=20 pass10: Serial Number CXB46H0716 =20 pass10: 80.000MB/s transfers (40.000MHz, offset 31, 16bit) [dan@knew:~] $=20 >=20 >>>=20 >>> Google didn't quickly produce a SCSI manual for the DEC drive, but = the >>> Quantum SDLT manual indicates that it supports long position data, = despite >>> identifying itself as a SCSI-2 drive. >>>=20 >>>>> =3D=3D=3D Append files test =3D=3D=3D >>>>>>=20 >>>>>> This test is essential to Bacula. >>>>>>=20 >>>>>> I'm going to write one record in file 0, >>>>>> two records in file 1, >>>>>> and three records in file 2 >>>>>>=20 >>>>>> btape: btape.c:574-0 Rewound "DLT" (/dev/nsa1) >>>>>> btape: btape.c:1907-0 Wrote one record of 64412 bytes. >>>>>> btape: btape.c:1909-0 Wrote block to device. >>>>>> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) >>>>>> btape: btape.c:1907-0 Wrote one record of 64412 bytes. >>>>>> btape: btape.c:1909-0 Wrote block to device. >>>>>> btape: btape.c:1907-0 Wrote one record of 64412 bytes. >>>>>> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) >>>>>> btape: btape.c:1907-0 Wrote one record of 64412 bytes. >>>>>> btape: btape.c:1909-0 Wrote block to device. >>>>>> btape: btape.c:1907-0 Wrote one record of 64412 bytes. >>>>>> btape: btape.c:1909-0 Wrote block to device. >>>>>> btape: btape.c:1907-0 Wrote one record of 64412 bytes. >>>>>> btape: btape.c:1909-0 Wrote block to device. >>>>>> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) >>>>>> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) >>>>>> btape: btape.c:469-0 open device "DLT" (/dev/nsa1): OK >>>>>> btape: btape.c:574-0 Rewound "DLT" (/dev/nsa1) >>>>>> btape: btape.c:1420-0 Now moving to end of medium. >>>>>> btape: btape.c:625-0 Moved to end of medium. >>>>>> We should be in file 3. I am at file 3. This is correct! >>>>>>=20 >>>>>> Now the important part, I am going to attempt to append to the = tape. >>>>>>=20 >>>>>> btape: btape.c:1907-0 Wrote one record of 64412 bytes. >>>>>> btape: btape.c:1909-0 Wrote block to device. >>>>>> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) >>>>>> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) >>>>>> btape: btape.c:574-0 Rewound "DLT" (/dev/nsa1) >>>>>> Done appending, there should be no I/O errors >>>>>>=20 >>>>>> Doing Bacula scan of blocks: >>>>>> 1 block of 64448 bytes in file 1 >>>>>> End of File mark. >>>>>> 2 blocks of 64448 bytes in file 2 >>>>>> End of File mark. >>>>>> 3 blocks of 64448 bytes in file 3 >>>>>> End of File mark. >>>>>> 1 block of 64448 bytes in file 4 >>>>>> End of File mark. >>>>>> Total files=3D4, blocks=3D7, bytes =3D 451,136 >>>>>> End scanning the tape. >>>>>> We should be in file 4. I am at file 4. This is correct! >>>>>>=20 >>>>>>=20 >>>>>> It looks like the test worked this time, please add: >>>>>>=20 >>>>>> Hardware End of Medium =3D No >>>>>>=20 >>>>>> Fast Forward Space File =3D No >>>>>> to your Device resource in the Storage conf file. >>>>>>=20 >>>>>> The above Bacula scan should have output identical to what = follows. >>>>>> Please double check it ... >>>>>> =3D=3D=3D Sample correct output =3D=3D=3D >>>>>> 1 block of 64448 bytes in file 1 >>>>>> End of File mark. >>>>>> 2 blocks of 64448 bytes in file 2 >>>>>> End of File mark. >>>>>> 3 blocks of 64448 bytes in file 3 >>>>>> End of File mark. >>>>>> 1 block of 64448 bytes in file 4 >>>>>> End of File mark. >>>>>> Total files=3D4, blocks=3D7, bytes =3D 451,136 >>>>>> =3D=3D=3D End sample correct output =3D=3D=3D >>>>>>=20 >>>>>> If the above scan output is not identical to the >>>>>> sample output, you MUST correct the problem >>>>>> or Bacula will not be able to write multiple Jobs to=20 >>>>>> the tape. >>>>>>=20 >>>>>> Skipping read backwards test because BSR turned off. >>>>>>=20 >>>>>>=20 >>>>>> =3D=3D=3D Forward space files test =3D=3D=3D >>>>>>=20 >>>>>> This test is essential to Bacula. >>>>>>=20 >>>>>> I'm going to write five files then test forward spacing >>>>>>=20 >>>>>> btape: btape.c:574-0 Rewound "DLT" (/dev/nsa1) >>>>>> btape: btape.c:1907-0 Wrote one record of 64412 bytes. >>>>>> btape: btape.c:1909-0 Wrote block to device. >>>>>> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) >>>>>> btape: btape.c:1907-0 Wrote one record of 64412 bytes. >>>>>> btape: btape.c:1909-0 Wrote block to device. >>>>>> btape: btape.c:1907-0 Wrote one record of 64412 bytes. >>>>>> btape: btape.c:1909-0 Wrote block to device. >>>>>> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) >>>>>> btape: btape.c:1907-0 Wrote one record of 64412 bytes. >>>>>> btape: btape.c:1909-0 Wrote block to device. >>>>>> btape: btape.c:1907-0 Wrote one record of 64412 bytes. >>>>>> btape: btape.c:1909-0 Wrote block to device. >>>>>> btape: btape.c:1907-0 Wrote one record of 64412 bytes. >>>>>> btape: btape.c:1909-0 Wrote block to device. >>>>>> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) >>>>>> btape: btape.c:1907-0 Wrote one record of 64412 bytes. >>>>>> btape: btape.c:1909-0 Wrote block to device. >>>>>> btape: btape.c:1907-0 Wrote one record of 64412 bytes. >>>>>> btape: btape.c:1909-0 Wrote block to device. >>>>>> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) >>>>>> btape: btape.c:1907-0 Wrote one record of 64412 bytes. >>>>>> btape: btape.c:1909-0 Wrote block to device. >>>>>> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) >>>>>> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) >>>>>> btape: btape.c:574-0 Rewound "DLT" (/dev/nsa1) >>>>>> btape: btape.c:1634-0 Now forward spacing 1 file. >>>>>> We should be in file 1. I am at file 1. This is correct! >>>>>> btape: btape.c:1646-0 Now forward spacing 2 files. >>>>>> We should be in file 3. I am at file 3. This is correct! >>>>>> btape: btape.c:574-0 Rewound "DLT" (/dev/nsa1) >>>>>> btape: btape.c:1659-0 Now forward spacing 4 files. >>>>>> We should be in file 4. I am at file 4. This is correct! >>>>>>=20 >>>>>> btape: btape.c:1677-0 Now forward spacing 1 more file. >>>>>> We should be in file 5. I am at file 5. This is correct! >>>>>>=20 >>>>>> =3D=3D=3D End Forward space files test =3D=3D=3D >>>>>>=20 >>>>>>=20 >>>>>> Ah, I see you have an autochanger configured. >>>>>> To test the autochanger you must have a blank tape >>>>>> that I can write on in Slot 1. >>>>>>=20 >>>>>> Do you wish to continue with the Autochanger test? (y/n): y >>>>>>=20 >>>>>>=20 >>>>>> =3D=3D=3D Autochanger test =3D=3D=3D >>>>>>=20 >>>>>> 3301 Issuing autochanger "loaded" command. >>>>>> Nothing loaded in the drive. OK. >>>>>> 3303 Issuing autochanger "load 1 0" command. >>>>>> 3303 Autochanger "load 1 0" status is OK. >>>>>> btape: btape.c:469-0 open device "DLT" (/dev/nsa1): OK >>>>>> btape: btape.c:1564-0 Rewound "DLT" (/dev/nsa1) >>>>>> btape: btape.c:1571-0 Wrote EOF to "DLT" (/dev/nsa1) >>>>>>=20 >>>>>> The test autochanger worked!! >>>>>=20 >>>>> Great, thanks for running the test! Looks like things are working = as well >>>>> as they were before. >>>>>=20 >>>>>>>=20 >>>>>>>> I'll let the other Bacula devs know about this. They deal with = the hardware. I work on PostgreSQL. >>>>>>>>=20 >>>>>>>=20 >>>>>>> Thanks! If there are additional features they would like out of = the tape >>>>>>> driver, I'm happy to talk about it. (Or help if they'd like to = use the new >>>>>>> status reporting ioctl, MTIOCEXTGET or any of the other new = ioctls.) >>>>>>=20 >>>>>> Errors are interesting to me. Especially corrected errors. They = are a good indicator of tape quality. >>>>>>=20 >>>>>=20 >>>>> Yes. At least on modern drives, there is a good bit available in = the log >>>>> pages. You can try seeing what logs your drive supports by = installing the >>>>> sg3_utils package and running 'sg_logs sa1 --all'. >>>>>=20 >>>>> Given the large amount of data available on some drives, and the = difficulty >>>>> distilling it down to a clear good/bad, I probably won't stick = that in the >>>>> 'mt status' output. >>>>>=20 >>>>> But you can certainly take a look at it and have an idea of = whether your >>>>> particular tape/drive are experiencing issues. >>>>=20 >>>> That's a lot of output: = https://gist.github.com/dlangille/0e15a7fbf7acab56fd32 >>>=20 >>> That is a lot. >>>=20 >>>> I will run some Bacula jobs soon. I'm still setting up config = files. >>>=20 >>> Thanks for all the testing, I really appreciate it! >>>=20 >>> Ken >>> --=20 >>> Kenneth Merry >>> ken@FreeBSD.ORG >>=20 >> ?=20 >> Dan Langille >> http://langille.org/ >>=20 >>=20 >>=20 >>=20 >>=20 >=20 > Ken > --=20 > Kenneth Merry > ken@FreeBSD.ORG =E2=80=94=20 Dan Langille http://langille.org/ From owner-freebsd-current@FreeBSD.ORG Mon Mar 2 16:46:01 2015 Return-Path: Delivered-To: current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3C08DED7; Mon, 2 Mar 2015 16:46:01 +0000 (UTC) Received: from clavin1.langille.org (clavin.langille.org [162.208.116.86]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "clavin.langille.org", Issuer "StartCom Class 2 Primary Intermediate Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 145A2A1; Mon, 2 Mar 2015 16:46:00 +0000 (UTC) Received: from (clavin1.int.langille.org (clavin1.int.unixathome.org [10.4.7.7]) (Authenticated sender: hidden) with ESMTPSA id D72D56D88 ; Mon, 2 Mar 2015 16:45:59 +0000 (UTC) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 8.2 \(2070.6\)) Subject: Re: sa(4) driver changes available for test From: Dan Langille In-Reply-To: <20150214003232.GA63990@mithlond.kdm.org> Date: Mon, 2 Mar 2015 11:45:59 -0500 Content-Transfer-Encoding: quoted-printable Message-Id: <8EFB1A84-96E6-4F63-A6D0-A87432C02B72@langille.org> References: <20150214003232.GA63990@mithlond.kdm.org> To: "Kenneth D. Merry" X-Mailer: Apple Mail (2.2070.6) Cc: current@freebsd.org, scsi@freebsd.org X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Mar 2015 16:46:01 -0000 > On Feb 13, 2015, at 7:32 PM, Kenneth D. Merry wrote: >=20 >=20 > I have a fairly large set of changes to the sa(4) driver and mt(1) = driver > that I'm planning to commit in the near future. >=20 > A description of the changes is here and below in this message. >=20 > If you have tape hardware and the inclination, I'd appreciate testing = and > feedback. This came to me today via the Bacula mailing lists. http://marc.info/?l=3Dbacula-users&m=3D142531236722693&w=3D2 > As far as I can tell ltfs support on linux sits on top of the standard = mt-st stuff \ > as a userspace (fuse) filesystem=20 > I'd hope it's much the same with BSD. Removing the standard interface = would be \ > counterproductive overall Can you answer that and I'll relay please? =E2=80=94=20 Dan Langille http://langille.org/ From owner-freebsd-current@FreeBSD.ORG Mon Mar 2 17:26:33 2015 Return-Path: Delivered-To: current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 350131C4; Mon, 2 Mar 2015 17:26:33 +0000 (UTC) Received: from mithlond.kdm.org (mithlond.kdm.org [70.56.43.85]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "A1-33714", Issuer "A1-33714" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id D2A9878D; Mon, 2 Mar 2015 17:26:32 +0000 (UTC) Received: from mithlond.kdm.org (localhost [127.0.0.1]) by mithlond.kdm.org (8.14.9/8.14.9) with ESMTP id t22HQTcu087076 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Mon, 2 Mar 2015 10:26:29 -0700 (MST) (envelope-from ken@mithlond.kdm.org) Received: (from ken@localhost) by mithlond.kdm.org (8.14.9/8.14.9/Submit) id t22HQT7p087075; Mon, 2 Mar 2015 10:26:29 -0700 (MST) (envelope-from ken) Date: Mon, 2 Mar 2015 10:26:29 -0700 From: "Kenneth D. Merry" To: Dan Langille Subject: Re: sa(4) driver changes available for test Message-ID: <20150302172629.GA87055@mithlond.kdm.org> References: <20150214003232.GA63990@mithlond.kdm.org> <4A478C5C-7965-498E-9F0F-80192265E310@langille.org> <20150302001833.GA71528@mithlond.kdm.org> <6C82281F-649A-4DA8-8ACF-17E81C04F730@langille.org> <20150302003658.GA72258@mithlond.kdm.org> <20150302020608.GA73433@mithlond.kdm.org> <30C55F00-2DE4-4596-96EA-2E3CC40B4DB6@langille.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <30C55F00-2DE4-4596-96EA-2E3CC40B4DB6@langille.org> User-Agent: Mutt/1.5.23 (2014-03-12) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (mithlond.kdm.org [127.0.0.1]); Mon, 02 Mar 2015 10:26:29 -0700 (MST) X-Spam-Status: No, score=-2.9 required=5.0 tests=ALL_TRUSTED,BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS autolearn=ham autolearn_force=no version=3.4.0 X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on mithlond.kdm.org Cc: current@freebsd.org, scsi@freebsd.org X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Mar 2015 17:26:33 -0000 On Mon, Mar 02, 2015 at 11:43:15 -0500, Dan Langille wrote: > > > On Mar 1, 2015, at 9:06 PM, Kenneth D. Merry wrote: > > > > On Sun, Mar 01, 2015 at 19:40:40 -0500, Dan Langille wrote: > >> > >>> On Mar 1, 2015, at 7:36 PM, Kenneth D. Merry wrote: > >>> > >>> On Sun, Mar 01, 2015 at 19:28:37 -0500, Dan Langille wrote: > >>>> > >>>>> On Mar 1, 2015, at 7:18 PM, Kenneth D. Merry wrote: > >>>>> > >>>>> On Sun, Mar 01, 2015 at 17:06:24 -0500, Dan Langille wrote: > >>>>>> > >>>>>>> On Feb 13, 2015, at 7:32 PM, Kenneth D. Merry wrote: > >>>>>>> > >>>>>>> > >>>>>>> I have a fairly large set of changes to the sa(4) driver and mt(1) driver > >>>>>>> that I'm planning to commit in the near future. > >>>>>>> > >>>>>>> A description of the changes is here and below in this message. > >>>>>>> > >>>>>>> If you have tape hardware and the inclination, I'd appreciate testing and > >>>>>>> feedback. > >>>>>>> > >>>>>>> ============ > >>>>>>> Rough draft commit message: > >>>>>>> > >>>>>>> http://people.freebsd.org/~ken/sa_changes_commitmsg.20150213.3.txt > >>>>>>> > >>>>>>> The patches against FreeBSD/head as of SVN revision 278706: > >>>>>>> > >>>>>>> http://people.freebsd.org/~ken/sa_changes.20150213.3.txt > >>>>>>> > >>>>>>> And (untested) patches against FreeBSD stable/10 as of SVN revision 278721. > >>>>>>> > >>>>>>> http://people.freebsd.org/~ken/sa_changes.stable_10.20150213.3.txt > >>>>>>> ============ > >>>>>>> > >>>>>>> The intent is to get the tape infrastructure more up to date, so we can > >>>>>>> support LTFS and more modern tape drives: > >>>>>>> > >>>>>>> http://www.ibm.com/systems/storage/tape/ltfs/ > >>>>>>> > >>>>>>> I have ported IBM's LTFS Single Drive Edition to FreeBSD. The port depends > >>>>>>> on the patches linked above. It isn't fully cleaned up and ready for > >>>>>>> redistribution. If you're interested, though, let me know and I'll tell > >>>>>>> you when it is ready to go out. You need an IBM LTO-5, LTO-6, TS1140 or > >>>>>>> TS1150 tape drive. HP drives aren't supported by IBM's LTFS, and older > >>>>>>> drives don't have the necessary features to support LTFS. > >>>>>>> > >>>>>>> The commit message below outlines most of the changes. > >>>>>>> > >>>>>>> A few comments: > >>>>>>> > >>>>>>> 1. I'm planning to commit the XPT_DEV_ADVINFO changes separately. > >>>>>>> > >>>>>>> 2. The XML output is similar to what GEOM and CTL do. It would be nice to > >>>>>>> figure out how to put a standard schema on it so that standard tools > >>>>>>> could read it. I don't know how feasible that is, since I haven't > >>>>>>> time to dig into it. If anyone has suggestions on whether that is > >>>>>>> feasible or advisable, I'd appreciate feedback. > >>>>>>> > >>>>>>> 3. I have tested with a reasonable amount of tape hardware (see below for a > >>>>>>> list), but more testing and feedback would be good. > >>>>>>> > >>>>>>> 4. Standard 'mt status' output looks like this: > >>>>>>> > >>>>>>> # mt -f /dev/nsa3 status -v > >>>>>>> Drive: sa3: Serial Number: 101500520A > >>>>>>> --------------------------------- > >>>>>>> Mode Density Blocksize bpi Compression > >>>>>>> Current: 0x5a:LTO-6 variable 384607 enabled (0xff) > >>>>>>> --------------------------------- > >>>>>>> Current Driver State: at rest. > >>>>>>> --------------------------------- > >>>>>>> Partition: 0 Calc File Number: 0 Calc Record Number: 0 > >>>>>>> Residual: 0 Reported File Number: 0 Reported Record Number: 0 > >>>>>>> Flags: BOP > >>>>>>> > >>>>>>> 5. 'mt status -v' looks like this: > >>>>>>> > >>>>>>> # mt -f /dev/nsa3 status -v > >>>>>>> Drive: sa3: Serial Number: 101500520A > >>>>>>> --------------------------------- > >>>>>>> Mode Density Blocksize bpi Compression > >>>>>>> Current: 0x5a:LTO-6 variable 384607 enabled (0xff) > >>>>>>> --------------------------------- > >>>>>>> Current Driver State: at rest. > >>>>>>> --------------------------------- > >>>>>>> Partition: 0 Calc File Number: 0 Calc Record Number: 0 > >>>>>>> Residual: 0 Reported File Number: 0 Reported Record Number: 0 > >>>>>>> Flags: BOP > >>>>>>> --------------------------------- > >>>>>>> Tape I/O parameters: > >>>>>>> Maximum I/O size allowed by driver and controller (maxio): 1081344 bytes > >>>>>>> Maximum I/O size reported by controller (cpi_maxio): 5197824 bytes > >>>>>>> Maximum block size supported by tape drive and media (max_blk): 8388608 bytes > >>>>>>> Minimum block size supported by tape drive and media (min_blk): 1 bytes > >>>>>>> Block granularity supported by tape drive and media (blk_gran): 0 bytes > >>>>>>> Maximum possible I/O size (max_effective_iosize): 1081344 bytes > >>>>>> > >>>>>> > >>>>>> # mtx -f /dev/pass0 status > >>>>>> Storage Changer /dev/pass0:2 Drives, 10 Slots ( 0 Import/Export ) > >>>>>> Data Transfer Element 0:Empty > >>>>>> Data Transfer Element 1:Empty > >>>>>> Storage Element 1:Empty > >>>>>> Storage Element 2:Empty > >>>>>> Storage Element 3:Empty > >>>>>> Storage Element 4:Full :VolumeTag=FAI260 > >>>>>> Storage Element 5:Full :VolumeTag=FAI261 > >>>>>> Storage Element 6:Full :VolumeTag=FAI262 > >>>>>> Storage Element 7:Full :VolumeTag=FAI263 > >>>>>> Storage Element 8:Empty > >>>>>> Storage Element 9:Empty > >>>>>> Storage Element 10:Empty > >>>>>> > >>>>>> > >>>>>> It was at this point I spent the next 90 minute trying to get the tape > >>>>>> drive out of the tape library to free a stuck tape. Some of this was spent > >>>>>> attempting, and failing, to undo a stripped screw. I stopped the attempt when > >>>>>> I noticed the screw did need to be removed. :/ > >>>>> > >>>>> Thanks for all of the effort! Looks like it is paying off! :) > >>>>> > >>>>>> When I do this command, I hear the drive move a bit, to read the tape: > >>>>>> > >>>>>> # mt -f /dev/nsa1 status > >>>>>> Drive: sa1: Serial Number: CXA09S1340 > >>>>>> --------------------------------- > >>>>>> Mode Density Blocksize bpi Compression > >>>>>> Current: 0x1b:DLTapeIV(35GB) variable 85937 enabled (IDRC) > >>>>>> --------------------------------- > >>>>>> Current Driver State: at rest. > >>>>>> --------------------------------- > >>>>>> Partition: 0 Calc File Number: 0 Calc Record Number: 0 > >>>>>> Residual: 0 Reported File Number: -1 Reported Record Number: -1 > >>>>>> Flags: None > >>>>> > >>>>> Looks like the drive isn't reporting position information. It will still > >>>>> be useful to try it with Bacula, though. > >>>>> > >>>>>> # mt -f /dev/nsa1 ostatus > >>>>>> Mode Density Blocksize bpi Compression > >>>>>> Current: 0x1b:DLTapeIV(35GB) variable 85937 IDRC > >>>>>> ---------available modes--------- > >>>>>> 0: 0x1b:DLTapeIV(35GB) variable 85937 IDRC > >>>>>> 1: 0x1b:DLTapeIV(35GB) variable 85937 IDRC > >>>>>> 2: 0x1b:DLTapeIV(35GB) variable 85937 IDRC > >>>>>> 3: 0x1b:DLTapeIV(35GB) variable 85937 IDRC > >>>>>> --------------------------------- > >>>>>> Current Driver State: at rest. > >>>>>> --------------------------------- > >>>>>> File Number: 0 Record Number: 0 Residual Count 0 > >>>>>> > >>>>>> > >>>>>> After doing a very small tar -c and tar -x, I have: > >>>>>> > >>>>>> # mt -f /dev/nsa1 /dev/nsa1 ostatus > >>>>>> Mode Density Blocksize bpi Compression > >>>>>> Current: 0x1b:DLTapeIV(35GB) variable 85937 IDRC > >>>>>> ---------available modes--------- > >>>>>> 0: 0x1b:DLTapeIV(35GB) variable 85937 IDRC > >>>>>> 1: 0x1b:DLTapeIV(35GB) variable 85937 IDRC > >>>>>> 2: 0x1b:DLTapeIV(35GB) variable 85937 IDRC > >>>>>> 3: 0x1b:DLTapeIV(35GB) variable 85937 IDRC > >>>>>> --------------------------------- > >>>>>> Current Driver State: at rest. > >>>>>> --------------------------------- > >>>>>> File Number: 0 Record Number: 7 Residual Count 0 > >>>>> > >>>>> Woohoo! It works. > >>>>> > >>>>>> # mt -f /dev/nsa1 status -v > >>>>>> Drive: sa1: Serial Number: CXA09S1340 > >>>>>> --------------------------------- > >>>>>> Mode Density Blocksize bpi Compression > >>>>>> Current: 0x1b:DLTapeIV(35GB) variable 85937 enabled (IDRC) > >>>>>> --------------------------------- > >>>>>> Current Driver State: at rest. > >>>>>> --------------------------------- > >>>>>> Partition: 0 Calc File Number: 0 Calc Record Number: 7 > >>>>>> Residual: 0 Reported File Number: -1 Reported Record Number: -1 > >>>>>> Flags: None > >>>>>> --------------------------------- > >>>>>> Tape I/O parameters: > >>>>>> Maximum I/O size allowed by driver and controller (maxio): 65536 bytes > >>>>>> Maximum I/O size reported by controller (cpi_maxio): 0 bytes > >>>>>> Maximum block size supported by tape drive and media (max_blk): 16777214 bytes > >>>>>> Minimum block size supported by tape drive and media (min_blk): 2 bytes > >>>>>> Block granularity supported by tape drive and media (blk_gran): 0 bytes > >>>>>> Maximum possible I/O size (max_effective_iosize): 65536 bytes > >>>>>> > >>>>>> I may not get to testing Bacula today. > >>>>>> > >>>>>> Based on the above, is there any commands you'd like me to try? > >>>>> > >>>>> Aside from making sure things work okay with Bacula, that is probably > >>>>> sufficient. These drives won't support density reports or position > >>>>> information. > >>>>> > >>>>>> Read below regarding two tape drives > >>>>>> > >>>>>>> > >>>>>>> 6. Existing applications should work without changes. If not, please let > >>>>>>> me know. Hopefully they will move over time to the new interfaces. > >>>>>>> > >>>>>>> 7. There are lots of additional features that could be added later. > >>>>>>> Append-only support, encryption, more log pages, etc. > >>>>>>> > >>>>>>> 8. I have SCSI READ ATTRIBUTE changes for camcontrol(8) that will go in > >>>>>>> separately. These changes allow displaying the contents of the MAM > >>>>>>> (Medium Auxiliary Memory) chips on LTO, TS and other modern tape drives. > >>>>>>> These are good, and a future possible direction is adding attributes > >>>>>>> to the status XML from the sa(4) driver. > >>>>>>> > >>>>>>> ============ > >>>>>>> Significant upgrades to sa(4) and mt(1). > >>>>>>> > >>>>>>> The primary focus of these changes is to modernize FreeBSD's > >>>>>>> tape infrastructure so that we can take advantage of some of the > >>>>>>> features of modern tape drives and allow support for LTFS. > >>>>>>> > >>>>>>> Significant changes and new features include: > >>>>>>> > >>>>>>> o sa(4) driver status and parameter information is now exported via an > >>>>>>> XML structure. This will allow for changes and improvements later > >>>>>>> on that will not break userland applications. The old MTIOCGET > >>>>>>> status ioctl remains, so applications using the existing interface > >>>>>>> will not break. > >>>>>>> > >>>>>>> o 'mt status' now reports drive-reported tape position information > >>>>>>> as well as the previously available calculated tape position > >>>>>>> information. These numbers will be different at times, because > >>>>>>> the drive-reported block numbers are relative to BOP (Beginning > >>>>>>> of Partition), but the block numbers calculated previously via > >>>>>>> sa(4) (and still provided) are relative to the last filemark. > >>>>>>> Both numbers are now provided. 'mt status' now also shows the > >>>>>>> drive INQUIRY information, serial number and any position flags > >>>>>>> (BOP, EOT, etc.) provided with the tape position information. > >>>>>>> 'mt status -v' adds information on the maximum possible I/O size, > >>>>>>> and the underlying values used to calculate it. > >>>>>>> > >>>>>>> o The extra sa(4) /dev entries (/dev/saN.[0-3]) have been removed. > >>>>>> > >>>>>> How does this affect a tape library with more than one tape drive? > >>>>>> > >>>>>> [root@cuppy:~] # camcontrol amcontrol devlist > >>>>>> at scbus0 target 0 lun 0 (pass0,ch0) > >>>>>> at scbus0 target 2 lun 0 (sa1,pass2) > >>>>>> at scbus1 target 0 lun 0 (pass3,ada0) > >>>>>> at scbus2 target 0 lun 0 (pass4,ada1) > >>>>>> at scbus3 target 0 lun 0 (pass5,ses0) > >>>>>> > >>>>>> This system has two tapes drives and I can access them through the front panel but: > >>>>>> > >>>>>> # ls -l /dev/*sa* > >>>>>> crw-rw---- 1 root operator 0x65 Feb 28 22:04 /dev/esa1 > >>>>>> crw-rw---- 1 root operator 0x64 Mar 1 22:43 /dev/nsa1 > >>>>>> crw-rw---- 1 root operator 0x63 Feb 28 22:04 /dev/sa1 > >>>>>> crw-rw---- 1 root operator 0x62 Feb 28 22:04 /dev/sa1.ctl > >>>>>> > >>>>>> ... only one tape drives shows up. > >>>>> > >>>>> > >>>>> Hmm. The tape drive is listed as sa1, which implies that there may be an > >>>>> sa0 that was there previously or is in the process of probing. What does > >>>>> dmesg show? How about 'camcontrol devlist -v'? > >>>> > >>>> # camcontrol devlist -v > >>>> scbus0 on ahc0 bus 0: > >>>> at scbus0 target 0 lun 0 (pass0,ch0) > >>>> at scbus0 target 2 lun 0 (sa1,pass2) > >>>> <> at scbus0 target -1 lun ffffffff () > >>>> scbus1 on ahcich2 bus 0: > >>>> at scbus1 target 0 lun 0 (pass3,ada0) > >>>> <> at scbus1 target -1 lun ffffffff () > >>>> scbus2 on ahcich4 bus 0: > >>>> at scbus2 target 0 lun 0 (pass4,ada1) > >>>> <> at scbus2 target -1 lun ffffffff () > >>>> scbus3 on ahciem0 bus 0: > >>>> at scbus3 target 0 lun 0 (pass5,ses0) > >>>> <> at scbus3 target -1 lun ffffffff () > >>>> scbus-1 on xpt0 bus 0: > >>>> <> at scbus-1 target -1 lun ffffffff (xpt0) > >>>> > >>>> > >>>> BUT! > >>>> > >>>> # grep sa /var/run/dmesg.boot > >>>> VT-x: (disabled in BIOS) PAT,HLT,MTF,PAUSE,EPT,UG,VPID > >>>> module_register_init: MOD_LOAD (vesa, 0xffffffff80de3720, 0) error 19 > >>>> alc0: Using 1 MSIX message(s). > >>>> isab0: at device 31.0 on pci0 > >>>> isa0: on isab0 > >>>> orm0: at iomem 0xce800-0xcefff on isa0 > >>>> atkbdc0: at port 0x60,0x64 on isa0 > >>>> sa0 at ahc0 bus 0 scbus0 target 1 lun 0 > >>>> sa0: Removable Sequential Access SCSI-2 device > >>>> sa0: Serial Number CXA22S2338 > >>>> sa0: 10.000MB/s transfers (10.000MHz, offset 15) > >>>> sa0: quirks=0x100 > >>>> sa1 at ahc0 bus 0 scbus0 target 2 lun 0 > >>>> sa1: Removable Sequential Access SCSI-2 device > >>>> sa1: Serial Number CXA09S1340 > >>>> sa1: 10.000MB/s transfers (10.000MHz, offset 15) > >>>> sa1: quirks=0x100 > >>> > >>> If you run 'dmesg', you should have seen a message when it went away. Perhaps > >>> there will be something preceding it that will give us a clue about the > >>> problem. (Generally a selection timeout.) At least this does show that > >>> sa0 is at target 1, and so should not conflict with the library or sa1. > >> > >> Ahh: > >> > >> Trying to mount root from zfs:system/bootenv/FreeBSDHEad []... > >> sa0 at ahc0 bus 0 scbus0 target 1 lun 0 > >> sa0: s/n CXA22S2338 detached > >> (sa0:ahc0:0:1:0): Periph destroyed > >> arp: 10.55.0.60 moved from e4:ce:8f:46:f1:98 to 78:ca:39:fe:d6:b3 on em0 > >> arp: 10.55.0.60 moved from e4:ce:8f:46:f1:98 to 78:ca:39:fe:d6:b3 on em0 > >> arp: 10.55.0.60 moved from 78:ca:39:fe:d6:b3 to e4:ce:8f:46:f1:98 on em0 > >> (sa1:ahc0:0:2:0): 64512-byte tape record bigger than supplied buffer > >> (sa1:ahc0:0:2:0): 10240-byte tape record bigger than supplied buffer > > > > Okay. Well, no indication of what happened. Perhaps boot -v will show it, > > perhaps not. > > > > Good news. After a reboot, both tape drives are present: > > $ ls -l /dev/*sa* > crw-rw---- 1 root operator 0x61 Mar 2 17:27 /dev/esa0 > crw-rw---- 1 root operator 0x65 Mar 2 17:27 /dev/esa1 > crw-rw---- 1 root operator 0x60 Mar 2 17:27 /dev/nsa0 > crw-rw---- 1 root operator 0x64 Mar 2 17:27 /dev/nsa1 > crw-rw---- 1 root operator 0x5f Mar 2 17:27 /dev/sa0 > crw-rw---- 1 root operator 0x5e Mar 2 17:27 /dev/sa0.ctl > crw-rw---- 1 root operator 0x63 Mar 2 17:27 /dev/sa1 > crw-rw---- 1 root operator 0x62 Mar 2 17:27 /dev/sa1.ctl > Ahh, good. Glad it is working now! Ken -- Kenneth Merry ken@FreeBSD.ORG From owner-freebsd-current@FreeBSD.ORG Mon Mar 2 17:28:50 2015 Return-Path: Delivered-To: current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 2D731322; Mon, 2 Mar 2015 17:28:50 +0000 (UTC) Received: from mithlond.kdm.org (mithlond.kdm.org [70.56.43.85]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "A1-33714", Issuer "A1-33714" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id C90017B7; Mon, 2 Mar 2015 17:28:49 +0000 (UTC) Received: from mithlond.kdm.org (localhost [127.0.0.1]) by mithlond.kdm.org (8.14.9/8.14.9) with ESMTP id t22HSlGp087099 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Mon, 2 Mar 2015 10:28:47 -0700 (MST) (envelope-from ken@mithlond.kdm.org) Received: (from ken@localhost) by mithlond.kdm.org (8.14.9/8.14.9/Submit) id t22HSksW087098; Mon, 2 Mar 2015 10:28:46 -0700 (MST) (envelope-from ken) Date: Mon, 2 Mar 2015 10:28:46 -0700 From: "Kenneth D. Merry" To: Dan Langille Subject: Re: sa(4) driver changes available for test Message-ID: <20150302172846.GB87055@mithlond.kdm.org> References: <20150214003232.GA63990@mithlond.kdm.org> <7CA52DF3-E073-4F50-BE4E-01C51CCDF2C7@langille.org> <20150217183645.GA30947@mithlond.kdm.org> <303C48AC-0DA8-447C-8DE7-0A1DD5DD7351@langille.org> <20150302003150.GB71528@mithlond.kdm.org> <20150302022957.GB73433@mithlond.kdm.org> <0F17CDDB-4CE9-4881-B948-6B9BEDEB6455@langille.org> <20150302163146.GA85515@mithlond.kdm.org> <85E2DDA0-4A2B-4316-A318-3706B3A05C11@langille.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <85E2DDA0-4A2B-4316-A318-3706B3A05C11@langille.org> User-Agent: Mutt/1.5.23 (2014-03-12) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (mithlond.kdm.org [127.0.0.1]); Mon, 02 Mar 2015 10:28:47 -0700 (MST) X-Spam-Status: No, score=-2.9 required=5.0 tests=ALL_TRUSTED,BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on mithlond.kdm.org Cc: current@freebsd.org, scsi@freebsd.org X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Mar 2015 17:28:50 -0000 On Mon, Mar 02, 2015 at 11:44:09 -0500, Dan Langille wrote: > > > On Mar 2, 2015, at 11:31 AM, Kenneth D. Merry wrote: > > > > On Mon, Mar 02, 2015 at 11:09:57 -0500, Dan Langille wrote: > >> > >>> On Mar 1, 2015, at 9:29 PM, Kenneth D. Merry wrote: > >>> > >>> On Sun, Mar 01, 2015 at 19:41:07 -0500, Dan Langille wrote: > >>>> > >>>>> On Mar 1, 2015, at 7:31 PM, Kenneth D. Merry wrote: > >>>>> > >>>>> On Sun, Mar 01, 2015 at 19:15:05 -0500, Dan Langille wrote: > >>>>>> > >>>>>>> On Feb 17, 2015, at 1:36 PM, Kenneth D. Merry wrote: > >>>>>>> > >>>>>>> On Sat, Feb 14, 2015 at 18:22:43 -0500, Dan Langille wrote: > >>>>>>>> > >>>>>>>>> On Feb 13, 2015, at 7:32 PM, Kenneth D. Merry wrote: > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> I have a fairly large set of changes to the sa(4) driver and mt(1) driver > >>>>>>>>> that I'm planning to commit in the near future. > >>>>>>>>> > >>>>>>>>> A description of the changes is here and below in this message. > >>>>>>>>> > >>>>>>>>> If you have tape hardware and the inclination, I'd appreciate testing and > >>>>>>>>> feedback. > >>>>>>>> > >>>>>>>> I have a DLT 8000 and an SDLT 220. > >>>>>>>> > >>>>>>>> I don't have anything running current, but I have a spare machine which I could use for testing. > >>>>>>>> > >>>>>>>> Do you see any value is tests with that hardware? I'd be testing it via Bacula. > >>>>>>>> > >>>>>>>> disclosure: I'm the sysutils/bacula-* maintainer and a Bacula committer. > >>>>>>>> > >>>>>>> > >>>>>>> Actually, yes. Bacula is a bit tricky to configure, so your trying it out > >>>>>>> would be helpful if you have the time. > >>>>>>> > >>>>>>> In looking at the manuals for both the SDLT 220 and the DLT 8000, they both > >>>>>>> claim to support long position information for the SCSI READ POSITION > >>>>>>> command. > >>>>>>> > >>>>>>> You can see what I'm talking about by doing: > >>>>>>> > >>>>>>> mt eod > >>>>>>> mt status > >>>>>>> > >>>>>>> On my DDS-4 tape drive, this shows: > >>>>>>> > >>>>>>> # mt -f /dev/nsa3 status > >>>>>>> Drive: sa3: Serial Number: HJ00YWY > >>>>>>> --------------------------------- > >>>>>>> Mode Density Blocksize bpi Compression > >>>>>>> Current: 0x26:DDS-4 1024 bytes 97000 enabled (DCLZ) > >>>>>>> --------------------------------- > >>>>>>> Current Driver State: at rest. > >>>>>>> --------------------------------- > >>>>>>> Partition: 0 Calc File Number: -1 Calc Record Number: -1 > >>>>>>> Residual: 0 Reported File Number: -1 Reported Record Number: -1 > >>>>>>> Flags: None > >>>>>>> > >>>>>>> But on an LTO-5, which will give long position information, I get: > >>>>>>> > >>>>>>> [root@doc ~]# mt status > >>>>>>> Drive: sa0: > >>>>>>> --------------------------------- > >>>>>>> Mode Density Blocksize bpi Compression > >>>>>>> Current: 0x58:LTO-5 variable 384607 enabled (0x1) > >>>>>>> --------------------------------- > >>>>>>> Current Driver State: at rest. > >>>>>>> --------------------------------- > >>>>>>> Partition: 0 Calc File Number: 2 Calc Record Number: -1 > >>>>>>> Residual: 0 Reported File Number: 2 Reported Record Number: 32373 > >>>>>>> Flags: None > >>>>>>> > >>>>>>> That, in combination with the changes I made to the position information > >>>>>>> code in the driver, mean that even the old MTIOCGET ioctl should return an > >>>>>>> accurate file number at end of data. e.g., on the LTO-5: > >>>>>>> > >>>>>>> [root@doc ~]# mt ostatus > >>>>>>> Mode Density Blocksize bpi Compression > >>>>>>> Current: 0x58:LTO-5 variable 384607 0x1 > >>>>>>> ---------available modes--------- > >>>>>>> 0: 0x58:LTO-5 variable 384607 0x1 > >>>>>>> 1: 0x58:LTO-5 variable 384607 0x1 > >>>>>>> 2: 0x58:LTO-5 variable 384607 0x1 > >>>>>>> 3: 0x58:LTO-5 variable 384607 0x1 > >>>>>>> --------------------------------- > >>>>>>> Current Driver State: at rest. > >>>>>>> --------------------------------- > >>>>>>> File Number: 2 Record Number: -1 Residual Count -1 > >>>>>>> > >>>>>>> So the thing to try, in addition to just making sure that Bacula continues > >>>>>>> to work properly, is to try setting this for the tape drive in > >>>>>>> bacula-sd.conf: > >>>>>>> > >>>>>>> Hardware End of Medium = yes > >>>>>>> > >>>>>>> It looks like the Bacula tape program (btape) has a test mode, and it would > >>>>>>> be good to run through the tests on one of the tape drives and see whether > >>>>>>> they work, and whether the results are different before and after the > >>>>>>> changes. I'm not sure how to enable the test mode. > >>>>>> > >>>>>> I have this in /usr/local/etc/bacula/bacula-sd.conf > >>>>>> > >>>>>> Device { > >>>>>> Name = DLT > >>>>>> Description = "QUANTUM DLT7000 1624" > >>>>>> Media Type = DLT > >>>>>> Archive Device = /dev/nsa1 > >>>>>> > >>>>>> Autochanger = YES > >>>>>> Drive Index = 0 > >>>>>> > >>>>>> Offline On Unmount = no > >>>>>> Hardware End of Medium = yes > >>>>>> BSF at EOM = yes > >>>>>> Backward Space Record = no > >>>>>> Fast Forward Space File = no > >>>>>> TWO EOF = yes > >>>>>> } > >>>>>> > >>>>>> FYI, http://www.freebsddiary.org/digital-tl891.php (from 2006) has a btape test on this same model. > >>>>>> > >>>>>> Here's the test I ran tonight: > >>>>>> > >>>>>> [root@cuppy:/usr/home/dan] # btape -c /usr/local/etc/bacula/bacula-sd.conf /dev/nsa1 > >>>>>> Tape block granularity is 1024 bytes. > >>>>>> btape: butil.c:287-0 Using device: "/dev/nsa1" for writing. > >>>>>> btape: btape.c:469-0 open device "DLT" (/dev/nsa1): OK > >>>>>> *test > >>>>>> > >>>>>> === Write, rewind, and re-read test === > >>>>>> > >>>>>> I'm going to write 10000 records and an EOF > >>>>>> then write 10000 records and an EOF, then rewind, > >>>>>> and re-read the data to verify that it is correct. > >>>>>> > >>>>>> This is an *essential* feature ... > >>>>>> > >>>>>> btape: btape.c:1152-0 Wrote 10000 blocks of 64412 bytes. > >>>>>> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) > >>>>>> btape: btape.c:1168-0 Wrote 10000 blocks of 64412 bytes. > >>>>>> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) > >>>>>> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) > >>>>>> btape: btape.c:1210-0 Rewind OK. > >>>>>> 10000 blocks re-read correctly. > >>>>>> Got EOF on tape. > >>>>>> 10000 blocks re-read correctly. > >>>>>> === Test Succeeded. End Write, rewind, and re-read test === > >>>>>> > >>>>>> btape: btape.c:1277-0 Block position test > >>>>>> btape: btape.c:1289-0 Rewind OK. > >>>>>> Reposition to file:block 0:4 > >>>>>> Block 5 re-read correctly. > >>>>>> Reposition to file:block 0:200 > >>>>>> Block 201 re-read correctly. > >>>>>> Reposition to file:block 0:9999 > >>>>>> Block 10000 re-read correctly. > >>>>>> Reposition to file:block 1:0 > >>>>>> Block 10001 re-read correctly. > >>>>>> Reposition to file:block 1:600 > >>>>>> Block 10601 re-read correctly. > >>>>>> Reposition to file:block 1:9999 > >>>>>> Block 20000 re-read correctly. > >>>>>> === Test Succeeded. End Write, rewind, and re-read test === > >>>>>> > >>>>>> > >>>>>> > >>>>>> === Append files test === > >>>>>> > >>>>>> This test is essential to Bacula. > >>>>>> > >>>>>> I'm going to write one record in file 0, > >>>>>> two records in file 1, > >>>>>> and three records in file 2 > >>>>>> > >>>>>> btape: btape.c:574-0 Rewound "DLT" (/dev/nsa1) > >>>>>> btape: btape.c:1907-0 Wrote one record of 64412 bytes. > >>>>>> btape: btape.c:1909-0 Wrote block to device. > >>>>>> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) > >>>>>> btape: btape.c:1907-0 Wrote one record of 64412 bytes. > >>>>>> btape: btape.c:1909-0 Wrote block to device. > >>>>>> btape: btape.c:1907-0 Wrote one record of 64412 bytes. > >>>>>> btape: btape.c:1909-0 Wrote block to device. > >>>>>> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) > >>>>>> btape: btape.c:1907-0 Wrote one record of 64412 bytes. > >>>>>> btape: btape.c:1909-0 Wrote block to device. > >>>>>> btape: btape.c:1907-0 Wrote one record of 64412 bytes. > >>>>>> btape: btape.c:1909-0 Wrote block to device. > >>>>>> btape: btape.c:1907-0 Wrote one record of 64412 bytes. > >>>>>> btape: btape.c:1909-0 Wrote block to device. > >>>>>> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) > >>>>>> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) > >>>>>> btape: btape.c:469-0 open device "DLT" (/dev/nsa1): OK > >>>>>> btape: btape.c:574-0 Rewound "DLT" (/dev/nsa1) > >>>>>> btape: btape.c:1420-0 Now moving to end of medium. > >>>>> > >>>>> This is the critical piece. The test moves the tape to the end of the > >>>>> medium. With hardware position information, you can tell what filemark > >>>>> you're on. Without it, you can't. > >>>>> > >>>>>> btape: btape.c:622-0 tape_dev.c:345 ioctl MTIOCGET error on "DLT" (/dev/nsa1). ERR=No error: 0. > >>>>>> We should be in file 3. I am at file 0. This is NOT correct!!!! > >>>>>> > >>>>>> Append test failed. Attempting again. > >>>>>> Setting "Hardware End of Medium = no > >>>>>> and "Fast Forward Space File = no > >>>>>> and retrying append test. > >>>>> > >>>>> This is not surprsing, given that the drive doesn't support long read > >>>>> position data. (It's a SCSI-2 device.) So that means that Bacula will > >>>>> need to do it manually. > >>>> > >>>> Yes, I have nothing newer than SCSI-2. Even my SDLT is SCSI-2 but that > >>>> tape library is hooked up to a different computer and was doing backups today. > >>> > >>> So, here is one thing that we can try to see whether these drives support > >>> long position information, even though they only claim to be SCSI-2. If > >>> they do, we can potentially add a quirk (or autodetection) to enable it. > >>> The code currently doesn't bother asking drives that claim to be SCSI-2 > >>> for long position information. (Because that feature was added in the > >>> SSC spec, which came after SCSI-2.) > >>> > >>> Issue a READ POSITION with the short form specified: > >>> > >>> camcontrol cmd sa1 -v -c "34 0 0 0 0 0 0 0 0 0" -i 20 - |hd > >>> > >>> Issue a READ POSITION with the vendor-specific block numbers: > >>> > >>> camcontrol cmd sa1 -v -c "34 1 0 0 0 0 0 0 0 0" -i 20 - |hd > >>> > >>> Issue a READ POSITION with the long form data: > >>> > >>> camcontrol cmd sa1 -v -c "34 6 0 0 0 0 0 0 0 0" -i 32 - |hd > >>> > >>> If it supports the last one, then I can put a quirk (or autodetection) in > >>> the driver and Bacula will get the hardware filemarks. You should try this > >>> on your SDLT as well. It may well support it. > >> > >> Sadly, no: > >> > >> [root@cuppy:~] # camcontrol cmd sa1 -v -c "34 0 0 0 0 0 0 0 0 0" -i 20 - |hd > >> 00000000 80 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| > >> 00000010 00 00 00 00 |....| > >> 00000014 > >> [root@cuppy:~] # camcontrol cmd sa1 -v -c "34 1 0 0 0 0 0 0 0 0" -i 20 - |hd > >> 00000000 80 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| > >> 00000010 00 00 00 00 |....| > >> 00000014 > >> [root@cuppy:~] # camcontrol cmd sa1 -v -c "34 6 0 0 0 0 0 0 0 0" -i 32 - |hd > >> camcontrol: error sending command > >> (pass2:ahc0:0:2:0): READ POSITION. CDB: 34 06 00 00 00 00 00 00 00 00 > >> (pass2:ahc0:0:2:0): CAM status: SCSI Status Error > >> (pass2:ahc0:0:2:0): SCSI status: Check Condition > >> (pass2:ahc0:0:2:0): SCSI sense: ILLEGAL REQUEST asc:24,0 (Invalid field in CDB) > >> (pass2:ahc0:0:2:0): Command byte 1 bit 2 is invalid > >> [root@cuppy:~] # > > > > Okay. Not too surprising I suppose. > > > >> The SDLT server is on 9.3 though: > >> > >> [root@knew:/usr/home/dan] # camcontrol cmd sa1 -v -c "34 0 0 0 0 0 0 0 0 0" -i 20 - |hd > >> camcontrol: cam_lookup_pass: CAMGETPASSTHRU ioctl failed > >> cam_lookup_pass: No such file or directory > >> cam_lookup_pass: either the pass driver isn't in your kernel > >> cam_lookup_pass: or sa1 doesn't exist > >> [root@knew:/usr/home/dan] # uname -a > >> FreeBSD knew.unixathome.org 9.3-RELEASE-p10 FreeBSD 9.3-RELEASE-p10 #0: Tue Feb 24 21:28:03 UTC 2015 root@amd64-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC amd64 > >> [root@knew:/usr/home/dan] # > >> > >> > >> It took me a while to figure that out... there is no sa1 on *this* system. > >> > >> But, my SDLT: > >> > >> [root@knew:/usr/home/dan] # camcontrol cmd sa0 -v -c "34 0 0 0 0 0 0 0 0 0" -i 20 - |hd > >> 00000000 80 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| > >> 00000010 00 00 00 00 |....| > >> 00000014 > >> [root@knew:/usr/home/dan] # camcontrol cmd sa0 -v -c "34 1 0 0 0 0 0 0 0 0" -i 20 - |hd > >> 00000000 80 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| > >> 00000010 00 00 00 00 |....| > >> 00000014 > >> [root@knew:/usr/home/dan] # camcontrol cmd sa0 -v -c "34 6 0 0 0 0 0 0 0 0" -i 32 - |hd > >> 00000000 80 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| > >> 00000010 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| > >> 00000020 > >> [root@knew:/usr/home/dan] # > > > > Just to confirm, can you send the output of: > > > > camcontrol inquiry sa0 -v > > > > I want to make certain it reports that it is SCSI-2. If so, I'll change > > the check in the driver to try asking for long position information on > > SCSI-2 devices. If it fails, it'll fall back to the regular method. > > [dan@knew:~] $ sudo camcontrol inquiry sa0 -v > pass10: Removable Sequential Access SCSI-2 device > pass10: Serial Number CXB46H0716 > pass10: 80.000MB/s transfers (40.000MHz, offset 31, 16bit) > [dan@knew:~] $ Okay. Doing the check doesn't cause any problems on my collection of old tape drives, so I'll go ahead and enable checking on SCSI-2 devices. The primary thing is just making sure it doesn't cause tape drive to choke if we request long position information. It doesn't appear to be an issue. If we do run into one, we can put in a quirk. Ken -- Kenneth Merry ken@FreeBSD.ORG From owner-freebsd-current@FreeBSD.ORG Mon Mar 2 17:39:17 2015 Return-Path: Delivered-To: current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 80281877; Mon, 2 Mar 2015 17:39:17 +0000 (UTC) Received: from mithlond.kdm.org (mithlond.kdm.org [70.56.43.85]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "A1-33714", Issuer "A1-33714" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 24E7C908; Mon, 2 Mar 2015 17:39:17 +0000 (UTC) Received: from mithlond.kdm.org (localhost [127.0.0.1]) by mithlond.kdm.org (8.14.9/8.14.9) with ESMTP id t22HdFki087288 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Mon, 2 Mar 2015 10:39:15 -0700 (MST) (envelope-from ken@mithlond.kdm.org) Received: (from ken@localhost) by mithlond.kdm.org (8.14.9/8.14.9/Submit) id t22HdFQc087287; Mon, 2 Mar 2015 10:39:15 -0700 (MST) (envelope-from ken) Date: Mon, 2 Mar 2015 10:39:15 -0700 From: "Kenneth D. Merry" To: Dan Langille Subject: Re: sa(4) driver changes available for test Message-ID: <20150302173914.GC87055@mithlond.kdm.org> References: <20150214003232.GA63990@mithlond.kdm.org> <8EFB1A84-96E6-4F63-A6D0-A87432C02B72@langille.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <8EFB1A84-96E6-4F63-A6D0-A87432C02B72@langille.org> User-Agent: Mutt/1.5.23 (2014-03-12) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (mithlond.kdm.org [127.0.0.1]); Mon, 02 Mar 2015 10:39:15 -0700 (MST) X-Spam-Status: No, score=-2.9 required=5.0 tests=ALL_TRUSTED,BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on mithlond.kdm.org Cc: current@freebsd.org, scsi@freebsd.org X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Mar 2015 17:39:17 -0000 On Mon, Mar 02, 2015 at 11:45:59 -0500, Dan Langille wrote: > > > On Feb 13, 2015, at 7:32 PM, Kenneth D. Merry wrote: > > > > > > I have a fairly large set of changes to the sa(4) driver and mt(1) driver > > that I'm planning to commit in the near future. > > > > A description of the changes is here and below in this message. > > > > If you have tape hardware and the inclination, I'd appreciate testing and > > feedback. > > This came to me today via the Bacula mailing lists. > > http://marc.info/?l=bacula-users&m=142531236722693&w=2 > > > As far as I can tell ltfs support on linux sits on top of the standard mt-st stuff \ > > as a userspace (fuse) filesystem > > I'd hope it's much the same with BSD. Removing the standard interface would be \ > > counterproductive overall > > Can you answer that and I'll relay please? Sure. In short, the current interface will stay in place. I have added additional ioctls that provide more features and information, but I don't see any issue with leaving the current ioctls in place. The MTIOCGET ioctl even gets an improvement in behavior when the tape drive supports long position information -- it will report the file number after a 'mt eod'. IBM's LTFS sits on top of their own Linux tape driver, and operates with a combination of tape driver ioctls (e.g. the standard MTIOTCOP ioctls) and SCSI passthrough. When I ported it to FreeBSD, I ran into several areas where we needed more information out of the tape driver. So that was the primary motivation behind adding the additional features. (Other features I implemented using SCSI passthrough.) He is correct that it runs with FUSE, although it can be linked into an application as a library as well. Ken -- Kenneth Merry ken@FreeBSD.ORG From owner-freebsd-current@FreeBSD.ORG Mon Mar 2 19:07:38 2015 Return-Path: Delivered-To: current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 83B58CA7; Mon, 2 Mar 2015 19:07:38 +0000 (UTC) Received: from clavin1.langille.org (clavin.langille.org [162.208.116.86]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "clavin.langille.org", Issuer "StartCom Class 2 Primary Intermediate Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 53479653; Mon, 2 Mar 2015 19:07:37 +0000 (UTC) Received: from (clavin1.int.langille.org (clavin1.int.unixathome.org [10.4.7.7]) (Authenticated sender: hidden) with ESMTPSA id 211FB6FEC ; Mon, 2 Mar 2015 19:07:36 +0000 (UTC) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 8.2 \(2070.6\)) Subject: Re: sa(4) driver changes available for test From: Dan Langille In-Reply-To: <20150302172846.GB87055@mithlond.kdm.org> Date: Mon, 2 Mar 2015 14:07:35 -0500 Content-Transfer-Encoding: quoted-printable Message-Id: <12EED94D-CC9D-44C4-91C2-62C8A51B839B@langille.org> References: <20150214003232.GA63990@mithlond.kdm.org> <7CA52DF3-E073-4F50-BE4E-01C51CCDF2C7@langille.org> <20150217183645.GA30947@mithlond.kdm.org> <303C48AC-0DA8-447C-8DE7-0A1DD5DD7351@langille.org> <20150302003150.GB71528@mithlond.kdm.org> <20150302022957.GB73433@mithlond.kdm.org> <0F17CDDB-4CE9-4881-B948-6B9BEDEB6455@langille.org> <20150302163146.GA85515@mithlond.kdm.org> <85E2DDA0-4A2B-4316-A318-3706B3A05C11@langille.org> <20150302172846.GB87055@mithlond.kdm.org> To: "Kenneth D. Merry" X-Mailer: Apple Mail (2.2070.6) Cc: current@freebsd.org, scsi@freebsd.org X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Mar 2015 19:07:38 -0000 > On Mar 2, 2015, at 12:28 PM, Kenneth D. Merry wrote: >=20 > On Mon, Mar 02, 2015 at 11:44:09 -0500, Dan Langille wrote: >>=20 >>> On Mar 2, 2015, at 11:31 AM, Kenneth D. Merry = wrote: >>>=20 >>> On Mon, Mar 02, 2015 at 11:09:57 -0500, Dan Langille wrote: >>>>=20 >>>>> On Mar 1, 2015, at 9:29 PM, Kenneth D. Merry = wrote: >>>>>=20 >>>>> On Sun, Mar 01, 2015 at 19:41:07 -0500, Dan Langille wrote: >>>>>>=20 >>>>>>> On Mar 1, 2015, at 7:31 PM, Kenneth D. Merry = wrote: >>>>>>>=20 >>>>>>> On Sun, Mar 01, 2015 at 19:15:05 -0500, Dan Langille wrote: >>>>>>>>=20 >>>>>>>>> On Feb 17, 2015, at 1:36 PM, Kenneth D. Merry = wrote: >>>>>>>>>=20 >>>>>>>>> On Sat, Feb 14, 2015 at 18:22:43 -0500, Dan Langille wrote: >>>>>>>>>>=20 >>>>>>>>>>> On Feb 13, 2015, at 7:32 PM, Kenneth D. Merry = wrote: >>>>>>>>>>>=20 >>>>>>>>>>>=20 >>>>>>>>>>> I have a fairly large set of changes to the sa(4) driver and = mt(1) driver >>>>>>>>>>> that I'm planning to commit in the near future. >>>>>>>>>>>=20 >>>>>>>>>>> A description of the changes is here and below in this = message. >>>>>>>>>>>=20 >>>>>>>>>>> If you have tape hardware and the inclination, I'd = appreciate testing and >>>>>>>>>>> feedback. >>>>>>>>>>=20 >>>>>>>>>> I have a DLT 8000 and an SDLT 220. >>>>>>>>>>=20 >>>>>>>>>> I don't have anything running current, but I have a spare = machine which I could use for testing. >>>>>>>>>>=20 >>>>>>>>>> Do you see any value is tests with that hardware? I'd be = testing it via Bacula. >>>>>>>>>>=20 >>>>>>>>>> disclosure: I'm the sysutils/bacula-* maintainer and a Bacula = committer. >>>>>>>>>>=20 >>>>>>>>>=20 >>>>>>>>> Actually, yes. Bacula is a bit tricky to configure, so your = trying it out >>>>>>>>> would be helpful if you have the time. >>>>>>>>>=20 >>>>>>>>> In looking at the manuals for both the SDLT 220 and the DLT = 8000, they both >>>>>>>>> claim to support long position information for the SCSI READ = POSITION >>>>>>>>> command. >>>>>>>>>=20 >>>>>>>>> You can see what I'm talking about by doing: >>>>>>>>>=20 >>>>>>>>> mt eod >>>>>>>>> mt status >>>>>>>>>=20 >>>>>>>>> On my DDS-4 tape drive, this shows: >>>>>>>>>=20 >>>>>>>>> # mt -f /dev/nsa3 status >>>>>>>>> Drive: sa3: Serial Number: = HJ00YWY >>>>>>>>> --------------------------------- >>>>>>>>> Mode Density Blocksize bpi = Compression >>>>>>>>> Current: 0x26:DDS-4 1024 bytes 97000 enabled = (DCLZ) >>>>>>>>> --------------------------------- >>>>>>>>> Current Driver State: at rest. >>>>>>>>> --------------------------------- >>>>>>>>> Partition: 0 Calc File Number: -1 Calc Record = Number: -1 >>>>>>>>> Residual: 0 Reported File Number: -1 Reported Record = Number: -1 >>>>>>>>> Flags: None >>>>>>>>>=20 >>>>>>>>> But on an LTO-5, which will give long position information, I = get: >>>>>>>>>=20 >>>>>>>>> [root@doc ~]# mt status >>>>>>>>> Drive: sa0: >>>>>>>>> --------------------------------- >>>>>>>>> Mode Density Blocksize bpi = Compression >>>>>>>>> Current: 0x58:LTO-5 variable 384607 enabled = (0x1) >>>>>>>>> --------------------------------- >>>>>>>>> Current Driver State: at rest. >>>>>>>>> --------------------------------- >>>>>>>>> Partition: 0 Calc File Number: 2 Calc Record = Number: -1 >>>>>>>>> Residual: 0 Reported File Number: 2 Reported Record = Number: 32373 >>>>>>>>> Flags: None >>>>>>>>>=20 >>>>>>>>> That, in combination with the changes I made to the position = information >>>>>>>>> code in the driver, mean that even the old MTIOCGET ioctl = should return an >>>>>>>>> accurate file number at end of data. e.g., on the LTO-5: >>>>>>>>>=20 >>>>>>>>> [root@doc ~]# mt ostatus >>>>>>>>> Mode Density Blocksize bpi = Compression >>>>>>>>> Current: 0x58:LTO-5 variable 384607 0x1 >>>>>>>>> ---------available modes--------- >>>>>>>>> 0: 0x58:LTO-5 variable 384607 0x1 >>>>>>>>> 1: 0x58:LTO-5 variable 384607 0x1 >>>>>>>>> 2: 0x58:LTO-5 variable 384607 0x1 >>>>>>>>> 3: 0x58:LTO-5 variable 384607 0x1 >>>>>>>>> --------------------------------- >>>>>>>>> Current Driver State: at rest. >>>>>>>>> --------------------------------- >>>>>>>>> File Number: 2 Record Number: -1 Residual Count -1 >>>>>>>>>=20 >>>>>>>>> So the thing to try, in addition to just making sure that = Bacula continues >>>>>>>>> to work properly, is to try setting this for the tape drive in >>>>>>>>> bacula-sd.conf: >>>>>>>>>=20 >>>>>>>>> Hardware End of Medium =3D yes >>>>>>>>>=20 >>>>>>>>> It looks like the Bacula tape program (btape) has a test mode, = and it would >>>>>>>>> be good to run through the tests on one of the tape drives and = see whether >>>>>>>>> they work, and whether the results are different before and = after the >>>>>>>>> changes. I'm not sure how to enable the test mode. >>>>>>>>=20 >>>>>>>> I have this in /usr/local/etc/bacula/bacula-sd.conf >>>>>>>>=20 >>>>>>>> Device { >>>>>>>> Name =3D DLT >>>>>>>> Description =3D "QUANTUM DLT7000 1624" >>>>>>>> Media Type =3D DLT >>>>>>>> Archive Device =3D /dev/nsa1 >>>>>>>>=20 >>>>>>>> Autochanger =3D YES >>>>>>>> Drive Index =3D 0 >>>>>>>>=20 >>>>>>>> Offline On Unmount =3D no >>>>>>>> Hardware End of Medium =3D yes >>>>>>>> BSF at EOM =3D yes >>>>>>>> Backward Space Record =3D no >>>>>>>> Fast Forward Space File =3D no >>>>>>>> TWO EOF =3D yes >>>>>>>> } >>>>>>>>=20 >>>>>>>> FYI, http://www.freebsddiary.org/digital-tl891.php (from 2006) = has a btape test on this same model. >>>>>>>>=20 >>>>>>>> Here's the test I ran tonight: >>>>>>>>=20 >>>>>>>> [root@cuppy:/usr/home/dan] # btape -c = /usr/local/etc/bacula/bacula-sd.conf /dev/nsa1 = =20 >>>>>>>> Tape block granularity is 1024 bytes. >>>>>>>> btape: butil.c:287-0 Using device: "/dev/nsa1" for writing. >>>>>>>> btape: btape.c:469-0 open device "DLT" (/dev/nsa1): OK >>>>>>>> *test >>>>>>>>=20 >>>>>>>> =3D=3D=3D Write, rewind, and re-read test =3D=3D=3D >>>>>>>>=20 >>>>>>>> I'm going to write 10000 records and an EOF >>>>>>>> then write 10000 records and an EOF, then rewind, >>>>>>>> and re-read the data to verify that it is correct. >>>>>>>>=20 >>>>>>>> This is an *essential* feature ... >>>>>>>>=20 >>>>>>>> btape: btape.c:1152-0 Wrote 10000 blocks of 64412 bytes. >>>>>>>> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) >>>>>>>> btape: btape.c:1168-0 Wrote 10000 blocks of 64412 bytes. >>>>>>>> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) >>>>>>>> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) >>>>>>>> btape: btape.c:1210-0 Rewind OK. >>>>>>>> 10000 blocks re-read correctly. >>>>>>>> Got EOF on tape. >>>>>>>> 10000 blocks re-read correctly. >>>>>>>> =3D=3D=3D Test Succeeded. End Write, rewind, and re-read test = =3D=3D=3D >>>>>>>>=20 >>>>>>>> btape: btape.c:1277-0 Block position test >>>>>>>> btape: btape.c:1289-0 Rewind OK. >>>>>>>> Reposition to file:block 0:4 >>>>>>>> Block 5 re-read correctly. >>>>>>>> Reposition to file:block 0:200 >>>>>>>> Block 201 re-read correctly. >>>>>>>> Reposition to file:block 0:9999 >>>>>>>> Block 10000 re-read correctly. >>>>>>>> Reposition to file:block 1:0 >>>>>>>> Block 10001 re-read correctly. >>>>>>>> Reposition to file:block 1:600 >>>>>>>> Block 10601 re-read correctly. >>>>>>>> Reposition to file:block 1:9999 >>>>>>>> Block 20000 re-read correctly. >>>>>>>> =3D=3D=3D Test Succeeded. End Write, rewind, and re-read test = =3D=3D=3D >>>>>>>>=20 >>>>>>>>=20 >>>>>>>>=20 >>>>>>>> =3D=3D=3D Append files test =3D=3D=3D >>>>>>>>=20 >>>>>>>> This test is essential to Bacula. >>>>>>>>=20 >>>>>>>> I'm going to write one record in file 0, >>>>>>>> two records in file 1, >>>>>>>> and three records in file 2 >>>>>>>>=20 >>>>>>>> btape: btape.c:574-0 Rewound "DLT" (/dev/nsa1) >>>>>>>> btape: btape.c:1907-0 Wrote one record of 64412 bytes. >>>>>>>> btape: btape.c:1909-0 Wrote block to device. >>>>>>>> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) >>>>>>>> btape: btape.c:1907-0 Wrote one record of 64412 bytes. >>>>>>>> btape: btape.c:1909-0 Wrote block to device. >>>>>>>> btape: btape.c:1907-0 Wrote one record of 64412 bytes. >>>>>>>> btape: btape.c:1909-0 Wrote block to device. >>>>>>>> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) >>>>>>>> btape: btape.c:1907-0 Wrote one record of 64412 bytes. >>>>>>>> btape: btape.c:1909-0 Wrote block to device. >>>>>>>> btape: btape.c:1907-0 Wrote one record of 64412 bytes. >>>>>>>> btape: btape.c:1909-0 Wrote block to device. >>>>>>>> btape: btape.c:1907-0 Wrote one record of 64412 bytes. >>>>>>>> btape: btape.c:1909-0 Wrote block to device. >>>>>>>> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) >>>>>>>> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) >>>>>>>> btape: btape.c:469-0 open device "DLT" (/dev/nsa1): OK >>>>>>>> btape: btape.c:574-0 Rewound "DLT" (/dev/nsa1) >>>>>>>> btape: btape.c:1420-0 Now moving to end of medium. >>>>>>>=20 >>>>>>> This is the critical piece. The test moves the tape to the end = of the >>>>>>> medium. With hardware position information, you can tell what = filemark >>>>>>> you're on. Without it, you can't. >>>>>>>=20 >>>>>>>> btape: btape.c:622-0 tape_dev.c:345 ioctl MTIOCGET error on = "DLT" (/dev/nsa1). ERR=3DNo error: 0. >>>>>>>> We should be in file 3. I am at file 0. This is NOT correct!!!! >>>>>>>>=20 >>>>>>>> Append test failed. Attempting again. >>>>>>>> Setting "Hardware End of Medium =3D no >>>>>>>> and "Fast Forward Space File =3D no >>>>>>>> and retrying append test. >>>>>>>=20 >>>>>>> This is not surprsing, given that the drive doesn't support long = read >>>>>>> position data. (It's a SCSI-2 device.) So that means that = Bacula will >>>>>>> need to do it manually. >>>>>>=20 >>>>>> Yes, I have nothing newer than SCSI-2. Even my SDLT is SCSI-2 = but that >>>>>> tape library is hooked up to a different computer and was doing = backups today. >>>>>=20 >>>>> So, here is one thing that we can try to see whether these drives = support >>>>> long position information, even though they only claim to be = SCSI-2. If >>>>> they do, we can potentially add a quirk (or autodetection) to = enable it. >>>>> The code currently doesn't bother asking drives that claim to be = SCSI-2 >>>>> for long position information. (Because that feature was added in = the >>>>> SSC spec, which came after SCSI-2.) >>>>>=20 >>>>> Issue a READ POSITION with the short form specified: >>>>>=20 >>>>> camcontrol cmd sa1 -v -c "34 0 0 0 0 0 0 0 0 0" -i 20 - |hd >>>>>=20 >>>>> Issue a READ POSITION with the vendor-specific block numbers: >>>>>=20 >>>>> camcontrol cmd sa1 -v -c "34 1 0 0 0 0 0 0 0 0" -i 20 - |hd >>>>>=20 >>>>> Issue a READ POSITION with the long form data: >>>>>=20 >>>>> camcontrol cmd sa1 -v -c "34 6 0 0 0 0 0 0 0 0" -i 32 - |hd >>>>>=20 >>>>> If it supports the last one, then I can put a quirk (or = autodetection) in >>>>> the driver and Bacula will get the hardware filemarks. You should = try this >>>>> on your SDLT as well. It may well support it. >>>>=20 >>>> Sadly, no: >>>>=20 >>>> [root@cuppy:~] # camcontrol cmd sa1 -v -c "34 0 0 0 0 0 0 0 0 0" -i = 20 - |hd >>>> 00000000 80 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 = |................| >>>> 00000010 00 00 00 00 |....| >>>> 00000014 >>>> [root@cuppy:~] # camcontrol cmd sa1 -v -c "34 1 0 0 0 0 0 0 0 0" -i = 20 - |hd >>>> 00000000 80 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 = |................| >>>> 00000010 00 00 00 00 |....| >>>> 00000014 >>>> [root@cuppy:~] # camcontrol cmd sa1 -v -c "34 6 0 0 0 0 0 0 0 0" -i = 32 - |hd >>>> camcontrol: error sending command >>>> (pass2:ahc0:0:2:0): READ POSITION. CDB: 34 06 00 00 00 00 00 00 00 = 00=20 >>>> (pass2:ahc0:0:2:0): CAM status: SCSI Status Error >>>> (pass2:ahc0:0:2:0): SCSI status: Check Condition >>>> (pass2:ahc0:0:2:0): SCSI sense: ILLEGAL REQUEST asc:24,0 (Invalid = field in CDB) >>>> (pass2:ahc0:0:2:0): Command byte 1 bit 2 is invalid >>>> [root@cuppy:~] #=20 >>>=20 >>> Okay. Not too surprising I suppose. Does this mean much to you? Mar 2 19:42:59 cuppy kernel: (ahc0:A:2:0): Sending SDTR period 19, = offset f Mar 2 19:42:59 cuppy kernel: (ahc0:A:2:0): Received SDTR period 19, = offset f Mar 2 19:42:59 cuppy kernel: Filtered to period 19, offset f Mar 2 19:42:59 cuppy kernel: (ahc0:A:2:0): Sending SDTR period 19, = offset f Mar 2 19:42:59 cuppy kernel: (ahc0:A:2:0): Received SDTR period 19, = offset f Mar 2 19:42:59 cuppy kernel: Filtered to period 19, offset f Mar 2 19:42:59 cuppy kernel: (ahc0:A:2:0): Sending SDTR period 19, = offset f Mar 2 19:42:59 cuppy kernel: (ahc0:A:2:0): Received SDTR period 19, = offset f Mar 2 19:42:59 cuppy kernel: Filtered to period 19, offset f I'm having trouble with labeling barcodes from Bacula and the above is = seen in /var/log/messages >>>=20 >>>> The SDLT server is on 9.3 though: >>>>=20 >>>> [root@knew:/usr/home/dan] # camcontrol cmd sa1 -v -c "34 0 0 0 0 0 = 0 0 0 0" -i 20 - |hd >>>> camcontrol: cam_lookup_pass: CAMGETPASSTHRU ioctl failed >>>> cam_lookup_pass: No such file or directory >>>> cam_lookup_pass: either the pass driver isn't in your kernel >>>> cam_lookup_pass: or sa1 doesn't exist >>>> [root@knew:/usr/home/dan] # uname -a >>>> FreeBSD knew.unixathome.org 9.3-RELEASE-p10 FreeBSD 9.3-RELEASE-p10 = #0: Tue Feb 24 21:28:03 UTC 2015 = root@amd64-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC amd64 >>>> [root@knew:/usr/home/dan] #=20 >>>>=20 >>>>=20 >>>> It took me a while to figure that out... there is no sa1 on *this* = system. >>>>=20 >>>> But, my SDLT: >>>>=20 >>>> [root@knew:/usr/home/dan] # camcontrol cmd sa0 -v -c "34 0 0 0 0 0 = 0 0 0 0" -i 20 - |hd >>>> 00000000 80 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 = |................| >>>> 00000010 00 00 00 00 |....| >>>> 00000014 >>>> [root@knew:/usr/home/dan] # camcontrol cmd sa0 -v -c "34 1 0 0 0 0 = 0 0 0 0" -i 20 - |hd >>>> 00000000 80 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 = |................| >>>> 00000010 00 00 00 00 |....| >>>> 00000014 >>>> [root@knew:/usr/home/dan] # camcontrol cmd sa0 -v -c "34 6 0 0 0 0 = 0 0 0 0" -i 32 - |hd >>>> 00000000 80 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 = |................| >>>> 00000010 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 = |................| >>>> 00000020 >>>> [root@knew:/usr/home/dan] #=20 >>>=20 >>> Just to confirm, can you send the output of: >>>=20 >>> camcontrol inquiry sa0 -v >>>=20 >>> I want to make certain it reports that it is SCSI-2. If so, I'll = change >>> the check in the driver to try asking for long position information = on >>> SCSI-2 devices. If it fails, it'll fall back to the regular method. >>=20 >> [dan@knew:~] $ sudo camcontrol inquiry sa0 -v >> pass10: Removable Sequential Access SCSI-2 = device=20 >> pass10: Serial Number CXB46H0716 =20 >> pass10: 80.000MB/s transfers (40.000MHz, offset 31, 16bit) >> [dan@knew:~] $=20 >=20 > Okay. Doing the check doesn't cause any problems on my collection of = old > tape drives, so I'll go ahead and enable checking on SCSI-2 devices. >=20 > The primary thing is just making sure it doesn't cause tape drive to = choke > if we request long position information. It doesn't appear to be an > issue. If we do run into one, we can put in a quirk. >=20 > Ken > --=20 > Kenneth Merry > ken@FreeBSD.ORG =E2=80=94=20 Dan Langille http://langille.org/ From owner-freebsd-current@FreeBSD.ORG Mon Mar 2 19:47:10 2015 Return-Path: Delivered-To: current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id BAA739BD; Mon, 2 Mar 2015 19:47:10 +0000 (UTC) Received: from clavin1.langille.org (clavin.langille.org [162.208.116.86]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "clavin.langille.org", Issuer "StartCom Class 2 Primary Intermediate Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 89FF9B8C; Mon, 2 Mar 2015 19:47:09 +0000 (UTC) Received: from (clavin1.int.langille.org (clavin1.int.unixathome.org [10.4.7.7]) (Authenticated sender: hidden) with ESMTPSA id 9871C6FF3 ; Mon, 2 Mar 2015 19:47:08 +0000 (UTC) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 8.2 \(2070.6\)) Subject: Re: sa(4) driver changes available for test From: Dan Langille In-Reply-To: <12EED94D-CC9D-44C4-91C2-62C8A51B839B@langille.org> Date: Mon, 2 Mar 2015 14:47:07 -0500 Content-Transfer-Encoding: quoted-printable Message-Id: References: <20150214003232.GA63990@mithlond.kdm.org> <7CA52DF3-E073-4F50-BE4E-01C51CCDF2C7@langille.org> <20150217183645.GA30947@mithlond.kdm.org> <303C48AC-0DA8-447C-8DE7-0A1DD5DD7351@langille.org> <20150302003150.GB71528@mithlond.kdm.org> <20150302022957.GB73433@mithlond.kdm.org> <0F17CDDB-4CE9-4881-B948-6B9BEDEB6455@langille.org> <20150302163146.GA85515@mithlond.kdm.org> <85E2DDA0-4A2B-4316-A318-3706B3A05C11@langille.org> <20150302172846.GB87055@mithlond.kdm.org> <12EED94D-CC9D-44C4-91C2-62C8A51B839B@langille.org> To: "Kenneth D. Merry" X-Mailer: Apple Mail (2.2070.6) Cc: current@freebsd.org, scsi@freebsd.org X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Mar 2015 19:47:10 -0000 > On Mar 2, 2015, at 2:07 PM, Dan Langille wrote: >=20 >=20 >> On Mar 2, 2015, at 12:28 PM, Kenneth D. Merry = wrote: >>=20 >> On Mon, Mar 02, 2015 at 11:44:09 -0500, Dan Langille wrote: >>>=20 >>>> On Mar 2, 2015, at 11:31 AM, Kenneth D. Merry = wrote: >>>>=20 >>>> On Mon, Mar 02, 2015 at 11:09:57 -0500, Dan Langille wrote: >>>>>=20 >>>>>> On Mar 1, 2015, at 9:29 PM, Kenneth D. Merry = wrote: >>>>>>=20 >>>>>> On Sun, Mar 01, 2015 at 19:41:07 -0500, Dan Langille wrote: >>>>>>>=20 >>>>>>>> On Mar 1, 2015, at 7:31 PM, Kenneth D. Merry = wrote: >>>>>>>>=20 >>>>>>>> On Sun, Mar 01, 2015 at 19:15:05 -0500, Dan Langille wrote: >>>>>>>>>=20 >>>>>>>>>> On Feb 17, 2015, at 1:36 PM, Kenneth D. Merry = wrote: >>>>>>>>>>=20 >>>>>>>>>> On Sat, Feb 14, 2015 at 18:22:43 -0500, Dan Langille wrote: >>>>>>>>>>>=20 >>>>>>>>>>>> On Feb 13, 2015, at 7:32 PM, Kenneth D. Merry = wrote: >>>>>>>>>>>>=20 >>>>>>>>>>>>=20 >>>>>>>>>>>> I have a fairly large set of changes to the sa(4) driver = and mt(1) driver >>>>>>>>>>>> that I'm planning to commit in the near future. >>>>>>>>>>>>=20 >>>>>>>>>>>> A description of the changes is here and below in this = message. >>>>>>>>>>>>=20 >>>>>>>>>>>> If you have tape hardware and the inclination, I'd = appreciate testing and >>>>>>>>>>>> feedback. >>>>>>>>>>>=20 >>>>>>>>>>> I have a DLT 8000 and an SDLT 220. >>>>>>>>>>>=20 >>>>>>>>>>> I don't have anything running current, but I have a spare = machine which I could use for testing. >>>>>>>>>>>=20 >>>>>>>>>>> Do you see any value is tests with that hardware? I'd be = testing it via Bacula. >>>>>>>>>>>=20 >>>>>>>>>>> disclosure: I'm the sysutils/bacula-* maintainer and a = Bacula committer. >>>>>>>>>>>=20 >>>>>>>>>>=20 >>>>>>>>>> Actually, yes. Bacula is a bit tricky to configure, so your = trying it out >>>>>>>>>> would be helpful if you have the time. >>>>>>>>>>=20 >>>>>>>>>> In looking at the manuals for both the SDLT 220 and the DLT = 8000, they both >>>>>>>>>> claim to support long position information for the SCSI READ = POSITION >>>>>>>>>> command. >>>>>>>>>>=20 >>>>>>>>>> You can see what I'm talking about by doing: >>>>>>>>>>=20 >>>>>>>>>> mt eod >>>>>>>>>> mt status >>>>>>>>>>=20 >>>>>>>>>> On my DDS-4 tape drive, this shows: >>>>>>>>>>=20 >>>>>>>>>> # mt -f /dev/nsa3 status >>>>>>>>>> Drive: sa3: Serial Number: = HJ00YWY >>>>>>>>>> --------------------------------- >>>>>>>>>> Mode Density Blocksize bpi = Compression >>>>>>>>>> Current: 0x26:DDS-4 1024 bytes 97000 = enabled (DCLZ) >>>>>>>>>> --------------------------------- >>>>>>>>>> Current Driver State: at rest. >>>>>>>>>> --------------------------------- >>>>>>>>>> Partition: 0 Calc File Number: -1 Calc Record = Number: -1 >>>>>>>>>> Residual: 0 Reported File Number: -1 Reported Record = Number: -1 >>>>>>>>>> Flags: None >>>>>>>>>>=20 >>>>>>>>>> But on an LTO-5, which will give long position information, I = get: >>>>>>>>>>=20 >>>>>>>>>> [root@doc ~]# mt status >>>>>>>>>> Drive: sa0: >>>>>>>>>> --------------------------------- >>>>>>>>>> Mode Density Blocksize bpi = Compression >>>>>>>>>> Current: 0x58:LTO-5 variable 384607 = enabled (0x1) >>>>>>>>>> --------------------------------- >>>>>>>>>> Current Driver State: at rest. >>>>>>>>>> --------------------------------- >>>>>>>>>> Partition: 0 Calc File Number: 2 Calc Record = Number: -1 >>>>>>>>>> Residual: 0 Reported File Number: 2 Reported Record = Number: 32373 >>>>>>>>>> Flags: None >>>>>>>>>>=20 >>>>>>>>>> That, in combination with the changes I made to the position = information >>>>>>>>>> code in the driver, mean that even the old MTIOCGET ioctl = should return an >>>>>>>>>> accurate file number at end of data. e.g., on the LTO-5: >>>>>>>>>>=20 >>>>>>>>>> [root@doc ~]# mt ostatus >>>>>>>>>> Mode Density Blocksize bpi = Compression >>>>>>>>>> Current: 0x58:LTO-5 variable 384607 0x1 >>>>>>>>>> ---------available modes--------- >>>>>>>>>> 0: 0x58:LTO-5 variable 384607 0x1 >>>>>>>>>> 1: 0x58:LTO-5 variable 384607 0x1 >>>>>>>>>> 2: 0x58:LTO-5 variable 384607 0x1 >>>>>>>>>> 3: 0x58:LTO-5 variable 384607 0x1 >>>>>>>>>> --------------------------------- >>>>>>>>>> Current Driver State: at rest. >>>>>>>>>> --------------------------------- >>>>>>>>>> File Number: 2 Record Number: -1 Residual Count -1 >>>>>>>>>>=20 >>>>>>>>>> So the thing to try, in addition to just making sure that = Bacula continues >>>>>>>>>> to work properly, is to try setting this for the tape drive = in >>>>>>>>>> bacula-sd.conf: >>>>>>>>>>=20 >>>>>>>>>> Hardware End of Medium =3D yes >>>>>>>>>>=20 >>>>>>>>>> It looks like the Bacula tape program (btape) has a test = mode, and it would >>>>>>>>>> be good to run through the tests on one of the tape drives = and see whether >>>>>>>>>> they work, and whether the results are different before and = after the >>>>>>>>>> changes. I'm not sure how to enable the test mode. >>>>>>>>>=20 >>>>>>>>> I have this in /usr/local/etc/bacula/bacula-sd.conf >>>>>>>>>=20 >>>>>>>>> Device { >>>>>>>>> Name =3D DLT >>>>>>>>> Description =3D "QUANTUM DLT7000 1624" >>>>>>>>> Media Type =3D DLT >>>>>>>>> Archive Device =3D /dev/nsa1 >>>>>>>>>=20 >>>>>>>>> Autochanger =3D YES >>>>>>>>> Drive Index =3D 0 >>>>>>>>>=20 >>>>>>>>> Offline On Unmount =3D no >>>>>>>>> Hardware End of Medium =3D yes >>>>>>>>> BSF at EOM =3D yes >>>>>>>>> Backward Space Record =3D no >>>>>>>>> Fast Forward Space File =3D no >>>>>>>>> TWO EOF =3D yes >>>>>>>>> } >>>>>>>>>=20 >>>>>>>>> FYI, http://www.freebsddiary.org/digital-tl891.php (from 2006) = has a btape test on this same model. >>>>>>>>>=20 >>>>>>>>> Here's the test I ran tonight: >>>>>>>>>=20 >>>>>>>>> [root@cuppy:/usr/home/dan] # btape -c = /usr/local/etc/bacula/bacula-sd.conf /dev/nsa1 = =20 >>>>>>>>> Tape block granularity is 1024 bytes. >>>>>>>>> btape: butil.c:287-0 Using device: "/dev/nsa1" for writing. >>>>>>>>> btape: btape.c:469-0 open device "DLT" (/dev/nsa1): OK >>>>>>>>> *test >>>>>>>>>=20 >>>>>>>>> =3D=3D=3D Write, rewind, and re-read test =3D=3D=3D >>>>>>>>>=20 >>>>>>>>> I'm going to write 10000 records and an EOF >>>>>>>>> then write 10000 records and an EOF, then rewind, >>>>>>>>> and re-read the data to verify that it is correct. >>>>>>>>>=20 >>>>>>>>> This is an *essential* feature ... >>>>>>>>>=20 >>>>>>>>> btape: btape.c:1152-0 Wrote 10000 blocks of 64412 bytes. >>>>>>>>> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) >>>>>>>>> btape: btape.c:1168-0 Wrote 10000 blocks of 64412 bytes. >>>>>>>>> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) >>>>>>>>> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) >>>>>>>>> btape: btape.c:1210-0 Rewind OK. >>>>>>>>> 10000 blocks re-read correctly. >>>>>>>>> Got EOF on tape. >>>>>>>>> 10000 blocks re-read correctly. >>>>>>>>> =3D=3D=3D Test Succeeded. End Write, rewind, and re-read test = =3D=3D=3D >>>>>>>>>=20 >>>>>>>>> btape: btape.c:1277-0 Block position test >>>>>>>>> btape: btape.c:1289-0 Rewind OK. >>>>>>>>> Reposition to file:block 0:4 >>>>>>>>> Block 5 re-read correctly. >>>>>>>>> Reposition to file:block 0:200 >>>>>>>>> Block 201 re-read correctly. >>>>>>>>> Reposition to file:block 0:9999 >>>>>>>>> Block 10000 re-read correctly. >>>>>>>>> Reposition to file:block 1:0 >>>>>>>>> Block 10001 re-read correctly. >>>>>>>>> Reposition to file:block 1:600 >>>>>>>>> Block 10601 re-read correctly. >>>>>>>>> Reposition to file:block 1:9999 >>>>>>>>> Block 20000 re-read correctly. >>>>>>>>> =3D=3D=3D Test Succeeded. End Write, rewind, and re-read test = =3D=3D=3D >>>>>>>>>=20 >>>>>>>>>=20 >>>>>>>>>=20 >>>>>>>>> =3D=3D=3D Append files test =3D=3D=3D >>>>>>>>>=20 >>>>>>>>> This test is essential to Bacula. >>>>>>>>>=20 >>>>>>>>> I'm going to write one record in file 0, >>>>>>>>> two records in file 1, >>>>>>>>> and three records in file 2 >>>>>>>>>=20 >>>>>>>>> btape: btape.c:574-0 Rewound "DLT" (/dev/nsa1) >>>>>>>>> btape: btape.c:1907-0 Wrote one record of 64412 bytes. >>>>>>>>> btape: btape.c:1909-0 Wrote block to device. >>>>>>>>> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) >>>>>>>>> btape: btape.c:1907-0 Wrote one record of 64412 bytes. >>>>>>>>> btape: btape.c:1909-0 Wrote block to device. >>>>>>>>> btape: btape.c:1907-0 Wrote one record of 64412 bytes. >>>>>>>>> btape: btape.c:1909-0 Wrote block to device. >>>>>>>>> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) >>>>>>>>> btape: btape.c:1907-0 Wrote one record of 64412 bytes. >>>>>>>>> btape: btape.c:1909-0 Wrote block to device. >>>>>>>>> btape: btape.c:1907-0 Wrote one record of 64412 bytes. >>>>>>>>> btape: btape.c:1909-0 Wrote block to device. >>>>>>>>> btape: btape.c:1907-0 Wrote one record of 64412 bytes. >>>>>>>>> btape: btape.c:1909-0 Wrote block to device. >>>>>>>>> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) >>>>>>>>> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) >>>>>>>>> btape: btape.c:469-0 open device "DLT" (/dev/nsa1): OK >>>>>>>>> btape: btape.c:574-0 Rewound "DLT" (/dev/nsa1) >>>>>>>>> btape: btape.c:1420-0 Now moving to end of medium. >>>>>>>>=20 >>>>>>>> This is the critical piece. The test moves the tape to the end = of the >>>>>>>> medium. With hardware position information, you can tell what = filemark >>>>>>>> you're on. Without it, you can't. >>>>>>>>=20 >>>>>>>>> btape: btape.c:622-0 tape_dev.c:345 ioctl MTIOCGET error on = "DLT" (/dev/nsa1). ERR=3DNo error: 0. >>>>>>>>> We should be in file 3. I am at file 0. This is NOT = correct!!!! >>>>>>>>>=20 >>>>>>>>> Append test failed. Attempting again. >>>>>>>>> Setting "Hardware End of Medium =3D no >>>>>>>>> and "Fast Forward Space File =3D no >>>>>>>>> and retrying append test. >>>>>>>>=20 >>>>>>>> This is not surprsing, given that the drive doesn't support = long read >>>>>>>> position data. (It's a SCSI-2 device.) So that means that = Bacula will >>>>>>>> need to do it manually. >>>>>>>=20 >>>>>>> Yes, I have nothing newer than SCSI-2. Even my SDLT is SCSI-2 = but that >>>>>>> tape library is hooked up to a different computer and was doing = backups today. >>>>>>=20 >>>>>> So, here is one thing that we can try to see whether these drives = support >>>>>> long position information, even though they only claim to be = SCSI-2. If >>>>>> they do, we can potentially add a quirk (or autodetection) to = enable it. >>>>>> The code currently doesn't bother asking drives that claim to be = SCSI-2 >>>>>> for long position information. (Because that feature was added = in the >>>>>> SSC spec, which came after SCSI-2.) >>>>>>=20 >>>>>> Issue a READ POSITION with the short form specified: >>>>>>=20 >>>>>> camcontrol cmd sa1 -v -c "34 0 0 0 0 0 0 0 0 0" -i 20 - |hd >>>>>>=20 >>>>>> Issue a READ POSITION with the vendor-specific block numbers: >>>>>>=20 >>>>>> camcontrol cmd sa1 -v -c "34 1 0 0 0 0 0 0 0 0" -i 20 - |hd >>>>>>=20 >>>>>> Issue a READ POSITION with the long form data: >>>>>>=20 >>>>>> camcontrol cmd sa1 -v -c "34 6 0 0 0 0 0 0 0 0" -i 32 - |hd >>>>>>=20 >>>>>> If it supports the last one, then I can put a quirk (or = autodetection) in >>>>>> the driver and Bacula will get the hardware filemarks. You = should try this >>>>>> on your SDLT as well. It may well support it. >>>>>=20 >>>>> Sadly, no: >>>>>=20 >>>>> [root@cuppy:~] # camcontrol cmd sa1 -v -c "34 0 0 0 0 0 0 0 0 0" = -i 20 - |hd >>>>> 00000000 80 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 = |................| >>>>> 00000010 00 00 00 00 |....| >>>>> 00000014 >>>>> [root@cuppy:~] # camcontrol cmd sa1 -v -c "34 1 0 0 0 0 0 0 0 0" = -i 20 - |hd >>>>> 00000000 80 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 = |................| >>>>> 00000010 00 00 00 00 |....| >>>>> 00000014 >>>>> [root@cuppy:~] # camcontrol cmd sa1 -v -c "34 6 0 0 0 0 0 0 0 0" = -i 32 - |hd >>>>> camcontrol: error sending command >>>>> (pass2:ahc0:0:2:0): READ POSITION. CDB: 34 06 00 00 00 00 00 00 00 = 00=20 >>>>> (pass2:ahc0:0:2:0): CAM status: SCSI Status Error >>>>> (pass2:ahc0:0:2:0): SCSI status: Check Condition >>>>> (pass2:ahc0:0:2:0): SCSI sense: ILLEGAL REQUEST asc:24,0 (Invalid = field in CDB) >>>>> (pass2:ahc0:0:2:0): Command byte 1 bit 2 is invalid >>>>> [root@cuppy:~] #=20 >>>>=20 >>>> Okay. Not too surprising I suppose. >=20 >=20 > Does this mean much to you? >=20 > Mar 2 19:42:59 cuppy kernel: (ahc0:A:2:0): Sending SDTR period 19, = offset f > Mar 2 19:42:59 cuppy kernel: (ahc0:A:2:0): Received SDTR period 19, = offset f > Mar 2 19:42:59 cuppy kernel: Filtered to period 19, offset f > Mar 2 19:42:59 cuppy kernel: (ahc0:A:2:0): Sending SDTR period 19, = offset f > Mar 2 19:42:59 cuppy kernel: (ahc0:A:2:0): Received SDTR period 19, = offset f > Mar 2 19:42:59 cuppy kernel: Filtered to period 19, offset f > Mar 2 19:42:59 cuppy kernel: (ahc0:A:2:0): Sending SDTR period 19, = offset f > Mar 2 19:42:59 cuppy kernel: (ahc0:A:2:0): Received SDTR period 19, = offset f > Mar 2 19:42:59 cuppy kernel: Filtered to period 19, offset f >=20 >=20 > I'm having trouble with labeling barcodes from Bacula and the above is = seen in /var/log/messages The barcodes issue is resolved. I changed to using drive 0 instead of = drive 1, now that we have both drives. *label barcodes slots=3D3,4 The defined Storage resources are: 1: File1 2: OldTL891 Select Storage resource (1-2): 2 Connecting to Storage daemon OldTL891 at 10.55.0.18:9103 ... 3306 Issuing autochanger "slots" command. Device "DTL03" has 10 slots. Connecting to Storage daemon OldTL891 at 10.55.0.18:9103 ... 3306 Issuing autochanger "list" command. The following Volumes will be labeled: Slot Volume =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D 3 FAI261 4 FAI262 Do you want to label these Volumes? (yes|no): yes Defined Pools: 1: Default 2: File 3: MYDLT 4: Scratch Select the Pool (1-4): 4 Connecting to Storage daemon OldTL891 at 10.55.0.18:9103 ... Sending label command for Volume "FAI261" Slot 3 ... 3000 OK label. VolBytes=3D64512 DVD=3D0 Volume=3D"FAI261" Device=3D"DTL03"= (/dev/nsa0) Catalog record for Volume "FAI261", Slot 3 successfully created. Sending label command for Volume "FAI262" Slot 4 ... 3307 Issuing autochanger "unload slot 3, drive 0" command. 3304 Issuing autochanger "load slot 4, drive 0" command. 3305 Autochanger "load slot 4, drive 0", status is OK. 3000 OK label. VolBytes=3D64512 DVD=3D0 Volume=3D"FAI262" Device=3D"DTL03"= (/dev/nsa0) Catalog record for Volume "FAI262", Slot 4 successfully created. *list volumes Pool: Default No results to list. Pool: File = +---------+------------+-----------+---------+------------+----------+----= ----------+---------+------+-----------+-----------+---------------------+= | mediaid | volumename | volstatus | enabled | volbytes | volfiles | = volretention | recycle | slot | inchanger | mediatype | lastwritten = | = +---------+------------+-----------+---------+------------+----------+----= ----------+---------+------+-----------+-----------+---------------------+= | 1 | Vol-0001 | Append | 1 | 19,835,982 | 0 | = 31,536,000 | 1 | 0 | 0 | File1 | 2015-03-02 = 17:50:40 | | 2 | Vol-0002 | Append | 1 | 0 | 0 | = 31,536,000 | 1 | 0 | 0 | DLT | = | = +---------+------------+-----------+---------+------------+----------+----= ----------+---------+------+-----------+-----------+---------------------+= Pool: MYDLT No results to list. Pool: Scratch = +---------+------------+-----------+---------+----------+----------+------= --------+---------+------+-----------+-----------+-------------+ | mediaid | volumename | volstatus | enabled | volbytes | volfiles | = volretention | recycle | slot | inchanger | mediatype | lastwritten | = +---------+------------+-----------+---------+----------+----------+------= --------+---------+------+-----------+-----------+-------------+ | 4 | FAI260 | Append | 1 | 64,512 | 0 | = 31,536,000 | 1 | 1 | 1 | DLT | | | 5 | FAI263 | Append | 1 | 64,512 | 0 | = 31,536,000 | 1 | 2 | 1 | DLT | | | 7 | FAI261 | Append | 1 | 64,512 | 0 | = 31,536,000 | 1 | 3 | 1 | DLT | | | 8 | FAI262 | Append | 1 | 64,512 | 0 | = 31,536,000 | 1 | 4 | 1 | DLT | | = +---------+------------+-----------+---------+----------+----------+------= --------+---------+------+-----------+-----------+-------------+ * But this is what arrives in /var/log/messages during the above: Mar 2 20:29:06 cuppy kernel: (ahc0:A:1:0): Sending SDTR period 19, = offset f Mar 2 20:29:06 cuppy kernel: (ahc0:A:1:0): Received SDTR period 19, = offset f Mar 2 20:29:06 cuppy kernel: Filtered to period 19, offset f Mar 2 20:30:22 cuppy kernel: (ahc0:A:1:0): Sending SDTR period 19, = offset f Mar 2 20:30:22 cuppy kernel: (ahc0:A:1:0): Received SDTR period 19, = offset f Mar 2 20:30:22 cuppy kernel: Filtered to period 19, offset f Mar 2 20:30:22 cuppy kernel: (ahc0:A:1:0): Sending SDTR period 19, = offset f Mar 2 20:30:22 cuppy kernel: (ahc0:A:1:0): Received SDTR period 19, = offset f Mar 2 20:30:22 cuppy kernel: Filtered to period 19, offset f Mar 2 20:30:59 cuppy kernel: (ahc0:A:1:0): Sending SDTR period 19, = offset f Mar 2 20:30:59 cuppy kernel: (ahc0:A:1:0): Received SDTR period 19, = offset f Mar 2 20:30:59 cuppy kernel: Filtered to period 19, offset f Mar 2 20:30:59 cuppy kernel: xpt_release_devq(): requested 1 > present = 0 Mar 2 20:31:07 cuppy kernel: (ahc0:A:1:0): Sending SDTR period 19, = offset f Mar 2 20:31:07 cuppy kernel: (ahc0:A:1:0): Received SDTR period 19, = offset f Mar 2 20:31:07 cuppy kernel: Filtered to period 19, offset f Anything to be concerned about? >=20 >>>>=20 >>>>> The SDLT server is on 9.3 though: >>>>>=20 >>>>> [root@knew:/usr/home/dan] # camcontrol cmd sa1 -v -c "34 0 0 0 0 0 = 0 0 0 0" -i 20 - |hd >>>>> camcontrol: cam_lookup_pass: CAMGETPASSTHRU ioctl failed >>>>> cam_lookup_pass: No such file or directory >>>>> cam_lookup_pass: either the pass driver isn't in your kernel >>>>> cam_lookup_pass: or sa1 doesn't exist >>>>> [root@knew:/usr/home/dan] # uname -a >>>>> FreeBSD knew.unixathome.org 9.3-RELEASE-p10 FreeBSD = 9.3-RELEASE-p10 #0: Tue Feb 24 21:28:03 UTC 2015 = root@amd64-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC amd64 >>>>> [root@knew:/usr/home/dan] #=20 >>>>>=20 >>>>>=20 >>>>> It took me a while to figure that out... there is no sa1 on *this* = system. >>>>>=20 >>>>> But, my SDLT: >>>>>=20 >>>>> [root@knew:/usr/home/dan] # camcontrol cmd sa0 -v -c "34 0 0 0 0 0 = 0 0 0 0" -i 20 - |hd >>>>> 00000000 80 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 = |................| >>>>> 00000010 00 00 00 00 |....| >>>>> 00000014 >>>>> [root@knew:/usr/home/dan] # camcontrol cmd sa0 -v -c "34 1 0 0 0 0 = 0 0 0 0" -i 20 - |hd >>>>> 00000000 80 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 = |................| >>>>> 00000010 00 00 00 00 |....| >>>>> 00000014 >>>>> [root@knew:/usr/home/dan] # camcontrol cmd sa0 -v -c "34 6 0 0 0 0 = 0 0 0 0" -i 32 - |hd >>>>> 00000000 80 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 = |................| >>>>> 00000010 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 = |................| >>>>> 00000020 >>>>> [root@knew:/usr/home/dan] #=20 >>>>=20 >>>> Just to confirm, can you send the output of: >>>>=20 >>>> camcontrol inquiry sa0 -v >>>>=20 >>>> I want to make certain it reports that it is SCSI-2. If so, I'll = change >>>> the check in the driver to try asking for long position information = on >>>> SCSI-2 devices. If it fails, it'll fall back to the regular = method. >>>=20 >>> [dan@knew:~] $ sudo camcontrol inquiry sa0 -v >>> pass10: Removable Sequential Access SCSI-2 = device=20 >>> pass10: Serial Number CXB46H0716 =20 >>> pass10: 80.000MB/s transfers (40.000MHz, offset 31, 16bit) >>> [dan@knew:~] $=20 >>=20 >> Okay. Doing the check doesn't cause any problems on my collection of = old >> tape drives, so I'll go ahead and enable checking on SCSI-2 devices. >>=20 >> The primary thing is just making sure it doesn't cause tape drive to = choke >> if we request long position information. It doesn't appear to be an >> issue. If we do run into one, we can put in a quirk. >>=20 >> Ken >> --=20 >> Kenneth Merry >> ken@FreeBSD.ORG >=20 > =E2=80=94=20 > Dan Langille > http://langille.org/ >=20 >=20 >=20 >=20 >=20 =E2=80=94=20 Dan Langille http://langille.org/ From owner-freebsd-current@FreeBSD.ORG Mon Mar 2 19:53:31 2015 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id DF448C8B for ; Mon, 2 Mar 2015 19:53:31 +0000 (UTC) Received: from vps1.elischer.org (vps1.elischer.org [204.109.63.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "vps1.elischer.org", Issuer "CA Cert Signing Authority" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id AD50EC6C for ; Mon, 2 Mar 2015 19:53:31 +0000 (UTC) Received: from Julian-MBP3.local ([12.179.176.146]) (authenticated bits=0) by vps1.elischer.org (8.14.9/8.14.9) with ESMTP id t22JrTK6006546 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO) for ; Mon, 2 Mar 2015 11:53:30 -0800 (PST) (envelope-from julian@freebsd.org) Message-ID: <54F4BFB4.1060307@freebsd.org> Date: Mon, 02 Mar 2015 11:53:24 -0800 From: Julian Elischer User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:31.0) Gecko/20100101 Thunderbird/31.5.0 MIME-Version: 1.0 To: freebsd-current@freebsd.org Subject: Re: Massive libxo-zation that breaks everything References: <54F31510.7050607@hot.ee> <54F34B6E.2040809@astrodoggroup.com> <54F35F29.4000603@astrodoggroup.com> <54F36431.30506@freebsd.org> <54F42A82.1020308@freebsd.org> <54F464D1.6060603@mu.org> In-Reply-To: <54F464D1.6060603@mu.org> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Mar 2015 19:53:32 -0000 On 3/2/15 5:25 AM, Alfred Perlstein wrote: > > > On 3/2/15 4:25 AM, David Chisnall wrote: >> On 2 Mar 2015, at 09:16, Julian Elischer wrote: >>> if we develop a suitable post processor with pluggable grammars, >>> we save a lot of work. >>> given enough examples you could almost have automatically >>> generated grammars. >> This decoupled approach is problematic. A large part of the point >> of libxo is to allow changing the human-readable output without >> breaking tools that consume the output. Now I need to keep the >> tool that consumes it and the tool that produces it in sync, so >> that's an extra set of moving parts. When you throw jails with >> multiple versions of world into the mix, it becomes a recipe for >> disaster. why? the jail has it own /usr/share? >> > +1 I think the risk is exactly opposite. That the human readable output will change subtly with bugs in the xo implementation. and people will not update the two output paths in exactly the same way, leading bugs. I'm not going to fight on it, but I am uncomfortable with it. You are increasing the complexity of every program you touch. > _______________________________________________ > freebsd-current@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-current > To unsubscribe, send any mail to > "freebsd-current-unsubscribe@freebsd.org" > From owner-freebsd-current@FreeBSD.ORG Mon Mar 2 19:55:57 2015 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 96F02DE9 for ; Mon, 2 Mar 2015 19:55:57 +0000 (UTC) Received: from vps1.elischer.org (vps1.elischer.org [204.109.63.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "vps1.elischer.org", Issuer "CA Cert Signing Authority" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4A5BBC8F for ; Mon, 2 Mar 2015 19:55:56 +0000 (UTC) Received: from Julian-MBP3.local (50-196-156-133-static.hfc.comcastbusiness.net [50.196.156.133]) (authenticated bits=0) by vps1.elischer.org (8.14.9/8.14.9) with ESMTP id t22Jtmji006599 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO); Mon, 2 Mar 2015 11:55:49 -0800 (PST) (envelope-from julian@freebsd.org) Message-ID: <54F4C03F.7030704@freebsd.org> Date: Mon, 02 Mar 2015 11:55:43 -0800 From: Julian Elischer User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:31.0) Gecko/20100101 Thunderbird/31.5.0 MIME-Version: 1.0 To: Alfred Perlstein , Harrison Grundy , freebsd-current@freebsd.org Subject: Re: Massive libxo-zation that breaks everything References: <54F31510.7050607@hot.ee> <54F34B6E.2040809@astrodoggroup.com> <54F35F29.4000603@astrodoggroup.com> <54F429EF.5050400@freebsd.org> <54F46536.8040607@mu.org> In-Reply-To: <54F46536.8040607@mu.org> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Mar 2015 19:55:57 -0000 On 3/2/15 5:27 AM, Alfred Perlstein wrote: > > > On 3/2/15 4:14 AM, Julian Elischer wrote: >> On 3/1/15 10:49 AM, Harrison Grundy wrote: >>> Thanks! >>> >>> That does seem useful, but I'm not sure I see the reasoning behind >>> putting into base, over a port or package, since processing XML in >>> base >>> is a pain, and it can't serve up JSON or HTML without additional >>> utilities anyway. >>> >>> (If I'm reviving a long-settled thing, let me know and I'll drop >>> it. I'm >>> trying to understand the use case for this.) >> >> To me it would almost seem more useful to have a programmable >> filter for which you could produce >> parse grammars to parse the output of various programs.. >> thus >> >> ifconfig -a | xmlize -g ifconfig | your-favourite-xml-parser >> with a set of grammars in /usr/share/xmlize/ >> then we could use it for out-of-tree programs as well if we wrote >> grammars for them.. >> >> The sentiment of machine-readable output is nice, but I think it's >> slightly off target. >> we shouldn't have to change all out utilities, and it isn't going >> to help at all with 3rd party apps, >> e.g. samba stuff. A generally easy to program output grammar parser >> would be truely useful. >> and not just for FreeBSD. >> >> I've been watching with an uncomfortable feeling, but it's taken me >> a while to put my >> finger on what it was.. >> >> > Are you sure it's not the hairs on the back of your neck standing up > due to NIH? > > Juniper has been doing this for years and it's very useful for them. I'm not saying the ability to generate machine readable output is wrong, but that the 'unix way' would be to make a filter for it. It seems that the noisy people don't agree with me so I will not stand in the way of progress.. > > -Alfred > > From owner-freebsd-current@FreeBSD.ORG Mon Mar 2 20:01:04 2015 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7528E2BA for ; Mon, 2 Mar 2015 20:01:04 +0000 (UTC) Received: from elvis.mu.org (elvis.mu.org [IPv6:2001:470:1f05:b76::196]) by mx1.freebsd.org (Postfix) with ESMTP id 5DF84CFC for ; Mon, 2 Mar 2015 20:01:03 +0000 (UTC) Received: from AlfredMacbookAir.local (unknown [12.130.117.99]) by elvis.mu.org (Postfix) with ESMTPSA id D2BE0341F90F for ; Mon, 2 Mar 2015 12:01:02 -0800 (PST) Message-ID: <54F4C250.7090704@mu.org> Date: Mon, 02 Mar 2015 15:04:32 -0500 From: Alfred Perlstein User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:31.0) Gecko/20100101 Thunderbird/31.5.0 MIME-Version: 1.0 To: freebsd-current@freebsd.org Subject: Re: Massive libxo-zation that breaks everything References: <54F31510.7050607@hot.ee> <54F34B6E.2040809@astrodoggroup.com> <54F35F29.4000603@astrodoggroup.com> <54F36431.30506@freebsd.org> <54F42A82.1020308@freebsd.org> <54F464D1.6060603@mu.org> <54F4BFB4.1060307@freebsd.org> In-Reply-To: <54F4BFB4.1060307@freebsd.org> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Mar 2015 20:01:04 -0000 On 3/2/15 2:53 PM, Julian Elischer wrote: > On 3/2/15 5:25 AM, Alfred Perlstein wrote: >> >> >> On 3/2/15 4:25 AM, David Chisnall wrote: >>> On 2 Mar 2015, at 09:16, Julian Elischer wrote: >>>> if we develop a suitable post processor with pluggable grammars, we >>>> save a lot of work. >>>> given enough examples you could almost have automatically generated >>>> grammars. >>> This decoupled approach is problematic. A large part of the point >>> of libxo is to allow changing the human-readable output without >>> breaking tools that consume the output. Now I need to keep the tool >>> that consumes it and the tool that produces it in sync, so that's an >>> extra set of moving parts. When you throw jails with multiple >>> versions of world into the mix, it becomes a recipe for disaster. > why? the jail has it own /usr/share? > >>> >> +1 > > I think the risk is exactly opposite. That the human readable output > will change subtly with bugs in the xo implementation. > and people will not update the two output paths in exactly the same > way, leading bugs. I'm not going to fight on it, but I am > uncomfortable with it. So you mean that we're going to have to act like mature software devs and have regression tests (atf) and such? I welcome such a change. > You are increasing the complexity of every program you touch. And its utility as well. Worth it. -Alfred From owner-freebsd-current@FreeBSD.ORG Mon Mar 2 20:42:43 2015 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C7DED864; Mon, 2 Mar 2015 20:42:43 +0000 (UTC) Received: from mail-ie0-x22f.google.com (mail-ie0-x22f.google.com [IPv6:2607:f8b0:4001:c03::22f]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8F96C279; Mon, 2 Mar 2015 20:42:43 +0000 (UTC) Received: by iecrp18 with SMTP id rp18so51433623iec.1; Mon, 02 Mar 2015 12:42:43 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:date:message-id:subject:from:to:content-type; bh=yI6xxNUgyElBECOfAJQZ4NxK5e7ECW+++wODeQwcnBg=; b=PqXbZWka+owY4MXLBkDTake90Y2xhJlkzNS05Jj9V1VsrMblnJ62Z+RUc/ZN/D2sEK wA1CfQ+B9nEhU4QfOWYxb3QgIpR+t/57ZY4/NHenevrExF6tr7JfkDNWXgxcJc32NMH4 Gj+rLFIoxdAUAqwgvgHCS6QPDtlg6ZzMEErMGRQHHfsP/Vf6J5Ex0xaLp9itXUuZjsST FPcLycTgXxumgajJr2yceZU6/YRAu8OIFfK1w792v8wQ8qXpMf2mxoFz3NsQuuaqN7pR KoJqXXIja8wXGrCfuz3qCMXlW6kXYjxfwNKr1Eyj/Gd3jN9ZeVg3yk/CwrBoKb7LXSNp Kk0w== MIME-Version: 1.0 X-Received: by 10.42.130.74 with SMTP id u10mr32332176ics.61.1425328962962; Mon, 02 Mar 2015 12:42:42 -0800 (PST) Sender: adrian.chadd@gmail.com Received: by 10.36.17.66 with HTTP; Mon, 2 Mar 2015 12:42:42 -0800 (PST) Date: Mon, 2 Mar 2015 12:42:42 -0800 X-Google-Sender-Auth: MV4jVvJ7JmxzeQyAQ3XtMFfPIXk Message-ID: Subject: Doing zero-copy stuff in drivers, or "is vm_fault_quick_hold_pages() enough" ? From: Adrian Chadd To: "freebsd-arch@freebsd.org" , freebsd-current Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Mar 2015 20:42:43 -0000 Hi, gonzo@ committed a fix (r278615) to the videocore driver for the raspberry pi. The fix involved doing an explicit wire of pages that were about to be passed down to the hardware to send to the videobuffer hardware. It turns out that doing vm_fault_quick_hold_pages() wasn't enough - the pages weren't being wired, and they may disappear before the hardware gets to them. I looked at vmapbuf() and how it uses vm_fault_quick_hold_pages(), but I can't find anything that wires the pages down before it hands the addresses to the hardware. So, am I missing something about how/where that's done? Thanks, -adrian From owner-freebsd-current@FreeBSD.ORG Mon Mar 2 20:03:14 2015 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3019B4A9 for ; Mon, 2 Mar 2015 20:03:14 +0000 (UTC) Received: from mailout.easymail.ca (mailout.easymail.ca [64.68.201.169]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id BD544DB7 for ; Mon, 2 Mar 2015 20:03:13 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by mailout.easymail.ca (Postfix) with ESMTP id 0D401E3C2 for ; Mon, 2 Mar 2015 15:03:05 -0500 (EST) X-Virus-Scanned: Debian amavisd-new at mailout.easymail.ca X-Spam-Flag: NO X-Spam-Score: -3.85 X-Spam-Level: X-Spam-Status: No, score=-3.85 required=5 tests=[ALL_TRUSTED=-1.8, AWL=-0.143, BAYES_00=-2.599, DNS_FROM_AHBL_RHSBL=0.692] Received: from mailout.easymail.ca ([127.0.0.1]) by localhost (easymail-mailout.easydns.vpn [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id KDmgwxjd1h4V for ; Mon, 2 Mar 2015 15:03:04 -0500 (EST) Received: from bsddt1241.lv01.astrodoggroup.com (unknown [40.141.24.126]) (using TLSv1 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by mailout.easymail.ca (Postfix) with ESMTPSA id A30C0E37C for ; Mon, 2 Mar 2015 15:03:04 -0500 (EST) Message-ID: <54F4C127.5020000@astrodoggroup.com> Date: Mon, 02 Mar 2015 11:59:35 -0800 From: Harrison Grundy User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:31.0) Gecko/20100101 Thunderbird/31.3.0 MIME-Version: 1.0 To: freebsd-current@freebsd.org Subject: Re: Massive libxo-zation that breaks everything References: <54F31510.7050607@hot.ee> <54F34B6E.2040809@astrodoggroup.com> <54F35F29.4000603@astrodoggroup.com> <54F429EF.5050400@freebsd.org> <54F46536.8040607@mu.org> <54F4C03F.7030704@freebsd.org> In-Reply-To: <54F4C03F.7030704@freebsd.org> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-Mailman-Approved-At: Mon, 02 Mar 2015 21:08:30 +0000 X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Mar 2015 20:03:14 -0000 On 03/02/15 11:55, Julian Elischer wrote: > On 3/2/15 5:27 AM, Alfred Perlstein wrote: >> >> >> On 3/2/15 4:14 AM, Julian Elischer wrote: >>> On 3/1/15 10:49 AM, Harrison Grundy wrote: >>>> Thanks! >>>> >>>> That does seem useful, but I'm not sure I see the reasoning behind >>>> putting into base, over a port or package, since processing XML in base >>>> is a pain, and it can't serve up JSON or HTML without additional >>>> utilities anyway. >>>> >>>> (If I'm reviving a long-settled thing, let me know and I'll drop it. >>>> I'm >>>> trying to understand the use case for this.) >>> >>> To me it would almost seem more useful to have a programmable filter >>> for which you could produce >>> parse grammars to parse the output of various programs.. >>> thus >>> >>> ifconfig -a | xmlize -g ifconfig | your-favourite-xml-parser >>> with a set of grammars in /usr/share/xmlize/ >>> then we could use it for out-of-tree programs as well if we wrote >>> grammars for them.. >>> >>> The sentiment of machine-readable output is nice, but I think it's >>> slightly off target. >>> we shouldn't have to change all out utilities, and it isn't going to >>> help at all with 3rd party apps, >>> e.g. samba stuff. A generally easy to program output grammar parser >>> would be truely useful. >>> and not just for FreeBSD. >>> >>> I've been watching with an uncomfortable feeling, but it's taken me a >>> while to put my >>> finger on what it was.. >>> >>> >> Are you sure it's not the hairs on the back of your neck standing up >> due to NIH? >> >> Juniper has been doing this for years and it's very useful for them. > I'm not saying the ability to generate machine readable output is wrong, > but that the 'unix way' would be to make a filter for it. It seems that > the noisy people don't > agree with me so I will not stand in the way of progress.. > Julian and Alfred, Does the "lib-ification" idea address your differing concerns? The idea is to split base binaries into a library that does the "work", and a binary that does the UI bits. This way, "df" becomes a CLI interface connected to "libdf" that handles the CLI output. "xodf" (or whatever we'd like to name it) becomes a libxo interface on top of libdf. This allows the CLI code to remain largely unchanged, and permits the XML output code to link across multiple utilities at the same time. (So that things like, say, ifconfig information can be incorporated into route and netstat results by linking to "libifconfig" "libroute" and "libnetstat", respectively. If we're going to have to touch every base binary anyway, it seems like this might be a useful abstraction to get at the same time. --- Harrison From owner-freebsd-current@FreeBSD.ORG Mon Mar 2 21:34:38 2015 Return-Path: Delivered-To: current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id BCC0DB0D; Mon, 2 Mar 2015 21:34:38 +0000 (UTC) Received: from clavin1.langille.org (clavin.langille.org [162.208.116.86]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "clavin.langille.org", Issuer "StartCom Class 2 Primary Intermediate Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8A5BFAA5; Mon, 2 Mar 2015 21:34:37 +0000 (UTC) Received: from (clavin1.int.langille.org (clavin1.int.unixathome.org [10.4.7.7]) (Authenticated sender: hidden) with ESMTPSA id 90CA81B3 ; Mon, 2 Mar 2015 21:34:35 +0000 (UTC) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 8.2 \(2070.6\)) Subject: Re: sa(4) driver changes available for test From: Dan Langille In-Reply-To: Date: Mon, 2 Mar 2015 16:34:34 -0500 Content-Transfer-Encoding: quoted-printable Message-Id: <95BCE742-8CC4-41FC-9FDD-D12C7A337951@langille.org> References: <20150214003232.GA63990@mithlond.kdm.org> <7CA52DF3-E073-4F50-BE4E-01C51CCDF2C7@langille.org> <20150217183645.GA30947@mithlond.kdm.org> <303C48AC-0DA8-447C-8DE7-0A1DD5DD7351@langille.org> <20150302003150.GB71528@mithlond.kdm.org> <20150302022957.GB73433@mithlond.kdm.org> <0F17CDDB-4CE9-4881-B948-6B9BEDEB6455@langille.org> <20150302163146.GA85515@mithlond.kdm.org> <85E2DDA0-4A2B-4316-A318-3706B3A05C11@langille.org> <20150302172846.GB87055@mithlond.kdm.org> <12EED94D-CC9D-44C4-91C2-62C8A51B839B@langille.org> To: "Kenneth D. Merry" X-Mailer: Apple Mail (2.2070.6) Cc: current@freebsd.org, scsi@freebsd.org X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Mar 2015 21:34:38 -0000 > On Mar 2, 2015, at 2:47 PM, Dan Langille wrote: >=20 >=20 >> On Mar 2, 2015, at 2:07 PM, Dan Langille wrote: >>=20 >>=20 >>> On Mar 2, 2015, at 12:28 PM, Kenneth D. Merry = wrote: >>>=20 >>> On Mon, Mar 02, 2015 at 11:44:09 -0500, Dan Langille wrote: >>>>=20 >>>>> On Mar 2, 2015, at 11:31 AM, Kenneth D. Merry = wrote: >>>>>=20 >>>>> On Mon, Mar 02, 2015 at 11:09:57 -0500, Dan Langille wrote: >>>>>>=20 >>>>>>> On Mar 1, 2015, at 9:29 PM, Kenneth D. Merry = wrote: >>>>>>>=20 >>>>>>> On Sun, Mar 01, 2015 at 19:41:07 -0500, Dan Langille wrote: >>>>>>>>=20 >>>>>>>>> On Mar 1, 2015, at 7:31 PM, Kenneth D. Merry = wrote: >>>>>>>>>=20 >>>>>>>>> On Sun, Mar 01, 2015 at 19:15:05 -0500, Dan Langille wrote: >>>>>>>>>>=20 >>>>>>>>>>> On Feb 17, 2015, at 1:36 PM, Kenneth D. Merry = wrote: >>>>>>>>>>>=20 >>>>>>>>>>> On Sat, Feb 14, 2015 at 18:22:43 -0500, Dan Langille wrote: >>>>>>>>>>>>=20 >>>>>>>>>>>>> On Feb 13, 2015, at 7:32 PM, Kenneth D. Merry = wrote: >>>>>>>>>>>>>=20 >>>>>>>>>>>>>=20 >>>>>>>>>>>>> I have a fairly large set of changes to the sa(4) driver = and mt(1) driver >>>>>>>>>>>>> that I'm planning to commit in the near future. >>>>>>>>>>>>>=20 >>>>>>>>>>>>> A description of the changes is here and below in this = message. >>>>>>>>>>>>>=20 >>>>>>>>>>>>> If you have tape hardware and the inclination, I'd = appreciate testing and >>>>>>>>>>>>> feedback. >>>>>>>>>>>>=20 >>>>>>>>>>>> I have a DLT 8000 and an SDLT 220. >>>>>>>>>>>>=20 >>>>>>>>>>>> I don't have anything running current, but I have a spare = machine which I could use for testing. >>>>>>>>>>>>=20 >>>>>>>>>>>> Do you see any value is tests with that hardware? I'd be = testing it via Bacula. >>>>>>>>>>>>=20 >>>>>>>>>>>> disclosure: I'm the sysutils/bacula-* maintainer and a = Bacula committer. >>>>>>>>>>>>=20 >>>>>>>>>>>=20 >>>>>>>>>>> Actually, yes. Bacula is a bit tricky to configure, so your = trying it out >>>>>>>>>>> would be helpful if you have the time. >>>>>>>>>>>=20 >>>>>>>>>>> In looking at the manuals for both the SDLT 220 and the DLT = 8000, they both >>>>>>>>>>> claim to support long position information for the SCSI READ = POSITION >>>>>>>>>>> command. >>>>>>>>>>>=20 >>>>>>>>>>> You can see what I'm talking about by doing: >>>>>>>>>>>=20 >>>>>>>>>>> mt eod >>>>>>>>>>> mt status >>>>>>>>>>>=20 >>>>>>>>>>> On my DDS-4 tape drive, this shows: >>>>>>>>>>>=20 >>>>>>>>>>> # mt -f /dev/nsa3 status >>>>>>>>>>> Drive: sa3: Serial Number: = HJ00YWY >>>>>>>>>>> --------------------------------- >>>>>>>>>>> Mode Density Blocksize bpi = Compression >>>>>>>>>>> Current: 0x26:DDS-4 1024 bytes 97000 = enabled (DCLZ) >>>>>>>>>>> --------------------------------- >>>>>>>>>>> Current Driver State: at rest. >>>>>>>>>>> --------------------------------- >>>>>>>>>>> Partition: 0 Calc File Number: -1 Calc Record = Number: -1 >>>>>>>>>>> Residual: 0 Reported File Number: -1 Reported Record = Number: -1 >>>>>>>>>>> Flags: None >>>>>>>>>>>=20 >>>>>>>>>>> But on an LTO-5, which will give long position information, = I get: >>>>>>>>>>>=20 >>>>>>>>>>> [root@doc ~]# mt status >>>>>>>>>>> Drive: sa0: >>>>>>>>>>> --------------------------------- >>>>>>>>>>> Mode Density Blocksize bpi = Compression >>>>>>>>>>> Current: 0x58:LTO-5 variable 384607 = enabled (0x1) >>>>>>>>>>> --------------------------------- >>>>>>>>>>> Current Driver State: at rest. >>>>>>>>>>> --------------------------------- >>>>>>>>>>> Partition: 0 Calc File Number: 2 Calc Record = Number: -1 >>>>>>>>>>> Residual: 0 Reported File Number: 2 Reported Record = Number: 32373 >>>>>>>>>>> Flags: None >>>>>>>>>>>=20 >>>>>>>>>>> That, in combination with the changes I made to the position = information >>>>>>>>>>> code in the driver, mean that even the old MTIOCGET ioctl = should return an >>>>>>>>>>> accurate file number at end of data. e.g., on the LTO-5: >>>>>>>>>>>=20 >>>>>>>>>>> [root@doc ~]# mt ostatus >>>>>>>>>>> Mode Density Blocksize bpi = Compression >>>>>>>>>>> Current: 0x58:LTO-5 variable 384607 0x1 >>>>>>>>>>> ---------available modes--------- >>>>>>>>>>> 0: 0x58:LTO-5 variable 384607 0x1 >>>>>>>>>>> 1: 0x58:LTO-5 variable 384607 0x1 >>>>>>>>>>> 2: 0x58:LTO-5 variable 384607 0x1 >>>>>>>>>>> 3: 0x58:LTO-5 variable 384607 0x1 >>>>>>>>>>> --------------------------------- >>>>>>>>>>> Current Driver State: at rest. >>>>>>>>>>> --------------------------------- >>>>>>>>>>> File Number: 2 Record Number: -1 Residual Count -1 >>>>>>>>>>>=20 >>>>>>>>>>> So the thing to try, in addition to just making sure that = Bacula continues >>>>>>>>>>> to work properly, is to try setting this for the tape drive = in >>>>>>>>>>> bacula-sd.conf: >>>>>>>>>>>=20 >>>>>>>>>>> Hardware End of Medium =3D yes >>>>>>>>>>>=20 >>>>>>>>>>> It looks like the Bacula tape program (btape) has a test = mode, and it would >>>>>>>>>>> be good to run through the tests on one of the tape drives = and see whether >>>>>>>>>>> they work, and whether the results are different before and = after the >>>>>>>>>>> changes. I'm not sure how to enable the test mode. >>>>>>>>>>=20 >>>>>>>>>> I have this in /usr/local/etc/bacula/bacula-sd.conf >>>>>>>>>>=20 >>>>>>>>>> Device { >>>>>>>>>> Name =3D DLT >>>>>>>>>> Description =3D "QUANTUM DLT7000 1624" >>>>>>>>>> Media Type =3D DLT >>>>>>>>>> Archive Device =3D /dev/nsa1 >>>>>>>>>>=20 >>>>>>>>>> Autochanger =3D YES >>>>>>>>>> Drive Index =3D 0 >>>>>>>>>>=20 >>>>>>>>>> Offline On Unmount =3D no >>>>>>>>>> Hardware End of Medium =3D yes >>>>>>>>>> BSF at EOM =3D yes >>>>>>>>>> Backward Space Record =3D no >>>>>>>>>> Fast Forward Space File =3D no >>>>>>>>>> TWO EOF =3D yes >>>>>>>>>> } >>>>>>>>>>=20 >>>>>>>>>> FYI, http://www.freebsddiary.org/digital-tl891.php (from = 2006) has a btape test on this same model. >>>>>>>>>>=20 >>>>>>>>>> Here's the test I ran tonight: >>>>>>>>>>=20 >>>>>>>>>> [root@cuppy:/usr/home/dan] # btape -c = /usr/local/etc/bacula/bacula-sd.conf /dev/nsa1 = =20 >>>>>>>>>> Tape block granularity is 1024 bytes. >>>>>>>>>> btape: butil.c:287-0 Using device: "/dev/nsa1" for writing. >>>>>>>>>> btape: btape.c:469-0 open device "DLT" (/dev/nsa1): OK >>>>>>>>>> *test >>>>>>>>>>=20 >>>>>>>>>> =3D=3D=3D Write, rewind, and re-read test =3D=3D=3D >>>>>>>>>>=20 >>>>>>>>>> I'm going to write 10000 records and an EOF >>>>>>>>>> then write 10000 records and an EOF, then rewind, >>>>>>>>>> and re-read the data to verify that it is correct. >>>>>>>>>>=20 >>>>>>>>>> This is an *essential* feature ... >>>>>>>>>>=20 >>>>>>>>>> btape: btape.c:1152-0 Wrote 10000 blocks of 64412 bytes. >>>>>>>>>> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) >>>>>>>>>> btape: btape.c:1168-0 Wrote 10000 blocks of 64412 bytes. >>>>>>>>>> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) >>>>>>>>>> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) >>>>>>>>>> btape: btape.c:1210-0 Rewind OK. >>>>>>>>>> 10000 blocks re-read correctly. >>>>>>>>>> Got EOF on tape. >>>>>>>>>> 10000 blocks re-read correctly. >>>>>>>>>> =3D=3D=3D Test Succeeded. End Write, rewind, and re-read test = =3D=3D=3D >>>>>>>>>>=20 >>>>>>>>>> btape: btape.c:1277-0 Block position test >>>>>>>>>> btape: btape.c:1289-0 Rewind OK. >>>>>>>>>> Reposition to file:block 0:4 >>>>>>>>>> Block 5 re-read correctly. >>>>>>>>>> Reposition to file:block 0:200 >>>>>>>>>> Block 201 re-read correctly. >>>>>>>>>> Reposition to file:block 0:9999 >>>>>>>>>> Block 10000 re-read correctly. >>>>>>>>>> Reposition to file:block 1:0 >>>>>>>>>> Block 10001 re-read correctly. >>>>>>>>>> Reposition to file:block 1:600 >>>>>>>>>> Block 10601 re-read correctly. >>>>>>>>>> Reposition to file:block 1:9999 >>>>>>>>>> Block 20000 re-read correctly. >>>>>>>>>> =3D=3D=3D Test Succeeded. End Write, rewind, and re-read test = =3D=3D=3D >>>>>>>>>>=20 >>>>>>>>>>=20 >>>>>>>>>>=20 >>>>>>>>>> =3D=3D=3D Append files test =3D=3D=3D >>>>>>>>>>=20 >>>>>>>>>> This test is essential to Bacula. >>>>>>>>>>=20 >>>>>>>>>> I'm going to write one record in file 0, >>>>>>>>>> two records in file 1, >>>>>>>>>> and three records in file 2 >>>>>>>>>>=20 >>>>>>>>>> btape: btape.c:574-0 Rewound "DLT" (/dev/nsa1) >>>>>>>>>> btape: btape.c:1907-0 Wrote one record of 64412 bytes. >>>>>>>>>> btape: btape.c:1909-0 Wrote block to device. >>>>>>>>>> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) >>>>>>>>>> btape: btape.c:1907-0 Wrote one record of 64412 bytes. >>>>>>>>>> btape: btape.c:1909-0 Wrote block to device. >>>>>>>>>> btape: btape.c:1907-0 Wrote one record of 64412 bytes. >>>>>>>>>> btape: btape.c:1909-0 Wrote block to device. >>>>>>>>>> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) >>>>>>>>>> btape: btape.c:1907-0 Wrote one record of 64412 bytes. >>>>>>>>>> btape: btape.c:1909-0 Wrote block to device. >>>>>>>>>> btape: btape.c:1907-0 Wrote one record of 64412 bytes. >>>>>>>>>> btape: btape.c:1909-0 Wrote block to device. >>>>>>>>>> btape: btape.c:1907-0 Wrote one record of 64412 bytes. >>>>>>>>>> btape: btape.c:1909-0 Wrote block to device. >>>>>>>>>> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) >>>>>>>>>> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) >>>>>>>>>> btape: btape.c:469-0 open device "DLT" (/dev/nsa1): OK >>>>>>>>>> btape: btape.c:574-0 Rewound "DLT" (/dev/nsa1) >>>>>>>>>> btape: btape.c:1420-0 Now moving to end of medium. >>>>>>>>>=20 >>>>>>>>> This is the critical piece. The test moves the tape to the = end of the >>>>>>>>> medium. With hardware position information, you can tell what = filemark >>>>>>>>> you're on. Without it, you can't. >>>>>>>>>=20 >>>>>>>>>> btape: btape.c:622-0 tape_dev.c:345 ioctl MTIOCGET error on = "DLT" (/dev/nsa1). ERR=3DNo error: 0. >>>>>>>>>> We should be in file 3. I am at file 0. This is NOT = correct!!!! >>>>>>>>>>=20 >>>>>>>>>> Append test failed. Attempting again. >>>>>>>>>> Setting "Hardware End of Medium =3D no >>>>>>>>>> and "Fast Forward Space File =3D no >>>>>>>>>> and retrying append test. >>>>>>>>>=20 >>>>>>>>> This is not surprsing, given that the drive doesn't support = long read >>>>>>>>> position data. (It's a SCSI-2 device.) So that means that = Bacula will >>>>>>>>> need to do it manually. >>>>>>>>=20 >>>>>>>> Yes, I have nothing newer than SCSI-2. Even my SDLT is SCSI-2 = but that >>>>>>>> tape library is hooked up to a different computer and was doing = backups today. >>>>>>>=20 >>>>>>> So, here is one thing that we can try to see whether these = drives support >>>>>>> long position information, even though they only claim to be = SCSI-2. If >>>>>>> they do, we can potentially add a quirk (or autodetection) to = enable it. >>>>>>> The code currently doesn't bother asking drives that claim to be = SCSI-2 >>>>>>> for long position information. (Because that feature was added = in the >>>>>>> SSC spec, which came after SCSI-2.) >>>>>>>=20 >>>>>>> Issue a READ POSITION with the short form specified: >>>>>>>=20 >>>>>>> camcontrol cmd sa1 -v -c "34 0 0 0 0 0 0 0 0 0" -i 20 - |hd >>>>>>>=20 >>>>>>> Issue a READ POSITION with the vendor-specific block numbers: >>>>>>>=20 >>>>>>> camcontrol cmd sa1 -v -c "34 1 0 0 0 0 0 0 0 0" -i 20 - |hd >>>>>>>=20 >>>>>>> Issue a READ POSITION with the long form data: >>>>>>>=20 >>>>>>> camcontrol cmd sa1 -v -c "34 6 0 0 0 0 0 0 0 0" -i 32 - |hd >>>>>>>=20 >>>>>>> If it supports the last one, then I can put a quirk (or = autodetection) in >>>>>>> the driver and Bacula will get the hardware filemarks. You = should try this >>>>>>> on your SDLT as well. It may well support it. >>>>>>=20 >>>>>> Sadly, no: >>>>>>=20 >>>>>> [root@cuppy:~] # camcontrol cmd sa1 -v -c "34 0 0 0 0 0 0 0 0 0" = -i 20 - |hd >>>>>> 00000000 80 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 = |................| >>>>>> 00000010 00 00 00 00 = |....| >>>>>> 00000014 >>>>>> [root@cuppy:~] # camcontrol cmd sa1 -v -c "34 1 0 0 0 0 0 0 0 0" = -i 20 - |hd >>>>>> 00000000 80 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 = |................| >>>>>> 00000010 00 00 00 00 = |....| >>>>>> 00000014 >>>>>> [root@cuppy:~] # camcontrol cmd sa1 -v -c "34 6 0 0 0 0 0 0 0 0" = -i 32 - |hd >>>>>> camcontrol: error sending command >>>>>> (pass2:ahc0:0:2:0): READ POSITION. CDB: 34 06 00 00 00 00 00 00 = 00 00=20 >>>>>> (pass2:ahc0:0:2:0): CAM status: SCSI Status Error >>>>>> (pass2:ahc0:0:2:0): SCSI status: Check Condition >>>>>> (pass2:ahc0:0:2:0): SCSI sense: ILLEGAL REQUEST asc:24,0 (Invalid = field in CDB) >>>>>> (pass2:ahc0:0:2:0): Command byte 1 bit 2 is invalid >>>>>> [root@cuppy:~] #=20 >>>>>=20 >>>>> Okay. Not too surprising I suppose. >>=20 >>=20 >> Does this mean much to you? >>=20 >> Mar 2 19:42:59 cuppy kernel: (ahc0:A:2:0): Sending SDTR period 19, = offset f >> Mar 2 19:42:59 cuppy kernel: (ahc0:A:2:0): Received SDTR period 19, = offset f >> Mar 2 19:42:59 cuppy kernel: Filtered to period 19, offset f >> Mar 2 19:42:59 cuppy kernel: (ahc0:A:2:0): Sending SDTR period 19, = offset f >> Mar 2 19:42:59 cuppy kernel: (ahc0:A:2:0): Received SDTR period 19, = offset f >> Mar 2 19:42:59 cuppy kernel: Filtered to period 19, offset f >> Mar 2 19:42:59 cuppy kernel: (ahc0:A:2:0): Sending SDTR period 19, = offset f >> Mar 2 19:42:59 cuppy kernel: (ahc0:A:2:0): Received SDTR period 19, = offset f >> Mar 2 19:42:59 cuppy kernel: Filtered to period 19, offset f >>=20 >>=20 >> I'm having trouble with labeling barcodes from Bacula and the above = is seen in /var/log/messages >=20 > The barcodes issue is resolved. I changed to using drive 0 instead of = drive 1, now that we have both drives. >=20 > *label barcodes slots=3D3,4 > The defined Storage resources are: > 1: File1 > 2: OldTL891 > Select Storage resource (1-2): 2 > Connecting to Storage daemon OldTL891 at 10.55.0.18:9103 ... > 3306 Issuing autochanger "slots" command. > Device "DTL03" has 10 slots. > Connecting to Storage daemon OldTL891 at 10.55.0.18:9103 ... > 3306 Issuing autochanger "list" command. > The following Volumes will be labeled: > Slot Volume > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > 3 FAI261 > 4 FAI262 > Do you want to label these Volumes? (yes|no): yes > Defined Pools: > 1: Default > 2: File > 3: MYDLT > 4: Scratch > Select the Pool (1-4): 4 > Connecting to Storage daemon OldTL891 at 10.55.0.18:9103 ... > Sending label command for Volume "FAI261" Slot 3 ... > 3000 OK label. VolBytes=3D64512 DVD=3D0 Volume=3D"FAI261" = Device=3D"DTL03" (/dev/nsa0) > Catalog record for Volume "FAI261", Slot 3 successfully created. > Sending label command for Volume "FAI262" Slot 4 ... > 3307 Issuing autochanger "unload slot 3, drive 0" command. > 3304 Issuing autochanger "load slot 4, drive 0" command. > 3305 Autochanger "load slot 4, drive 0", status is OK. > 3000 OK label. VolBytes=3D64512 DVD=3D0 Volume=3D"FAI262" = Device=3D"DTL03" (/dev/nsa0) > Catalog record for Volume "FAI262", Slot 4 successfully created. > *list volumes > Pool: Default > No results to list. > Pool: File > = +---------+------------+-----------+---------+------------+----------+----= ----------+---------+------+-----------+-----------+---------------------+= > | mediaid | volumename | volstatus | enabled | volbytes | volfiles | = volretention | recycle | slot | inchanger | mediatype | lastwritten = | > = +---------+------------+-----------+---------+------------+----------+----= ----------+---------+------+-----------+-----------+---------------------+= > | 1 | Vol-0001 | Append | 1 | 19,835,982 | 0 | = 31,536,000 | 1 | 0 | 0 | File1 | 2015-03-02 = 17:50:40 | > | 2 | Vol-0002 | Append | 1 | 0 | 0 | = 31,536,000 | 1 | 0 | 0 | DLT | = | > = +---------+------------+-----------+---------+------------+----------+----= ----------+---------+------+-----------+-----------+---------------------+= > Pool: MYDLT > No results to list. > Pool: Scratch > = +---------+------------+-----------+---------+----------+----------+------= --------+---------+------+-----------+-----------+-------------+ > | mediaid | volumename | volstatus | enabled | volbytes | volfiles | = volretention | recycle | slot | inchanger | mediatype | lastwritten | > = +---------+------------+-----------+---------+----------+----------+------= --------+---------+------+-----------+-----------+-------------+ > | 4 | FAI260 | Append | 1 | 64,512 | 0 | = 31,536,000 | 1 | 1 | 1 | DLT | | > | 5 | FAI263 | Append | 1 | 64,512 | 0 | = 31,536,000 | 1 | 2 | 1 | DLT | | > | 7 | FAI261 | Append | 1 | 64,512 | 0 | = 31,536,000 | 1 | 3 | 1 | DLT | | > | 8 | FAI262 | Append | 1 | 64,512 | 0 | = 31,536,000 | 1 | 4 | 1 | DLT | | > = +---------+------------+-----------+---------+----------+----------+------= --------+---------+------+-----------+-----------+-------------+ > * >=20 > But this is what arrives in /var/log/messages during the above: >=20 > Mar 2 20:29:06 cuppy kernel: (ahc0:A:1:0): Sending SDTR period 19, = offset f > Mar 2 20:29:06 cuppy kernel: (ahc0:A:1:0): Received SDTR period 19, = offset f > Mar 2 20:29:06 cuppy kernel: Filtered to period 19, offset f > Mar 2 20:30:22 cuppy kernel: (ahc0:A:1:0): Sending SDTR period 19, = offset f > Mar 2 20:30:22 cuppy kernel: (ahc0:A:1:0): Received SDTR period 19, = offset f > Mar 2 20:30:22 cuppy kernel: Filtered to period 19, offset f > Mar 2 20:30:22 cuppy kernel: (ahc0:A:1:0): Sending SDTR period 19, = offset f > Mar 2 20:30:22 cuppy kernel: (ahc0:A:1:0): Received SDTR period 19, = offset f > Mar 2 20:30:22 cuppy kernel: Filtered to period 19, offset f > Mar 2 20:30:59 cuppy kernel: (ahc0:A:1:0): Sending SDTR period 19, = offset f > Mar 2 20:30:59 cuppy kernel: (ahc0:A:1:0): Received SDTR period 19, = offset f > Mar 2 20:30:59 cuppy kernel: Filtered to period 19, offset f > Mar 2 20:30:59 cuppy kernel: xpt_release_devq(): requested 1 > = present 0 > Mar 2 20:31:07 cuppy kernel: (ahc0:A:1:0): Sending SDTR period 19, = offset f > Mar 2 20:31:07 cuppy kernel: (ahc0:A:1:0): Received SDTR period 19, = offset f > Mar 2 20:31:07 cuppy kernel: Filtered to period 19, offset f >=20 > Anything to be concerned about? When adding a 2nd job to a tape: 02-Mar 21:29 cuppy-sd JobId 19: Error: Unable to position to end of data = on tape device "DTL03" (/dev/nsa0): ERR=3Dtape_dev.c:345 ioctl MTIOCGET = error on "DTL03" (/dev/nsa0). ERR=3DNo error: 0. I've gotten this on three consecutive tapes. NOTE: These *are* tapes I = no longer use because of higher rates of corrected errors. The problem seems to be locating the end of the data. I can run a 4 or 5 jobs in a row, but if I restore a job, then add a = job, it fails with the above message. =E2=80=94=20 Dan Langille http://langille.org/ From owner-freebsd-current@FreeBSD.ORG Mon Mar 2 21:42:44 2015 Return-Path: Delivered-To: current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 98DDF57F; Mon, 2 Mar 2015 21:42:44 +0000 (UTC) Received: from mithlond.kdm.org (mithlond.kdm.org [70.56.43.85]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "A1-33714", Issuer "A1-33714" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 40D9CC15; Mon, 2 Mar 2015 21:42:44 +0000 (UTC) Received: from mithlond.kdm.org (localhost [127.0.0.1]) by mithlond.kdm.org (8.14.9/8.14.9) with ESMTP id t22LgeEk091238 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Mon, 2 Mar 2015 14:42:40 -0700 (MST) (envelope-from ken@mithlond.kdm.org) Received: (from ken@localhost) by mithlond.kdm.org (8.14.9/8.14.9/Submit) id t22LgehR091237; Mon, 2 Mar 2015 14:42:40 -0700 (MST) (envelope-from ken) Date: Mon, 2 Mar 2015 14:42:40 -0700 From: "Kenneth D. Merry" To: Dan Langille Subject: Re: sa(4) driver changes available for test Message-ID: <20150302214240.GA91133@mithlond.kdm.org> References: <20150302003150.GB71528@mithlond.kdm.org> <20150302022957.GB73433@mithlond.kdm.org> <0F17CDDB-4CE9-4881-B948-6B9BEDEB6455@langille.org> <20150302163146.GA85515@mithlond.kdm.org> <85E2DDA0-4A2B-4316-A318-3706B3A05C11@langille.org> <20150302172846.GB87055@mithlond.kdm.org> <12EED94D-CC9D-44C4-91C2-62C8A51B839B@langille.org> <95BCE742-8CC4-41FC-9FDD-D12C7A337951@langille.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <95BCE742-8CC4-41FC-9FDD-D12C7A337951@langille.org> User-Agent: Mutt/1.5.23 (2014-03-12) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (mithlond.kdm.org [127.0.0.1]); Mon, 02 Mar 2015 14:42:40 -0700 (MST) X-Spam-Status: No, score=-2.9 required=5.0 tests=ALL_TRUSTED,BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on mithlond.kdm.org Cc: current@freebsd.org, scsi@freebsd.org X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Mar 2015 21:42:44 -0000 On Mon, Mar 02, 2015 at 16:34:34 -0500, Dan Langille wrote: > > > On Mar 2, 2015, at 2:47 PM, Dan Langille wrote: > > > > > >> On Mar 2, 2015, at 2:07 PM, Dan Langille wrote: > >> > >> > >>> On Mar 2, 2015, at 12:28 PM, Kenneth D. Merry wrote: > >>> > >>> On Mon, Mar 02, 2015 at 11:44:09 -0500, Dan Langille wrote: > >>>> > >>>>> On Mar 2, 2015, at 11:31 AM, Kenneth D. Merry wrote: > >>>>> > >>>>> On Mon, Mar 02, 2015 at 11:09:57 -0500, Dan Langille wrote: > >>>>>> > >>>>>>> On Mar 1, 2015, at 9:29 PM, Kenneth D. Merry wrote: > >>>>>>> > >>>>>>> On Sun, Mar 01, 2015 at 19:41:07 -0500, Dan Langille wrote: > >>>>>>>> > >>>>>>>>> On Mar 1, 2015, at 7:31 PM, Kenneth D. Merry wrote: > >>>>>>>>> > >>>>>>>>> On Sun, Mar 01, 2015 at 19:15:05 -0500, Dan Langille wrote: > >>>>>>>>>> > >>>>>>>>>>> On Feb 17, 2015, at 1:36 PM, Kenneth D. Merry wrote: > >>>>>>>>>>> > >>>>>>>>>>> On Sat, Feb 14, 2015 at 18:22:43 -0500, Dan Langille wrote: > >>>>>>>>>>>> > >>>>>>>>>>>>> On Feb 13, 2015, at 7:32 PM, Kenneth D. Merry wrote: > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>> I have a fairly large set of changes to the sa(4) driver and mt(1) driver > >>>>>>>>>>>>> that I'm planning to commit in the near future. > >>>>>>>>>>>>> > >>>>>>>>>>>>> A description of the changes is here and below in this message. > >>>>>>>>>>>>> > >>>>>>>>>>>>> If you have tape hardware and the inclination, I'd appreciate testing and > >>>>>>>>>>>>> feedback. > >>>>>>>>>>>> > >>>>>>>>>>>> I have a DLT 8000 and an SDLT 220. > >>>>>>>>>>>> > >>>>>>>>>>>> I don't have anything running current, but I have a spare machine which I could use for testing. > >>>>>>>>>>>> > >>>>>>>>>>>> Do you see any value is tests with that hardware? I'd be testing it via Bacula. > >>>>>>>>>>>> > >>>>>>>>>>>> disclosure: I'm the sysutils/bacula-* maintainer and a Bacula committer. > >>>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> Actually, yes. Bacula is a bit tricky to configure, so your trying it out > >>>>>>>>>>> would be helpful if you have the time. > >>>>>>>>>>> > >>>>>>>>>>> In looking at the manuals for both the SDLT 220 and the DLT 8000, they both > >>>>>>>>>>> claim to support long position information for the SCSI READ POSITION > >>>>>>>>>>> command. > >>>>>>>>>>> > >>>>>>>>>>> You can see what I'm talking about by doing: > >>>>>>>>>>> > >>>>>>>>>>> mt eod > >>>>>>>>>>> mt status > >>>>>>>>>>> > >>>>>>>>>>> On my DDS-4 tape drive, this shows: > >>>>>>>>>>> > >>>>>>>>>>> # mt -f /dev/nsa3 status > >>>>>>>>>>> Drive: sa3: Serial Number: HJ00YWY > >>>>>>>>>>> --------------------------------- > >>>>>>>>>>> Mode Density Blocksize bpi Compression > >>>>>>>>>>> Current: 0x26:DDS-4 1024 bytes 97000 enabled (DCLZ) > >>>>>>>>>>> --------------------------------- > >>>>>>>>>>> Current Driver State: at rest. > >>>>>>>>>>> --------------------------------- > >>>>>>>>>>> Partition: 0 Calc File Number: -1 Calc Record Number: -1 > >>>>>>>>>>> Residual: 0 Reported File Number: -1 Reported Record Number: -1 > >>>>>>>>>>> Flags: None > >>>>>>>>>>> > >>>>>>>>>>> But on an LTO-5, which will give long position information, I get: > >>>>>>>>>>> > >>>>>>>>>>> [root@doc ~]# mt status > >>>>>>>>>>> Drive: sa0: > >>>>>>>>>>> --------------------------------- > >>>>>>>>>>> Mode Density Blocksize bpi Compression > >>>>>>>>>>> Current: 0x58:LTO-5 variable 384607 enabled (0x1) > >>>>>>>>>>> --------------------------------- > >>>>>>>>>>> Current Driver State: at rest. > >>>>>>>>>>> --------------------------------- > >>>>>>>>>>> Partition: 0 Calc File Number: 2 Calc Record Number: -1 > >>>>>>>>>>> Residual: 0 Reported File Number: 2 Reported Record Number: 32373 > >>>>>>>>>>> Flags: None > >>>>>>>>>>> > >>>>>>>>>>> That, in combination with the changes I made to the position information > >>>>>>>>>>> code in the driver, mean that even the old MTIOCGET ioctl should return an > >>>>>>>>>>> accurate file number at end of data. e.g., on the LTO-5: > >>>>>>>>>>> > >>>>>>>>>>> [root@doc ~]# mt ostatus > >>>>>>>>>>> Mode Density Blocksize bpi Compression > >>>>>>>>>>> Current: 0x58:LTO-5 variable 384607 0x1 > >>>>>>>>>>> ---------available modes--------- > >>>>>>>>>>> 0: 0x58:LTO-5 variable 384607 0x1 > >>>>>>>>>>> 1: 0x58:LTO-5 variable 384607 0x1 > >>>>>>>>>>> 2: 0x58:LTO-5 variable 384607 0x1 > >>>>>>>>>>> 3: 0x58:LTO-5 variable 384607 0x1 > >>>>>>>>>>> --------------------------------- > >>>>>>>>>>> Current Driver State: at rest. > >>>>>>>>>>> --------------------------------- > >>>>>>>>>>> File Number: 2 Record Number: -1 Residual Count -1 > >>>>>>>>>>> > >>>>>>>>>>> So the thing to try, in addition to just making sure that Bacula continues > >>>>>>>>>>> to work properly, is to try setting this for the tape drive in > >>>>>>>>>>> bacula-sd.conf: > >>>>>>>>>>> > >>>>>>>>>>> Hardware End of Medium = yes > >>>>>>>>>>> > >>>>>>>>>>> It looks like the Bacula tape program (btape) has a test mode, and it would > >>>>>>>>>>> be good to run through the tests on one of the tape drives and see whether > >>>>>>>>>>> they work, and whether the results are different before and after the > >>>>>>>>>>> changes. I'm not sure how to enable the test mode. > >>>>>>>>>> > >>>>>>>>>> I have this in /usr/local/etc/bacula/bacula-sd.conf > >>>>>>>>>> > >>>>>>>>>> Device { > >>>>>>>>>> Name = DLT > >>>>>>>>>> Description = "QUANTUM DLT7000 1624" > >>>>>>>>>> Media Type = DLT > >>>>>>>>>> Archive Device = /dev/nsa1 > >>>>>>>>>> > >>>>>>>>>> Autochanger = YES > >>>>>>>>>> Drive Index = 0 > >>>>>>>>>> > >>>>>>>>>> Offline On Unmount = no > >>>>>>>>>> Hardware End of Medium = yes > >>>>>>>>>> BSF at EOM = yes > >>>>>>>>>> Backward Space Record = no > >>>>>>>>>> Fast Forward Space File = no > >>>>>>>>>> TWO EOF = yes > >>>>>>>>>> } > >>>>>>>>>> > >>>>>>>>>> FYI, http://www.freebsddiary.org/digital-tl891.php (from 2006) has a btape test on this same model. > >>>>>>>>>> > >>>>>>>>>> Here's the test I ran tonight: > >>>>>>>>>> > >>>>>>>>>> [root@cuppy:/usr/home/dan] # btape -c /usr/local/etc/bacula/bacula-sd.conf /dev/nsa1 > >>>>>>>>>> Tape block granularity is 1024 bytes. > >>>>>>>>>> btape: butil.c:287-0 Using device: "/dev/nsa1" for writing. > >>>>>>>>>> btape: btape.c:469-0 open device "DLT" (/dev/nsa1): OK > >>>>>>>>>> *test > >>>>>>>>>> > >>>>>>>>>> === Write, rewind, and re-read test === > >>>>>>>>>> > >>>>>>>>>> I'm going to write 10000 records and an EOF > >>>>>>>>>> then write 10000 records and an EOF, then rewind, > >>>>>>>>>> and re-read the data to verify that it is correct. > >>>>>>>>>> > >>>>>>>>>> This is an *essential* feature ... > >>>>>>>>>> > >>>>>>>>>> btape: btape.c:1152-0 Wrote 10000 blocks of 64412 bytes. > >>>>>>>>>> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) > >>>>>>>>>> btape: btape.c:1168-0 Wrote 10000 blocks of 64412 bytes. > >>>>>>>>>> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) > >>>>>>>>>> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) > >>>>>>>>>> btape: btape.c:1210-0 Rewind OK. > >>>>>>>>>> 10000 blocks re-read correctly. > >>>>>>>>>> Got EOF on tape. > >>>>>>>>>> 10000 blocks re-read correctly. > >>>>>>>>>> === Test Succeeded. End Write, rewind, and re-read test === > >>>>>>>>>> > >>>>>>>>>> btape: btape.c:1277-0 Block position test > >>>>>>>>>> btape: btape.c:1289-0 Rewind OK. > >>>>>>>>>> Reposition to file:block 0:4 > >>>>>>>>>> Block 5 re-read correctly. > >>>>>>>>>> Reposition to file:block 0:200 > >>>>>>>>>> Block 201 re-read correctly. > >>>>>>>>>> Reposition to file:block 0:9999 > >>>>>>>>>> Block 10000 re-read correctly. > >>>>>>>>>> Reposition to file:block 1:0 > >>>>>>>>>> Block 10001 re-read correctly. > >>>>>>>>>> Reposition to file:block 1:600 > >>>>>>>>>> Block 10601 re-read correctly. > >>>>>>>>>> Reposition to file:block 1:9999 > >>>>>>>>>> Block 20000 re-read correctly. > >>>>>>>>>> === Test Succeeded. End Write, rewind, and re-read test === > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> === Append files test === > >>>>>>>>>> > >>>>>>>>>> This test is essential to Bacula. > >>>>>>>>>> > >>>>>>>>>> I'm going to write one record in file 0, > >>>>>>>>>> two records in file 1, > >>>>>>>>>> and three records in file 2 > >>>>>>>>>> > >>>>>>>>>> btape: btape.c:574-0 Rewound "DLT" (/dev/nsa1) > >>>>>>>>>> btape: btape.c:1907-0 Wrote one record of 64412 bytes. > >>>>>>>>>> btape: btape.c:1909-0 Wrote block to device. > >>>>>>>>>> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) > >>>>>>>>>> btape: btape.c:1907-0 Wrote one record of 64412 bytes. > >>>>>>>>>> btape: btape.c:1909-0 Wrote block to device. > >>>>>>>>>> btape: btape.c:1907-0 Wrote one record of 64412 bytes. > >>>>>>>>>> btape: btape.c:1909-0 Wrote block to device. > >>>>>>>>>> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) > >>>>>>>>>> btape: btape.c:1907-0 Wrote one record of 64412 bytes. > >>>>>>>>>> btape: btape.c:1909-0 Wrote block to device. > >>>>>>>>>> btape: btape.c:1907-0 Wrote one record of 64412 bytes. > >>>>>>>>>> btape: btape.c:1909-0 Wrote block to device. > >>>>>>>>>> btape: btape.c:1907-0 Wrote one record of 64412 bytes. > >>>>>>>>>> btape: btape.c:1909-0 Wrote block to device. > >>>>>>>>>> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) > >>>>>>>>>> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) > >>>>>>>>>> btape: btape.c:469-0 open device "DLT" (/dev/nsa1): OK > >>>>>>>>>> btape: btape.c:574-0 Rewound "DLT" (/dev/nsa1) > >>>>>>>>>> btape: btape.c:1420-0 Now moving to end of medium. > >>>>>>>>> > >>>>>>>>> This is the critical piece. The test moves the tape to the end of the > >>>>>>>>> medium. With hardware position information, you can tell what filemark > >>>>>>>>> you're on. Without it, you can't. > >>>>>>>>> > >>>>>>>>>> btape: btape.c:622-0 tape_dev.c:345 ioctl MTIOCGET error on "DLT" (/dev/nsa1). ERR=No error: 0. > >>>>>>>>>> We should be in file 3. I am at file 0. This is NOT correct!!!! > >>>>>>>>>> > >>>>>>>>>> Append test failed. Attempting again. > >>>>>>>>>> Setting "Hardware End of Medium = no > >>>>>>>>>> and "Fast Forward Space File = no > >>>>>>>>>> and retrying append test. > >>>>>>>>> > >>>>>>>>> This is not surprsing, given that the drive doesn't support long read > >>>>>>>>> position data. (It's a SCSI-2 device.) So that means that Bacula will > >>>>>>>>> need to do it manually. > >>>>>>>> > >>>>>>>> Yes, I have nothing newer than SCSI-2. Even my SDLT is SCSI-2 but that > >>>>>>>> tape library is hooked up to a different computer and was doing backups today. > >>>>>>> > >>>>>>> So, here is one thing that we can try to see whether these drives support > >>>>>>> long position information, even though they only claim to be SCSI-2. If > >>>>>>> they do, we can potentially add a quirk (or autodetection) to enable it. > >>>>>>> The code currently doesn't bother asking drives that claim to be SCSI-2 > >>>>>>> for long position information. (Because that feature was added in the > >>>>>>> SSC spec, which came after SCSI-2.) > >>>>>>> > >>>>>>> Issue a READ POSITION with the short form specified: > >>>>>>> > >>>>>>> camcontrol cmd sa1 -v -c "34 0 0 0 0 0 0 0 0 0" -i 20 - |hd > >>>>>>> > >>>>>>> Issue a READ POSITION with the vendor-specific block numbers: > >>>>>>> > >>>>>>> camcontrol cmd sa1 -v -c "34 1 0 0 0 0 0 0 0 0" -i 20 - |hd > >>>>>>> > >>>>>>> Issue a READ POSITION with the long form data: > >>>>>>> > >>>>>>> camcontrol cmd sa1 -v -c "34 6 0 0 0 0 0 0 0 0" -i 32 - |hd > >>>>>>> > >>>>>>> If it supports the last one, then I can put a quirk (or autodetection) in > >>>>>>> the driver and Bacula will get the hardware filemarks. You should try this > >>>>>>> on your SDLT as well. It may well support it. > >>>>>> > >>>>>> Sadly, no: > >>>>>> > >>>>>> [root@cuppy:~] # camcontrol cmd sa1 -v -c "34 0 0 0 0 0 0 0 0 0" -i 20 - |hd > >>>>>> 00000000 80 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| > >>>>>> 00000010 00 00 00 00 |....| > >>>>>> 00000014 > >>>>>> [root@cuppy:~] # camcontrol cmd sa1 -v -c "34 1 0 0 0 0 0 0 0 0" -i 20 - |hd > >>>>>> 00000000 80 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| > >>>>>> 00000010 00 00 00 00 |....| > >>>>>> 00000014 > >>>>>> [root@cuppy:~] # camcontrol cmd sa1 -v -c "34 6 0 0 0 0 0 0 0 0" -i 32 - |hd > >>>>>> camcontrol: error sending command > >>>>>> (pass2:ahc0:0:2:0): READ POSITION. CDB: 34 06 00 00 00 00 00 00 00 00 > >>>>>> (pass2:ahc0:0:2:0): CAM status: SCSI Status Error > >>>>>> (pass2:ahc0:0:2:0): SCSI status: Check Condition > >>>>>> (pass2:ahc0:0:2:0): SCSI sense: ILLEGAL REQUEST asc:24,0 (Invalid field in CDB) > >>>>>> (pass2:ahc0:0:2:0): Command byte 1 bit 2 is invalid > >>>>>> [root@cuppy:~] # > >>>>> > >>>>> Okay. Not too surprising I suppose. > >> > >> > >> Does this mean much to you? > >> > >> Mar 2 19:42:59 cuppy kernel: (ahc0:A:2:0): Sending SDTR period 19, offset f > >> Mar 2 19:42:59 cuppy kernel: (ahc0:A:2:0): Received SDTR period 19, offset f > >> Mar 2 19:42:59 cuppy kernel: Filtered to period 19, offset f > >> Mar 2 19:42:59 cuppy kernel: (ahc0:A:2:0): Sending SDTR period 19, offset f > >> Mar 2 19:42:59 cuppy kernel: (ahc0:A:2:0): Received SDTR period 19, offset f > >> Mar 2 19:42:59 cuppy kernel: Filtered to period 19, offset f > >> Mar 2 19:42:59 cuppy kernel: (ahc0:A:2:0): Sending SDTR period 19, offset f > >> Mar 2 19:42:59 cuppy kernel: (ahc0:A:2:0): Received SDTR period 19, offset f > >> Mar 2 19:42:59 cuppy kernel: Filtered to period 19, offset f > >> > >> > >> I'm having trouble with labeling barcodes from Bacula and the above is seen in /var/log/messages > > > > The barcodes issue is resolved. I changed to using drive 0 instead of drive 1, now that we have both drives. > > > > *label barcodes slots=3,4 > > The defined Storage resources are: > > 1: File1 > > 2: OldTL891 > > Select Storage resource (1-2): 2 > > Connecting to Storage daemon OldTL891 at 10.55.0.18:9103 ... > > 3306 Issuing autochanger "slots" command. > > Device "DTL03" has 10 slots. > > Connecting to Storage daemon OldTL891 at 10.55.0.18:9103 ... > > 3306 Issuing autochanger "list" command. > > The following Volumes will be labeled: > > Slot Volume > > ============== > > 3 FAI261 > > 4 FAI262 > > Do you want to label these Volumes? (yes|no): yes > > Defined Pools: > > 1: Default > > 2: File > > 3: MYDLT > > 4: Scratch > > Select the Pool (1-4): 4 > > Connecting to Storage daemon OldTL891 at 10.55.0.18:9103 ... > > Sending label command for Volume "FAI261" Slot 3 ... > > 3000 OK label. VolBytes=64512 DVD=0 Volume="FAI261" Device="DTL03" (/dev/nsa0) > > Catalog record for Volume "FAI261", Slot 3 successfully created. > > Sending label command for Volume "FAI262" Slot 4 ... > > 3307 Issuing autochanger "unload slot 3, drive 0" command. > > 3304 Issuing autochanger "load slot 4, drive 0" command. > > 3305 Autochanger "load slot 4, drive 0", status is OK. > > 3000 OK label. VolBytes=64512 DVD=0 Volume="FAI262" Device="DTL03" (/dev/nsa0) > > Catalog record for Volume "FAI262", Slot 4 successfully created. > > *list volumes > > Pool: Default > > No results to list. > > Pool: File > > +---------+------------+-----------+---------+------------+----------+--------------+---------+------+-----------+-----------+---------------------+ > > | mediaid | volumename | volstatus | enabled | volbytes | volfiles | volretention | recycle | slot | inchanger | mediatype | lastwritten | > > +---------+------------+-----------+---------+------------+----------+--------------+---------+------+-----------+-----------+---------------------+ > > | 1 | Vol-0001 | Append | 1 | 19,835,982 | 0 | 31,536,000 | 1 | 0 | 0 | File1 | 2015-03-02 17:50:40 | > > | 2 | Vol-0002 | Append | 1 | 0 | 0 | 31,536,000 | 1 | 0 | 0 | DLT | | > > +---------+------------+-----------+---------+------------+----------+--------------+---------+------+-----------+-----------+---------------------+ > > Pool: MYDLT > > No results to list. > > Pool: Scratch > > +---------+------------+-----------+---------+----------+----------+--------------+---------+------+-----------+-----------+-------------+ > > | mediaid | volumename | volstatus | enabled | volbytes | volfiles | volretention | recycle | slot | inchanger | mediatype | lastwritten | > > +---------+------------+-----------+---------+----------+----------+--------------+---------+------+-----------+-----------+-------------+ > > | 4 | FAI260 | Append | 1 | 64,512 | 0 | 31,536,000 | 1 | 1 | 1 | DLT | | > > | 5 | FAI263 | Append | 1 | 64,512 | 0 | 31,536,000 | 1 | 2 | 1 | DLT | | > > | 7 | FAI261 | Append | 1 | 64,512 | 0 | 31,536,000 | 1 | 3 | 1 | DLT | | > > | 8 | FAI262 | Append | 1 | 64,512 | 0 | 31,536,000 | 1 | 4 | 1 | DLT | | > > +---------+------------+-----------+---------+----------+----------+--------------+---------+------+-----------+-----------+-------------+ > > * > > > > But this is what arrives in /var/log/messages during the above: > > > > Mar 2 20:29:06 cuppy kernel: (ahc0:A:1:0): Sending SDTR period 19, offset f > > Mar 2 20:29:06 cuppy kernel: (ahc0:A:1:0): Received SDTR period 19, offset f > > Mar 2 20:29:06 cuppy kernel: Filtered to period 19, offset f > > Mar 2 20:30:22 cuppy kernel: (ahc0:A:1:0): Sending SDTR period 19, offset f > > Mar 2 20:30:22 cuppy kernel: (ahc0:A:1:0): Received SDTR period 19, offset f > > Mar 2 20:30:22 cuppy kernel: Filtered to period 19, offset f > > Mar 2 20:30:22 cuppy kernel: (ahc0:A:1:0): Sending SDTR period 19, offset f > > Mar 2 20:30:22 cuppy kernel: (ahc0:A:1:0): Received SDTR period 19, offset f > > Mar 2 20:30:22 cuppy kernel: Filtered to period 19, offset f > > Mar 2 20:30:59 cuppy kernel: (ahc0:A:1:0): Sending SDTR period 19, offset f > > Mar 2 20:30:59 cuppy kernel: (ahc0:A:1:0): Received SDTR period 19, offset f > > Mar 2 20:30:59 cuppy kernel: Filtered to period 19, offset f > > Mar 2 20:30:59 cuppy kernel: xpt_release_devq(): requested 1 > present 0 > > Mar 2 20:31:07 cuppy kernel: (ahc0:A:1:0): Sending SDTR period 19, offset f > > Mar 2 20:31:07 cuppy kernel: (ahc0:A:1:0): Received SDTR period 19, offset f > > Mar 2 20:31:07 cuppy kernel: Filtered to period 19, offset f > > > > Anything to be concerned about? > > When adding a 2nd job to a tape: > > 02-Mar 21:29 cuppy-sd JobId 19: Error: Unable to position to end of data on tape device "DTL03" (/dev/nsa0): ERR=tape_dev.c:345 ioctl MTIOCGET error on "DTL03" (/dev/nsa0). ERR=No error: 0. > > I've gotten this on three consecutive tapes. NOTE: These *are* tapes I no longer use because of higher rates of corrected errors. > > The problem seems to be locating the end of the data. Do you have 'Hardware End of Medium = no' in the configuration file? In looking at the code, it may be set to yes, and that could be the issue here. > > I can run a 4 or 5 jobs in a row, but if I restore a job, then add a job, it fails with the above message. > I would expect that on these particular tape drives because they don't support long position information. Ken -- Kenneth Merry ken@FreeBSD.ORG From owner-freebsd-current@FreeBSD.ORG Mon Mar 2 21:57:01 2015 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 59A18A33 for ; Mon, 2 Mar 2015 21:57:01 +0000 (UTC) Received: from mail-ig0-x231.google.com (mail-ig0-x231.google.com [IPv6:2607:f8b0:4001:c05::231]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 1C22AD8D for ; Mon, 2 Mar 2015 21:57:01 +0000 (UTC) Received: by igdh15 with SMTP id h15so21359844igd.4 for ; Mon, 02 Mar 2015 13:57:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=Eq90M6GPACz+RzrI1cCO8ZkFLKQakkEYECJ9sKFkdtU=; b=DtqV+IixiIRN0fSe8X6Uok9n7OWlLQkAAyngPwAU+8GSE0rYVSosEQ07/RKB9kjkRA MVKhiwOvJsndMykXxHCaHDkI5jG01EJtXrZYa6SJPTxoiincdgcqUOzXW/IjEVwoBAce Am4zNkUSuyPONR575nyxTJo11SZ5xnEZx4vvz25Fo+Bt0dDIuadiPs/Mli4XVYWx8a+q 1+rFyP3v9+FOHC6aXOsHeMpLgi5xpRlpog7WiEtyk6Qk5cFm6sHt4eN5gexPJx7/O3Et 6we9zozkKYXnqWfMEdTZ2Ml8KQuOr7P1VsTLqhaU1VH01hmblTqNHPzldl2kJkljJdtb tCbQ== MIME-Version: 1.0 X-Received: by 10.42.188.133 with SMTP id da5mr33434174icb.37.1425333420509; Mon, 02 Mar 2015 13:57:00 -0800 (PST) Sender: adrian.chadd@gmail.com Received: by 10.36.17.66 with HTTP; Mon, 2 Mar 2015 13:57:00 -0800 (PST) In-Reply-To: <54F4C127.5020000@astrodoggroup.com> References: <54F31510.7050607@hot.ee> <54F34B6E.2040809@astrodoggroup.com> <54F35F29.4000603@astrodoggroup.com> <54F429EF.5050400@freebsd.org> <54F46536.8040607@mu.org> <54F4C03F.7030704@freebsd.org> <54F4C127.5020000@astrodoggroup.com> Date: Mon, 2 Mar 2015 13:57:00 -0800 X-Google-Sender-Auth: ns3dTJA05SLyBKP1DyO_LQLhd9E Message-ID: Subject: Re: Massive libxo-zation that breaks everything From: Adrian Chadd To: Harrison Grundy Content-Type: text/plain; charset=UTF-8 Cc: freebsd-current X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Mar 2015 21:57:01 -0000 .. we can/should do both. Just make sure the json/html/xml output is versioned, otherwise you're going to end up with /exactly the same problems/ you have with the current format. -adrian From owner-freebsd-current@FreeBSD.ORG Mon Mar 2 22:03:57 2015 Return-Path: Delivered-To: current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 00595D48; Mon, 2 Mar 2015 22:03:56 +0000 (UTC) Received: from clavin1.langille.org (clavin.langille.org [162.208.116.86]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "clavin.langille.org", Issuer "StartCom Class 2 Primary Intermediate Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C4630E6D; Mon, 2 Mar 2015 22:03:56 +0000 (UTC) Received: from (clavin1.int.langille.org (clavin1.int.unixathome.org [10.4.7.7]) (Authenticated sender: hidden) with ESMTPSA id 7957237F ; Mon, 2 Mar 2015 22:03:54 +0000 (UTC) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 8.2 \(2070.6\)) Subject: Re: sa(4) driver changes available for test From: Dan Langille In-Reply-To: <20150302214240.GA91133@mithlond.kdm.org> Date: Mon, 2 Mar 2015 17:03:53 -0500 Content-Transfer-Encoding: quoted-printable Message-Id: <4FD3E56B-2B55-4C1F-95C2-58366EAD1180@langille.org> References: <20150302003150.GB71528@mithlond.kdm.org> <20150302022957.GB73433@mithlond.kdm.org> <0F17CDDB-4CE9-4881-B948-6B9BEDEB6455@langille.org> <20150302163146.GA85515@mithlond.kdm.org> <85E2DDA0-4A2B-4316-A318-3706B3A05C11@langille.org> <20150302172846.GB87055@mithlond.kdm.org> <12EED94D-CC9D-44C4-91C2-62C8A51B839B@langille.org> <95BCE742-8CC4-41FC-9FDD-D12C7A337951@langille.org> <20150302214240.GA91133@mithlond.kdm.org> To: "Kenneth D. Merry" X-Mailer: Apple Mail (2.2070.6) Cc: current@freebsd.org, scsi@freebsd.org X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Mar 2015 22:03:57 -0000 > On Mar 2, 2015, at 4:42 PM, Kenneth D. Merry wrote: >=20 > On Mon, Mar 02, 2015 at 16:34:34 -0500, Dan Langille wrote: >>=20 >>> On Mar 2, 2015, at 2:47 PM, Dan Langille wrote: >>>=20 >>>=20 >>>> On Mar 2, 2015, at 2:07 PM, Dan Langille wrote: >>>>=20 >>>>=20 >>>>> On Mar 2, 2015, at 12:28 PM, Kenneth D. Merry = wrote: >>>>>=20 >>>>> On Mon, Mar 02, 2015 at 11:44:09 -0500, Dan Langille wrote: >>>>>>=20 >>>>>>> On Mar 2, 2015, at 11:31 AM, Kenneth D. Merry = wrote: >>>>>>>=20 >>>>>>> On Mon, Mar 02, 2015 at 11:09:57 -0500, Dan Langille wrote: >>>>>>>>=20 >>>>>>>>> On Mar 1, 2015, at 9:29 PM, Kenneth D. Merry = wrote: >>>>>>>>>=20 >>>>>>>>> On Sun, Mar 01, 2015 at 19:41:07 -0500, Dan Langille wrote: >>>>>>>>>>=20 >>>>>>>>>>> On Mar 1, 2015, at 7:31 PM, Kenneth D. Merry = wrote: >>>>>>>>>>>=20 >>>>>>>>>>> On Sun, Mar 01, 2015 at 19:15:05 -0500, Dan Langille wrote: >>>>>>>>>>>>=20 >>>>>>>>>>>>> On Feb 17, 2015, at 1:36 PM, Kenneth D. Merry = wrote: >>>>>>>>>>>>>=20 >>>>>>>>>>>>> On Sat, Feb 14, 2015 at 18:22:43 -0500, Dan Langille = wrote: >>>>>>>>>>>>>>=20 >>>>>>>>>>>>>>> On Feb 13, 2015, at 7:32 PM, Kenneth D. Merry = wrote: >>>>>>>>>>>>>>>=20 >>>>>>>>>>>>>>>=20 >>>>>>>>>>>>>>> I have a fairly large set of changes to the sa(4) driver = and mt(1) driver >>>>>>>>>>>>>>> that I'm planning to commit in the near future. >>>>>>>>>>>>>>>=20 >>>>>>>>>>>>>>> A description of the changes is here and below in this = message. >>>>>>>>>>>>>>>=20 >>>>>>>>>>>>>>> If you have tape hardware and the inclination, I'd = appreciate testing and >>>>>>>>>>>>>>> feedback. >>>>>>>>>>>>>>=20 >>>>>>>>>>>>>> I have a DLT 8000 and an SDLT 220. >>>>>>>>>>>>>>=20 >>>>>>>>>>>>>> I don't have anything running current, but I have a spare = machine which I could use for testing. >>>>>>>>>>>>>>=20 >>>>>>>>>>>>>> Do you see any value is tests with that hardware? I'd be = testing it via Bacula. >>>>>>>>>>>>>>=20 >>>>>>>>>>>>>> disclosure: I'm the sysutils/bacula-* maintainer and a = Bacula committer. >>>>>>>>>>>>>>=20 >>>>>>>>>>>>>=20 >>>>>>>>>>>>> Actually, yes. Bacula is a bit tricky to configure, so = your trying it out >>>>>>>>>>>>> would be helpful if you have the time. >>>>>>>>>>>>>=20 >>>>>>>>>>>>> In looking at the manuals for both the SDLT 220 and the = DLT 8000, they both >>>>>>>>>>>>> claim to support long position information for the SCSI = READ POSITION >>>>>>>>>>>>> command. >>>>>>>>>>>>>=20 >>>>>>>>>>>>> You can see what I'm talking about by doing: >>>>>>>>>>>>>=20 >>>>>>>>>>>>> mt eod >>>>>>>>>>>>> mt status >>>>>>>>>>>>>=20 >>>>>>>>>>>>> On my DDS-4 tape drive, this shows: >>>>>>>>>>>>>=20 >>>>>>>>>>>>> # mt -f /dev/nsa3 status >>>>>>>>>>>>> Drive: sa3: Serial Number: = HJ00YWY >>>>>>>>>>>>> --------------------------------- >>>>>>>>>>>>> Mode Density Blocksize bpi = Compression >>>>>>>>>>>>> Current: 0x26:DDS-4 1024 bytes 97000 = enabled (DCLZ) >>>>>>>>>>>>> --------------------------------- >>>>>>>>>>>>> Current Driver State: at rest. >>>>>>>>>>>>> --------------------------------- >>>>>>>>>>>>> Partition: 0 Calc File Number: -1 Calc Record = Number: -1 >>>>>>>>>>>>> Residual: 0 Reported File Number: -1 Reported Record = Number: -1 >>>>>>>>>>>>> Flags: None >>>>>>>>>>>>>=20 >>>>>>>>>>>>> But on an LTO-5, which will give long position = information, I get: >>>>>>>>>>>>>=20 >>>>>>>>>>>>> [root@doc ~]# mt status >>>>>>>>>>>>> Drive: sa0: >>>>>>>>>>>>> --------------------------------- >>>>>>>>>>>>> Mode Density Blocksize bpi = Compression >>>>>>>>>>>>> Current: 0x58:LTO-5 variable 384607 = enabled (0x1) >>>>>>>>>>>>> --------------------------------- >>>>>>>>>>>>> Current Driver State: at rest. >>>>>>>>>>>>> --------------------------------- >>>>>>>>>>>>> Partition: 0 Calc File Number: 2 Calc Record = Number: -1 >>>>>>>>>>>>> Residual: 0 Reported File Number: 2 Reported Record = Number: 32373 >>>>>>>>>>>>> Flags: None >>>>>>>>>>>>>=20 >>>>>>>>>>>>> That, in combination with the changes I made to the = position information >>>>>>>>>>>>> code in the driver, mean that even the old MTIOCGET ioctl = should return an >>>>>>>>>>>>> accurate file number at end of data. e.g., on the LTO-5: >>>>>>>>>>>>>=20 >>>>>>>>>>>>> [root@doc ~]# mt ostatus >>>>>>>>>>>>> Mode Density Blocksize bpi = Compression >>>>>>>>>>>>> Current: 0x58:LTO-5 variable 384607 0x1 >>>>>>>>>>>>> ---------available modes--------- >>>>>>>>>>>>> 0: 0x58:LTO-5 variable 384607 0x1 >>>>>>>>>>>>> 1: 0x58:LTO-5 variable 384607 0x1 >>>>>>>>>>>>> 2: 0x58:LTO-5 variable 384607 0x1 >>>>>>>>>>>>> 3: 0x58:LTO-5 variable 384607 0x1 >>>>>>>>>>>>> --------------------------------- >>>>>>>>>>>>> Current Driver State: at rest. >>>>>>>>>>>>> --------------------------------- >>>>>>>>>>>>> File Number: 2 Record Number: -1 Residual Count -1 >>>>>>>>>>>>>=20 >>>>>>>>>>>>> So the thing to try, in addition to just making sure that = Bacula continues >>>>>>>>>>>>> to work properly, is to try setting this for the tape = drive in >>>>>>>>>>>>> bacula-sd.conf: >>>>>>>>>>>>>=20 >>>>>>>>>>>>> Hardware End of Medium =3D yes >>>>>>>>>>>>>=20 >>>>>>>>>>>>> It looks like the Bacula tape program (btape) has a test = mode, and it would >>>>>>>>>>>>> be good to run through the tests on one of the tape drives = and see whether >>>>>>>>>>>>> they work, and whether the results are different before = and after the >>>>>>>>>>>>> changes. I'm not sure how to enable the test mode. >>>>>>>>>>>>=20 >>>>>>>>>>>> I have this in /usr/local/etc/bacula/bacula-sd.conf >>>>>>>>>>>>=20 >>>>>>>>>>>> Device { >>>>>>>>>>>> Name =3D DLT >>>>>>>>>>>> Description =3D "QUANTUM DLT7000 1624" >>>>>>>>>>>> Media Type =3D DLT >>>>>>>>>>>> Archive Device =3D /dev/nsa1 >>>>>>>>>>>>=20 >>>>>>>>>>>> Autochanger =3D YES >>>>>>>>>>>> Drive Index =3D 0 >>>>>>>>>>>>=20 >>>>>>>>>>>> Offline On Unmount =3D no >>>>>>>>>>>> Hardware End of Medium =3D yes >>>>>>>>>>>> BSF at EOM =3D yes >>>>>>>>>>>> Backward Space Record =3D no >>>>>>>>>>>> Fast Forward Space File =3D no >>>>>>>>>>>> TWO EOF =3D yes >>>>>>>>>>>> } >>>>>>>>>>>>=20 >>>>>>>>>>>> FYI, http://www.freebsddiary.org/digital-tl891.php (from = 2006) has a btape test on this same model. >>>>>>>>>>>>=20 >>>>>>>>>>>> Here's the test I ran tonight: >>>>>>>>>>>>=20 >>>>>>>>>>>> [root@cuppy:/usr/home/dan] # btape -c = /usr/local/etc/bacula/bacula-sd.conf /dev/nsa1 = =20 >>>>>>>>>>>> Tape block granularity is 1024 bytes. >>>>>>>>>>>> btape: butil.c:287-0 Using device: "/dev/nsa1" for writing. >>>>>>>>>>>> btape: btape.c:469-0 open device "DLT" (/dev/nsa1): OK >>>>>>>>>>>> *test >>>>>>>>>>>>=20 >>>>>>>>>>>> =3D=3D=3D Write, rewind, and re-read test =3D=3D=3D >>>>>>>>>>>>=20 >>>>>>>>>>>> I'm going to write 10000 records and an EOF >>>>>>>>>>>> then write 10000 records and an EOF, then rewind, >>>>>>>>>>>> and re-read the data to verify that it is correct. >>>>>>>>>>>>=20 >>>>>>>>>>>> This is an *essential* feature ... >>>>>>>>>>>>=20 >>>>>>>>>>>> btape: btape.c:1152-0 Wrote 10000 blocks of 64412 bytes. >>>>>>>>>>>> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) >>>>>>>>>>>> btape: btape.c:1168-0 Wrote 10000 blocks of 64412 bytes. >>>>>>>>>>>> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) >>>>>>>>>>>> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) >>>>>>>>>>>> btape: btape.c:1210-0 Rewind OK. >>>>>>>>>>>> 10000 blocks re-read correctly. >>>>>>>>>>>> Got EOF on tape. >>>>>>>>>>>> 10000 blocks re-read correctly. >>>>>>>>>>>> =3D=3D=3D Test Succeeded. End Write, rewind, and re-read = test =3D=3D=3D >>>>>>>>>>>>=20 >>>>>>>>>>>> btape: btape.c:1277-0 Block position test >>>>>>>>>>>> btape: btape.c:1289-0 Rewind OK. >>>>>>>>>>>> Reposition to file:block 0:4 >>>>>>>>>>>> Block 5 re-read correctly. >>>>>>>>>>>> Reposition to file:block 0:200 >>>>>>>>>>>> Block 201 re-read correctly. >>>>>>>>>>>> Reposition to file:block 0:9999 >>>>>>>>>>>> Block 10000 re-read correctly. >>>>>>>>>>>> Reposition to file:block 1:0 >>>>>>>>>>>> Block 10001 re-read correctly. >>>>>>>>>>>> Reposition to file:block 1:600 >>>>>>>>>>>> Block 10601 re-read correctly. >>>>>>>>>>>> Reposition to file:block 1:9999 >>>>>>>>>>>> Block 20000 re-read correctly. >>>>>>>>>>>> =3D=3D=3D Test Succeeded. End Write, rewind, and re-read = test =3D=3D=3D >>>>>>>>>>>>=20 >>>>>>>>>>>>=20 >>>>>>>>>>>>=20 >>>>>>>>>>>> =3D=3D=3D Append files test =3D=3D=3D >>>>>>>>>>>>=20 >>>>>>>>>>>> This test is essential to Bacula. >>>>>>>>>>>>=20 >>>>>>>>>>>> I'm going to write one record in file 0, >>>>>>>>>>>> two records in file 1, >>>>>>>>>>>> and three records in file 2 >>>>>>>>>>>>=20 >>>>>>>>>>>> btape: btape.c:574-0 Rewound "DLT" (/dev/nsa1) >>>>>>>>>>>> btape: btape.c:1907-0 Wrote one record of 64412 bytes. >>>>>>>>>>>> btape: btape.c:1909-0 Wrote block to device. >>>>>>>>>>>> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) >>>>>>>>>>>> btape: btape.c:1907-0 Wrote one record of 64412 bytes. >>>>>>>>>>>> btape: btape.c:1909-0 Wrote block to device. >>>>>>>>>>>> btape: btape.c:1907-0 Wrote one record of 64412 bytes. >>>>>>>>>>>> btape: btape.c:1909-0 Wrote block to device. >>>>>>>>>>>> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) >>>>>>>>>>>> btape: btape.c:1907-0 Wrote one record of 64412 bytes. >>>>>>>>>>>> btape: btape.c:1909-0 Wrote block to device. >>>>>>>>>>>> btape: btape.c:1907-0 Wrote one record of 64412 bytes. >>>>>>>>>>>> btape: btape.c:1909-0 Wrote block to device. >>>>>>>>>>>> btape: btape.c:1907-0 Wrote one record of 64412 bytes. >>>>>>>>>>>> btape: btape.c:1909-0 Wrote block to device. >>>>>>>>>>>> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) >>>>>>>>>>>> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) >>>>>>>>>>>> btape: btape.c:469-0 open device "DLT" (/dev/nsa1): OK >>>>>>>>>>>> btape: btape.c:574-0 Rewound "DLT" (/dev/nsa1) >>>>>>>>>>>> btape: btape.c:1420-0 Now moving to end of medium. >>>>>>>>>>>=20 >>>>>>>>>>> This is the critical piece. The test moves the tape to the = end of the >>>>>>>>>>> medium. With hardware position information, you can tell = what filemark >>>>>>>>>>> you're on. Without it, you can't. >>>>>>>>>>>=20 >>>>>>>>>>>> btape: btape.c:622-0 tape_dev.c:345 ioctl MTIOCGET error on = "DLT" (/dev/nsa1). ERR=3DNo error: 0. >>>>>>>>>>>> We should be in file 3. I am at file 0. This is NOT = correct!!!! >>>>>>>>>>>>=20 >>>>>>>>>>>> Append test failed. Attempting again. >>>>>>>>>>>> Setting "Hardware End of Medium =3D no >>>>>>>>>>>> and "Fast Forward Space File =3D no >>>>>>>>>>>> and retrying append test. >>>>>>>>>>>=20 >>>>>>>>>>> This is not surprsing, given that the drive doesn't support = long read >>>>>>>>>>> position data. (It's a SCSI-2 device.) So that means that = Bacula will >>>>>>>>>>> need to do it manually. >>>>>>>>>>=20 >>>>>>>>>> Yes, I have nothing newer than SCSI-2. Even my SDLT is = SCSI-2 but that >>>>>>>>>> tape library is hooked up to a different computer and was = doing backups today. >>>>>>>>>=20 >>>>>>>>> So, here is one thing that we can try to see whether these = drives support >>>>>>>>> long position information, even though they only claim to be = SCSI-2. If >>>>>>>>> they do, we can potentially add a quirk (or autodetection) to = enable it. >>>>>>>>> The code currently doesn't bother asking drives that claim to = be SCSI-2 >>>>>>>>> for long position information. (Because that feature was = added in the >>>>>>>>> SSC spec, which came after SCSI-2.) >>>>>>>>>=20 >>>>>>>>> Issue a READ POSITION with the short form specified: >>>>>>>>>=20 >>>>>>>>> camcontrol cmd sa1 -v -c "34 0 0 0 0 0 0 0 0 0" -i 20 - |hd >>>>>>>>>=20 >>>>>>>>> Issue a READ POSITION with the vendor-specific block numbers: >>>>>>>>>=20 >>>>>>>>> camcontrol cmd sa1 -v -c "34 1 0 0 0 0 0 0 0 0" -i 20 - |hd >>>>>>>>>=20 >>>>>>>>> Issue a READ POSITION with the long form data: >>>>>>>>>=20 >>>>>>>>> camcontrol cmd sa1 -v -c "34 6 0 0 0 0 0 0 0 0" -i 32 - |hd >>>>>>>>>=20 >>>>>>>>> If it supports the last one, then I can put a quirk (or = autodetection) in >>>>>>>>> the driver and Bacula will get the hardware filemarks. You = should try this >>>>>>>>> on your SDLT as well. It may well support it. >>>>>>>>=20 >>>>>>>> Sadly, no: >>>>>>>>=20 >>>>>>>> [root@cuppy:~] # camcontrol cmd sa1 -v -c "34 0 0 0 0 0 0 0 0 = 0" -i 20 - |hd >>>>>>>> 00000000 80 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 = |................| >>>>>>>> 00000010 00 00 00 00 = |....| >>>>>>>> 00000014 >>>>>>>> [root@cuppy:~] # camcontrol cmd sa1 -v -c "34 1 0 0 0 0 0 0 0 = 0" -i 20 - |hd >>>>>>>> 00000000 80 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 = |................| >>>>>>>> 00000010 00 00 00 00 = |....| >>>>>>>> 00000014 >>>>>>>> [root@cuppy:~] # camcontrol cmd sa1 -v -c "34 6 0 0 0 0 0 0 0 = 0" -i 32 - |hd >>>>>>>> camcontrol: error sending command >>>>>>>> (pass2:ahc0:0:2:0): READ POSITION. CDB: 34 06 00 00 00 00 00 00 = 00 00=20 >>>>>>>> (pass2:ahc0:0:2:0): CAM status: SCSI Status Error >>>>>>>> (pass2:ahc0:0:2:0): SCSI status: Check Condition >>>>>>>> (pass2:ahc0:0:2:0): SCSI sense: ILLEGAL REQUEST asc:24,0 = (Invalid field in CDB) >>>>>>>> (pass2:ahc0:0:2:0): Command byte 1 bit 2 is invalid >>>>>>>> [root@cuppy:~] #=20 >>>>>>>=20 >>>>>>> Okay. Not too surprising I suppose. >>>>=20 >>>>=20 >>>> Does this mean much to you? >>>>=20 >>>> Mar 2 19:42:59 cuppy kernel: (ahc0:A:2:0): Sending SDTR period 19, = offset f >>>> Mar 2 19:42:59 cuppy kernel: (ahc0:A:2:0): Received SDTR period = 19, offset f >>>> Mar 2 19:42:59 cuppy kernel: Filtered to period 19, offset f >>>> Mar 2 19:42:59 cuppy kernel: (ahc0:A:2:0): Sending SDTR period 19, = offset f >>>> Mar 2 19:42:59 cuppy kernel: (ahc0:A:2:0): Received SDTR period = 19, offset f >>>> Mar 2 19:42:59 cuppy kernel: Filtered to period 19, offset f >>>> Mar 2 19:42:59 cuppy kernel: (ahc0:A:2:0): Sending SDTR period 19, = offset f >>>> Mar 2 19:42:59 cuppy kernel: (ahc0:A:2:0): Received SDTR period = 19, offset f >>>> Mar 2 19:42:59 cuppy kernel: Filtered to period 19, offset f >>>>=20 >>>>=20 >>>> I'm having trouble with labeling barcodes from Bacula and the above = is seen in /var/log/messages >>>=20 >>> The barcodes issue is resolved. I changed to using drive 0 instead = of drive 1, now that we have both drives. >>>=20 >>> *label barcodes slots=3D3,4 >>> The defined Storage resources are: >>> 1: File1 >>> 2: OldTL891 >>> Select Storage resource (1-2): 2 >>> Connecting to Storage daemon OldTL891 at 10.55.0.18:9103 ... >>> 3306 Issuing autochanger "slots" command. >>> Device "DTL03" has 10 slots. >>> Connecting to Storage daemon OldTL891 at 10.55.0.18:9103 ... >>> 3306 Issuing autochanger "list" command. >>> The following Volumes will be labeled: >>> Slot Volume >>> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D >>> 3 FAI261 >>> 4 FAI262 >>> Do you want to label these Volumes? (yes|no): yes >>> Defined Pools: >>> 1: Default >>> 2: File >>> 3: MYDLT >>> 4: Scratch >>> Select the Pool (1-4): 4 >>> Connecting to Storage daemon OldTL891 at 10.55.0.18:9103 ... >>> Sending label command for Volume "FAI261" Slot 3 ... >>> 3000 OK label. VolBytes=3D64512 DVD=3D0 Volume=3D"FAI261" = Device=3D"DTL03" (/dev/nsa0) >>> Catalog record for Volume "FAI261", Slot 3 successfully created. >>> Sending label command for Volume "FAI262" Slot 4 ... >>> 3307 Issuing autochanger "unload slot 3, drive 0" command. >>> 3304 Issuing autochanger "load slot 4, drive 0" command. >>> 3305 Autochanger "load slot 4, drive 0", status is OK. >>> 3000 OK label. VolBytes=3D64512 DVD=3D0 Volume=3D"FAI262" = Device=3D"DTL03" (/dev/nsa0) >>> Catalog record for Volume "FAI262", Slot 4 successfully created. >>> *list volumes >>> Pool: Default >>> No results to list. >>> Pool: File >>> = +---------+------------+-----------+---------+------------+----------+----= ----------+---------+------+-----------+-----------+---------------------+= >>> | mediaid | volumename | volstatus | enabled | volbytes | volfiles = | volretention | recycle | slot | inchanger | mediatype | lastwritten = | >>> = +---------+------------+-----------+---------+------------+----------+----= ----------+---------+------+-----------+-----------+---------------------+= >>> | 1 | Vol-0001 | Append | 1 | 19,835,982 | 0 = | 31,536,000 | 1 | 0 | 0 | File1 | 2015-03-02 = 17:50:40 | >>> | 2 | Vol-0002 | Append | 1 | 0 | 0 = | 31,536,000 | 1 | 0 | 0 | DLT | = | >>> = +---------+------------+-----------+---------+------------+----------+----= ----------+---------+------+-----------+-----------+---------------------+= >>> Pool: MYDLT >>> No results to list. >>> Pool: Scratch >>> = +---------+------------+-----------+---------+----------+----------+------= --------+---------+------+-----------+-----------+-------------+ >>> | mediaid | volumename | volstatus | enabled | volbytes | volfiles | = volretention | recycle | slot | inchanger | mediatype | lastwritten | >>> = +---------+------------+-----------+---------+----------+----------+------= --------+---------+------+-----------+-----------+-------------+ >>> | 4 | FAI260 | Append | 1 | 64,512 | 0 | = 31,536,000 | 1 | 1 | 1 | DLT | | >>> | 5 | FAI263 | Append | 1 | 64,512 | 0 | = 31,536,000 | 1 | 2 | 1 | DLT | | >>> | 7 | FAI261 | Append | 1 | 64,512 | 0 | = 31,536,000 | 1 | 3 | 1 | DLT | | >>> | 8 | FAI262 | Append | 1 | 64,512 | 0 | = 31,536,000 | 1 | 4 | 1 | DLT | | >>> = +---------+------------+-----------+---------+----------+----------+------= --------+---------+------+-----------+-----------+-------------+ >>> * >>>=20 >>> But this is what arrives in /var/log/messages during the above: >>>=20 >>> Mar 2 20:29:06 cuppy kernel: (ahc0:A:1:0): Sending SDTR period 19, = offset f >>> Mar 2 20:29:06 cuppy kernel: (ahc0:A:1:0): Received SDTR period 19, = offset f >>> Mar 2 20:29:06 cuppy kernel: Filtered to period 19, offset f >>> Mar 2 20:30:22 cuppy kernel: (ahc0:A:1:0): Sending SDTR period 19, = offset f >>> Mar 2 20:30:22 cuppy kernel: (ahc0:A:1:0): Received SDTR period 19, = offset f >>> Mar 2 20:30:22 cuppy kernel: Filtered to period 19, offset f >>> Mar 2 20:30:22 cuppy kernel: (ahc0:A:1:0): Sending SDTR period 19, = offset f >>> Mar 2 20:30:22 cuppy kernel: (ahc0:A:1:0): Received SDTR period 19, = offset f >>> Mar 2 20:30:22 cuppy kernel: Filtered to period 19, offset f >>> Mar 2 20:30:59 cuppy kernel: (ahc0:A:1:0): Sending SDTR period 19, = offset f >>> Mar 2 20:30:59 cuppy kernel: (ahc0:A:1:0): Received SDTR period 19, = offset f >>> Mar 2 20:30:59 cuppy kernel: Filtered to period 19, offset f >>> Mar 2 20:30:59 cuppy kernel: xpt_release_devq(): requested 1 > = present 0 >>> Mar 2 20:31:07 cuppy kernel: (ahc0:A:1:0): Sending SDTR period 19, = offset f >>> Mar 2 20:31:07 cuppy kernel: (ahc0:A:1:0): Received SDTR period 19, = offset f >>> Mar 2 20:31:07 cuppy kernel: Filtered to period 19, offset f >>>=20 >>> Anything to be concerned about? >>=20 >> When adding a 2nd job to a tape: >>=20 >> 02-Mar 21:29 cuppy-sd JobId 19: Error: Unable to position to end of = data on tape device "DTL03" (/dev/nsa0): ERR=3Dtape_dev.c:345 ioctl = MTIOCGET error on "DTL03" (/dev/nsa0). ERR=3DNo error: 0. >>=20 >> I've gotten this on three consecutive tapes. NOTE: These *are* tapes = I no longer use because of higher rates of corrected errors. >>=20 >> The problem seems to be locating the end of the data. >=20 > Do you have 'Hardware End of Medium =3D no' in the configuration = file? >=20 > In looking at the code, it may be set to yes, and that could be the = issue > here. I had it set to yes. Now I'm testing with no and I do not encounter = that error. =20 >> I can run a 4 or 5 jobs in a row, but if I restore a job, then add a = job, it fails with the above message. >>=20 >=20 > I would expect that on these particular tape drives because they don't > support long position information. Ahh, good. =E2=80=94=20 Dan Langille http://langille.org/ From owner-freebsd-current@FreeBSD.ORG Mon Mar 2 23:06:00 2015 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 61522CFB; Mon, 2 Mar 2015 23:06:00 +0000 (UTC) Received: from mail-yh0-x230.google.com (mail-yh0-x230.google.com [IPv6:2607:f8b0:4002:c01::230]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 1C0436DA; Mon, 2 Mar 2015 23:06:00 +0000 (UTC) Received: by yhoa41 with SMTP id a41so16372674yho.9; Mon, 02 Mar 2015 15:05:59 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=I2W/TD3ky1JofeJdLHe2L3621FCeCd8zVSnl7JqeqQ0=; b=A7VF9zwIdYK2zI+9Pszrfr0F/3fxfVNuOsl5u+7byaDagcFyiFe3rJjvFJXBWn7Pbt vQZ+DheDyf5gX7Fu+jvU6AvTk4a6XUuX2yPg2sszuOKZs4tekRsutZ/JXVQEW/z8+p5y tO4sch7uy2CBtlZZSsxVgpOw0PYsdwTZ9YymnGyWp6RouKyq1OJJPBvhoweF1Xw/7tl2 S/2D6hW+tMNT9awJgUGGt2+wDAfPvF75OiP/8VZcmtgHtVCrSOEyClzd4yiSKNW/mFS3 4UmNn9zf3IenJtcttxTGjZY+4Cg3jxBVnNurVNAn0EW9pGnufZroVjn632H8efeduI1o V8Yw== MIME-Version: 1.0 X-Received: by 10.170.185.71 with SMTP id b68mr13529604yke.25.1425337559164; Mon, 02 Mar 2015 15:05:59 -0800 (PST) Sender: kmacybsd@gmail.com Received: by 10.170.76.66 with HTTP; Mon, 2 Mar 2015 15:05:59 -0800 (PST) In-Reply-To: References: Date: Mon, 2 Mar 2015 15:05:59 -0800 X-Google-Sender-Auth: 1k4rEY3CGKDdAhTV-jSklRrpPyY Message-ID: Subject: Re: Doing zero-copy stuff in drivers, or "is vm_fault_quick_hold_pages() enough" ? From: "K. Macy" To: Adrian Chadd Content-Type: text/plain; charset=UTF-8 Cc: freebsd-current , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Mar 2015 23:06:00 -0000 On Mon, Mar 2, 2015 at 12:42 PM, Adrian Chadd wrote: > Hi, > > gonzo@ committed a fix (r278615) to the videocore driver for the > raspberry pi. The fix involved doing an explicit wire of pages that > were about to be passed down to the hardware to send to the > videobuffer hardware. > > It turns out that doing vm_fault_quick_hold_pages() wasn't enough - > the pages weren't being wired, and they may disappear before the > hardware gets to them. Then your code is buggy or you've hit a bug in the VM. Holding a page is a short-term wiring. Right above vm_page_hold(): /* * Keep page from being freed by the page daemon * much of the same effect as wiring, except much lower * overhead and should be used only for *very* temporary * holding ("wiring"). */ > I looked at vmapbuf() and how it uses vm_fault_quick_hold_pages(), but > I can't find anything that wires the pages down before it hands the > addresses to the hardware. > > So, am I missing something about how/where that's done? Yes. > Thanks, > > > -adrian > _______________________________________________ > freebsd-current@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-current > To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org" From owner-freebsd-current@FreeBSD.ORG Mon Mar 2 23:41:38 2015 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 766CCAF7 for ; Mon, 2 Mar 2015 23:41:38 +0000 (UTC) Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by mx1.freebsd.org (Postfix) with ESMTP id 6044EAEA for ; Mon, 2 Mar 2015 23:41:38 +0000 (UTC) Received: from AlfredMacbookAir.local (unknown [12.130.117.99]) by elvis.mu.org (Postfix) with ESMTPSA id E1FA3341F861 for ; Mon, 2 Mar 2015 15:41:35 -0800 (PST) Message-ID: <54F4F606.3070104@mu.org> Date: Mon, 02 Mar 2015 18:45:10 -0500 From: Alfred Perlstein User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:31.0) Gecko/20100101 Thunderbird/31.5.0 MIME-Version: 1.0 To: freebsd-current@freebsd.org Subject: Re: Massive libxo-zation that breaks everything References: <54F31510.7050607@hot.ee> <54F34B6E.2040809@astrodoggroup.com> <54F35F29.4000603@astrodoggroup.com> <54F429EF.5050400@freebsd.org> <54F46536.8040607@mu.org> <54F4C03F.7030704@freebsd.org> <54F4C127.5020000@astrodoggroup.com> In-Reply-To: Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Mar 2015 23:41:38 -0000 On 3/2/15 4:57 PM, Adrian Chadd wrote: > .. we can/should do both. > > Just make sure the json/html/xml output is versioned, otherwise you're > going to end up with /exactly the same problems/ you have with the > current format. > +1 -Alfred From owner-freebsd-current@FreeBSD.ORG Mon Mar 2 23:49:57 2015 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E9BF8E6E; Mon, 2 Mar 2015 23:49:57 +0000 (UTC) Received: from mail-ie0-x233.google.com (mail-ie0-x233.google.com [IPv6:2607:f8b0:4001:c03::233]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id AE18BBA3; Mon, 2 Mar 2015 23:49:57 +0000 (UTC) Received: by ierx19 with SMTP id x19so52709628ier.3; Mon, 02 Mar 2015 15:49:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=vHsCoYnt8fm4lByVHNuUSFBJeE7qzH2FEL3+Z6dRCNU=; b=JXRanK2TPV4Yf0OkI+hm+7+zkWvXeINfOUhk+9hQNIQDgnsEoZAbTGnce0FpzFF6O8 YUdrSwVPuqyUe7v0ri0sg1v7f9Pe8j17w1eTRasKv95REoZgQqXQt6v22hhvy7oFhiaq tnYqFmXOwsKtaqbHXfWI3+eMjpB/LXtRF3HOWlNor8HLvHplzn0lJzwn7QerriECXLd+ EyA0s/FZLi5NLzl48uPFAmFLvaoKdTsmq7KVqFyF7e7++egHxfTjljzsa14oZcLc/f8S MAeNQr0wq6pnNoku0fXQQ10ThXAHBfNE4QeIJqaUWClirwyF3VUGZ9XQUIy65MhADquq imDw== MIME-Version: 1.0 X-Received: by 10.107.155.13 with SMTP id d13mr39898709ioe.29.1425340196978; Mon, 02 Mar 2015 15:49:56 -0800 (PST) Sender: adrian.chadd@gmail.com Received: by 10.36.17.66 with HTTP; Mon, 2 Mar 2015 15:49:56 -0800 (PST) In-Reply-To: References: Date: Mon, 2 Mar 2015 15:49:56 -0800 X-Google-Sender-Auth: GszTLDc9d24aTj_AC-IR8XmeAWg Message-ID: Subject: Re: Doing zero-copy stuff in drivers, or "is vm_fault_quick_hold_pages() enough" ? From: Adrian Chadd To: "K. Macy" Content-Type: text/plain; charset=UTF-8 Cc: freebsd-current , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Mar 2015 23:49:58 -0000 On 2 March 2015 at 15:05, K. Macy wrote: > On Mon, Mar 2, 2015 at 12:42 PM, Adrian Chadd wrote: >> Hi, >> >> gonzo@ committed a fix (r278615) to the videocore driver for the >> raspberry pi. The fix involved doing an explicit wire of pages that >> were about to be passed down to the hardware to send to the >> videobuffer hardware. >> >> It turns out that doing vm_fault_quick_hold_pages() wasn't enough - >> the pages weren't being wired, and they may disappear before the >> hardware gets to them. > > > Then your code is buggy or you've hit a bug in the VM. Holding a page > is a short-term wiring. Well, you can look at what's going on in the vchiq code. gonzo made it an explicit wire in order to avoid issues. > Right above vm_page_hold(): > /* > * Keep page from being freed by the page daemon > * much of the same effect as wiring, except much lower > * overhead and should be used only for *very* temporary > * holding ("wiring"). > */ What's the definition of "very temporary holding" ? What's the behavioural difference? -adrian From owner-freebsd-current@FreeBSD.ORG Tue Mar 3 00:12:28 2015 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B460C55A; Tue, 3 Mar 2015 00:12:28 +0000 (UTC) Received: from mail-yh0-x22a.google.com (mail-yh0-x22a.google.com [IPv6:2607:f8b0:4002:c01::22a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6DAE9E2D; Tue, 3 Mar 2015 00:12:28 +0000 (UTC) Received: by yhzz6 with SMTP id z6so16540643yhz.5; Mon, 02 Mar 2015 16:12:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=ZBOXDfpNH5/5TPuolhIszIGBiZQ9Scrk+w26AthIBXI=; b=hzHH1Ax+VVK3VnnsIM0Px/H3t+Y7EoNT7eSS0ONS8Rfj3UA9HryauuikdHt2jbia+f NbxA/9cSaBYxJs+xao9865xQUSZqADpF6iU78sZep+2AokKcSNDWYl/m6WzYViF+2oPb nViIuXYSazbDhFVBgTZRyoXDQ+JesbXkhIM882jFE4uMwBhcrIMUbVnUdm6C0WJ47Va6 PnnN1mLp4yEr3VupJusW3+seB+qjpRUrBQTJoy9sGFd34qBc+hUZpfqZ3a2z7R59eab0 F5WINDu/7N9deIHen71+EoAgAmIabUszFY/SKMzmSP4P0vkC05nlXDux5tI1iAjQQiMr AkEg== MIME-Version: 1.0 X-Received: by 10.236.25.68 with SMTP id y44mr28269936yhy.4.1425341547450; Mon, 02 Mar 2015 16:12:27 -0800 (PST) Sender: kmacybsd@gmail.com Received: by 10.170.76.66 with HTTP; Mon, 2 Mar 2015 16:12:27 -0800 (PST) In-Reply-To: References: Date: Mon, 2 Mar 2015 16:12:27 -0800 X-Google-Sender-Auth: MSiKnxdPMREYCl4AuzX1lBOFQh0 Message-ID: Subject: Re: Doing zero-copy stuff in drivers, or "is vm_fault_quick_hold_pages() enough" ? From: "K. Macy" To: Adrian Chadd Content-Type: text/plain; charset=UTF-8 Cc: freebsd-current , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 03 Mar 2015 00:12:28 -0000 >> Right above vm_page_hold(): >> /* >> * Keep page from being freed by the page daemon >> * much of the same effect as wiring, except much lower >> * overhead and should be used only for *very* temporary >> * holding ("wiring"). >> */ > > What's the definition of "very temporary holding" ? What's the > behavioural difference? Long enough to complete a DMA operation versus the lifetime of an executing program. From owner-freebsd-current@FreeBSD.ORG Tue Mar 3 00:17:04 2015 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 6421E7C8; Tue, 3 Mar 2015 00:17:04 +0000 (UTC) Received: from mail-ig0-x22c.google.com (mail-ig0-x22c.google.com [IPv6:2607:f8b0:4001:c05::22c]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 265D1E61; Tue, 3 Mar 2015 00:17:04 +0000 (UTC) Received: by igal13 with SMTP id l13so20554386iga.1; Mon, 02 Mar 2015 16:17:03 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=O6ma+twoQqaH6CIF07PMVGVcafyydbyRFgPdmcxQ/ik=; b=WJNjyD8JM5yVSjifWXw04itVBGbXCOTSf20q+TcyfBwvk/qOXJTQJgYB7Qzwp90NWu +BFCy5qtOJR8+BEg5VjwptJemlrWr7NDYd6MMdS1HI63Hca+FSM0GsjyMv/Sh6ivebiO TLNVmlZLDrFGjPN44Uj3KRwz+XMalyGGfPdoiGyrid8znimeRJaukdH1HL5EaktdWsGF 7ZAg6bi+Oc0V3nekBHGoQW4ujmPIsFnyrC6iPArJpgy8kOE/b+feVi6w4NzxXPxj9F/E N6nU9ofW+ot476wR+B2n63D/yOvD3DJZYvfIjx/daE32GackFO14pXfS4tsMxvZXI45j SwRA== MIME-Version: 1.0 X-Received: by 10.50.43.201 with SMTP id y9mr25934985igl.6.1425341823612; Mon, 02 Mar 2015 16:17:03 -0800 (PST) Sender: adrian.chadd@gmail.com Received: by 10.36.17.66 with HTTP; Mon, 2 Mar 2015 16:17:03 -0800 (PST) In-Reply-To: References: Date: Mon, 2 Mar 2015 16:17:03 -0800 X-Google-Sender-Auth: LKAbD33Rc4Sd6D7PP2BMOdeFCGQ Message-ID: Subject: Re: Doing zero-copy stuff in drivers, or "is vm_fault_quick_hold_pages() enough" ? From: Adrian Chadd To: "K. Macy" Content-Type: text/plain; charset=UTF-8 Cc: freebsd-current , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 03 Mar 2015 00:17:04 -0000 On 2 March 2015 at 16:12, K. Macy wrote: >>> Right above vm_page_hold(): >>> /* >>> * Keep page from being freed by the page daemon >>> * much of the same effect as wiring, except much lower >>> * overhead and should be used only for *very* temporary >>> * holding ("wiring"). >>> */ >> >> What's the definition of "very temporary holding" ? What's the >> behavioural difference? > > Long enough to complete a DMA operation versus the lifetime of an > executing program. Ok, but is there a specific time length that this should be? A DMA operation to a slow device could be up to hundreds of milliseconds; or seconds if things are really backed up. Using wire instead of hold definitely made things work without having the page disappear from underneath it. Oleksander knows more about the details of that. -adrian From owner-freebsd-current@FreeBSD.ORG Tue Mar 3 00:22:46 2015 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id CC9779B5 for ; Tue, 3 Mar 2015 00:22:46 +0000 (UTC) Received: from mail-lb0-f170.google.com (mail-lb0-f170.google.com [209.85.217.170]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4E927F29 for ; Tue, 3 Mar 2015 00:22:45 +0000 (UTC) Received: by lbvn10 with SMTP id n10so33757181lbv.4 for ; Mon, 02 Mar 2015 16:22:38 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:message-id:date:from:user-agent:mime-version:to :subject:references:in-reply-to:content-type :content-transfer-encoding; bh=XO33tki1AyQBZoDrNQ11Ih0Tf6jH/EWF0HXaIK533ME=; b=fFfQN4gzUYGc1ZTVCZ+QZfo/dZjDmpV82NncOJ3MiKQqOOwzQWEfROkQEGlYYN5ow/ iLPtHW87nah5DLk8dp4fm0DyRHC92DEpNRCrc5zJERYV+WUd6lAmzkNnKf1+doirrZdh 7T0sOs+LmviqDe4G8gmD42LXWsRjj/EMU2jUlxwboMlK4qjVkhlfjjs96RoWw1T7Arvx C67Y9a3lmFtJhvXG2XkHQUF+ybTZf18FPjda0g9NuvKGhBxjYuQG0T4tzDAyAW4lF8r/ 2A8FrLtacxkaJDyEg3tMuY6oc5USQEbPXEZJnwzu4NVMz/bZAq9/Fk6IP4WXCIkk/H6Y +Z5w== X-Gm-Message-State: ALoCoQnBQPJ0SBUG1JPFisE6xWaIDPHr9Fawfniz7LhUfHnKurhkdvqXZl8LVs7vedhhWa/IhUxB X-Received: by 10.152.184.100 with SMTP id et4mr26824028lac.8.1425342158189; Mon, 02 Mar 2015 16:22:38 -0800 (PST) Received: from [192.168.1.2] ([89.169.173.68]) by mx.google.com with ESMTPSA id d6sm2832407lbp.16.2015.03.02.16.22.36 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 02 Mar 2015 16:22:37 -0800 (PST) Message-ID: <54F4FECB.90501@freebsd.org> Date: Tue, 03 Mar 2015 03:22:35 +0300 From: Andrey Chernov User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:31.0) Gecko/20100101 Thunderbird/31.5.0 MIME-Version: 1.0 To: Julian Elischer , Alfred Perlstein , Harrison Grundy , freebsd-current@freebsd.org Subject: Re: Massive libxo-zation that breaks everything References: <54F31510.7050607@hot.ee> <54F34B6E.2040809@astrodoggroup.com> <54F35F29.4000603@astrodoggroup.com> <54F429EF.5050400@freebsd.org> <54F46536.8040607@mu.org> <54F4C03F.7030704@freebsd.org> In-Reply-To: <54F4C03F.7030704@freebsd.org> Content-Type: text/plain; charset=koi8-r Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 03 Mar 2015 00:22:47 -0000 On 02.03.2015 22:55, Julian Elischer wrote: > On 3/2/15 5:27 AM, Alfred Perlstein wrote: >> >> >> On 3/2/15 4:14 AM, Julian Elischer wrote: >>> On 3/1/15 10:49 AM, Harrison Grundy wrote: >>>> Thanks! >>>> >>>> That does seem useful, but I'm not sure I see the reasoning behind >>>> putting into base, over a port or package, since processing XML in base >>>> is a pain, and it can't serve up JSON or HTML without additional >>>> utilities anyway. >>>> >>>> (If I'm reviving a long-settled thing, let me know and I'll drop it. >>>> I'm >>>> trying to understand the use case for this.) >>> >>> To me it would almost seem more useful to have a programmable filter >>> for which you could produce >>> parse grammars to parse the output of various programs.. >>> thus >>> >>> ifconfig -a | xmlize -g ifconfig | your-favourite-xml-parser >>> with a set of grammars in /usr/share/xmlize/ >>> then we could use it for out-of-tree programs as well if we wrote >>> grammars for them.. >>> >>> The sentiment of machine-readable output is nice, but I think it's >>> slightly off target. >>> we shouldn't have to change all out utilities, and it isn't going to >>> help at all with 3rd party apps, >>> e.g. samba stuff. A generally easy to program output grammar parser >>> would be truely useful. >>> and not just for FreeBSD. >>> >>> I've been watching with an uncomfortable feeling, but it's taken me a >>> while to put my >>> finger on what it was.. >>> >>> >> Are you sure it's not the hairs on the back of your neck standing up >> due to NIH? >> >> Juniper has been doing this for years and it's very useful for them. > I'm not saying the ability to generate machine readable output is wrong, > but that the 'unix way' would be to make a filter for it. It seems that > the noisy people don't > agree with me so I will not stand in the way of progress.. I agree. Even if someone starts with json and xml only, it will need some 3rd format soon, and adding any new format have real possibility to break all already existent (like adding json+xml breaks plain text in pipes). Moreover, it violates Unix principle 'one tool == one general function' and lots of other rules like Eric Raymond ones, making each program looks like systemd. It makes harder to merge changes from other BSDs too. Proper way to do this thing is to back out all changes and write completely separate templates-based parser - xml/json writer. -- http://ache.vniz.net/ From owner-freebsd-current@FreeBSD.ORG Tue Mar 3 00:25:42 2015 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C510DD80; Tue, 3 Mar 2015 00:25:42 +0000 (UTC) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 30E58F5D; Tue, 3 Mar 2015 00:25:41 +0000 (UTC) Received: from tom.home (kostik@localhost [127.0.0.1]) by kib.kiev.ua (8.14.9/8.14.9) with ESMTP id t230PaeG079640 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 3 Mar 2015 02:25:36 +0200 (EET) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.9.2 kib.kiev.ua t230PaeG079640 Received: (from kostik@localhost) by tom.home (8.14.9/8.14.9/Submit) id t230PaZK079639; Tue, 3 Mar 2015 02:25:36 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Tue, 3 Mar 2015 02:25:36 +0200 From: Konstantin Belousov To: Adrian Chadd Subject: Re: Doing zero-copy stuff in drivers, or "is vm_fault_quick_hold_pages() enough" ? Message-ID: <20150303002536.GE2379@kib.kiev.ua> References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.23 (2014-03-12) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.0 X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on tom.home Cc: freebsd-current , "K. Macy" , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 03 Mar 2015 00:25:42 -0000 On Mon, Mar 02, 2015 at 04:17:03PM -0800, Adrian Chadd wrote: > On 2 March 2015 at 16:12, K. Macy wrote: > >>> Right above vm_page_hold(): > >>> /* > >>> * Keep page from being freed by the page daemon > >>> * much of the same effect as wiring, except much lower > >>> * overhead and should be used only for *very* temporary > >>> * holding ("wiring"). > >>> */ > >> > >> What's the definition of "very temporary holding" ? What's the > >> behavioural difference? > > > > Long enough to complete a DMA operation versus the lifetime of an > > executing program. > > Ok, but is there a specific time length that this should be? Difference between hold and wire is effectively that held pages are still kept on the page queues, providing potentially uneccessary work for pagedaemon to find them and skip. Wired pages are removed from the queues. This means that holding a page is much cheaper, by the cost of leaving slightly more work to the system. Also, holding a page only requires the page lock, while wiring contend on the page queue lock, in addition to the page lock. > > A DMA operation to a slow device could be up to hundreds of > milliseconds; or seconds if things are really backed up. > > Using wire instead of hold definitely made things work without having > the page disappear from underneath it. Oleksander knows more about the > details of that. Page cannot 'disappear'. The only thing which could happen with the memory page is reuse, when the page is removed from the previous object and re-purposed for some other object, loosing old content. Your terminology suggests that something unrelated happen. From owner-freebsd-current@FreeBSD.ORG Tue Mar 3 00:37:14 2015 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id F2894103; Tue, 3 Mar 2015 00:37:13 +0000 (UTC) Received: from mail-ig0-x22d.google.com (mail-ig0-x22d.google.com [IPv6:2607:f8b0:4001:c05::22d]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B2C63E1; Tue, 3 Mar 2015 00:37:13 +0000 (UTC) Received: by igbhn18 with SMTP id hn18so22318172igb.2; Mon, 02 Mar 2015 16:37:13 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=/ywBmXikn/5JrH1PePyqOs1uXBgi5ufbpnJ1XlzX2A4=; b=G2JdhU4ZYJ94WCukjJldctNbC4WunxmZvy3gKRSoN4YTrEbYdWvT+eccXHhICYZAYH swdtjcEF1AIzaWz7vqcjivMmYK1uE9g7KMp1585Iaw+QswDOgTGy95FppsYJm4KWdqxz 38Os+5EtX/X5FuFzgtzPc3F4RIsNBj2nXLjJ0XdO+fWCI8VVF7yVX6o4C8F6i7guuNP+ ebQp9qZacjGu5ReH0g86bATcCRisJpbtHad4ifsVpRy0VOkxH7bvzzrL0PMfdFUEoLx2 FArys+znk6lhkbcCIP03jTAXut8jSb0g/WwWXEtAzvwQMO6lNHahhK7knIZvQ2r1eSky vq4w== MIME-Version: 1.0 X-Received: by 10.42.188.133 with SMTP id da5mr34081038icb.37.1425343033199; Mon, 02 Mar 2015 16:37:13 -0800 (PST) Sender: adrian.chadd@gmail.com Received: by 10.36.17.66 with HTTP; Mon, 2 Mar 2015 16:37:13 -0800 (PST) In-Reply-To: <20150303002536.GE2379@kib.kiev.ua> References: <20150303002536.GE2379@kib.kiev.ua> Date: Mon, 2 Mar 2015 16:37:13 -0800 X-Google-Sender-Auth: 3Um-zdb7MZ5WGA0o8Yr7A8iCP74 Message-ID: Subject: Re: Doing zero-copy stuff in drivers, or "is vm_fault_quick_hold_pages() enough" ? From: Adrian Chadd To: Konstantin Belousov Content-Type: text/plain; charset=UTF-8 Cc: freebsd-current , "K. Macy" , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 03 Mar 2015 00:37:14 -0000 On 2 March 2015 at 16:25, Konstantin Belousov wrote: >> Ok, but is there a specific time length that this should be? > Difference between hold and wire is effectively that held pages are > still kept on the page queues, providing potentially uneccessary work > for pagedaemon to find them and skip. Wired pages are removed from the > queues. > > This means that holding a page is much cheaper, by the cost of leaving > slightly more work to the system. Also, holding a page only requires the > page lock, while wiring contend on the page queue lock, in addition to > the page lock. Thanks for the description - that makes things a lot clearer! >> >> A DMA operation to a slow device could be up to hundreds of >> milliseconds; or seconds if things are really backed up. >> >> Using wire instead of hold definitely made things work without having >> the page disappear from underneath it. Oleksander knows more about the >> details of that. > > Page cannot 'disappear'. The only thing which could happen with the > memory page is reuse, when the page is removed from the previous object > and re-purposed for some other object, loosing old content. > > Your terminology suggests that something unrelated happen. Yup, and that's what I'm worried about :( -adrian From owner-freebsd-current@FreeBSD.ORG Tue Mar 3 00:48:45 2015 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 154298CD for ; Tue, 3 Mar 2015 00:48:45 +0000 (UTC) Received: from mx1.scaleengine.net (beauharnois2.bhs1.scaleengine.net [142.4.218.15]) by mx1.freebsd.org (Postfix) with ESMTP id DEB08214 for ; Tue, 3 Mar 2015 00:48:44 +0000 (UTC) Received: from [192.168.1.2] (Seawolf.HML3.ScaleEngine.net [209.51.186.28]) (Authenticated sender: allanjude.freebsd@scaleengine.com) by mx1.scaleengine.net (Postfix) with ESMTPSA id 7C7A699DA0 for ; Tue, 3 Mar 2015 00:48:43 +0000 (UTC) Message-ID: <54F504EF.3030303@freebsd.org> Date: Mon, 02 Mar 2015 19:48:47 -0500 From: Allan Jude User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:31.0) Gecko/20100101 Thunderbird/31.5.0 MIME-Version: 1.0 To: freebsd-current@freebsd.org Subject: Re: Massive libxo-zation that breaks everything References: <54F31510.7050607@hot.ee> <54F34B6E.2040809@astrodoggroup.com> <54F35F29.4000603@astrodoggroup.com> <54F429EF.5050400@freebsd.org> <54F46536.8040607@mu.org> <54F4C03F.7030704@freebsd.org> <54F4FECB.90501@freebsd.org> In-Reply-To: <54F4FECB.90501@freebsd.org> Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="hTBgLk1DQhGgiQX5TX39EdfXIT18qClsd" X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 03 Mar 2015 00:48:45 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --hTBgLk1DQhGgiQX5TX39EdfXIT18qClsd Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable On 2015-03-02 19:22, Andrey Chernov wrote: > On 02.03.2015 22:55, Julian Elischer wrote: >> On 3/2/15 5:27 AM, Alfred Perlstein wrote: >>> >>> >>> On 3/2/15 4:14 AM, Julian Elischer wrote: >>>> On 3/1/15 10:49 AM, Harrison Grundy wrote: >>>>> Thanks! >>>>> >>>>> That does seem useful, but I'm not sure I see the reasoning behind >>>>> putting into base, over a port or package, since processing XML in = base >>>>> is a pain, and it can't serve up JSON or HTML without additional >>>>> utilities anyway. >>>>> >>>>> (If I'm reviving a long-settled thing, let me know and I'll drop it= =2E >>>>> I'm >>>>> trying to understand the use case for this.) >>>> >>>> To me it would almost seem more useful to have a programmable filter= >>>> for which you could produce >>>> parse grammars to parse the output of various programs.. >>>> thus >>>> >>>> ifconfig -a | xmlize -g ifconfig | your-favourite-xml-parser >>>> with a set of grammars in /usr/share/xmlize/ >>>> then we could use it for out-of-tree programs as well if we wrote >>>> grammars for them.. >>>> >>>> The sentiment of machine-readable output is nice, but I think it's >>>> slightly off target. >>>> we shouldn't have to change all out utilities, and it isn't going to= >>>> help at all with 3rd party apps, >>>> e.g. samba stuff. A generally easy to program output grammar parser >>>> would be truely useful. >>>> and not just for FreeBSD. >>>> >>>> I've been watching with an uncomfortable feeling, but it's taken me = a >>>> while to put my >>>> finger on what it was.. >>>> >>>> >>> Are you sure it's not the hairs on the back of your neck standing up >>> due to NIH? >>> >>> Juniper has been doing this for years and it's very useful for them. >> I'm not saying the ability to generate machine readable output is wron= g, >> but that the 'unix way' would be to make a filter for it. It seems tha= t >> the noisy people don't >> agree with me so I will not stand in the way of progress.. >=20 > I agree. Even if someone starts with json and xml only, it will need > some 3rd format soon, and adding any new format have real possibility t= o > break all already existent (like adding json+xml breaks plain text in > pipes). Moreover, it violates Unix principle 'one tool =3D=3D one gener= al > function' and lots of other rules like Eric Raymond ones, making each > program looks like systemd. It makes harder to merge changes from other= > BSDs too. > Proper way to do this thing is to back out all changes and write > completely separate templates-based parser - xml/json writer. >=20 Have you actually looked at libxo? It isn't really specific to xmj/json, and could handle adding another format without the need to modify wc xo_open_container("wc"); xo_open_list("file"); xo_open_instance("file"); if (cnt(*argv) !=3D 0) ++errors; xo_close_instance("file"); xo_close_list("file"); xo_close_container("wc"); xo_finish(); with cnt() being the existing functions in wc, but with the printf replaced with an xo wrapper. There is nothing specific to json or xml, that is all handled in libxo. Obviously, more care needs to be taken when converting the utilities to ensure that the existing output is not changed, but it is not quite as invasive as some people seem to think. --=20 Allan Jude --hTBgLk1DQhGgiQX5TX39EdfXIT18qClsd Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (MingW32) iQIcBAEBAgAGBQJU9QTzAAoJEJrBFpNRJZKfL/oQAJSyeCAAu9DEToFS831/3eJQ W5mozxRYp6ve1+BMzeay2fXS9nWQCeU+ShtHB1y1IS01rZZs8+lKU0e9dqFdJOWN SoqufKL12CAieAprso36g176pjpxShHAh9NRCxHeKfwO0vdmHzCcvf0xmOP9/Fjp ecL1nVGxu1pD+DUAHWzrG+PdQHTFTq5ehF4N2Far8CYbI9ikhOSWfQozlPU7obQj cxQ8XvRDvhyhnXbmkg3BrPD8EAeAUzs/sz46XtFiYMttuKb1m7FDFfv4ZI+Nz9Rd gEdLBhS2aN8eRCPkLGmcCC6KM7ReHMAHcteSWFGUbgfpdRkQduz/h4Gz66THXNkl 3NaczV4ADKKMjsclPx/lufvZQ9NWkLFMr4rjcp0mhgZ90Q32qj5PHmgeV3e571xT qRiyVfahzhCbRRxPxqcuShSCA6KOlDIFQsl/xseK/FIzrs4l6zuKOxRugL4Cbp9S xTLBAzdHVqWIQetXaDs9KGfW+Rb3bJo29Y44JaW9Ojffs1G0C0ttc7PIG8F/9J7i 78FnA8dHuFg2AJTXjM0Be7QBXaXcjIuyyM6BHBi/GbF9tMoo6tP4ogClVO9ypdEv s7bdV+OvPwFSTCk4nfcXVJZC8AJ9WnEghZgaQh5v26QyOgE5GzyP3RudFTmfGvvg GhiOCD2p38BODspHN/jH =CiIg -----END PGP SIGNATURE----- --hTBgLk1DQhGgiQX5TX39EdfXIT18qClsd-- From owner-freebsd-current@FreeBSD.ORG Tue Mar 3 01:03:47 2015 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1F356DFE for ; Tue, 3 Mar 2015 01:03:47 +0000 (UTC) Received: from mail-yk0-x22b.google.com (mail-yk0-x22b.google.com [IPv6:2607:f8b0:4002:c07::22b]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C8B965E2 for ; Tue, 3 Mar 2015 01:03:46 +0000 (UTC) Received: by ykt10 with SMTP id 10so15170999ykt.1 for ; Mon, 02 Mar 2015 17:03:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=DuMJBVRCbaJfRX9O0TWQIDYHphUo3JePPrgkibFn9nk=; b=aUb2SkkiIlESlytKY+Ns/Eo49xuIRLmJMPnSg7QQqXTPmyoYmUY16Ic6HQ668DQ3r2 WCcuw1AzE+mTZfF652d6cqXwnB2b7ptvqz2xZ/fAy3fo/qPSqcjIDFwYSN1o9LFJe8E3 2u3bPYzzYHP/bH0xm00/VJ1Yy1s4xXz0XoDOPq3Er+eXnN1CMLJ6dy9t2+xSOZJ2D7Am fFWcJ5trG3UEmlIer0Yv2MKISzXBTrfTqkBBFGblZYAF19Bq0LA16+VEXp/vph9c61bv CftRflJHUmSECDhHcIx7JHltmXBa3bsfqq6bFplEQ8whJAObyo3zk2sZ6bvwWHIlH/1e cAcQ== MIME-Version: 1.0 X-Received: by 10.236.26.143 with SMTP id c15mr28442190yha.192.1425344626032; Mon, 02 Mar 2015 17:03:46 -0800 (PST) Received: by 10.170.60.85 with HTTP; Mon, 2 Mar 2015 17:03:45 -0800 (PST) In-Reply-To: <54F4F606.3070104@mu.org> References: <54F31510.7050607@hot.ee> <54F34B6E.2040809@astrodoggroup.com> <54F35F29.4000603@astrodoggroup.com> <54F429EF.5050400@freebsd.org> <54F46536.8040607@mu.org> <54F4C03F.7030704@freebsd.org> <54F4C127.5020000@astrodoggroup.com> <54F4F606.3070104@mu.org> Date: Mon, 2 Mar 2015 17:03:45 -0800 Message-ID: Subject: Re: Massive libxo-zation that breaks everything From: Mehmet Erol Sanliturk To: Alfred Perlstein Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 Cc: freebsd-current X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 03 Mar 2015 01:03:47 -0000 On Mon, Mar 2, 2015 at 3:45 PM, Alfred Perlstein wrote: > > > On 3/2/15 4:57 PM, Adrian Chadd wrote: > >> .. we can/should do both. >> >> Just make sure the json/html/xml output is versioned, otherwise you're >> going to end up with /exactly the same problems/ you have with the >> current format. >> >> +1 > > > -Alfred > > _______________________________________________ > > To process the generated XML or any other structured file automatically , at least the following three items should be present in a suitable enclosed owner item : generated_by_the_program = "..." version_of_the_file = "..." kind_of_the_file = { "Data" | "Result" | "Variables" | ... } where "Data" for statistical analyses , "Result" as "Success" or "Failure" "Variables" as Internal values of some variables . By using these values of items , the rest of the file will be parsed accordingly . These item values may be contained in the generated file name , but this requires another routine to process file names to extract these information , which is very unnecessary and does not save anything other than cause much trouble . Thank you very much . Mehmet Erol Sanliturk From owner-freebsd-current@FreeBSD.ORG Tue Mar 3 01:30:33 2015 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id ACA601BC; Tue, 3 Mar 2015 01:30:33 +0000 (UTC) Received: from elvis.mu.org (elvis.mu.org [IPv6:2001:470:1f05:b76::196]) by mx1.freebsd.org (Postfix) with ESMTP id 96D5483B; Tue, 3 Mar 2015 01:30:33 +0000 (UTC) Received: from [100.81.16.196] (173.sub-70-197-67.myvzw.com [70.197.67.173]) by elvis.mu.org (Postfix) with ESMTPSA id 4B71C341F861; Mon, 2 Mar 2015 17:30:33 -0800 (PST) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (1.0) Subject: Re: Massive libxo-zation that breaks everything From: Alfred Perlstein X-Mailer: iPhone Mail (12B440) In-Reply-To: <54F4FECB.90501@freebsd.org> Date: Mon, 2 Mar 2015 17:30:31 -0800 Content-Transfer-Encoding: quoted-printable Message-Id: <197AA5DC-0591-4F71-BF10-51A5D8104C11@mu.org> References: <54F31510.7050607@hot.ee> <54F34B6E.2040809@astrodoggroup.com> <54F35F29.4000603@astrodoggroup.com> <54F429EF.5050400@freebsd.org> <54F46536.8040607@mu.org> <54F4C03F.7030704@freebsd.org> <54F4FECB.90501@freebsd.org> To: Andrey Chernov Cc: Harrison Grundy , "freebsd-current@freebsd.org" X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 03 Mar 2015 01:30:33 -0000 > On Mar 2, 2015, at 4:22 PM, Andrey Chernov wrote: >=20 >> On 02.03.2015 22:55, Julian Elischer wrote: >>> On 3/2/15 5:27 AM, Alfred Perlstein wrote: >>>=20 >>>=20 >>>> On 3/2/15 4:14 AM, Julian Elischer wrote: >>>>> On 3/1/15 10:49 AM, Harrison Grundy wrote: >>>>> Thanks! >>>>>=20 >>>>> That does seem useful, but I'm not sure I see the reasoning behind >>>>> putting into base, over a port or package, since processing XML in bas= e >>>>> is a pain, and it can't serve up JSON or HTML without additional >>>>> utilities anyway. >>>>>=20 >>>>> (If I'm reviving a long-settled thing, let me know and I'll drop it. >>>>> I'm >>>>> trying to understand the use case for this.) >>>>=20 >>>> To me it would almost seem more useful to have a programmable filter >>>> for which you could produce >>>> parse grammars to parse the output of various programs.. >>>> thus >>>>=20 >>>> ifconfig -a | xmlize -g ifconfig | your-favourite-xml-parser >>>> with a set of grammars in /usr/share/xmlize/ >>>> then we could use it for out-of-tree programs as well if we wrote >>>> grammars for them.. >>>>=20 >>>> The sentiment of machine-readable output is nice, but I think it's >>>> slightly off target. >>>> we shouldn't have to change all out utilities, and it isn't going to >>>> help at all with 3rd party apps, >>>> e.g. samba stuff. A generally easy to program output grammar parser >>>> would be truely useful. >>>> and not just for FreeBSD. >>>>=20 >>>> I've been watching with an uncomfortable feeling, but it's taken me a >>>> while to put my >>>> finger on what it was.. >>> Are you sure it's not the hairs on the back of your neck standing up >>> due to NIH? >>>=20 >>> Juniper has been doing this for years and it's very useful for them. >> I'm not saying the ability to generate machine readable output is wrong, >> but that the 'unix way' would be to make a filter for it. It seems that >> the noisy people don't >> agree with me so I will not stand in the way of progress.. >=20 > I agree. Even if someone starts with json and xml only, it will need > some 3rd format soon, and adding any new format have real possibility to > break all already existent (like adding json+xml breaks plain text in > pipes). Moreover, it violates Unix principle 'one tool =3D=3D one general > function' and lots of other rules like Eric Raymond ones, making each > program looks like systemd. It makes harder to merge changes from other > BSDs too. > Proper way to do this thing is to back out all changes and write > completely separate templates-based parser - xml/json writer. Read the library. It doesn't care what output format it needs. It is up to t= he translation layer to do it. You could even do a csv format or most any ot= her structured output format without changing the userland utils.=20 >=20 > --=20 > http://ache.vniz.net/ > _______________________________________________ > freebsd-current@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-current > To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org"= >=20 From owner-freebsd-current@FreeBSD.ORG Tue Mar 3 01:37:10 2015 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 487B1356 for ; Tue, 3 Mar 2015 01:37:10 +0000 (UTC) Received: from mail-la0-f54.google.com (mail-la0-f54.google.com [209.85.215.54]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E192190B for ; Tue, 3 Mar 2015 01:37:09 +0000 (UTC) Received: by labhs14 with SMTP id hs14so34290406lab.1 for ; Mon, 02 Mar 2015 17:37:01 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:message-id:date:from:user-agent:mime-version:to :cc:subject:references:in-reply-to:content-type :content-transfer-encoding; bh=0Pfm1dXvYZ9qAlV+q52IP/TCSc8UfX7FWMV/Prp8LZA=; b=BO6YtvMJLExHE5MLompIAOkOPYDeahM+GvTTlf+Ok20iRAT7EaaCyuKdX5olhzfLEH QRkI1+OqdQqLYJhOvOrO1VJcEUk94a4SKUaIkdLD2RPXwvRoyBWD0+Rw1LQHBOGJkjK3 x5Wx2ReB96CqNTne1fUIL6j0rfMDY5VwXgK9Te+vrJ57NEef07GAlAK+tIwpTkYawRh7 zFS6kG7zjoAdI/Qu+0QsUJJF8OOJt4r7c3uCT4fWrngKetZ3s1u3eZa8WfBTErf5/Xf0 Tb36BFOUC0HatxrpyNxijcDFkE4LIGINTh3Nnufm5ERG6z52DblVqykQZ0UD07iKpTM8 EjIg== X-Gm-Message-State: ALoCoQlgfgwuKNcDJ6YyZWi13XC2SiETKi63c+49HUj/7orLSZF9gHQHvcaQmpJnQZSteHc3R5Vd X-Received: by 10.152.88.71 with SMTP id be7mr5802491lab.119.1425346621836; Mon, 02 Mar 2015 17:37:01 -0800 (PST) Received: from [192.168.1.2] ([89.169.173.68]) by mx.google.com with ESMTPSA id ks4sm2852119lac.23.2015.03.02.17.37.00 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 02 Mar 2015 17:37:01 -0800 (PST) Message-ID: <54F5103C.4020108@freebsd.org> Date: Tue, 03 Mar 2015 04:37:00 +0300 From: Andrey Chernov User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:31.0) Gecko/20100101 Thunderbird/31.5.0 MIME-Version: 1.0 To: Alfred Perlstein Subject: Re: Massive libxo-zation that breaks everything References: <54F31510.7050607@hot.ee> <54F34B6E.2040809@astrodoggroup.com> <54F35F29.4000603@astrodoggroup.com> <54F429EF.5050400@freebsd.org> <54F46536.8040607@mu.org> <54F4C03F.7030704@freebsd.org> <54F4FECB.90501@freebsd.org> <197AA5DC-0591-4F71-BF10-51A5D8104C11@mu.org> In-Reply-To: <197AA5DC-0591-4F71-BF10-51A5D8104C11@mu.org> Content-Type: text/plain; charset=koi8-r Content-Transfer-Encoding: 7bit Cc: Harrison Grundy , "freebsd-current@freebsd.org" X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 03 Mar 2015 01:37:10 -0000 On 03.03.2015 4:30, Alfred Perlstein wrote: > > >> On Mar 2, 2015, at 4:22 PM, Andrey Chernov wrote: >> >>> On 02.03.2015 22:55, Julian Elischer wrote: >>>> On 3/2/15 5:27 AM, Alfred Perlstein wrote: >>>> >>>> >>>>> On 3/2/15 4:14 AM, Julian Elischer wrote: >>>>>> On 3/1/15 10:49 AM, Harrison Grundy wrote: >>>>>> Thanks! >>>>>> >>>>>> That does seem useful, but I'm not sure I see the reasoning behind >>>>>> putting into base, over a port or package, since processing XML in base >>>>>> is a pain, and it can't serve up JSON or HTML without additional >>>>>> utilities anyway. >>>>>> >>>>>> (If I'm reviving a long-settled thing, let me know and I'll drop it. >>>>>> I'm >>>>>> trying to understand the use case for this.) >>>>> >>>>> To me it would almost seem more useful to have a programmable filter >>>>> for which you could produce >>>>> parse grammars to parse the output of various programs.. >>>>> thus >>>>> >>>>> ifconfig -a | xmlize -g ifconfig | your-favourite-xml-parser >>>>> with a set of grammars in /usr/share/xmlize/ >>>>> then we could use it for out-of-tree programs as well if we wrote >>>>> grammars for them.. >>>>> >>>>> The sentiment of machine-readable output is nice, but I think it's >>>>> slightly off target. >>>>> we shouldn't have to change all out utilities, and it isn't going to >>>>> help at all with 3rd party apps, >>>>> e.g. samba stuff. A generally easy to program output grammar parser >>>>> would be truely useful. >>>>> and not just for FreeBSD. >>>>> >>>>> I've been watching with an uncomfortable feeling, but it's taken me a >>>>> while to put my >>>>> finger on what it was.. >>>> Are you sure it's not the hairs on the back of your neck standing up >>>> due to NIH? >>>> >>>> Juniper has been doing this for years and it's very useful for them. >>> I'm not saying the ability to generate machine readable output is wrong, >>> but that the 'unix way' would be to make a filter for it. It seems that >>> the noisy people don't >>> agree with me so I will not stand in the way of progress.. >> >> I agree. Even if someone starts with json and xml only, it will need >> some 3rd format soon, and adding any new format have real possibility to >> break all already existent (like adding json+xml breaks plain text in >> pipes). Moreover, it violates Unix principle 'one tool == one general >> function' and lots of other rules like Eric Raymond ones, making each >> program looks like systemd. It makes harder to merge changes from other >> BSDs too. >> Proper way to do this thing is to back out all changes and write >> completely separate templates-based parser - xml/json writer. > > > Read the library. It doesn't care what output format it needs. It is up to the translation layer to do it. You could even do a csv format or most any other structured output format without changing the userland utils. I am happy the library can do it. So please stop to change userland utils and back out all libxo changes from them. My concern is userland utils, feel free to implement anything you need/want without changing them in this ugly way. -- http://ache.vniz.net/ From owner-freebsd-current@FreeBSD.ORG Tue Mar 3 01:37:56 2015 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3DE404E2 for ; Tue, 3 Mar 2015 01:37:56 +0000 (UTC) Received: from mail-la0-f52.google.com (mail-la0-f52.google.com [209.85.215.52]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A7F2291E for ; Tue, 3 Mar 2015 01:37:55 +0000 (UTC) Received: by labhs14 with SMTP id hs14so34339752lab.4 for ; Mon, 02 Mar 2015 17:37:54 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:message-id:date:from:user-agent:mime-version:to :subject:references:in-reply-to:content-type; bh=UVEka7y+KKggYVhwuuMBDMy/l3b1YQMJ0B3dc803r+0=; b=HXzJXUni+qMNI3hW4cOXTt01iBAUchqhmvd4EXnPGvE3c211VrdUvk7ocKcC/DInZI kXcJn8m5Ajj78lyd+yOMVvJ/dSoOwnPhxzolmtwjUaclEzKE/OqRbL2C7EWrKwW81BYi e73Yxd9bKebCx3NeGd7ZSAvkkvjkDITNk7bqoKq0OQfs+KOd+akDvxUejAHgMLUL/BZB wfhSBC71wbOhcClGpdMeyB+H8kmQs8poETkRXbnGpgchHb9cv6are2dg3dlH2SFuz4Zi 7Ob7BtoXjaC5L5q5RRUKbiuwYIPMbLW1oMVUNy8uMlrRTm+VmnVVu8fLVUiUrlhuNHTR 7d1A== X-Gm-Message-State: ALoCoQlolnqQQ//gB9hNl/exOXJNbi4l8Z8O40Kbygd/0IVnzXwgOYsBOQE6ksnHqDmdrYCWkSHV X-Received: by 10.152.184.198 with SMTP id ew6mr7774058lac.46.1425346337395; Mon, 02 Mar 2015 17:32:17 -0800 (PST) Received: from [192.168.1.2] ([89.169.173.68]) by mx.google.com with ESMTPSA id ci4sm2852650lad.16.2015.03.02.17.32.15 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 02 Mar 2015 17:32:16 -0800 (PST) Message-ID: <54F50F15.4050300@freebsd.org> Date: Tue, 03 Mar 2015 04:32:05 +0300 From: Andrey Chernov User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:31.0) Gecko/20100101 Thunderbird/31.5.0 MIME-Version: 1.0 To: Allan Jude , freebsd-current@freebsd.org Subject: Re: Massive libxo-zation that breaks everything References: <54F31510.7050607@hot.ee> <54F34B6E.2040809@astrodoggroup.com> <54F35F29.4000603@astrodoggroup.com> <54F429EF.5050400@freebsd.org> <54F46536.8040607@mu.org> <54F4C03F.7030704@freebsd.org> <54F4FECB.90501@freebsd.org> <54F504EF.3030303@freebsd.org> In-Reply-To: <54F504EF.3030303@freebsd.org> Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="x7vPIp1MaM7fxIJd0xniGlIruAErHh1qO" X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 03 Mar 2015 01:37:56 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --x7vPIp1MaM7fxIJd0xniGlIruAErHh1qO Content-Type: text/plain; charset=koi8-r Content-Transfer-Encoding: quoted-printable On 03.03.2015 3:48, Allan Jude wrote: > On 2015-03-02 19:22, Andrey Chernov wrote: >> On 02.03.2015 22:55, Julian Elischer wrote: >>> On 3/2/15 5:27 AM, Alfred Perlstein wrote: >>>> >>>> >>>> On 3/2/15 4:14 AM, Julian Elischer wrote: >>>>> On 3/1/15 10:49 AM, Harrison Grundy wrote: >>>>>> Thanks! >>>>>> >>>>>> That does seem useful, but I'm not sure I see the reasoning behind= >>>>>> putting into base, over a port or package, since processing XML in= base >>>>>> is a pain, and it can't serve up JSON or HTML without additional >>>>>> utilities anyway. >>>>>> >>>>>> (If I'm reviving a long-settled thing, let me know and I'll drop i= t. >>>>>> I'm >>>>>> trying to understand the use case for this.) >>>>> >>>>> To me it would almost seem more useful to have a programmable filte= r >>>>> for which you could produce >>>>> parse grammars to parse the output of various programs.. >>>>> thus >>>>> >>>>> ifconfig -a | xmlize -g ifconfig | your-favourite-xml-parser >>>>> with a set of grammars in /usr/share/xmlize/ >>>>> then we could use it for out-of-tree programs as well if we wrote >>>>> grammars for them.. >>>>> >>>>> The sentiment of machine-readable output is nice, but I think it's >>>>> slightly off target. >>>>> we shouldn't have to change all out utilities, and it isn't going t= o >>>>> help at all with 3rd party apps, >>>>> e.g. samba stuff. A generally easy to program output grammar parser= >>>>> would be truely useful. >>>>> and not just for FreeBSD. >>>>> >>>>> I've been watching with an uncomfortable feeling, but it's taken me= a >>>>> while to put my >>>>> finger on what it was.. >>>>> >>>>> >>>> Are you sure it's not the hairs on the back of your neck standing up= >>>> due to NIH? >>>> >>>> Juniper has been doing this for years and it's very useful for them.= >>> I'm not saying the ability to generate machine readable output is wro= ng, >>> but that the 'unix way' would be to make a filter for it. It seems th= at >>> the noisy people don't >>> agree with me so I will not stand in the way of progress.. >> >> I agree. Even if someone starts with json and xml only, it will need >> some 3rd format soon, and adding any new format have real possibility = to >> break all already existent (like adding json+xml breaks plain text in >> pipes). Moreover, it violates Unix principle 'one tool =3D=3D one gene= ral >> function' and lots of other rules like Eric Raymond ones, making each >> program looks like systemd. It makes harder to merge changes from othe= r >> BSDs too. >> Proper way to do this thing is to back out all changes and write >> completely separate templates-based parser - xml/json writer. >> >=20 > Have you actually looked at libxo? It isn't really specific to xmj/json= , > and could handle adding another format without the need to modify wc >=20 > xo_open_container("wc"); > xo_open_list("file"); > xo_open_instance("file"); > if (cnt(*argv) !=3D 0) > ++errors; > xo_close_instance("file"); > xo_close_list("file"); > xo_close_container("wc"); > xo_finish(); >=20 >=20 > with cnt() being the existing functions in wc, but with the printf > replaced with an xo wrapper. There is nothing specific to json or xml, > that is all handled in libxo. So, why you ever need to modify wc? Just load wc inside your json/xml/etc writer, replacing its printf at the ld-elf.so level. If you can't get json/xml/whatever from wc directly, it is not wc problem, so don't touch it. The problem is on your side, feel free to implement anything you need without affecting source code of others. > Obviously, more care needs to be taken when converting the utilities to= > ensure that the existing output is not changed, but it is not quite as > invasive as some people seem to think. I think yes, it is very invasive and leads to dead end as architectural design. --=20 http://ache.vniz.net/ --x7vPIp1MaM7fxIJd0xniGlIruAErHh1qO Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- iQEcBAEBCAAGBQJU9Q8fAAoJEKUckv0MjfbK5ioH/0Ds1EvyHevGoLq6KIhZHhcS CwicLW445TzI5IkvPXtSmxzVv0vjT6ArDluDfB2mDTFTY7ktqfS73Tk1IhpGDSht 4g4Q6QZwXg3s6DudyH98UCJdZLg2u/vfpniwdYXqhJqbDdRvoktGqixx9uzJUk/k GxE6QwJu4sJFesm1y5PrEYo2hkCPUw2RDpElefwh4pgavlu0xgRDdcZA3sNafgsn NwJYC9YDTwhp5/Q9FDiDYL/I6W7GiHG/LMLOkVQ0EuhvInRrNXxeSwHCkhZCy1c0 GMRRCCGr//y+XAQN1fgkjoxoCXeRdcXFF9eAZbB3wLQIfP5Dof/RbxeY4GDT7Fo= =1MiE -----END PGP SIGNATURE----- --x7vPIp1MaM7fxIJd0xniGlIruAErHh1qO-- From owner-freebsd-current@FreeBSD.ORG Tue Mar 3 01:45:59 2015 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 94A4571A; Tue, 3 Mar 2015 01:45:59 +0000 (UTC) Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by mx1.freebsd.org (Postfix) with ESMTP id 75F0EA05; Tue, 3 Mar 2015 01:45:59 +0000 (UTC) Received: from [100.81.16.196] (173.sub-70-197-67.myvzw.com [70.197.67.173]) by elvis.mu.org (Postfix) with ESMTPSA id BA5FC341F867; Mon, 2 Mar 2015 17:45:58 -0800 (PST) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (1.0) Subject: Re: Massive libxo-zation that breaks everything From: Alfred Perlstein X-Mailer: iPhone Mail (12B440) In-Reply-To: <54F5103C.4020108@freebsd.org> Date: Mon, 2 Mar 2015 17:45:56 -0800 Content-Transfer-Encoding: quoted-printable Message-Id: References: <54F31510.7050607@hot.ee> <54F34B6E.2040809@astrodoggroup.com> <54F35F29.4000603@astrodoggroup.com> <54F429EF.5050400@freebsd.org> <54F46536.8040607@mu.org> <54F4C03F.7030704@freebsd.org> <54F4FECB.90501@freebsd.org> <197AA5DC-0591-4F71-BF10-51A5D8104C11@mu.org> <54F5103C.4020108@freebsd.org> To: Andrey Chernov Cc: Harrison Grundy , "freebsd-current@freebsd.org" X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 03 Mar 2015 01:45:59 -0000 > On Mar 2, 2015, at 5:37 PM, Andrey Chernov wrote: >=20 >> On 03.03.2015 4:30, Alfred Perlstein wrote: >>=20 >>=20 >>>> On Mar 2, 2015, at 4:22 PM, Andrey Chernov wrote: >>>>=20 >>>>> On 02.03.2015 22:55, Julian Elischer wrote: >>>>> On 3/2/15 5:27 AM, Alfred Perlstein wrote: >>>>>=20 >>>>>=20 >>>>>>> On 3/2/15 4:14 AM, Julian Elischer wrote: >>>>>>> On 3/1/15 10:49 AM, Harrison Grundy wrote: >>>>>>> Thanks! >>>>>>>=20 >>>>>>> That does seem useful, but I'm not sure I see the reasoning behind >>>>>>> putting into base, over a port or package, since processing XML in b= ase >>>>>>> is a pain, and it can't serve up JSON or HTML without additional >>>>>>> utilities anyway. >>>>>>>=20 >>>>>>> (If I'm reviving a long-settled thing, let me know and I'll drop it.= >>>>>>> I'm >>>>>>> trying to understand the use case for this.) >>>>>>=20 >>>>>> To me it would almost seem more useful to have a programmable filter >>>>>> for which you could produce >>>>>> parse grammars to parse the output of various programs.. >>>>>> thus >>>>>>=20 >>>>>> ifconfig -a | xmlize -g ifconfig | your-favourite-xml-parser >>>>>> with a set of grammars in /usr/share/xmlize/ >>>>>> then we could use it for out-of-tree programs as well if we wrote >>>>>> grammars for them.. >>>>>>=20 >>>>>> The sentiment of machine-readable output is nice, but I think it's >>>>>> slightly off target. >>>>>> we shouldn't have to change all out utilities, and it isn't going to >>>>>> help at all with 3rd party apps, >>>>>> e.g. samba stuff. A generally easy to program output grammar parser >>>>>> would be truely useful. >>>>>> and not just for FreeBSD. >>>>>>=20 >>>>>> I've been watching with an uncomfortable feeling, but it's taken me a= >>>>>> while to put my >>>>>> finger on what it was.. >>>>> Are you sure it's not the hairs on the back of your neck standing up >>>>> due to NIH? >>>>>=20 >>>>> Juniper has been doing this for years and it's very useful for them. >>>> I'm not saying the ability to generate machine readable output is wrong= , >>>> but that the 'unix way' would be to make a filter for it. It seems that= >>>> the noisy people don't >>>> agree with me so I will not stand in the way of progress.. >>>=20 >>> I agree. Even if someone starts with json and xml only, it will need >>> some 3rd format soon, and adding any new format have real possibility to= >>> break all already existent (like adding json+xml breaks plain text in >>> pipes). Moreover, it violates Unix principle 'one tool =3D=3D one genera= l >>> function' and lots of other rules like Eric Raymond ones, making each >>> program looks like systemd. It makes harder to merge changes from other >>> BSDs too. >>> Proper way to do this thing is to back out all changes and write >>> completely separate templates-based parser - xml/json writer. >>=20 >>=20 >> Read the library. It doesn't care what output format it needs. It is up t= o the translation layer to do it. You could even do a csv format or most any= other structured output format without changing the userland utils. >=20 > I am happy the library can do it. So please stop to change userland > utils and back out all libxo changes from them. My concern is userland > utils, feel free to implement anything you need/want without changing > them in this ugly way. The responsibility is on you to provide something better, both the architect= ure AND code. So if you want it backed out, then write something better. Oth= erwise step back and let progress happen.=20 -Alfred.=20 From owner-freebsd-current@FreeBSD.ORG Tue Mar 3 02:27:45 2015 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D2337D21 for ; Tue, 3 Mar 2015 02:27:45 +0000 (UTC) Received: from mail-lb0-f176.google.com (mail-lb0-f176.google.com [209.85.217.176]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5098DDD2 for ; Tue, 3 Mar 2015 02:27:45 +0000 (UTC) Received: by lbiv13 with SMTP id v13so15419388lbi.11 for ; Mon, 02 Mar 2015 18:27:42 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:message-id:date:from:user-agent:mime-version:to :cc:subject:references:in-reply-to:content-type :content-transfer-encoding; bh=ngQ+eqq78CHmlc+jbfiKm1t3V8iIQUBudHqkWMgqSf8=; b=SCQZOBm5YWHnYptfoeDx7n7L7UsEP2/TuzVilpi3Wf5xuIQX0WbkinH3wReccy2YF/ iotWRfxQAu9tUtsRTb/F8tqOQmcoavFAHNtehuEbqnqyxhiYuFJnvFCOw6TBl60Mhjxy o3Az0B5rwZvET5cOZDINMgcmFV1jJqrBp7rJR90E2WrY4xev+YkjFDHp1hrGpec9aOGr 7rftz+LOlKCvbpRbyrHWjyyIjtvfeOBd5PI5vHfHrMi/Fl7t1eC0/83D9whxFu+lpC1j m40t43V7hh31EdHHlKLLwDxXs/ynWkVkeid+zNi5kG/fgymsTXw6yYnHpAP7owhn3hMx IA3w== X-Gm-Message-State: ALoCoQnNUaMsaSdEiGlYlsvV00cCh989II11cqWkmWeERCX7/UtiyX/sWVs0mHSizOT+X6or/A8t X-Received: by 10.152.206.70 with SMTP id lm6mr27502420lac.35.1425349662633; Mon, 02 Mar 2015 18:27:42 -0800 (PST) Received: from [192.168.1.2] ([89.169.173.68]) by mx.google.com with ESMTPSA id i13sm2863961lab.38.2015.03.02.18.27.41 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 02 Mar 2015 18:27:41 -0800 (PST) Message-ID: <54F51C1C.1050609@freebsd.org> Date: Tue, 03 Mar 2015 05:27:40 +0300 From: Andrey Chernov User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:31.0) Gecko/20100101 Thunderbird/31.5.0 MIME-Version: 1.0 To: Alfred Perlstein Subject: Re: Massive libxo-zation that breaks everything References: <54F31510.7050607@hot.ee> <54F34B6E.2040809@astrodoggroup.com> <54F35F29.4000603@astrodoggroup.com> <54F429EF.5050400@freebsd.org> <54F46536.8040607@mu.org> <54F4C03F.7030704@freebsd.org> <54F4FECB.90501@freebsd.org> <197AA5DC-0591-4F71-BF10-51A5D8104C11@mu.org> <54F5103C.4020108@freebsd.org> In-Reply-To: Content-Type: text/plain; charset=koi8-r Content-Transfer-Encoding: 7bit Cc: Harrison Grundy , "freebsd-current@freebsd.org" X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 03 Mar 2015 02:27:46 -0000 On 03.03.2015 4:45, Alfred Perlstein wrote: > > >> On Mar 2, 2015, at 5:37 PM, Andrey Chernov wrote: >> >>> On 03.03.2015 4:30, Alfred Perlstein wrote: >>> >>> >>>>> On Mar 2, 2015, at 4:22 PM, Andrey Chernov wrote: >>>>> >>>>>> On 02.03.2015 22:55, Julian Elischer wrote: >>>>>> On 3/2/15 5:27 AM, Alfred Perlstein wrote: >>>>>> >>>>>> >>>>>>>> On 3/2/15 4:14 AM, Julian Elischer wrote: >>>>>>>> On 3/1/15 10:49 AM, Harrison Grundy wrote: >>>>>>>> Thanks! >>>>>>>> >>>>>>>> That does seem useful, but I'm not sure I see the reasoning behind >>>>>>>> putting into base, over a port or package, since processing XML in base >>>>>>>> is a pain, and it can't serve up JSON or HTML without additional >>>>>>>> utilities anyway. >>>>>>>> >>>>>>>> (If I'm reviving a long-settled thing, let me know and I'll drop it. >>>>>>>> I'm >>>>>>>> trying to understand the use case for this.) >>>>>>> >>>>>>> To me it would almost seem more useful to have a programmable filter >>>>>>> for which you could produce >>>>>>> parse grammars to parse the output of various programs.. >>>>>>> thus >>>>>>> >>>>>>> ifconfig -a | xmlize -g ifconfig | your-favourite-xml-parser >>>>>>> with a set of grammars in /usr/share/xmlize/ >>>>>>> then we could use it for out-of-tree programs as well if we wrote >>>>>>> grammars for them.. >>>>>>> >>>>>>> The sentiment of machine-readable output is nice, but I think it's >>>>>>> slightly off target. >>>>>>> we shouldn't have to change all out utilities, and it isn't going to >>>>>>> help at all with 3rd party apps, >>>>>>> e.g. samba stuff. A generally easy to program output grammar parser >>>>>>> would be truely useful. >>>>>>> and not just for FreeBSD. >>>>>>> >>>>>>> I've been watching with an uncomfortable feeling, but it's taken me a >>>>>>> while to put my >>>>>>> finger on what it was.. >>>>>> Are you sure it's not the hairs on the back of your neck standing up >>>>>> due to NIH? >>>>>> >>>>>> Juniper has been doing this for years and it's very useful for them. >>>>> I'm not saying the ability to generate machine readable output is wrong, >>>>> but that the 'unix way' would be to make a filter for it. It seems that >>>>> the noisy people don't >>>>> agree with me so I will not stand in the way of progress.. >>>> >>>> I agree. Even if someone starts with json and xml only, it will need >>>> some 3rd format soon, and adding any new format have real possibility to >>>> break all already existent (like adding json+xml breaks plain text in >>>> pipes). Moreover, it violates Unix principle 'one tool == one general >>>> function' and lots of other rules like Eric Raymond ones, making each >>>> program looks like systemd. It makes harder to merge changes from other >>>> BSDs too. >>>> Proper way to do this thing is to back out all changes and write >>>> completely separate templates-based parser - xml/json writer. >>> >>> >>> Read the library. It doesn't care what output format it needs. It is up to the translation layer to do it. You could even do a csv format or most any other structured output format without changing the userland utils. >> >> I am happy the library can do it. So please stop to change userland >> utils and back out all libxo changes from them. My concern is userland >> utils, feel free to implement anything you need/want without changing >> them in this ugly way. > > > The responsibility is on you to provide something better, both the architecture AND code. So if you want it backed out, then write something better. Otherwise step back and let progress happen. > As it seems you know a lot about my responsibility and duty, I am very surprised by your broad interests. If you let me to speak for myself a bit, I can tell what I feel. In this particular case my responsibility is just to give good advice at the road fork with one way leads to obvious hell, nothing more. I already express my opinion from the technical point of view and don't want to participate in the flame war, so continue this thread without me, please. -- http://ache.vniz.net/ From owner-freebsd-current@FreeBSD.ORG Tue Mar 3 02:55:06 2015 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 9A25F3B0; Tue, 3 Mar 2015 02:55:06 +0000 (UTC) Received: from troutmask.apl.washington.edu (troutmask.apl.washington.edu [128.95.76.21]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "troutmask", Issuer "troutmask" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 78A1C103A; Tue, 3 Mar 2015 02:55:06 +0000 (UTC) Received: from troutmask.apl.washington.edu (localhost [127.0.0.1]) by troutmask.apl.washington.edu (8.14.9/8.14.9) with ESMTP id t232svIN081022 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Mon, 2 Mar 2015 18:54:57 -0800 (PST) (envelope-from sgk@troutmask.apl.washington.edu) Received: (from sgk@localhost) by troutmask.apl.washington.edu (8.14.9/8.14.9/Submit) id t232svZs081021; Mon, 2 Mar 2015 18:54:57 -0800 (PST) (envelope-from sgk) Date: Mon, 2 Mar 2015 18:54:57 -0800 From: Steve Kargl To: Alfred Perlstein Subject: Re: Massive libxo-zation that breaks everything Message-ID: <20150303025456.GA81001@troutmask.apl.washington.edu> References: <54F31510.7050607@hot.ee> <54F34B6E.2040809@astrodoggroup.com> <54F35F29.4000603@astrodoggroup.com> <54F429EF.5050400@freebsd.org> <54F46536.8040607@mu.org> <54F4C03F.7030704@freebsd.org> <54F4FECB.90501@freebsd.org> <197AA5DC-0591-4F71-BF10-51A5D8104C11@mu.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <197AA5DC-0591-4F71-BF10-51A5D8104C11@mu.org> User-Agent: Mutt/1.5.23 (2014-03-12) Cc: Harrison Grundy , "freebsd-current@freebsd.org" X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 03 Mar 2015 02:55:06 -0000 On Mon, Mar 02, 2015 at 05:30:31PM -0800, Alfred Perlstein wrote: > > Read the library. It doesn't care what output format it needs. > It is up to the translation layer to do it. You could even do > a csv format or most any other structured output format without > changing the userland utils. > Yeah, read the library code because the manpages are atrocious. There are 12 manpages apparently documenting xo_emit(). -- Steve Index: xo_attr.3 =================================================================== --- xo_attr.3 (revision 279526) +++ xo_attr.3 (working copy) @@ -11,8 +11,8 @@ .Dt LIBXO 3 .Os .Sh NAME -.Nm xo_emit -.Nd emit formatted output based on format string and arguments +.Nm xo_attr +.Nd FUBAR .Sh LIBRARY .Lb libxo .Sh SYNOPSIS Index: xo_create.3 =================================================================== --- xo_create.3 (revision 279526) +++ xo_create.3 (working copy) @@ -11,8 +11,8 @@ .Dt LIBXO 3 .Os .Sh NAME -.Nm xo_emit -.Nd emit formatted output based on format string and arguments +.Nm xo_create +.Nd FUBAR .Sh LIBRARY .Lb libxo .Sh SYNOPSIS Index: xo_finish.3 =================================================================== --- xo_finish.3 (revision 279526) +++ xo_finish.3 (working copy) @@ -11,8 +11,8 @@ .Dt LIBXO 3 .Os .Sh NAME -.Nm xo_emit -.Nd emit formatted output based on format string and arguments +.Nm xo_finish +.Nd FUBAR .Sh LIBRARY .Lb libxo .Sh SYNOPSIS Index: xo_flush.3 =================================================================== --- xo_flush.3 (revision 279526) +++ xo_flush.3 (working copy) @@ -11,8 +11,8 @@ .Dt LIBXO 3 .Os .Sh NAME -.Nm xo_emit -.Nd emit formatted output based on format string and arguments +.Nm xo_flush +.Nd FUBAR .Sh LIBRARY .Lb libxo .Sh SYNOPSIS Index: xo_open_container.3 =================================================================== --- xo_open_container.3 (revision 279526) +++ xo_open_container.3 (working copy) @@ -11,8 +11,8 @@ .Dt LIBXO 3 .Os .Sh NAME -.Nm xo_emit -.Nd emit formatted output based on format string and arguments +.Nm xo_open_container +.Nd FUBAR .Sh LIBRARY .Lb libxo .Sh SYNOPSIS Index: xo_open_list.3 =================================================================== --- xo_open_list.3 (revision 279526) +++ xo_open_list.3 (working copy) @@ -11,8 +11,8 @@ .Dt LIBXO 3 .Os .Sh NAME -.Nm xo_emit -.Nd emit formatted output based on format string and arguments +.Nm xo_open_list +.Nd FUBAR .Sh LIBRARY .Lb libxo .Sh SYNOPSIS Index: xo_set_allocator.3 =================================================================== --- xo_set_allocator.3 (revision 279526) +++ xo_set_allocator.3 (working copy) @@ -11,8 +11,8 @@ .Dt LIBXO 3 .Os .Sh NAME -.Nm xo_emit -.Nd emit formatted output based on format string and arguments +.Nm xo_set_allocator +.Nd FUBAR .Sh LIBRARY .Lb libxo .Sh SYNOPSIS Index: xo_set_flags.3 =================================================================== --- xo_set_flags.3 (revision 279526) +++ xo_set_flags.3 (working copy) @@ -11,8 +11,8 @@ .Dt LIBXO 3 .Os .Sh NAME -.Nm xo_emit -.Nd emit formatted output based on format string and arguments +.Nm xo_set_flags +.Nd FUBAR .Sh LIBRARY .Lb libxo .Sh SYNOPSIS Index: xo_set_info.3 =================================================================== --- xo_set_info.3 (revision 279526) +++ xo_set_info.3 (working copy) @@ -11,8 +11,8 @@ .Dt LIBXO 3 .Os .Sh NAME -.Nm xo_emit -.Nd emit formatted output based on format string and arguments +.Nm xo_set_info +.Nd FUBAR .Sh LIBRARY .Lb libxo .Sh SYNOPSIS Index: xo_set_style.3 =================================================================== --- xo_set_style.3 (revision 279526) +++ xo_set_style.3 (working copy) @@ -11,8 +11,8 @@ .Dt LIBXO 3 .Os .Sh NAME -.Nm xo_emit -.Nd emit formatted output based on format string and arguments +.Nm xo_set_style +.Nd FUBAR .Sh LIBRARY .Lb libxo .Sh SYNOPSIS Index: xo_set_writer.3 =================================================================== --- xo_set_writer.3 (revision 279526) +++ xo_set_writer.3 (working copy) @@ -11,8 +11,8 @@ .Dt LIBXO 3 .Os .Sh NAME -.Nm xo_emit -.Nd emit formatted output based on format string and arguments +.Nm xo_set_writer +.Nd FUBAR .Sh LIBRARY .Lb libxo .Sh SYNOPSIS From owner-freebsd-current@FreeBSD.ORG Tue Mar 3 02:56:50 2015 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 080269EC; Tue, 3 Mar 2015 02:56:50 +0000 (UTC) Received: from elvis.mu.org (elvis.mu.org [IPv6:2001:470:1f05:b76::196]) by mx1.freebsd.org (Postfix) with ESMTP id E4E201066; Tue, 3 Mar 2015 02:56:49 +0000 (UTC) Received: from [172.20.17.11] (unknown [12.167.51.131]) by elvis.mu.org (Postfix) with ESMTPSA id 4CC1E341F867; Mon, 2 Mar 2015 18:56:49 -0800 (PST) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (1.0) Subject: Re: Massive libxo-zation that breaks everything From: Alfred Perlstein X-Mailer: iPhone Mail (12B440) In-Reply-To: <54F51C1C.1050609@freebsd.org> Date: Mon, 2 Mar 2015 18:56:48 -0800 Content-Transfer-Encoding: quoted-printable Message-Id: <487A99D3-8620-48D2-9342-3D36E3445FB9@mu.org> References: <54F31510.7050607@hot.ee> <54F34B6E.2040809@astrodoggroup.com> <54F35F29.4000603@astrodoggroup.com> <54F429EF.5050400@freebsd.org> <54F46536.8040607@mu.org> <54F4C03F.7030704@freebsd.org> <54F4FECB.90501@freebsd.org> <197AA5DC-0591-4F71-BF10-51A5D8104C11@mu.org> <54F5103C.4020108@freebsd.org> <54F51C1C.1050609@freebsd.org> To: Andrey Chernov Cc: Harrison Grundy , "freebsd-current@freebsd.org" X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 03 Mar 2015 02:56:50 -0000 > On Mar 2, 2015, at 6:27 PM, Andrey Chernov wrote: >>=20 >>=20 >> The responsibility is on you to provide something better, both the archit= ecture AND code. So if you want it backed out, then write something better. O= therwise step back and let progress happen. >=20 > As it seems you know a lot about my responsibility and duty, I am very > surprised by your broad interests. If you let me to speak for myself a > bit, I can tell what I feel. In this particular case my responsibility > is just to give good advice at the road fork with one way leads to > obvious hell, nothing more. I already express my opinion from the > technical point of view and don't want to participate in the flame war, > so continue this thread without me, please. >=20 Other bad ideas (in the past) were support for threads, kernel threads, SMP,= etc in the project's past. So maybe trying something new might be a good id= ea. At the end of the day, if in a year we think it's really terrible then i= t can be replaced, ripped out, or better yet can be fixed. =20 What's worse than a bad idea is not moving on any ideas.=20 From owner-freebsd-current@FreeBSD.ORG Tue Mar 3 03:14:30 2015 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 335E61DF; Tue, 3 Mar 2015 03:14:30 +0000 (UTC) Received: from vps1.elischer.org (vps1.elischer.org [204.109.63.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "vps1.elischer.org", Issuer "CA Cert Signing Authority" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id F2FD1134E; Tue, 3 Mar 2015 03:14:29 +0000 (UTC) Received: from Julian-MBP3.local (50-196-156-133-static.hfc.comcastbusiness.net [50.196.156.133]) (authenticated bits=0) by vps1.elischer.org (8.14.9/8.14.9) with ESMTP id t233EQOi009013 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO); Mon, 2 Mar 2015 19:14:26 -0800 (PST) (envelope-from julian@freebsd.org) Message-ID: <54F5270D.1010009@freebsd.org> Date: Mon, 02 Mar 2015 19:14:21 -0800 From: Julian Elischer User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:31.0) Gecko/20100101 Thunderbird/31.5.0 MIME-Version: 1.0 To: Alfred Perlstein , Andrey Chernov Subject: Re: Massive libxo-zation that breaks everything References: <54F31510.7050607@hot.ee> <54F34B6E.2040809@astrodoggroup.com> <54F35F29.4000603@astrodoggroup.com> <54F429EF.5050400@freebsd.org> <54F46536.8040607@mu.org> <54F4C03F.7030704@freebsd.org> <54F4FECB.90501@freebsd.org> <197AA5DC-0591-4F71-BF10-51A5D8104C11@mu.org> In-Reply-To: <197AA5DC-0591-4F71-BF10-51A5D8104C11@mu.org> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit Cc: Harrison Grundy , "freebsd-current@freebsd.org" X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 03 Mar 2015 03:14:30 -0000 On 3/2/15 5:30 PM, Alfred Perlstein wrote: > >> On Mar 2, 2015, at 4:22 PM, Andrey Chernov wrote: >> >>> On 02.03.2015 22:55, Julian Elischer wrote: >>>> On 3/2/15 5:27 AM, Alfred Perlstein wrote: >>>> >>>> >>>>> On 3/2/15 4:14 AM, Julian Elischer wrote: >>>>>> On 3/1/15 10:49 AM, Harrison Grundy wrote: >>>>>> Thanks! >>>>>> >>>>>> That does seem useful, but I'm not sure I see the reasoning behind >>>>>> putting into base, over a port or package, since processing XML in base >>>>>> is a pain, and it can't serve up JSON or HTML without additional >>>>>> utilities anyway. >>>>>> >>>>>> (If I'm reviving a long-settled thing, let me know and I'll drop it. >>>>>> I'm >>>>>> trying to understand the use case for this.) >>>>> To me it would almost seem more useful to have a programmable filter >>>>> for which you could produce >>>>> parse grammars to parse the output of various programs.. >>>>> thus >>>>> >>>>> ifconfig -a | xmlize -g ifconfig | your-favourite-xml-parser >>>>> with a set of grammars in /usr/share/xmlize/ >>>>> then we could use it for out-of-tree programs as well if we wrote >>>>> grammars for them.. >>>>> >>>>> The sentiment of machine-readable output is nice, but I think it's >>>>> slightly off target. >>>>> we shouldn't have to change all out utilities, and it isn't going to >>>>> help at all with 3rd party apps, >>>>> e.g. samba stuff. A generally easy to program output grammar parser >>>>> would be truely useful. >>>>> and not just for FreeBSD. >>>>> >>>>> I've been watching with an uncomfortable feeling, but it's taken me a >>>>> while to put my >>>>> finger on what it was.. >>>> Are you sure it's not the hairs on the back of your neck standing up >>>> due to NIH? >>>> >>>> Juniper has been doing this for years and it's very useful for them. >>> I'm not saying the ability to generate machine readable output is wrong, >>> but that the 'unix way' would be to make a filter for it. It seems that >>> the noisy people don't >>> agree with me so I will not stand in the way of progress.. >> I agree. Even if someone starts with json and xml only, it will need >> some 3rd format soon, and adding any new format have real possibility to >> break all already existent (like adding json+xml breaks plain text in >> pipes). Moreover, it violates Unix principle 'one tool == one general >> function' and lots of other rules like Eric Raymond ones, making each >> program looks like systemd. It makes harder to merge changes from other >> BSDs too. >> Proper way to do this thing is to back out all changes and write >> completely separate templates-based parser - xml/json writer. > > Read the library. It doesn't care what output format it needs. It is up to the translation layer to do it. You could even do a csv format or most any other structured output format without changing the userland utils. As far as I can see that's not an argument either way. I just think it makes more sense to spend more time writing one generic converter and grammar files than to mess up the insides of every utility in the system. If we had a tool, we could have grammar templates for 3rd party tools easily.. do YOU want to make libxo changes to 3rd party ports? of course not. so you are going off here solving a half of the problem. > > > > > >> -- >> http://ache.vniz.net/ >> _______________________________________________ >> freebsd-current@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-current >> To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org" >> > > From owner-freebsd-current@FreeBSD.ORG Tue Mar 3 03:33:39 2015 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3A4394DD; Tue, 3 Mar 2015 03:33:39 +0000 (UTC) Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by mx1.freebsd.org (Postfix) with ESMTP id 1B1831596; Tue, 3 Mar 2015 03:33:38 +0000 (UTC) Received: from [172.20.17.11] (unknown [12.167.51.131]) by elvis.mu.org (Postfix) with ESMTPSA id 522BF341F861; Mon, 2 Mar 2015 19:33:37 -0800 (PST) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (1.0) Subject: Re: Massive libxo-zation that breaks everything From: Alfred Perlstein X-Mailer: iPhone Mail (12B440) In-Reply-To: <54F5270D.1010009@freebsd.org> Date: Mon, 2 Mar 2015 19:33:36 -0800 Content-Transfer-Encoding: quoted-printable Message-Id: References: <54F31510.7050607@hot.ee> <54F34B6E.2040809@astrodoggroup.com> <54F35F29.4000603@astrodoggroup.com> <54F429EF.5050400@freebsd.org> <54F46536.8040607@mu.org> <54F4C03F.7030704@freebsd.org> <54F4FECB.90501@freebsd.org> <197AA5DC-0591-4F71-BF10-51A5D8104C11@mu.org> <54F5270D.1010009@freebsd.org> To: Julian Elischer Cc: Harrison Grundy , "freebsd-current@freebsd.org" X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 03 Mar 2015 03:33:39 -0000 > On Mar 2, 2015, at 7:14 PM, Julian Elischer wrote: >=20 >> On 3/2/15 5:30 PM, Alfred Perlstein wrote: >>=20 >>>> On Mar 2, 2015, at 4:22 PM, Andrey Chernov wrote: >>>>=20 >>>>> On 02.03.2015 22:55, Julian Elischer wrote: >>>>> On 3/2/15 5:27 AM, Alfred Perlstein wrote: >>>>>=20 >>>>>=20 >>>>>>> On 3/2/15 4:14 AM, Julian Elischer wrote: >>>>>>> On 3/1/15 10:49 AM, Harrison Grundy wrote: >>>>>>> Thanks! >>>>>>>=20 >>>>>>> That does seem useful, but I'm not sure I see the reasoning behind >>>>>>> putting into base, over a port or package, since processing XML in b= ase >>>>>>> is a pain, and it can't serve up JSON or HTML without additional >>>>>>> utilities anyway. >>>>>>>=20 >>>>>>> (If I'm reviving a long-settled thing, let me know and I'll drop it.= >>>>>>> I'm >>>>>>> trying to understand the use case for this.) >>>>>> To me it would almost seem more useful to have a programmable filter >>>>>> for which you could produce >>>>>> parse grammars to parse the output of various programs.. >>>>>> thus >>>>>>=20 >>>>>> ifconfig -a | xmlize -g ifconfig | your-favourite-xml-parser >>>>>> with a set of grammars in /usr/share/xmlize/ >>>>>> then we could use it for out-of-tree programs as well if we wrote >>>>>> grammars for them.. >>>>>>=20 >>>>>> The sentiment of machine-readable output is nice, but I think it's >>>>>> slightly off target. >>>>>> we shouldn't have to change all out utilities, and it isn't going to >>>>>> help at all with 3rd party apps, >>>>>> e.g. samba stuff. A generally easy to program output grammar parser >>>>>> would be truely useful. >>>>>> and not just for FreeBSD. >>>>>>=20 >>>>>> I've been watching with an uncomfortable feeling, but it's taken me a= >>>>>> while to put my >>>>>> finger on what it was.. >>>>> Are you sure it's not the hairs on the back of your neck standing up >>>>> due to NIH? >>>>>=20 >>>>> Juniper has been doing this for years and it's very useful for them. >>>> I'm not saying the ability to generate machine readable output is wrong= , >>>> but that the 'unix way' would be to make a filter for it. It seems that= >>>> the noisy people don't >>>> agree with me so I will not stand in the way of progress.. >>> I agree. Even if someone starts with json and xml only, it will need >>> some 3rd format soon, and adding any new format have real possibility to= >>> break all already existent (like adding json+xml breaks plain text in >>> pipes). Moreover, it violates Unix principle 'one tool =3D=3D one genera= l >>> function' and lots of other rules like Eric Raymond ones, making each >>> program looks like systemd. It makes harder to merge changes from other >>> BSDs too. >>> Proper way to do this thing is to back out all changes and write >>> completely separate templates-based parser - xml/json writer. >>=20 >> Read the library. It doesn't care what output format it needs. It is up t= o the translation layer to do it. You could even do a csv format or most any= other structured output format without changing the userland utils. > As far as I can see that's not an argument either way. > I just think it makes more sense to spend more time writing one generic co= nverter and grammar files than to mess up the insides of every utility in th= e system. If we had a tool, we could have grammar templates for 3rd party to= ols easily.. do YOU want to make libxo changes to 3rd party ports? of course= not. so you are going off here solving a half of the problem. Actually I want to shame third party ports into adopting libxo (or at least p= roviding machine readable output).=20 I know it's scary to try to lead the pack, after all we could be wrong, but m= aybe it's time to try something new and see what happens.=20 And no, your idea doesn't make sense it just will lead to those files bit ro= tting. =20 Bedsides that you don't even have a real spec other than "it should be done d= ifferently".=20 Again, show the code.=20 From owner-freebsd-current@FreeBSD.ORG Tue Mar 3 05:01:57 2015 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1F52BE86 for ; Tue, 3 Mar 2015 05:01:57 +0000 (UTC) Received: from jenkins-9.freebsd.org (jenkins-9.freebsd.org [8.8.178.209]) by mx1.freebsd.org (Postfix) with ESMTP id 0FDD31D38 for ; Tue, 3 Mar 2015 05:01:57 +0000 (UTC) Received: from jenkins-9.freebsd.org (localhost [127.0.0.1]) by jenkins-9.freebsd.org (Postfix) with ESMTP id 44E5822B9 for ; Tue, 3 Mar 2015 05:01:57 +0000 (UTC) Date: Tue, 3 Mar 2015 05:01:57 +0000 (GMT) From: jenkins-admin@freebsd.org To: freebsd-current@freebsd.org Message-ID: <1687262409.55.1425358917244.JavaMail.jenkins@jenkins-9.freebsd.org> Subject: Jenkins build became unstable: FreeBSD_HEAD-tests2 #798 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Jenkins-Job: FreeBSD_HEAD-tests2 X-Jenkins-Result: UNSTABLE X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 03 Mar 2015 05:01:57 -0000 See From owner-freebsd-current@FreeBSD.ORG Tue Mar 3 06:24:40 2015 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C2F2DA58; Tue, 3 Mar 2015 06:24:40 +0000 (UTC) Received: from mail-wi0-x236.google.com (mail-wi0-x236.google.com [IPv6:2a00:1450:400c:c05::236]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 68F146AF; Tue, 3 Mar 2015 06:24:40 +0000 (UTC) Received: by wiwl15 with SMTP id l15so20194565wiw.5; Mon, 02 Mar 2015 22:24:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=date:from:to:cc:subject:message-id:mime-version:content-type :content-transfer-encoding; bh=cPyLB+jgudACSN0Z97Jm+YPNmg77kG42kunZuL/nzfw=; b=QkLgH5oQK2MkJS6VeFB9xJhTa0EmU3H0AUdbkUuUg0MNVvZZJ+o46q5zNC4EoY8gEa JOTQr1d433ugN4FB3XWIk+8d4NlMzR97p+W0plawj9vvsVP8dAdiSv5mM+lAQLGxV6Eb UhUgtUzGAYVBqTdlAmWhzPpt4iyPs9BU/kDD5P2m5J9PAQFB5xsw2vWIeofL/YE5NS5m 0GXcOGkxYgsRc2gGmKBUublJzkQci9gLA3tI4oZ/VmPB9z3i/5YVqCIblxk9DNvT4XkU cM3YNodzALArd9QkPJ7i5A5ZvYtWv2FFJwednMIEF1qB9+q2Rvs2TCBEVivxwLYxMzjD p0KA== X-Received: by 10.195.13.104 with SMTP id ex8mr64461098wjd.12.1425363878658; Mon, 02 Mar 2015 22:24:38 -0800 (PST) Received: from laptop.minsk.domain (minsk.nivalnetwork.com. [86.57.144.74]) by mx.google.com with ESMTPSA id s5sm19686690wia.1.2015.03.02.22.24.37 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 02 Mar 2015 22:24:37 -0800 (PST) Date: Tue, 3 Mar 2015 09:23:37 +0300 From: "Sergey V. Dyatko" To: freebsd-current@freebsd.org Subject: nvidia+HEAD amd64 trouble Message-ID: <20150303092337.05aeb928@laptop.minsk.domain> X-Mailer: Claws Mail 3.11.1 (GTK+ 2.24.25; amd64-portbld-freebsd11.0) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Cc: freebsd-x11@freebsd.org X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 03 Mar 2015 06:24:40 -0000 Hi, I have Lenovo IdeaPad z400 touch laptop. It have GeForce GT 635M graphics card. Also it have integrated Intel card agp0: on vgapci0 agp0: aperture size is 256M, detected 65532k stolen memory when I enable nvidia on laptop bios and trying to use it I got following panic: NVRM: failed to copy vbios to system memory. NVRM: RmInitAdapter failed! (0x30:0xffff:800) nvidia0: NVRM: rm_init_adapter() failed! full core.txt file located here: http://svn.freebsd.by/files/coredump-nvidia.txt tell me what other information I need to provide It is HEAD@r279306 amd64 [tiger@laptop]:~/tmpdir>pkg info -x nvidia nvidia-driver-346.47 nvidia-settings-340.24_1 [tiger@laptop]:~/tmpdir>pkg info -x ^xorg xorg-7.7_1 xorg-apps-7.7_2 xorg-cf-files-1.0.5_1 xorg-docs-1.7,1 xorg-drivers-7.7_2 xorg-fonts-7.7 xorg-fonts-100dpi-7.7 xorg-fonts-75dpi-7.7 xorg-fonts-cyrillic-7.7 xorg-fonts-miscbitmaps-7.7 xorg-fonts-truetype-7.7_1 xorg-fonts-type1-7.7 xorg-libraries-7.7_2 xorg-macros-1.19.0 xorg-server-1.14.7_2,1 I would be very appreciated for any help -- wbr, tiger From owner-freebsd-current@FreeBSD.ORG Tue Mar 3 07:53:28 2015 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7C601883 for ; Tue, 3 Mar 2015 07:53:28 +0000 (UTC) Received: from jenkins-9.freebsd.org (jenkins-9.freebsd.org [8.8.178.209]) by mx1.freebsd.org (Postfix) with ESMTP id 6AB58F33 for ; Tue, 3 Mar 2015 07:53:28 +0000 (UTC) Received: from jenkins-9.freebsd.org (localhost [127.0.0.1]) by jenkins-9.freebsd.org (Postfix) with ESMTP id 8DD3D2511 for ; Tue, 3 Mar 2015 07:53:28 +0000 (UTC) Date: Tue, 3 Mar 2015 07:53:28 +0000 (GMT) From: jenkins-admin@freebsd.org To: freebsd-current@freebsd.org Message-ID: <470765445.63.1425369208540.JavaMail.jenkins@jenkins-9.freebsd.org> In-Reply-To: <1687262409.55.1425358917244.JavaMail.jenkins@jenkins-9.freebsd.org> References: <1687262409.55.1425358917244.JavaMail.jenkins@jenkins-9.freebsd.org> Subject: Jenkins build is back to stable : FreeBSD_HEAD-tests2 #799 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Jenkins-Job: FreeBSD_HEAD-tests2 X-Jenkins-Result: SUCCESS X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 03 Mar 2015 07:53:28 -0000 See From owner-freebsd-current@FreeBSD.ORG Tue Mar 3 09:05:16 2015 Return-Path: Delivered-To: freebsd-current@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id BAF8FB73; Tue, 3 Mar 2015 09:05:16 +0000 (UTC) Received: from mail.turbocat.net (mail.turbocat.net [IPv6:2a01:4f8:d16:4514::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 6120D924; Tue, 3 Mar 2015 09:05:16 +0000 (UTC) Received: from laptop015.home.selasky.org (cm-176.74.213.204.customer.telag.net [176.74.213.204]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.turbocat.net (Postfix) with ESMTPSA id DA83A1FE022; Tue, 3 Mar 2015 10:05:13 +0100 (CET) Message-ID: <54F57979.3060008@selasky.org> Date: Tue, 03 Mar 2015 10:06:01 +0100 From: Hans Petter Selasky User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:31.0) Gecko/20100101 Thunderbird/31.4.0 MIME-Version: 1.0 To: Brett Wynkoop , Warner Losh , freebsd-arm@freebsd.org, freebsd-current@FreeBSD.ORG Subject: Re: crash on writing usbstick References: <20150301041855.5352663e@ivory.wynn.com> <20150301144653.63b38cdf@ivory.wynn.com> <20150301184456.7b5e6487@ivory.wynn.com> <1DC8221F-64EA-418C-8CE5-5FFA4F3DBC64@bsdimp.com> <20150301203244.55578413@ivory.wynn.com> <20150302214352.5143d72e@ivory.wynn.com> In-Reply-To: <20150302214352.5143d72e@ivory.wynn.com> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 03 Mar 2015 09:05:16 -0000 Hi, On 03/03/15 03:43, Brett Wynkoop wrote: > So do we think this is an ARM specific thing, or is it a UFS thing? > > I am thinking maybe I should format as ext or ntfs and see if we have > the same issue. If we do then we can rule out a UFS bug. I just caught this issue with amd64 while building a kernel. > Fatal trap 12: page fault while in kernel mode > cpuid = 12; apic id = 20 > fault virtual address = 0xffffffffffffffff > fault code = supervisor read data, page not present > instruction pointer = 0x20:0xffffffff80bba91d > stack pointer = 0x28:0xfffffe0466e04120 > frame pointer = 0x28:0xfffffe0466e04150 > code segment = base 0x0, limit 0xfffff, type 0x1b > = DPL 0, pres 1, long 1, def32 0, gran 1 > processor eflags = interrupt enabled, resume, IOPL = 0 > current process = 83323 (objcopy) > (kgdb) bt > #0 doadump (textdump=Unhandled dwarf expression opcode 0x93 > ) at pcpu.h:219 > #1 0xffffffff803530ae in db_dump (dummy=, dummy2=Unhandled dwarf expression opcode 0x93 > ) > at /usr/img/freebsd/sys/ddb/db_command.c:533 > #2 0xffffffff80352b2c in db_command (cmd_table=0x0) at /usr/img/freebsd/sys/ddb/db_command.c:440 > #3 0xffffffff80352894 in db_command_loop () at /usr/img/freebsd/sys/ddb/db_command.c:493 > #4 0xffffffff803553f0 in db_trap (type=, code=Unhandled dwarf expression opcode 0x93 > ) > at /usr/img/freebsd/sys/ddb/db_main.c:251 > #5 0xffffffff80994e8e in kdb_trap (type=Unhandled dwarf expression opcode 0x93 > ) at /usr/img/freebsd/sys/kern/subr_kdb.c:654 > #6 0xffffffff80d79fe9 in trap_fatal (frame=0xfffffe0466e04070, eva=) > at /usr/img/freebsd/sys/amd64/amd64/trap.c:856 > #7 0xffffffff80d7a281 in trap_pfault (frame=0xfffffe0466e04070, usermode=) > at /usr/img/freebsd/sys/amd64/amd64/trap.c:678 > #8 0xffffffff80d79942 in trap (frame=0xfffffe0466e04070) at /usr/img/freebsd/sys/amd64/amd64/trap.c:426 > #9 0xffffffff80d57e72 in calltrap () at /usr/img/freebsd/sys/amd64/amd64/exception.S:235 > #10 0xffffffff80bba91d in add_to_worklist (wk=0xfffff801670d0680, flags=Unhandled dwarf expression opcode 0x93 > ) > at /usr/img/freebsd/sys/ufs/ffs/ffs_softdep.c:1513 > #11 0xffffffff80bc1813 in free_newblk (newblk=0xfffff8006d6f3700) > at /usr/img/freebsd/sys/ufs/ffs/ffs_softdep.c:7414 > #12 0xffffffff80bb1a80 in softdep_setup_allocdirect (ip=0xfffff8024ffbadc8, off=, > newblkno=, oldblkno=, newsize=32768, oldsize=32768, > bp=) at /usr/img/freebsd/sys/ufs/ffs/ffs_softdep.c:5361 > #13 0xffffffff80b9a6e4 in ffs_reallocblks (ap=) > at /usr/img/freebsd/sys/ufs/ffs/ffs_alloc.c:870 > #14 0xffffffff80eb0f67 in VOP_REALLOCBLKS_APV (vop=, a=) > at vnode_if.c:2727 > #15 0xffffffff809f7574 in cluster_write (vp=0xfffff802a44493b0, bp=0xfffffe03e1b3a590, filesize=393216, > seqcount=0, gbflags=) at vnode_if.h:1122 > #16 0xffffffff80bca11e in ffs_write (ap=0xfffffe0466e04690) at /usr/img/freebsd/sys/ufs/ffs/ffs_vnops.c:810 > #17 0xffffffff80eaeac3 in VOP_WRITE_APV (vop=, a=0xfffffe0466e04690) at vnode_if.c:997 > #18 0xffffffff80a1acce in vn_write (fp=0xfffff801675d0280, uio=0xfffffe0466e04970, > active_cred=, flags=, td=0xfffff8041cb8f980) at vnode_if.h:413 > #19 0xffffffff80a18805 in vn_io_fault1 () at /usr/img/freebsd/sys/kern/vfs_vnops.c:1053 > #20 0xffffffff80a16e03 in vn_io_fault (fp=0xfffff801675d0280, uio=0xfffffe0466e04970, > active_cred=, flags=0, td=0xfffff8041cb8f980) > at /usr/img/freebsd/sys/kern/vfs_vnops.c:1158 > #21 0xffffffff809b654a in dofilewrite (td=0xfffff8041cb8f980, fd=4, fp=0xfffff801675d0280, > auio=0xfffffe0466e04970, offset=, flags=Unhandled dwarf expression opcode 0x93 > ) at file.h:304 > #22 0xffffffff809b6258 in kern_writev (td=0xfffff8041cb8f980, fd=Unhandled dwarf expression opcode 0x93 > ) > ---Type to continue, or q to quit--- > at /usr/img/freebsd/sys/kern/sys_generic.c:481 > #23 0xffffffff809b61e3 in sys_write (td=0xfffff80167997000, uap=) > at /usr/img/freebsd/sys/kern/sys_generic.c:396 > #24 0xffffffff80d7a84f in amd64_syscall (td=0xfffff8041cb8f980, traced=0) at subr_syscall.c:133 > #25 0xffffffff80d5815b in Xfast_syscall () at /usr/img/freebsd/sys/amd64/amd64/exception.S:395 > #26 0x00000000004cae5a in ?? () > (kgdb) print wk > $1 = (struct worklist *) 0xfffff801670d0680 > (kgdb) print /x *((struct ufsmount *)wk->wk_mp->mnt_data) > $5 = { > um_mountp = 0xfffff80167201000, > um_dev = 0xfffff80167107600, > um_cp = 0xfffff80167214100, > um_bo = 0xfffff801671e4830, > um_devvp = 0xfffff801671e4760, > um_fstype = 0x2, > um_fs = 0xfffff8016722a000, > um_extattr = { > uepm_lock = { > lock_object = { > lo_name = 0x0, > lo_flags = 0x0, > lo_data = 0x0, > lo_witness = 0x0 > }, > sx_lock = 0x0 > }, > uepm_list = { > lh_first = 0x0 > }, > uepm_ucred = 0x0, > uepm_flags = 0x0 > }, > um_nindir = 0x1000, > um_bptrtodb = 0x3, > um_seqinc = 0x8, > um_lock = { > lock_object = { > lo_name = 0xffffffff81053eb2, > lo_flags = 0x1030000, > lo_data = 0x0, > lo_witness = 0xfffffe0000b1c900 > }, > mtx_lock = 0x4 > }, > um_fsckpid = 0x0, > um_softdep = 0xfffff80167997000, > ---Type to continue, or q to quit--- > um_quotas = {0x0, 0x0}, > um_cred = {0x0, 0x0}, > um_btime = {0x0, 0x0}, > um_itime = {0x0, 0x0}, > um_qflags = {0x0, 0x0}, > um_savedmaxfilesize = 0x0, > um_candelete = 0x0, > um_writesuspended = 0x0, > um_balloc = 0xffffffff80b9ff30, > um_blkatoff = 0xffffffff80bc4950, > um_truncate = 0xffffffff80ba1fd0, > um_update = 0xffffffff80ba1c60, > um_valloc = 0xffffffff80b9ac70, > um_vfree = 0xffffffff80b9ba70, > um_ifree = 0xffffffff80bc9210, > um_rdonly = 0xffffffff80ba3bc0, > um_snapgone = 0xffffffff80ba7580 > } > (kgdb) print /x ((struct ufsmount *)wk->wk_mp->mnt_data)[0].um_softdep[0] > $6 = { > sd_fslock = { > lock_object = { > lo_name = 0xffffffff810502ea, > lo_flags = 0x5230000, > lo_data = 0x0, > lo_witness = 0xfffffe0000b1ca80 > }, > rw_lock = 0xfffff8041cb8f980 > }, > sd_workitem_pending = { > lh_first = 0xfffff801670f6a00 > }, > sd_worklist_tail = 0xffffffffffffffff, > sd_journal_pending = { > lh_first = 0x0 > }, > sd_journal_tail = 0x0, > sd_jblocks = 0x0, > sd_unlinked = { > tqh_first = 0x0, > tqh_last = 0xfffff80167997048 > }, > sd_dirtycg = { > lh_first = 0xfffff801672de900 > }, > sd_mkdirlisthd = { > lh_first = 0x0 > }, > sd_pdhash = 0xfffffe0002467000, > sd_pdhashsize = 0xffff, > sd_pdnextclean = 0x0, > sd_idhash = 0xfffffe00024e7000, > sd_idhashsize = 0x3ffff, > sd_idnextclean = 0x0, > sd_newblkhash = 0xfffffe00026e7000, > sd_newblkhashsize = 0x7ffff, > sd_bmhash = 0xfffffe0002207000, > ---Type to continue, or q to quit--- > sd_bmhashsize = 0x3ff, > sd_indirhash = 0xfffff801679a5070, > sd_indirhashsize = 0x0, > sd_on_journal = 0x0, > sd_on_worklist = 0x2, > sd_deps = 0x76e7, > sd_accdeps = 0xa1d66, > sd_req = 0x0, > sd_flags = 0x0, > sd_cleanups = 0x0, > sd_flushtd = 0xfffff801679264c0, > sd_next = { > tqe_next = 0x0, > tqe_prev = 0xffffffff817d8798 > }, > sd_ump = 0xfffff80167229200, > sd_curdeps = {0x1a3, 0x105f, 0x5, 0x1, 0x30e4, 0x172, 0x22c4, 0x1, 0x2f, 0x32, 0xf21, 0x0, 0x8, 0x0, > 0x3a, 0x0 } > } > (kgdb) frame 11 > #11 0xffffffff80bc1813 in free_newblk (newblk=0xfffff8006d6f3700) > at /usr/img/freebsd/sys/ufs/ffs/ffs_softdep.c:7414 > 7414 add_to_worklist(&freefrag->ff_list, 0); > (kgdb) print *newblk > $7 = { > nb_list = { > wk_list = { > le_next = 0xffffffffffffffff, > le_prev = 0xffffffffffffffff > }, > wk_mp = 0xfffff80167201000, > wk_type = 4, > wk_state = 257 > }, > nb_hash = { > le_next = 0x0, > le_prev = 0xfffffe00029963c0 > }, > nb_deps = { > le_next = 0xffffffffffffffff, > le_prev = 0xffffffffffffffff > }, > nb_jnewblk = 0x0, > nb_bmsafemap = 0xfffff801672de900, > nb_freefrag = 0x0, > nb_indirdeps = { > lh_first = 0x0 > }, > nb_newdirblk = { > lh_first = 0x0 > }, > nb_jwork = { > lh_first = 0x0 > }, > nb_newblkno = 66412152 > } --HPS From owner-freebsd-current@FreeBSD.ORG Tue Mar 3 09:10:00 2015 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D6184CDB; Tue, 3 Mar 2015 09:09:59 +0000 (UTC) Received: from theravensnest.org (theraven.freebsd.your.org [216.14.102.27]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "cloud.theravensnest.org", Issuer "StartCom Class 1 Primary Intermediate Server CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id A8C91952; Tue, 3 Mar 2015 09:09:58 +0000 (UTC) Received: from [192.168.0.7] (cpc16-cmbg15-2-0-cust60.5-4.cable.virginm.net [86.5.162.61]) (authenticated bits=0) by theravensnest.org (8.15.1/8.14.9) with ESMTPSA id t2399nsb032154 (version=TLSv1 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 3 Mar 2015 09:09:51 GMT (envelope-from theraven@FreeBSD.org) X-Authentication-Warning: theravensnest.org: Host cpc16-cmbg15-2-0-cust60.5-4.cable.virginm.net [86.5.162.61] claimed to be [192.168.0.7] Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 8.2 \(2070.6\)) Subject: Re: Massive libxo-zation that breaks everything From: David Chisnall In-Reply-To: <54F50F15.4050300@freebsd.org> Date: Tue, 3 Mar 2015 09:09:43 +0000 Content-Transfer-Encoding: quoted-printable Message-Id: References: <54F31510.7050607@hot.ee> <54F34B6E.2040809@astrodoggroup.com> <54F35F29.4000603@astrodoggroup.com> <54F429EF.5050400@freebsd.org> <54F46536.8040607@mu.org> <54F4C03F.7030704@freebsd.org> <54F4FECB.90501@freebsd.org> <54F504EF.3030303@freebsd.org> <54F50F15.4050300@freebsd.org> To: Andrey Chernov X-Mailer: Apple Mail (2.2070.6) Cc: freebsd-current@freebsd.org, Allan Jude X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 03 Mar 2015 09:10:00 -0000 On 3 Mar 2015, at 01:32, Andrey Chernov wrote: >=20 > So, why you ever need to modify wc? Just load wc inside your > json/xml/etc writer, replacing its printf at the ld-elf.so level. You can't get structured output from printf() because printf() takes = unstructured input. It's a string with some variables pasted in, but no = awareness of context. The libxo changes to the tools are simply marking the output as having a = defined structure. The library then translates this abstract structure = into something that can be parsed easily by external tools. If your argument is that the UNIX philosophy is simple tools doing one = thing well, then please remember the context of this philosophy: It = dates back to the original UNIX systems *that did not support shared = libraries* and was an argument used to justify not implementing them. = This is why globbing is in the shell instead of a shared library and why = some variant of mv *.a *.b works on every command-line interface except = for a UNIX shell. Even with that in mind, small changes to individual tools are a *lot* = simpler than one massive monolithic tool that understands the output = formats of every other tool in the base system and can transform them. = Why do you think a few library calls in each application is more complex = *than an entire parser per tool*? I am reminded of the time about ten years ago when I was writing a = system info widget for a DE. Getting the total amount of RAM, number of = cores, and CPU speed on FreeBSD, OpenBSD, NetBSD, and Solaris proved to = be a single sysctl for each attribute. On Linux, the information was = all there, in /proc/cpuinfo, so I had to write a parser. Unfortunately, = the format changed across kernel revisions and across architectures. In = the end, I had to link in a 300KB library to parse information that the = kernel already had in a machine-readable format and present it to me in = a machine-readable format. Hopefully there's a lesson here that we can learn from: human-readable = formats do not make good intermediate representations when communicating = between tools. If your argument is about maintainability of these changes, then please = point to concrete instances where the changes are complex and difficult = to maintain. If your argument is about binary size, then it would be relatively easy = for us to add a version of libxo for static linking into the versions in = /rescue that only supported plain-text output, but again, please = quantify your objections: What is the size increase that you're seeing = and what kind of devices is this causing a problem for? The general approach of making tools produce machine-readable output has = been under discussion for *years* and the integration of libxo has been = underway for several months, prior to which it was used inside Juniper. = If you had a better approach, then the correct time to raise it was back = when the requirement to produce machine-readable output was raised. The = subject of this thread is just plain trolling (is everything broken by = libxo? No. Is anything broken by libxo? Occasionally, but new = features often come with temporary breakage that's found and fixed in = testing: that's why we have -CURRENT). David From owner-freebsd-current@FreeBSD.ORG Tue Mar 3 17:32:32 2015 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id DA4E53A8; Tue, 3 Mar 2015 17:32:31 +0000 (UTC) Received: from mail.strugglingcoder.info (strugglingcoder.info [65.19.130.35]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id BD2EDCC1; Tue, 3 Mar 2015 17:32:31 +0000 (UTC) Received: from localhost (unknown [10.1.1.3]) (Authenticated sender: hiren@strugglingcoder.info) by mail.strugglingcoder.info (Postfix) with ESMTPSA id 953CFD227E; Tue, 3 Mar 2015 09:32:23 -0800 (PST) Date: Tue, 3 Mar 2015 09:32:23 -0800 From: hiren panchasara To: Alfred Perlstein Subject: Re: Massive libxo-zation that breaks everything Message-ID: <20150303173223.GQ95752@strugglingcoder.info> References: <54F34B6E.2040809@astrodoggroup.com> <54F35F29.4000603@astrodoggroup.com> <54F429EF.5050400@freebsd.org> <54F46536.8040607@mu.org> <54F4C03F.7030704@freebsd.org> <54F4FECB.90501@freebsd.org> <197AA5DC-0591-4F71-BF10-51A5D8104C11@mu.org> <54F5270D.1010009@freebsd.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="V2tfspbppmK1TQo2" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.23 (2014-03-12) Cc: Harrison Grundy , "freebsd-current@freebsd.org" X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 03 Mar 2015 17:32:32 -0000 --V2tfspbppmK1TQo2 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On 03/02/15 at 07:33P, Alfred Perlstein wrote: >=20 >=20 > > On Mar 2, 2015, at 7:14 PM, Julian Elischer wrote: > >=20 > >> On 3/2/15 5:30 PM, Alfred Perlstein wrote: > >>=20 > >>>> On Mar 2, 2015, at 4:22 PM, Andrey Chernov wrote: > >>>>=20 > >>>>> On 02.03.2015 22:55, Julian Elischer wrote: > >>>>> On 3/2/15 5:27 AM, Alfred Perlstein wrote: > >>>>>=20 > >>>>>=20 > >>>>>>> On 3/2/15 4:14 AM, Julian Elischer wrote: > >>>>>>> On 3/1/15 10:49 AM, Harrison Grundy wrote: > >>>>>>> Thanks! > >>>>>>>=20 > >>>>>>> That does seem useful, but I'm not sure I see the reasoning behind > >>>>>>> putting into base, over a port or package, since processing XML i= n base > >>>>>>> is a pain, and it can't serve up JSON or HTML without additional > >>>>>>> utilities anyway. > >>>>>>>=20 > >>>>>>> (If I'm reviving a long-settled thing, let me know and I'll drop = it. > >>>>>>> I'm > >>>>>>> trying to understand the use case for this.) > >>>>>> To me it would almost seem more useful to have a programmable filt= er > >>>>>> for which you could produce > >>>>>> parse grammars to parse the output of various programs.. > >>>>>> thus > >>>>>>=20 > >>>>>> ifconfig -a | xmlize -g ifconfig | your-favourite-xml-parser > >>>>>> with a set of grammars in /usr/share/xmlize/ > >>>>>> then we could use it for out-of-tree programs as well if we wrote > >>>>>> grammars for them.. > >>>>>>=20 > >>>>>> The sentiment of machine-readable output is nice, but I think it's > >>>>>> slightly off target. > >>>>>> we shouldn't have to change all out utilities, and it isn't going = to > >>>>>> help at all with 3rd party apps, > >>>>>> e.g. samba stuff. A generally easy to program output grammar parser > >>>>>> would be truely useful. > >>>>>> and not just for FreeBSD. > >>>>>>=20 > >>>>>> I've been watching with an uncomfortable feeling, but it's taken m= e a > >>>>>> while to put my > >>>>>> finger on what it was.. > >>>>> Are you sure it's not the hairs on the back of your neck standing up > >>>>> due to NIH? > >>>>>=20 > >>>>> Juniper has been doing this for years and it's very useful for them. > >>>> I'm not saying the ability to generate machine readable output is wr= ong, > >>>> but that the 'unix way' would be to make a filter for it. It seems t= hat > >>>> the noisy people don't > >>>> agree with me so I will not stand in the way of progress.. > >>> I agree. Even if someone starts with json and xml only, it will need > >>> some 3rd format soon, and adding any new format have real possibility= to > >>> break all already existent (like adding json+xml breaks plain text in > >>> pipes). Moreover, it violates Unix principle 'one tool =3D=3D one gen= eral > >>> function' and lots of other rules like Eric Raymond ones, making each > >>> program looks like systemd. It makes harder to merge changes from oth= er > >>> BSDs too. > >>> Proper way to do this thing is to back out all changes and write > >>> completely separate templates-based parser - xml/json writer. > >>=20 > >> Read the library. It doesn't care what output format it needs. It is u= p to the translation layer to do it. You could even do a csv format or most= any other structured output format without changing the userland utils. > > As far as I can see that's not an argument either way. > > I just think it makes more sense to spend more time writing one generic= converter and grammar files than to mess up the insides of every utility i= n the system. If we had a tool, we could have grammar templates for 3rd par= ty tools easily.. do YOU want to make libxo changes to 3rd party ports? of = course not. so you are going off here solving a half of the problem. >=20 > Actually I want to shame third party ports into adopting libxo (or at lea= st providing machine readable output).=20 >=20 > I know it's scary to try to lead the pack, after all we could be wrong, b= ut maybe it's time to try something new and see what happens.=20 >=20 > And no, your idea doesn't make sense it just will lead to those files bit= rotting. =20 >=20 > Bedsides that you don't even have a real spec other than "it should be do= ne differently".=20 >=20 > Again, show the code.=20 Wow. He told you want he didn't like and he moved on. I hope/wish we as a project can take criticism more positively than this. Answer to every criticism should not be "show me your code". We (should) know better than that. Hiren --V2tfspbppmK1TQo2 Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (FreeBSD) iQF8BAEBCgBmBQJU9fAmXxSAAAAAAC4AKGlzc3Vlci1mcHJAbm90YXRpb25zLm9w ZW5wZ3AuZmlmdGhob3JzZW1hbi5uZXRBNEUyMEZBMUQ4Nzg4RjNGMTdFNjZGMDI4 QjkyNTBFMTU2M0VERkU1AAoJEIuSUOFWPt/ljOkH/j5fjhvUYv2iihHU7EONnYO/ gPG5QTbFJ6ivRN6IzLxS7UvgFltlZYx4nWhCTuH7l0JxUH5IFEXhJFpMHuXfP+DZ ZhcZNGqZ+Hw24ZewnE2eiLTYtpWWHRrnZPcjqfpbnzUReYojFFWTUIJy35sfbmed t8+DGEvPyZ6a5dneu/hrTp7ekNyhs3748d3/pdO3UqTj0IwrvjyL2uKxEUdiT61q Eh62eKZ6DEV6ltoHNqArzG/7fKM/uGFOAYXRYKEItnQDTJCIQwoLaPJGE1s4r9lr 9g0tmgB4NfA9XsnBfZlweThT93tircgL2c2oZ6rIdbYy9fhh9nbm5lS5apRT9CU= =y3vq -----END PGP SIGNATURE----- --V2tfspbppmK1TQo2-- From owner-freebsd-current@FreeBSD.ORG Tue Mar 3 18:40:05 2015 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 4F5CC4DD; Tue, 3 Mar 2015 18:40:05 +0000 (UTC) Received: from elvis.mu.org (elvis.mu.org [IPv6:2001:470:1f05:b76::196]) by mx1.freebsd.org (Postfix) with ESMTP id 35E8B686; Tue, 3 Mar 2015 18:40:05 +0000 (UTC) Received: from [10.211.109.92] (unknown [216.205.235.226]) by elvis.mu.org (Postfix) with ESMTPSA id 01191341F87C; Tue, 3 Mar 2015 10:40:03 -0800 (PST) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (1.0) Subject: Re: Massive libxo-zation that breaks everything From: Alfred Perlstein X-Mailer: iPhone Mail (12B440) In-Reply-To: <20150303173223.GQ95752@strugglingcoder.info> Date: Tue, 3 Mar 2015 10:39:53 -0800 Content-Transfer-Encoding: quoted-printable Message-Id: <93307065-F026-42DE-A2D2-3843619BAD1A@mu.org> References: <54F34B6E.2040809@astrodoggroup.com> <54F35F29.4000603@astrodoggroup.com> <54F429EF.5050400@freebsd.org> <54F46536.8040607@mu.org> <54F4C03F.7030704@freebsd.org> <54F4FECB.90501@freebsd.org> <197AA5DC-0591-4F71-BF10-51A5D8104C11@mu.org> <54F5270D.1010009@freebsd.org> <20150303173223.GQ95752@strugglingcoder.info> To: hiren panchasara Cc: Harrison Grundy , "freebsd-current@freebsd.org" X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 03 Mar 2015 18:40:05 -0000 Sent from my iPhone > On Mar 3, 2015, at 9:32 AM, hiren panchasara w= rote: >=20 >> On 03/02/15 at 07:33P, Alfred Perlstein wrote: >>=20 >>>>=20 >>=20 >> Actually I want to shame third party ports into adopting libxo (or at lea= st providing machine readable output).=20 >>=20 >> I know it's scary to try to lead the pack, after all we could be wrong, b= ut maybe it's time to try something new and see what happens.=20 >>=20 >> And no, your idea doesn't make sense it just will lead to those files bit= rotting. =20 >>=20 >> Bedsides that you don't even have a real spec other than "it should be do= ne differently".=20 >>=20 >> Again, show the code. >=20 > Wow. He told you want he didn't like and he moved on. I hope/wish we as > a project can take criticism more positively than this. >=20 > Answer to every criticism should not be "show me your code". We (should) > know better than that. Maybe I am too old school but the motd on freefall is still "now shut up and= code". I still believe that's right. =20 We ALL need to know when to step back from an issue we really do not have th= e bandwidth to constructively contribute to.=20 This doesn't mean ideas or concerns aren't valid, but that unless you truly h= ave the bandwidth to contribute, taking a step back is a good idea.=20 -Alfred.=20 From owner-freebsd-current@FreeBSD.ORG Tue Mar 3 19:07:23 2015 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E925ED0F for ; Tue, 3 Mar 2015 19:07:23 +0000 (UTC) Received: from mail-lb0-x236.google.com (mail-lb0-x236.google.com [IPv6:2a00:1450:4010:c04::236]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 625D3A10 for ; Tue, 3 Mar 2015 19:07:23 +0000 (UTC) Received: by lbiz12 with SMTP id z12so11383719lbi.12 for ; Tue, 03 Mar 2015 11:07:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=QSrbWR9fPUWrL9suYJ/SUtCLqcSz+t7qxEmpMxmpe3A=; b=n5zE3Mf6RMdwDhjwSEFjCfgWs3DZilF+BOOvZHt2RbVXkt0DK2rcPyLfTx3ks87gvA F/njGc6+yjv7exg9CPE6ZxLnZ64Bsi/wEr5crBgWdeA9vLX9+yJ6l5nke/X1Ry+y/1Ct svnt2Rjt5rQTMxOF7cTni4Sv7Dvh1ov+dMmFXmAm+EJF8FCZRK0nPv0E2lA7vCrujFbb pMlQumw3wY2XhmSgbNf0BHjPkahEI3mH59EnpaQUd69WzupGfUwYpEsMpHaIzRsEywG3 ejZlM7lpF9l5Odct6+IGhHbs608wgJnn+RbGkM7uOXP3Qg8bzlX2PBb2Jd49sEYHcYc/ 8Q+w== MIME-Version: 1.0 X-Received: by 10.152.5.6 with SMTP id o6mr238139lao.59.1425409640945; Tue, 03 Mar 2015 11:07:20 -0800 (PST) Sender: chmeeedalf@gmail.com Received: by 10.25.145.65 with HTTP; Tue, 3 Mar 2015 11:07:20 -0800 (PST) In-Reply-To: <93307065-F026-42DE-A2D2-3843619BAD1A@mu.org> References: <54F34B6E.2040809@astrodoggroup.com> <54F35F29.4000603@astrodoggroup.com> <54F429EF.5050400@freebsd.org> <54F46536.8040607@mu.org> <54F4C03F.7030704@freebsd.org> <54F4FECB.90501@freebsd.org> <197AA5DC-0591-4F71-BF10-51A5D8104C11@mu.org> <54F5270D.1010009@freebsd.org> <20150303173223.GQ95752@strugglingcoder.info> <93307065-F026-42DE-A2D2-3843619BAD1A@mu.org> Date: Tue, 3 Mar 2015 11:07:20 -0800 X-Google-Sender-Auth: bHY44UKm8AHmQuOR1vQcUTEugGY Message-ID: Subject: Re: Massive libxo-zation that breaks everything From: Justin Hibbits To: Alfred Perlstein Content-Type: text/plain; charset=UTF-8 Cc: Harrison Grundy , "freebsd-current@freebsd.org" , hiren panchasara X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 03 Mar 2015 19:07:24 -0000 On Tue, Mar 3, 2015 at 10:39 AM, Alfred Perlstein wrote: > > > Sent from my iPhone > >> On Mar 3, 2015, at 9:32 AM, hiren panchasara wrote: >> >>> On 03/02/15 at 07:33P, Alfred Perlstein wrote: >>> >>>>> >>> >>> Actually I want to shame third party ports into adopting libxo (or at least providing machine readable output). >>> >>> I know it's scary to try to lead the pack, after all we could be wrong, but maybe it's time to try something new and see what happens. >>> >>> And no, your idea doesn't make sense it just will lead to those files bit rotting. >>> >>> Bedsides that you don't even have a real spec other than "it should be done differently". >>> >>> Again, show the code. >> >> Wow. He told you want he didn't like and he moved on. I hope/wish we as >> a project can take criticism more positively than this. >> >> Answer to every criticism should not be "show me your code". We (should) >> know better than that. > > Maybe I am too old school but the motd on freefall is still "now shut up and code". I still believe that's right. > > We ALL need to know when to step back from an issue we really do not have the bandwidth to constructively contribute to. > > This doesn't mean ideas or concerns aren't valid, but that unless you truly have the bandwidth to contribute, taking a step back is a good idea. > > -Alfred. You're both right. Maybe we need a hackathon on this. Is BSDCan too late? - Justin From owner-freebsd-current@FreeBSD.ORG Tue Mar 3 19:14:29 2015 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id CCA0DF8E for ; Tue, 3 Mar 2015 19:14:29 +0000 (UTC) Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by mx1.freebsd.org (Postfix) with ESMTP id B26E1B4F for ; Tue, 3 Mar 2015 19:14:29 +0000 (UTC) Received: from [10.211.109.92] (unknown [216.205.235.226]) by elvis.mu.org (Postfix) with ESMTPSA id C96DF341F869; Tue, 3 Mar 2015 11:14:28 -0800 (PST) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (1.0) Subject: Re: Massive libxo-zation that breaks everything From: Alfred Perlstein X-Mailer: iPhone Mail (12B440) In-Reply-To: Date: Tue, 3 Mar 2015 11:14:19 -0800 Content-Transfer-Encoding: quoted-printable Message-Id: <569FDEF4-D6D1-4994-A30B-9D9BCE8B1858@mu.org> References: <54F34B6E.2040809@astrodoggroup.com> <54F35F29.4000603@astrodoggroup.com> <54F429EF.5050400@freebsd.org> <54F46536.8040607@mu.org> <54F4C03F.7030704@freebsd.org> <54F4FECB.90501@freebsd.org> <197AA5DC-0591-4F71-BF10-51A5D8104C11@mu.org> <54F5270D.1010009@freebsd.org> <20150303173223.GQ95752@strugglingcoder.info> <93307065-F026-42DE-A2D2-3843619BAD1A@mu.org> To: Justin Hibbits Cc: Harrison Grundy , "freebsd-current@freebsd.org" , hiren panchasara X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 03 Mar 2015 19:14:29 -0000 > On Mar 3, 2015, at 11:07 AM, Justin Hibbits wrote:= >=20 >> On Tue, Mar 3, 2015 at 10:39 AM, Alfred Perlstein wrote: >>=20 >>=20 >> Sent from my iPhone >>=20 >>>> On Mar 3, 2015, at 9:32 AM, hiren panchasara wrote: >>>>=20 >>>> On 03/02/15 at 07:33P, Alfred Perlstein wrote: >>>>=20 >>>>=20 >>>> Actually I want to shame third party ports into adopting libxo (or at l= east providing machine readable output). >>>>=20 >>>> I know it's scary to try to lead the pack, after all we could be wrong,= but maybe it's time to try something new and see what happens. >>>>=20 >>>> And no, your idea doesn't make sense it just will lead to those files b= it rotting. >>>>=20 >>>> Bedsides that you don't even have a real spec other than "it should be d= one differently". >>>>=20 >>>> Again, show the code. >>>=20 >>> Wow. He told you want he didn't like and he moved on. I hope/wish we as >>> a project can take criticism more positively than this. >>>=20 >>> Answer to every criticism should not be "show me your code". We (should)= >>> know better than that. >>=20 >> Maybe I am too old school but the motd on freefall is still "now shut up a= nd code". I still believe that's right. >>=20 >> We ALL need to know when to step back from an issue we really do not have= the bandwidth to constructively contribute to. >>=20 >> This doesn't mean ideas or concerns aren't valid, but that unless you tru= ly have the bandwidth to contribute, taking a step back is a good idea. >>=20 >> -Alfred. >=20 > You're both right. Maybe we need a hackathon on this. Is BSDCan too late= ? Hackathon on what exactly? -Alfred=20= From owner-freebsd-current@FreeBSD.ORG Tue Mar 3 20:02:22 2015 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 45719704 for ; Tue, 3 Mar 2015 20:02:22 +0000 (UTC) Received: from mail-lb0-x234.google.com (mail-lb0-x234.google.com [IPv6:2a00:1450:4010:c04::234]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B0107D5 for ; Tue, 3 Mar 2015 20:02:21 +0000 (UTC) Received: by lbiz11 with SMTP id z11so13652758lbi.3 for ; Tue, 03 Mar 2015 12:02:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=QTs2TY24yppJ/z1qUedRBoP9hhGW7coHvwot/ZbFOJg=; b=QVpdjYTcpjRCeOrNp+sPExMMPuZlBjlzZL/a8wZu7hHunbAQhVOcP1BppD+eubjDzK +sofRPPsaJrhEAc7jfQzPXzTdSwEE+sRlfRDH3xJx5AGQmroAnDkzuVNCthqZeKVj6VU 9SZiEEp3KmeyKmzGqanN6TQDegsqKi9wJEWn3iwm31qg3zhU27x7wS+KU9EK4pgLlBT8 W0i6SQRe6YHvH+QpiJOSEWgxPunyM/A9CtsI+5MJnOON1Jo4A45QuCytvLnIRBrBKHKu 1ofgPJgrOCv9+njy8UuGikx2VFTMDa03X/n1IenULLUNe49csBxAxqEL2TdmW1Blrxrd 4OVA== MIME-Version: 1.0 X-Received: by 10.152.20.201 with SMTP id p9mr555104lae.50.1425412939679; Tue, 03 Mar 2015 12:02:19 -0800 (PST) Sender: chmeeedalf@gmail.com Received: by 10.25.145.65 with HTTP; Tue, 3 Mar 2015 12:02:19 -0800 (PST) In-Reply-To: <569FDEF4-D6D1-4994-A30B-9D9BCE8B1858@mu.org> References: <54F34B6E.2040809@astrodoggroup.com> <54F35F29.4000603@astrodoggroup.com> <54F429EF.5050400@freebsd.org> <54F46536.8040607@mu.org> <54F4C03F.7030704@freebsd.org> <54F4FECB.90501@freebsd.org> <197AA5DC-0591-4F71-BF10-51A5D8104C11@mu.org> <54F5270D.1010009@freebsd.org> <20150303173223.GQ95752@strugglingcoder.info> <93307065-F026-42DE-A2D2-3843619BAD1A@mu.org> <569FDEF4-D6D1-4994-A30B-9D9BCE8B1858@mu.org> Date: Tue, 3 Mar 2015 12:02:19 -0800 X-Google-Sender-Auth: X2Ne_VBO5cEMRShlAX0XzKpqb-o Message-ID: Subject: Re: Massive libxo-zation that breaks everything From: Justin Hibbits To: Alfred Perlstein Content-Type: text/plain; charset=UTF-8 Cc: Harrison Grundy , "freebsd-current@freebsd.org" , hiren panchasara X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 03 Mar 2015 20:02:22 -0000 On Tue, Mar 3, 2015 at 11:14 AM, Alfred Perlstein wrote: > > >> On Mar 3, 2015, at 11:07 AM, Justin Hibbits wrote: >> >>> On Tue, Mar 3, 2015 at 10:39 AM, Alfred Perlstein wrote: >>> >>> >>> Sent from my iPhone >>> >>>>> On Mar 3, 2015, at 9:32 AM, hiren panchasara wrote: >>>>> >>>>> On 03/02/15 at 07:33P, Alfred Perlstein wrote: >>>>> >>>>> >>>>> Actually I want to shame third party ports into adopting libxo (or at least providing machine readable output). >>>>> >>>>> I know it's scary to try to lead the pack, after all we could be wrong, but maybe it's time to try something new and see what happens. >>>>> >>>>> And no, your idea doesn't make sense it just will lead to those files bit rotting. >>>>> >>>>> Bedsides that you don't even have a real spec other than "it should be done differently". >>>>> >>>>> Again, show the code. >>>> >>>> Wow. He told you want he didn't like and he moved on. I hope/wish we as >>>> a project can take criticism more positively than this. >>>> >>>> Answer to every criticism should not be "show me your code". We (should) >>>> know better than that. >>> >>> Maybe I am too old school but the motd on freefall is still "now shut up and code". I still believe that's right. >>> >>> We ALL need to know when to step back from an issue we really do not have the bandwidth to constructively contribute to. >>> >>> This doesn't mean ideas or concerns aren't valid, but that unless you truly have the bandwidth to contribute, taking a step back is a good idea. >>> >>> -Alfred. >> >> You're both right. Maybe we need a hackathon on this. Is BSDCan too late? > > Hackathon on what exactly? > > -Alfred libifying base. And, if there's still stuff that needs libxo'd, do that, too. Looking at both could help to derive a clean library API that libxo'd interfaces can use. libifying base has been on my "interesting TODO" list for the last 10 years or so, I just haven't had the inclination. Now that there's been a swell of support for it, getting people together to actually make it happen seems like a good idea. - Justin From owner-freebsd-current@FreeBSD.ORG Tue Mar 3 20:27:25 2015 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 5A9063CC; Tue, 3 Mar 2015 20:27:25 +0000 (UTC) Received: from wa3yre.wynn.com (wa3yre.wynn.com [199.89.147.3]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 10FB53F4; Tue, 3 Mar 2015 20:27:24 +0000 (UTC) Received: from ivory.wynn.com (mail.wynn.com [199.89.147.3]) (authenticated bits=0) by wa3yre.wynn.com (8.14.3/8.12.6) with ESMTP id t23KREcY086663; Tue, 3 Mar 2015 15:27:14 -0500 (EST) (envelope-from freebsd-arm@wynn.com) Date: Tue, 3 Mar 2015 15:27:13 -0500 From: Brett Wynkoop To: Hans Petter Selasky , freebsd-arm@freebsd.org Subject: Re: crash on writing usbstick Message-ID: <20150303152713.5b97c487@ivory.wynn.com> In-Reply-To: <54F57979.3060008@selasky.org> References: <20150301041855.5352663e@ivory.wynn.com> <20150301144653.63b38cdf@ivory.wynn.com> <20150301184456.7b5e6487@ivory.wynn.com> <1DC8221F-64EA-418C-8CE5-5FFA4F3DBC64@bsdimp.com> <20150301203244.55578413@ivory.wynn.com> <20150302214352.5143d72e@ivory.wynn.com> <54F57979.3060008@selasky.org> X-Mailer: Claws Mail 3.10.1 (GTK+ 2.24.25; x86_64-apple-darwin10.8.0) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Mailman-Approved-At: Tue, 03 Mar 2015 20:50:15 +0000 Cc: Brett Wynkoop , freebsd-current@freebsd.org, Warner Losh X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 03 Mar 2015 20:27:25 -0000 Greeting- So can others duplicate my results, or should I give some kernel dev access to my console server and my BeagleBone? -Brett -- wynkoop@wynn.com http://prd4.wynn.com/wynkoop/pgp-keys.txt 917-642-6925 929-272-0000 Amendment IV The right of the people to be secure in their persons, houses, papers, and effects, against unreasonable searches and seizures, shall not be violated, and no warrants shall issue, but upon probable cause, supported by oath or affirmation, and particularly describing the place to be searched, and the persons or things to be seized. From owner-freebsd-current@FreeBSD.ORG Tue Mar 3 22:14:15 2015 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C620BD93; Tue, 3 Mar 2015 22:14:15 +0000 (UTC) Received: from mail-pa0-x22a.google.com (mail-pa0-x22a.google.com [IPv6:2607:f8b0:400e:c03::22a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8E42321C; Tue, 3 Mar 2015 22:14:15 +0000 (UTC) Received: by pablf10 with SMTP id lf10so56965680pab.12; Tue, 03 Mar 2015 14:14:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; bh=R4fmBldnCJVk+J6QXbMoHg5+sjnmiTeHcHD5ImjKmkU=; b=LRDYDzDlqXAnY3KvlN3bHpugWlSzP2kaD2+ddiCV4ILVrb15ZeeqQsVz8bux/+Uyw9 152ECd9hXHaB9ndLb4uISEIrHcU6o77VxJkdKpk7ztjyFaRSLnp498GE5ipEvTNbmipA ipTdu2SHUfjs1YZ4mTMeKHaAsQ2cinmM/e2fgIExZVvN2IkmG23xT4OzStkRD5oAtntH 6t5o9zcA8GwdDjrVaY/rwixN8pWJrdR7vU4byXddsHwICT6a1oN31ZnuObI+QSB70oPP Afmo9wcIZxZBGUDyZnwlW/HjMR1BTgw4fJzfhrhU3OykevegGn+Z2rrSSaFQ2Vcn6XQc E77Q== X-Received: by 10.66.218.129 with SMTP id pg1mr1411542pac.65.1425420855005; Tue, 03 Mar 2015 14:14:15 -0800 (PST) MIME-Version: 1.0 Received: by 10.66.237.39 with HTTP; Tue, 3 Mar 2015 14:13:54 -0800 (PST) In-Reply-To: References: <54F31510.7050607@hot.ee> <54F34B6E.2040809@astrodoggroup.com> <54F35F29.4000603@astrodoggroup.com> From: Arseny Nasokin Date: Wed, 4 Mar 2015 02:13:54 +0400 Message-ID: Subject: Re: Massive libxo-zation that breaks everything To: Craig Rodrigues Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 Cc: Harrison Grundy , freebsd-current Current X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 03 Mar 2015 22:14:15 -0000 On 2 March 2015 at 00:25, Craig Rodrigues wrote: > On Sun, Mar 1, 2015 at 10:49 AM, Harrison Grundy < > harrison.grundy@astrodoggroup.com> wrote: > > > Thanks! > > > > That does seem useful, but I'm not sure I see the reasoning behind > > putting into base, over a port or package > > > > > > , since processing XML in base is a pain, and it can't serve up JSON or > > HTML without additional > > utilities anyway. > > > > > I think if you take another pass at reading the entire thread of responses > to see the discussion for the motivation behind libxo, you will get the > idea: > https://lists.freebsd.org/pipermail/freebsd-arch/2014-July/015633.html > > > There are many people who are building products on top of FreeBSD. > For these people, it is super useful for the base utilities in FreeBSD > to emit output in properly formatted XML or JSON. > > That way, they do not need to write scripts to take the output of > say, netstat, and use awk/sed/whatever scripts to take the human readable > netstat output and convert it to a form which can be used in a script. > > There are many, many parsers for XML and JSON not in the base system. > For people building products on top of FreeBSD, they don't care > if these parsers are not in the base, since they can add these parsers on > top of base FreeBSD. > > For example, languages like Python and Ruby have excellent parsers > for JSON and XML. Many people build products using these languages. > There are JSON and XML parsers in C, C++, and other languages as well. > > In addition to people building products, the other audience of people who > benefit from libxo are devops people. > These days, devops folks have no problem using Python, Ruby, Perl, > whatever to write scripts to interact with Unix boxes and pull information > off of it. Having the base utilities emit info in native JSON or XML > greatly facilitates this. > > I talked to one person who is improving FreeBSD support for Saltstack (a > devops framework). He told me > that if more base utilities such as sysctl, could use libxo to emit output > in JSON, that would greatly facilitate improving FreeBSD support for > these devops frameworks. That is because Saltstack would require > less FreeBSD-specific parsing code for getting info from base utilities. > > -- > Craig > _______________________________________________ > freebsd-current@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-current > To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org" > Craig, I think it's bad idea. If you look deep inside Mac OS X, you'll find plist(5) or similar format output of few utilities. They leave application output as is, and give library access (with Objective C Frameworks) to everyone who want to build anything. It includes Scripting Bridge interface to have public API to any program which exposes it. Yes, programs has libraries to be able to read and output JSON/XML/Plist formats natively. The better idea is give well-designed API and language bindings like py-objc or jpype for example. I don't think you really need libdf if you have libutil(3) to have information about free space. -- Eir Nym From owner-freebsd-current@FreeBSD.ORG Tue Mar 3 22:15:31 2015 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 55FB6EEA; Tue, 3 Mar 2015 22:15:31 +0000 (UTC) Received: from mail.made4.biz (mail.made4.biz [IPv6:2001:41d0:2:c018::1:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 16DAF23A; Tue, 3 Mar 2015 22:15:31 +0000 (UTC) Received: from 2a02-8428-011b-e000-0290-f5ff-fe9d-b78c.rev.sfr.net ([2a02:8428:11b:e000:290:f5ff:fe9d:b78c] helo=magellan.dumbbell.fr) by mail.made4.biz with esmtpsa (TLSv1.2:DHE-RSA-AES128-SHA:128) (Exim 4.85 (FreeBSD)) (envelope-from ) id 1YSv69-000FnA-4B; Tue, 03 Mar 2015 23:15:29 +0100 Message-ID: <54F6327C.1000801@FreeBSD.org> Date: Tue, 03 Mar 2015 23:15:24 +0100 From: =?windows-1252?Q?Jean-S=E9bastien_P=E9dron?= User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:31.0) Gecko/20100101 Thunderbird/31.4.0 MIME-Version: 1.0 To: freebsd-current@freebsd.org Subject: Re: system failure: cannot kill webkit-gtk3 if using dumbell/radeon References: <20150227183143.708a0656@rsbsd.rsb> In-Reply-To: <20150227183143.708a0656@rsbsd.rsb> Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="fUmIdmuEp3Klfwc296qI1bpqlX3vGueIN" Cc: Koop Mast X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 03 Mar 2015 22:15:31 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --fUmIdmuEp3Klfwc296qI1bpqlX3vGueIN Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable Hi! Sorry for the delay to get back to you (including your private emails). I'll answer to all at once here, I hope you don't mind :) I'm not sure the issues you have with the 'h' letter in Midori, the flickering you see with some applications/desktop environments or the slim/GDM failure are related to this DRM update or the kernel video drivers in general. I CC'd Koop Mast, in case he can help debug GDM. You mentionned "ghost text" with nano in vt(4). Could you please file a bug in Bugzilla with a picture of what you get? Please join a dmesg too. However, the unkillable processes are interesting. If you can reproduce this, could you please post the output of "procstat -kk -a" when this happens? --=20 Jean-S=E9bastien P=E9dron --fUmIdmuEp3Klfwc296qI1bpqlX3vGueIN Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQJ8BAEBCgBmBQJU9jKAXxSAAAAAAC4AKGlzc3Vlci1mcHJAbm90YXRpb25zLm9w ZW5wZ3AuZmlmdGhob3JzZW1hbi5uZXQ2NzA4N0ZEMUFFQUUwRTEyREJDNkE2RjAz OUU5OTc2MUE1RkQ5NENDAAoJEDnpl2Gl/ZTM+RYP/1n3hwjV+tlvCgkXdwd3+VKp lCQEWdrOBvLLAAs9l5Fp/ctwEGO1fZl4gfgCCM7S8Bv2aPaNNb0AGY6rUxtDWuZM /7QwE/LjUNxKHW3X/Q/Bj4w4FCHgHGsa/OszBV6jLSrrkJx5oP9OSELgmnL1lvgP b4/u0PvJfNJQUrXlw+S9kGOv7+25FYr9FWY0yCkym26UyJFhv+LQA9wYTSN/Jq+i EmVVsFzeOP8lyym+XV4PNPcTPQJrjOHOVS9FHU6jY0ef0tdoljCDFbdGYzoYIMLD fJNwfX+5haHrWCIanGwrRaC+0cmfvVAzbowg4xco56JOa3fFq1jcaj+/Gn8GG2Jn dpn7b4094V7BmVCj9Nsa3ZPp+qfYqvxSFuvD5lFd70gJ26GU8epj3ttMIbozoNvP e6kqwJ8AgOuyuRvpEGDVsdvg3DJLxuzcdwb16sZrF4iita/Jfqa7MALyftj4oUNl K7naxsQ+gwG5Or/dTnvW9o85ruq9jxQzoEC49G2jSkRdJ5kEuvyGzh2Vc6dndNXB BJEJUwpJ1SE222O45QSsLbk9Q6uyIlcTeSs7nafDuNPSXrVr4/sNXB7ypKtTBvPQ FqYY7g0d0Tgy1ERiwZs+pWq43jx25CCxFQtES8tNczRmJqXYcn3XFB7v+aXH7vHg i+hjNadZ3Gh52fRbd2z4 =H0fl -----END PGP SIGNATURE----- --fUmIdmuEp3Klfwc296qI1bpqlX3vGueIN-- From owner-freebsd-current@FreeBSD.ORG Tue Mar 3 22:33:30 2015 Return-Path: Delivered-To: freebsd-current@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 2BF7A1C1; Tue, 3 Mar 2015 22:33:30 +0000 (UTC) Received: from mail.made4.biz (mail.made4.biz [IPv6:2001:41d0:2:c018::1:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id E117760D; Tue, 3 Mar 2015 22:33:29 +0000 (UTC) Received: from 2a02-8428-011b-e000-0290-f5ff-fe9d-b78c.rev.sfr.net ([2a02:8428:11b:e000:290:f5ff:fe9d:b78c] helo=magellan.dumbbell.fr) by mail.made4.biz with esmtpsa (TLSv1.2:DHE-RSA-AES128-SHA:128) (Exim 4.85 (FreeBSD)) (envelope-from ) id 1YSvNX-000FtF-Vc; Tue, 03 Mar 2015 23:33:28 +0100 Message-ID: <54F636B3.90701@FreeBSD.org> Date: Tue, 03 Mar 2015 23:33:23 +0100 From: =?UTF-8?B?SmVhbi1Tw6liYXN0aWVuIFDDqWRyb24=?= Reply-To: freebsd-x11@FreeBSD.org User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:31.0) Gecko/20100101 Thunderbird/31.4.0 MIME-Version: 1.0 To: freebsd-x11@FreeBSD.org, freebsd-current@FreeBSD.org, Hans Petter Selasky , Ed Maste Subject: [Call for testers] DRM device-independent code update to Linux 3.8 (take #2) Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="dGgxjfWjVtuAajoaoFSBsSLtcaiulFvxf" X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 03 Mar 2015 22:33:30 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --dGgxjfWjVtuAajoaoFSBsSLtcaiulFvxf Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Hi! Here is a new patch to based on HEAD r279508: https://people.freebsd.org/~dumbbell/graphics/drm-update-38.i.patch You can apply it to a Subversion checkout using the following command: svn patch drm-update-38.i.patch There are few changes: o The panic reported by J.R. Oldroyd is fixed, but not the CP init problem. o A lock assert was added, suggested by Konstantin Belousov I had several panics ("Stray timeout") with a taskqueue used by TTM, but it didn't occur in the past 6 days. Maybe it was a problem outside of DRM= =2E I would like people to test again and report :) If something fails, please post a full dmesg, taken after loading the relevant *kms module, or the core.txt.$N file if it's a panic. Hans Petter and Ed, I couldn't reproduce neither of your problems (HDMI hotplug events storm and VT-switch misbehaviour). However, they both involve callouts, like the "Stray timeout" panic I had with TTM. I suspect there was a transient problem with callouts in HEAD at the same time. Could you please test again with this patch and a very recent HEAD?= Thank you to everyone! --=20 Jean-S=C3=A9bastien P=C3=A9dron --dGgxjfWjVtuAajoaoFSBsSLtcaiulFvxf Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQJ8BAEBCgBmBQJU9jazXxSAAAAAAC4AKGlzc3Vlci1mcHJAbm90YXRpb25zLm9w ZW5wZ3AuZmlmdGhob3JzZW1hbi5uZXQ2NzA4N0ZEMUFFQUUwRTEyREJDNkE2RjAz OUU5OTc2MUE1RkQ5NENDAAoJEDnpl2Gl/ZTM6+8QANDGUSiPTgvieM88jF3c7XZL rL6RYTuMNbZuNevAcoQzEAPsiuIDbSfjBr5FHaRhGm/mtCtODGS2QHe2SYunNo1k gaQflZ2EwrQrOMHa7lzMzSVlY7lNu2G1BMs5i5bU38+5CxoSFNXVlTMFbH7Wc36k LIqdvjXgEVSJV/RhBkTq6g8HmWQZz5TGKjUTRIUhlg6VD6V7cOkcoBjNh2oJppN/ tcwGADYVbguYJX2eJ1s9wkWRIaYc81bxmDrvcYg28YxHMUIhtGrCl7g85x+Aef17 4Y+B24wRREw9W+Z4jAwvbKBUhFAIhJI9pz7cg2Xnd5HHoLx+mRW6XVFoT0u8kHBA MBqXuZe/uNycyJjg+pHcq8rHESnb31rwPKAex9SqnSTORu4nI8ByYaUG7hT5Y0/e 0EIzvczrglUoYdm+o4vFhvuN039dAMwLjsR1YIUzUtOTfeRdadv3YBMV8IxjNHQa phQ3vtXf4Shkk2hZ345opV+cjfjIEkaFM8kXtCL2hSvepq42Bcy1lTt0ZQOHNPUr mat26iNaR1Je7MCKfD3sBicpuDKlxhmkSY9BHJUHLTFgQPl+pEJQ1vXYrxUgPOoT uxQyJakAhqVj5Yag8RAzGobWi50C+ZDqCD5Nc0dq4n6J40UV1kVb0iWuH9PQ6rPW TyGDJYSTpvCymk5VlmMA =ixrP -----END PGP SIGNATURE----- --dGgxjfWjVtuAajoaoFSBsSLtcaiulFvxf-- From owner-freebsd-current@FreeBSD.ORG Wed Mar 4 03:29:48 2015 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E02526EB; Wed, 4 Mar 2015 03:29:48 +0000 (UTC) Received: from troutmask.apl.washington.edu (troutmask.apl.washington.edu [128.95.76.21]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "troutmask", Issuer "troutmask" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id B95CC8CB; Wed, 4 Mar 2015 03:29:48 +0000 (UTC) Received: from troutmask.apl.washington.edu (localhost [127.0.0.1]) by troutmask.apl.washington.edu (8.14.9/8.14.9) with ESMTP id t243TgAY092800 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Tue, 3 Mar 2015 19:29:42 -0800 (PST) (envelope-from sgk@troutmask.apl.washington.edu) Received: (from sgk@localhost) by troutmask.apl.washington.edu (8.14.9/8.14.9/Submit) id t243Tgxi092799; Tue, 3 Mar 2015 19:29:42 -0800 (PST) (envelope-from sgk) Date: Tue, 3 Mar 2015 19:29:41 -0800 From: Steve Kargl To: David Chisnall Subject: Re: Massive libxo-zation that breaks everything Message-ID: <20150304032941.GA92784@troutmask.apl.washington.edu> References: <54F34B6E.2040809@astrodoggroup.com> <54F35F29.4000603@astrodoggroup.com> <54F429EF.5050400@freebsd.org> <54F46536.8040607@mu.org> <54F4C03F.7030704@freebsd.org> <54F4FECB.90501@freebsd.org> <54F504EF.3030303@freebsd.org> <54F50F15.4050300@freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.23 (2014-03-12) Cc: freebsd-current@freebsd.org, Allan Jude X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 04 Mar 2015 03:29:49 -0000 On Tue, Mar 03, 2015 at 09:09:43AM +0000, David Chisnall wrote: > > If your argument is about binary size, then it would be relatively > easy for us to add a version of libxo for static linking into the > versions in /rescue that only supported plain-text output, but > again, please quantify your objections: What is the size increase > that you're seeing and what kind of devices is this causing a problem for? > Well, since you asked, with libxo -rwxr-xr-x 1 root wheel - 20104 Mar 3 19:16 w* dynamic -rwxr-xr-x 1 root wheel - 1011688 Mar 3 19:17 w* static without libxo -rwxr-xr-x 1 root wheel - 16856 Mar 3 19:18 w* dynamic -rwxr-xr-x 1 root wheel - 974680 Mar 3 19:19 w* static 3.17 kB dynamic 36.14 kB static The output from w is fairly simple compared to what ifconfig or netstat or vmstat or ps or any number of other utilities generated. The use of libxo may have an impact on embedded systems with limited resources. Too bad the libxo supporters did not do the damage with a WITH_LIBXO knob. -- Steve From owner-freebsd-current@FreeBSD.ORG Wed Mar 4 06:04:48 2015 Return-Path: Delivered-To: freebsd-current@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D4DE74C6; Wed, 4 Mar 2015 06:04:48 +0000 (UTC) Received: from gw.catspoiler.org (cl-1657.chi-02.us.sixxs.net [IPv6:2001:4978:f:678::2]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 647B5A70; Wed, 4 Mar 2015 06:04:48 +0000 (UTC) Received: from FreeBSD.org (mousie.catspoiler.org [192.168.101.2]) by gw.catspoiler.org (8.13.3/8.13.3) with ESMTP id t2464ZPd057825; Tue, 3 Mar 2015 22:04:39 -0800 (PST) (envelope-from truckman@FreeBSD.org) Message-Id: <201503040604.t2464ZPd057825@gw.catspoiler.org> Date: Tue, 3 Mar 2015 22:04:35 -0800 (PST) From: Don Lewis Subject: Re: Massive libxo-zation that breaks everything To: theraven@FreeBSD.org In-Reply-To: MIME-Version: 1.0 Content-Type: TEXT/plain; charset=us-ascii Cc: freebsd-current@FreeBSD.org, allanjude@FreeBSD.org X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 04 Mar 2015 06:04:48 -0000 On 3 Mar, David Chisnall wrote: > On 3 Mar 2015, at 01:32, Andrey Chernov wrote: >> >> So, why you ever need to modify wc? Just load wc inside your >> json/xml/etc writer, replacing its printf at the ld-elf.so level. > > You can't get structured output from printf() because printf() takes > unstructured input. It's a string with some variables pasted in, but > no awareness of context. > > The libxo changes to the tools are simply marking the output as having > a defined structure. The library then translates this abstract > structure into something that can be parsed easily by external tools. > > If your argument is that the UNIX philosophy is simple tools doing one > thing well, then please remember the context of this philosophy: It > dates back to the original UNIX systems *that did not support shared > libraries* and was an argument used to justify not implementing them. > This is why globbing is in the shell instead of a shared library and > why some variant of mv *.a *.b works on every command-line interface > except for a UNIX shell. > > Even with that in mind, small changes to individual tools are a *lot* > simpler than one massive monolithic tool that understands the output > formats of every other tool in the base system and can transform them. > Why do you think a few library calls in each application is more > complex *than an entire parser per tool*? The proper *nix way is probably for all the tools to only generate machine parsable output in XML or whatever. When human readable output is desired, the output would be piped to a formatter that takes as an argument a file describing the output format that it would use as a template. The formatter should be simple because it only has to understand two input formats: the machine readable output common to all the base tools, and the template file format. netstat is a complicated example because it has a bunch of different output formats to handle all of the different reports that it can generate. Multiple output template files would be required. #!/bin/sh [complicated argument handling here] /usr/bin/netstat_xml $* | /usr/bin/humanize_xml -f $netstatformatfile This scheme makes it easy to generate customized human-readable reports because they only require new template files. No messing around with the source code and recompiling is necessary. As it is, ordinary users can't do that with netstat because it it setgid. Unfortunately this doesn't map well to /rescue. Translating human readable output to a more machine friendly one is a lot more difficult and more likely to be buggy. What happens when the columns run together in the pretty human-oriented output because some of the values got too large? How do you do regression testing on that? From owner-freebsd-current@FreeBSD.ORG Wed Mar 4 11:07:48 2015 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 6B515927 for ; Wed, 4 Mar 2015 11:07:48 +0000 (UTC) Received: from dec.sakura.ne.jp (dec.sakura.ne.jp [210.188.226.8]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id E1369FF3 for ; Wed, 4 Mar 2015 11:07:47 +0000 (UTC) Received: from fortune.joker.local (180-199-46-187.nagoya1.commufa.jp [180.199.46.187]) (authenticated bits=0) by dec.sakura.ne.jp (8.14.3/8.14.2/[SAKURA-WEB]/20080708) with ESMTP id t24B7d9N042523 for ; Wed, 4 Mar 2015 20:07:39 +0900 (JST) (envelope-from junchoon@dec.sakura.ne.jp) Date: Wed, 4 Mar 2015 20:07:39 +0900 From: Tomoaki AOKI To: freebsd-current@freebsd.org Subject: sysutils/lsof does not build (maybe) after r279433 Message-Id: <20150304200739.e8e2bcf2a1d517dac626e601@dec.sakura.ne.jp> Organization: Junchoon corps X-Mailer: Sylpheed 3.4.2 (GTK+ 2.24.25; amd64-portbld-freebsd10.1) Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="Multipart=_Wed__4_Mar_2015_20_07_39_+0900_uGS=LZUaxwDjZaQt" X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 04 Mar 2015 11:07:48 -0000 This is a multi-part message in MIME format. --Multipart=_Wed__4_Mar_2015_20_07_39_+0900_uGS=LZUaxwDjZaQt Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Hi. Today I upgraded my -head (amd64) VM from r279417 to r279579, and sysutils/lsof does not build any more. Looking into build log (attached), it seems that r279433 broke build. (Not actually bi-sected) Fix: Not known. But maybe renaming asprintf() and vasprintf() in sys/sys/systm.h and its consumers to avoid conflicts would help from base side. Can sysutils/lsof side support this? -- Tomoaki AOKI junchoon@dec.sakura.ne.jp --Multipart=_Wed__4_Mar_2015_20_07_39_+0900_uGS=LZUaxwDjZaQt Content-Type: application/octet-stream; name="lsof_20150304.log" Content-Disposition: attachment; filename="lsof_20150304.log" Content-Transfer-Encoding: base64 PT09PiAgQ2xlYW5pbmcgZm9yIGxzb2YtNC44OS5jLDgNCj09PT4gICBsc29mLTQuODkuYyw4IGRl cGVuZHMgb24gZmlsZTogL3Vzci9sb2NhbC9zYmluL3BrZyAtIGZvdW5kDQo9PT0+IEZldGNoaW5n IGFsbCBkaXN0ZmlsZXMgcmVxdWlyZWQgYnkgbHNvZi00Ljg5LmMsOCBmb3IgYnVpbGRpbmcNCj09 PT4gIEV4dHJhY3RpbmcgZm9yIGxzb2YtNC44OS5jLDgNCj0+IFNIQTI1NiBDaGVja3N1bSBPSyBm b3IgbHNvZl80Ljg5Qy5mcmVlYnNkLnRhci5iejIuDQo9PT0+ICBQYXRjaGluZyBmb3IgbHNvZi00 Ljg5LmMsOA0KPT09PiAgQ29uZmlndXJpbmcgZm9yIGxzb2YtNC44OS5jLDgNCkNyZWF0aW5nIC4v bG9ja2Zfb3duZXIuaCBmcm9tIC91c3Ivc3JjL3N5cy9rZXJuL2tlcm5fbG9ja2YuYw0KLi9sb2Nr Zl9vd25lci5oIGNyZWF0aW9uIHN1Y2NlZWRlZC4NCnJtIC1mIGRkZXYuYyBkZmlsZS5jIGRsc29m LmggZG1udC5jIGRub2RlKi5jIGRwcm9jLmMgZHByb3RvLmggZHNvY2suYyBkc3RvcmUuYyBkemZz Lmgga2VybmVsYmFzZS5oIG1hY2hpbmUuaCBtYWNoaW5lLmgub2xkIG5ld19tYWNoaW5lLmggX19s c2Vlay5zIE1ha2VmaWxlIE1ha2VmaWxlLnpmcyAuL3Rlc3RzL2NvbmZpZy5jZmxhZ3MNCnJtIC1m IC4vdGVzdHMvY29uZmlnLmNjIC4vdGVzdHMvY29uZmlnLnhvYmogLi90ZXN0cy9jb25maWcubGRm bGFncw0KVGVzdGluZyBDIGxpYnJhcnkgZm9yIGxvY2FsdGltZSgpIGFuZCBzdHJmdGltZSgpLCB1 c2luZyBjYyAuLi4gcHJlc2VudA0KbG4gLXMgZGlhbGVjdHMvZnJlZWJzZC9kbHNvZi5oIGRsc29m LmgNCmxuIC1zIGRpYWxlY3RzL2ZyZWVic2QvZG1udC5jIGRtbnQuYw0KbG4gLXMgZGlhbGVjdHMv ZnJlZWJzZC9kbm9kZS5jIGRub2RlLmMNCmxuIC1zIGRpYWxlY3RzL2ZyZWVic2QvZG5vZGUxLmMg ZG5vZGUxLmMNCmxuIC1zIGRpYWxlY3RzL2ZyZWVic2QvZG5vZGUyLmMgZG5vZGUyLmMNCmxuIC1z IGRpYWxlY3RzL2ZyZWVic2QvZHByb2MuYyBkcHJvYy5jDQpsbiAtcyBkaWFsZWN0cy9mcmVlYnNk L2Rwcm90by5oIGRwcm90by5oDQpsbiAtcyBkaWFsZWN0cy9mcmVlYnNkL2Rzb2NrLmMgZHNvY2su Yw0KbG4gLXMgZGlhbGVjdHMvZnJlZWJzZC9kc3RvcmUuYyBkc3RvcmUuYw0KbG4gLXMgZGlhbGVj dHMvZnJlZWJzZC9kemZzLmggZHpmcy5oDQpsbiAtcyBkaWFsZWN0cy9mcmVlYnNkL21hY2hpbmUu aCBtYWNoaW5lLmgNCk1ha2VmaWxlIGFuZCBsaWIvTWFrZWZpbGUgY3JlYXRlZC4NCk1ha2VmaWxl LnpmcyBjcmVhdGVkLg0KLi90ZXN0cy9jb25maWcuY2MgY3JlYXRlZA0KLi90ZXN0cy9jb25maWcu Y2ZsYWdzIGNyZWF0ZWQNCi4vdGVzdHMvY29uZmlnLmxkZmxhZ3MgY3JlYXRlZA0KLi90ZXN0cy9j b25maWcueG9iaiBjcmVhdGVkDQo9PT0+ICBCdWlsZGluZyBmb3IgbHNvZi00Ljg5LmMsOA0KLS0t IHZlcnNpb24uaCAtLS0NCi0tLSBsaWIvbGlibHNvZi5hIC0tLQ0KLS0tIHZlcnNpb24uaCAtLS0N CkNvbnN0cnVjdGluZyB2ZXJzaW9uLmgNCi0tLSBsaWIvbGlibHNvZi5hIC0tLQ0KKGNkIGxpYjsg L3Vzci9iaW4vbWFrZSBERUJVRz0iLU8yIiBDRkdGPSItcGlwZSAtZnN0YWNrLXByb3RlY3RvciAt Zm5vLXN0cmljdC1hbGlhc2luZyAtRE5FRURTX0JPT0xfVFlQRURFRiAtREhBU1RBU0tTIC1ESEFT X1BBVVNFX1NCVCAtREhBU0VGRk5MSU5LPWlfZWZmbmxpbmsgLURIQVNGX1ZOT0RFIC1ESEFTX0ZJ TEVERVNDRU5UIC1ESEFTX1RNUEZTIC1ESEFTV0NUWVBFX0ggLURIQVNTQlNUQVRFIC1ESEFTX0tW TV9WTk9ERSAtREhBU19VRlMxXzIgLURIQVNfVk1fTUVNQVRUUl9UIC1ESEFTX0NERVYyUFJJViAt REhBU19OT19TSV9VREVWIC1ESEFTX1NZU19TWF9IIC1ESEFTX1pGUyAtREhBU19WX0xPQ0tGIC1E SEFTX0xPQ0tGX0VOVFJZIC1ESEFTX05PXzZQT1JUIC1ESEFTX05PXzZQUENCIC1ETkVFRFNfQk9P TEVBTl9UIC1ESEFTX1NCX0NDQyAtREhBU19GREVTQ0VOVFRCTCAtREZSRUVCU0RWPTExMDAwIC1E SEFTRkRFU0NGUz0yIC1ESEFTUFNFVURPRlMgLURIQVNOVUxMRlMgLURIQVNJUHY2IC1ESEFTVVRN UFggLURIQVNfU1RSRlRJTUUgLURMU09GX1ZTVFI9XCIxMS4wLUNVUlJFTlRcIiIpDQotLS0gY2tr di5vIC0tLQ0KY2MgIC1waXBlIC1mc3RhY2stcHJvdGVjdG9yIC1mbm8tc3RyaWN0LWFsaWFzaW5n IC1ETkVFRFNfQk9PTF9UWVBFREVGIC1ESEFTVEFTS1MgLURIQVNfUEFVU0VfU0JUIC1ESEFTRUZG TkxJTks9aV9lZmZubGluayAtREhBU0ZfVk5PREUgLURIQVNfRklMRURFU0NFTlQgLURIQVNfVE1Q RlMgLURIQVNXQ1RZUEVfSCAtREhBU1NCU1RBVEUgLURIQVNfS1ZNX1ZOT0RFIC1ESEFTX1VGUzFf MiAtREhBU19WTV9NRU1BVFRSX1QgLURIQVNfQ0RFVjJQUklWIC1ESEFTX05PX1NJX1VERVYgLURI QVNfU1lTX1NYX0ggLURIQVNfWkZTIC1ESEFTX1ZfTE9DS0YgLURIQVNfTE9DS0ZfRU5UUlkgLURI QVNfTk9fNlBPUlQgLURIQVNfTk9fNlBQQ0IgLURORUVEU19CT09MRUFOX1QgLURIQVNfU0JfQ0ND IC1ESEFTX0ZERVNDRU5UVEJMIC1ERlJFRUJTRFY9MTEwMDAgLURIQVNGREVTQ0ZTPTIgLURIQVNQ U0VVRE9GUyAtREhBU05VTExGUyAtREhBU0lQdjYgLURIQVNVVE1QWCAtREhBU19TVFJGVElNRSAt RExTT0ZfVlNUUj0iMTEuMC1DVVJSRU5UIiAtSS91c3Ivc3JjL3N5cyAtTzIgLWMgY2trdi5jDQot LS0gY3Zmcy5vIC0tLQ0KY2MgIC1waXBlIC1mc3RhY2stcHJvdGVjdG9yIC1mbm8tc3RyaWN0LWFs aWFzaW5nIC1ETkVFRFNfQk9PTF9UWVBFREVGIC1ESEFTVEFTS1MgLURIQVNfUEFVU0VfU0JUIC1E SEFTRUZGTkxJTks9aV9lZmZubGluayAtREhBU0ZfVk5PREUgLURIQVNfRklMRURFU0NFTlQgLURI QVNfVE1QRlMgLURIQVNXQ1RZUEVfSCAtREhBU1NCU1RBVEUgLURIQVNfS1ZNX1ZOT0RFIC1ESEFT X1VGUzFfMiAtREhBU19WTV9NRU1BVFRSX1QgLURIQVNfQ0RFVjJQUklWIC1ESEFTX05PX1NJX1VE RVYgLURIQVNfU1lTX1NYX0ggLURIQVNfWkZTIC1ESEFTX1ZfTE9DS0YgLURIQVNfTE9DS0ZfRU5U UlkgLURIQVNfTk9fNlBPUlQgLURIQVNfTk9fNlBQQ0IgLURORUVEU19CT09MRUFOX1QgLURIQVNf U0JfQ0NDIC1ESEFTX0ZERVNDRU5UVEJMIC1ERlJFRUJTRFY9MTEwMDAgLURIQVNGREVTQ0ZTPTIg LURIQVNQU0VVRE9GUyAtREhBU05VTExGUyAtREhBU0lQdjYgLURIQVNVVE1QWCAtREhBU19TVFJG VElNRSAtRExTT0ZfVlNUUj0iMTEuMC1DVVJSRU5UIiAtSS91c3Ivc3JjL3N5cyAtTzIgLWMgY3Zm cy5jDQotLS0gZG1udC5vIC0tLQ0KY2MgIC1waXBlIC1mc3RhY2stcHJvdGVjdG9yIC1mbm8tc3Ry aWN0LWFsaWFzaW5nIC1ETkVFRFNfQk9PTF9UWVBFREVGIC1ESEFTVEFTS1MgLURIQVNfUEFVU0Vf U0JUIC1ESEFTRUZGTkxJTks9aV9lZmZubGluayAtREhBU0ZfVk5PREUgLURIQVNfRklMRURFU0NF TlQgLURIQVNfVE1QRlMgLURIQVNXQ1RZUEVfSCAtREhBU1NCU1RBVEUgLURIQVNfS1ZNX1ZOT0RF IC1ESEFTX1VGUzFfMiAtREhBU19WTV9NRU1BVFRSX1QgLURIQVNfQ0RFVjJQUklWIC1ESEFTX05P X1NJX1VERVYgLURIQVNfU1lTX1NYX0ggLURIQVNfWkZTIC1ESEFTX1ZfTE9DS0YgLURIQVNfTE9D S0ZfRU5UUlkgLURIQVNfTk9fNlBPUlQgLURIQVNfTk9fNlBQQ0IgLURORUVEU19CT09MRUFOX1Qg LURIQVNfU0JfQ0NDIC1ESEFTX0ZERVNDRU5UVEJMIC1ERlJFRUJTRFY9MTEwMDAgLURIQVNGREVT Q0ZTPTIgLURIQVNQU0VVRE9GUyAtREhBU05VTExGUyAtREhBU0lQdjYgLURIQVNVVE1QWCAtREhB U19TVFJGVElNRSAtRExTT0ZfVlNUUj1cIjExLjAtQ1VSUkVOVFwiIC1JL3Vzci9zcmMvc3lzIC1P MiAtYyBkbW50LmMNCkluIGZpbGUgaW5jbHVkZWQgZnJvbSBkbW50LmM6Mzk6DQotLS0gbGliL2xp Ymxzb2YuYSAtLS0NCi0tLSBja2t2Lm8gLS0tDQpJbiBmaWxlIGluY2x1ZGVkIGZyb20gY2trdi5j OjQzOg0KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vLi4vbHNvZi5oOjE5NToNCi0tLSBkbW50Lm8g LS0tDQpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi9sc29mLmg6MTk1Og0KSW4gZmlsZSBpbmNsdWRl ZCBmcm9tIC4vZGxzb2YuaDozNzY6DQpJbiBmaWxlIGluY2x1ZGVkIGZyb20gL3Vzci9zcmMvc3lz L3N5cy9maWxlLmg6NDI6DQpJbiBmaWxlIGluY2x1ZGVkIGZyb20gL3Vzci9zcmMvc3lzL3N5cy9y ZWZjb3VudC5oOjM2Og0KL3Vzci9zcmMvc3lzL3N5cy9zeXN0bS5oOjIwODo1OiBlcnJvcjogY29u ZmxpY3RpbmcgdHlwZXMgZm9yICdhc3ByaW50ZicNCmludCAgICAgYXNwcmludGYoY2hhciAqKnJl dCwgc3RydWN0IG1hbGxvY190eXBlICptdHAsIGNvbnN0IGNoYXIgKmZvcm1hdCwgDQogICAgICAg IF4NCi91c3IvaW5jbHVkZS9zdGRpby5oOjM5Njo2OiBub3RlOiBwcmV2aW91cyBkZWNsYXJhdGlv biBpcyBoZXJlDQppbnQgICAgICBhc3ByaW50ZihjaGFyICoqLCBjb25zdCBjaGFyICosIC4uLikg X19wcmludGZsaWtlKDIsIDMpOw0KICAgICAgICAgXg0KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIGRt bnQuYzozOToNCkluIGZpbGUgaW5jbHVkZWQgZnJvbSAuL2xzb2YuaDoxOTU6DQpJbiBmaWxlIGlu Y2x1ZGVkIGZyb20gLi9kbHNvZi5oOjM3NjoNCkluIGZpbGUgaW5jbHVkZWQgZnJvbSAvdXNyL3Ny Yy9zeXMvc3lzL2ZpbGUuaDo0MjoNCkluIGZpbGUgaW5jbHVkZWQgZnJvbSAvdXNyL3NyYy9zeXMv c3lzL3JlZmNvdW50Lmg6MzY6DQovdXNyL3NyYy9zeXMvc3lzL3N5c3RtLmg6MjE1OjU6IGVycm9y OiBjb25mbGljdGluZyB0eXBlcyBmb3IgJ3Zhc3ByaW50ZicNCmludCAgICAgdmFzcHJpbnRmKGNo YXIgKipyZXQsIHN0cnVjdCBtYWxsb2NfdHlwZSAqbXRwLCBjb25zdCBjaGFyICpmb3JtYXQsDQog ICAgICAgIF4NCi91c3IvaW5jbHVkZS9zdGRpby5oOjQwNDo2OiBub3RlOiBwcmV2aW91cyBkZWNs YXJhdGlvbiBpcyBoZXJlDQppbnQgICAgICB2YXNwcmludGYoY2hhciAqKiwgY29uc3QgY2hhciAq LCBfX3ZhX2xpc3QpDQogICAgICAgICBeDQotLS0gbGliL2xpYmxzb2YuYSAtLS0NCkluIGZpbGUg aW5jbHVkZWQgZnJvbSAuLy4uL2Rsc29mLmg6Mzc2Og0KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC91 c3Ivc3JjL3N5cy9zeXMvZmlsZS5oOjQyOg0KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC91c3Ivc3Jj L3N5cy9zeXMvcmVmY291bnQuaDozNjoNCi91c3Ivc3JjL3N5cy9zeXMvc3lzdG0uaDoyMDg6NTog ZXJyb3I6IGNvbmZsaWN0aW5nIHR5cGVzIGZvciAnYXNwcmludGYnDQppbnQgICAgIGFzcHJpbnRm KGNoYXIgKipyZXQsIHN0cnVjdCBtYWxsb2NfdHlwZSAqbXRwLCBjb25zdCBjaGFyICpmb3JtYXQs IA0KICAgICAgICBeDQovdXNyL2luY2x1ZGUvc3RkaW8uaDozOTY6Njogbm90ZTogcHJldmlvdXMg ZGVjbGFyYXRpb24gaXMgaGVyZQ0KaW50ICAgICAgYXNwcmludGYoY2hhciAqKiwgY29uc3QgY2hh ciAqLCAuLi4pIF9fcHJpbnRmbGlrZSgyLCAzKTsNCiAgICAgICAgIF4NCkluIGZpbGUgaW5jbHVk ZWQgZnJvbSBja2t2LmM6NDM6DQpJbiBmaWxlIGluY2x1ZGVkIGZyb20gLi8uLi9sc29mLmg6MTk1 Og0KSW4gZmlsZSBpbmNsdWRlZCBmcm9tIC4vLi4vZGxzb2YuaDozNzY6DQpJbiBmaWxlIGluY2x1 ZGVkIGZyb20gL3Vzci9zcmMvc3lzL3N5cy9maWxlLmg6NDI6DQpJbiBmaWxlIGluY2x1ZGVkIGZy b20gL3Vzci9zcmMvc3lzL3N5cy9yZWZjb3VudC5oOjM2Og0KL3Vzci9zcmMvc3lzL3N5cy9zeXN0 bS5oOjIxNTo1OiBlcnJvcjogY29uZmxpY3RpbmcgdHlwZXMgZm9yICd2YXNwcmludGYnDQppbnQg ICAgIHZhc3ByaW50ZihjaGFyICoqcmV0LCBzdHJ1Y3QgbWFsbG9jX3R5cGUgKm10cCwgY29uc3Qg Y2hhciAqZm9ybWF0LA0KICAgICAgICBeDQovdXNyL2luY2x1ZGUvc3RkaW8uaDo0MDQ6Njogbm90 ZTogcHJldmlvdXMgZGVjbGFyYXRpb24gaXMgaGVyZQ0KaW50ICAgICAgdmFzcHJpbnRmKGNoYXIg KiosIGNvbnN0IGNoYXIgKiwgX192YV9saXN0KQ0KICAgICAgICAgXg0KLS0tIGRtbnQubyAtLS0N CjIgZXJyb3JzIGdlbmVyYXRlZC4NCioqKiBbZG1udC5vXSBFcnJvciBjb2RlIDENCg0KbWFrZVsx XTogc3RvcHBlZCBpbiAvdXNyL3BvcnRzL3N5c3V0aWxzL2xzb2Yvd29yay9sc29mXzQuODlDLmZy ZWVic2QNCi0tLSBsaWIvbGlibHNvZi5hIC0tLQ0KMiBlcnJvcnMgZ2VuZXJhdGVkLg0KKioqIFtj a2t2Lm9dIEVycm9yIGNvZGUgMQ0KDQptYWtlWzJdOiBzdG9wcGVkIGluIC91c3IvcG9ydHMvc3lz dXRpbHMvbHNvZi93b3JrL2xzb2ZfNC44OUMuZnJlZWJzZC9saWINCjEgZXJyb3INCg0KbWFrZVsy XTogc3RvcHBlZCBpbiAvdXNyL3BvcnRzL3N5c3V0aWxzL2xzb2Yvd29yay9sc29mXzQuODlDLmZy ZWVic2QvbGliDQoqKiogW2xpYi9saWJsc29mLmFdIEVycm9yIGNvZGUgMg0KDQptYWtlWzFdOiBz dG9wcGVkIGluIC91c3IvcG9ydHMvc3lzdXRpbHMvbHNvZi93b3JrL2xzb2ZfNC44OUMuZnJlZWJz ZA0KMiBlcnJvcnMNCg0KbWFrZVsxXTogc3RvcHBlZCBpbiAvdXNyL3BvcnRzL3N5c3V0aWxzL2xz b2Yvd29yay9sc29mXzQuODlDLmZyZWVic2QNCj09PT4gQ29tcGlsYXRpb24gZmFpbGVkIHVuZXhw ZWN0ZWRseS4NClRyeSB0byBzZXQgTUFLRV9KT0JTX1VOU0FGRT15ZXMgYW5kIHJlYnVpbGQgYmVm b3JlIHJlcG9ydGluZyB0aGUgZmFpbHVyZSB0bw0KdGhlIG1haW50YWluZXIuDQoqKiogRXJyb3Ig Y29kZSAxDQoNClN0b3AuDQptYWtlOiBzdG9wcGVkIGluIC91c3IvcG9ydHMvc3lzdXRpbHMvbHNv Zg0K --Multipart=_Wed__4_Mar_2015_20_07_39_+0900_uGS=LZUaxwDjZaQt-- From owner-freebsd-current@FreeBSD.ORG Wed Mar 4 11:16:13 2015 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A3D6DCB7; Wed, 4 Mar 2015 11:16:13 +0000 (UTC) Received: from mail-yh0-x22b.google.com (mail-yh0-x22b.google.com [IPv6:2607:f8b0:4002:c01::22b]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5E6EB1BB; Wed, 4 Mar 2015 11:16:13 +0000 (UTC) Received: by yhnv1 with SMTP id v1so4924680yhn.2; Wed, 04 Mar 2015 03:16:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=date:from:to:cc:subject:message-id:mime-version:content-type :content-disposition:user-agent; bh=3rat4aMXUsfK2hzXq2au0gnNlkBfrZ/QQEpftQBvMhM=; b=CsM9ML9zl0kWOthLDxqL3hv86AblXJcnloh8ANvI2BSDxDMdRQ1Z8uCxHt6aws63BL AbICBhmzQIo+1f6iGQdr4ONZsEWeBeqdh6PwAHpA6U+zo9pb1C37kOVc7ZI82jlwEAY1 D0MaI6Bl0jHSCTGqry5WbUE/kwocFpHYe5y5rJXHroPRX8UVGdCqX6+mQ2ouHyMsqh6C YGt2QftBjS3kPxAOH//S8wYnNcj++qVwJq+pgbDieRnMkefQpfZpsfzKn782Hv8yG3Py WhoFEOwT+OjLz825aZ7Wk9oDJwFEMf0lmXi+QmbF6nGSK/gUP9BovB8Gur6idtMAJ9jP bmDg== X-Received: by 10.170.189.141 with SMTP id g135mr2584068yke.37.1425467772512; Wed, 04 Mar 2015 03:16:12 -0800 (PST) Received: from k8-bsd.hsd1.ga.comcast.net ([2601:0:4600:611:2ed0:5aff:fe78:91ca]) by mx.google.com with ESMTPSA id 48sm2856974yhi.0.2015.03.04.03.16.11 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 04 Mar 2015 03:16:11 -0800 (PST) Date: Wed, 4 Mar 2015 06:17:01 -0500 From: R0B_ROD To: jbeich@FreeBSD.org Subject: ports/audio/alsa-lib Error code 74 Message-ID: <20150304111701.GA42872@k8-bsd.hsd1.ga.comcast.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.5.23 (2014-03-12) Cc: freebsd-current@freebsd.org X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 04 Mar 2015 11:16:13 -0000 Hello, First ld could not find python. I removed a $. Now the following: Dunno what to do. PS: Thanks to all for such amazing software! FreeBSD rocks! Willing to be a tester. I have this hobby of breaking things ;) -Roberto Rodriguez Jr /* # diff Makefile.R0B Makefile.11.0-CURRENT 18c18 < CONFIGURE_ARGS= --with-pkgconfdir="\${prefix}/libdata/pkgconfig" --- > CONFIGURE_ARGS= --with-pkgconfdir="\$${prefix}/libdata/pkgconfig" # make install clean ===> Installing for alsa-lib-1.0.29 ===> Checking if alsa-lib already installed ===> Registering installation for alsa-lib-1.0.29 pkg-static: Unable to access file /usr/ports/audio/alsa-lib/work/stage/usr/local /libdata/pkgconfig/alsa.pc: No such file or directory *** Error code 74 Stop. make[1]: stopped in /usr/ports/audio/alsa-lib *** Error code 1 Stop. make: stopped in /usr/ports/audio/alsa-lib From owner-freebsd-current@FreeBSD.ORG Wed Mar 4 11:18:53 2015 Return-Path: Delivered-To: freebsd-current@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 02C04DF7; Wed, 4 Mar 2015 11:18:53 +0000 (UTC) Received: from mail.turbocat.net (mail.turbocat.net [IPv6:2a01:4f8:d16:4514::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 8970D1D5; Wed, 4 Mar 2015 11:18:52 +0000 (UTC) Received: from laptop015.home.selasky.org (cm-176.74.213.204.customer.telag.net [176.74.213.204]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.turbocat.net (Postfix) with ESMTPSA id 1C5931FE022; Wed, 4 Mar 2015 12:18:49 +0100 (CET) Message-ID: <54F6EA48.5050806@selasky.org> Date: Wed, 04 Mar 2015 12:19:36 +0100 From: Hans Petter Selasky User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:31.0) Gecko/20100101 Thunderbird/31.4.0 MIME-Version: 1.0 To: freebsd-x11@FreeBSD.org, freebsd-current@FreeBSD.org, Ed Maste Subject: Re: [Call for testers] DRM device-independent code update to Linux 3.8 (take #2) References: <54F636B3.90701@FreeBSD.org> In-Reply-To: <54F636B3.90701@FreeBSD.org> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 04 Mar 2015 11:18:53 -0000 On 03/03/15 23:33, Jean-Sébastien Pédron wrote: > Hi! > > Here is a new patch to based on HEAD r279508: > https://people.freebsd.org/~dumbbell/graphics/drm-update-38.i.patch > > You can apply it to a Subversion checkout using the following command: > svn patch drm-update-38.i.patch > > There are few changes: > o The panic reported by J.R. Oldroyd is fixed, but not the CP init > problem. > o A lock assert was added, suggested by Konstantin Belousov > > I had several panics ("Stray timeout") with a taskqueue used by TTM, but > it didn't occur in the past 6 days. Maybe it was a problem outside of DRM. > > I would like people to test again and report :) If something fails, > please post a full dmesg, taken after loading the relevant *kms module, > or the core.txt.$N file if it's a panic. > > Hans Petter and Ed, I couldn't reproduce neither of your problems (HDMI > hotplug events storm and VT-switch misbehaviour). However, they both > involve callouts, like the "Stray timeout" panic I had with TTM. I > suspect there was a transient problem with callouts in HEAD at the same > time. Could you please test again with this patch and a very recent HEAD? 1) Callouts are not yet entirely fixed in HEAD yet. Fixes and discussions are still ongoing, see D1894. BTW: I'm using: https://reviews.freebsd.org/D1438 2) Maybe you could consider adding "SUBDIR_PARALLEL=" to some of the drm2 Makefiles? 3) After a lot of digging I found the only way to get flicker free video with XVideo using the Intel driver was to configure X.org like this: Section "Device" Identifier "Device0" Driver "intel" Option "AccelMethod" "sna" Option "TearFree" "true" EndSection Maybe that could be the default? 4) High Xorg usage is not solved: 1026 root 1 26 0 99M 26928K 915gbr 0 1:03 8.89% Xorg 1026 root 1 102 0 99M 26936K CPU0 0 1:52 98.58% Xorg info: [drm] Initialized drm 1.1.0 20060810 drmn0: on vgapci0 iicbus0: on iicbb0 addr 0xff iic0: on iicbus0 iic1: on iicbus1 iicbus2: on iicbb1 addr 0x0 iic2: on iicbus2 iic3: on iicbus3 iicbus4: on iicbb2 addr 0x0 iic4: on iicbus4 iic5: on iicbus5 iicbus6: on iicbb3 addr 0x0 iic6: on iicbus6 iic7: on iicbus7 iicbus8: on iicbb4 addr 0x0 iic8: on iicbus8 iic9: on iicbus9 iicbus10: on iicbb5 addr 0x0 iic10: on iicbus10 iic11: on iicbus11 iicbus12: on iicbb6 addr 0x0 iic12: on iicbus12 iic13: on iicbus13 info: [drm] MSI enabled 1 message(s) info: [drm] Supports vblank timestamp caching Rev 1 (10.10.2010). info: [drm] Driver supports precise vblank timestamp query. drmn0: taking over the fictitious range 0xe0000000-0xf0000000 info: [drm] Enabling RC6 states: RC6 off, RC6p off, RC6pp off info: [drm] Connector VGA-1: get mode from tunables: info: [drm] - kern.vt.fb.modes.VGA-1 info: [drm] - kern.vt.fb.default_mode info: [drm] Connector HDMI-A-1: get mode from tunables: info: [drm] - kern.vt.fb.modes.HDMI-A-1 info: [drm] - kern.vt.fb.default_mode info: [drm] Connector DP-1: get mode from tunables: info: [drm] - kern.vt.fb.modes.DP-1 info: [drm] - kern.vt.fb.default_mode fbd0 on drmn0 VT: Replacing driver "vga" with new "fb". info: [drm] Initialized i915 1.6.0 20080730 for drmn0 on minor 0 error: [drm:pid1026:gen6_sanitize_pm] *ERROR* Power management discrepancy: GEN6_RP_INTERRUPT_LIMITS expected 14000000, was 12060000 info: [drm] GMBUS [gmbus dpd] timed out, falling back to bit banging on pin 6 --HPS From owner-freebsd-current@FreeBSD.ORG Wed Mar 4 11:33:22 2015 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D5E902BB for ; Wed, 4 Mar 2015 11:33:22 +0000 (UTC) Received: from outpost1.zedat.fu-berlin.de (outpost1.zedat.fu-berlin.de [130.133.4.66]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 8E7DF3E2 for ; Wed, 4 Mar 2015 11:33:22 +0000 (UTC) Received: from inpost2.zedat.fu-berlin.de ([130.133.4.69]) by outpost.zedat.fu-berlin.de (Exim 4.85) with esmtp (envelope-from ) id <1YT7YA-000Z7g-HC>; Wed, 04 Mar 2015 12:33:14 +0100 Received: from p578a69f9.dip0.t-ipconnect.de ([87.138.105.249] helo=prometheus) by inpost2.zedat.fu-berlin.de (Exim 4.85) with esmtpsa (envelope-from ) id <1YT7YA-00030o-Eb>; Wed, 04 Mar 2015 12:33:14 +0100 Date: Wed, 4 Mar 2015 12:31:10 +0100 From: "O. Hartmann" To: Ryan Stone Subject: Re: r279514: buildworld failure: /usr/src/lib/libnv/tests/dnv_tests.cc:453:2: error: use of overloaded operator '<<' is ambiguous Message-ID: <20150304123110.48aa4abb@prometheus> In-Reply-To: References: <20150302115057.62d2c74c@prometheus> Organization: FU Berlin X-Mailer: Claws Mail 3.11.1 (GTK+ 2.24.25; amd64-portbld-freebsd11.0) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: base64 X-Originating-IP: 87.138.105.249 Cc: freebsd-current X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 04 Mar 2015 11:33:22 -0000 LS0tLS1CRUdJTiBQR1AgU0lHTkVEIE1FU1NBR0UtLS0tLQ0KSGFzaDogU0hBMjU2DQoNCk9uIE1v biwgMiBNYXIgMjAxNSAwODo1ODowNSAtMDUwMA0KUnlhbiBTdG9uZSA8cnlzdG8zMkBnbWFpbC5j b20+IHdyb3RlOg0KDQo+IENhbiB5b3UgcG9zdCB0aGUgY29udGVudHMgb2YgeW91ciBtYWtlLmNv bmYgYW5kIHNyYy5jb25mPyAgSSBkaWRuJ3QNCj4gc2VlIHRoaXMgaW4gYW55IG9mIG15ICJtYWtl IHRpbmRlcmJveCIgcnVucw0KPiBfX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fXw0KPiBmcmVlYnNkLWN1cnJlbnRAZnJlZWJzZC5vcmcgbWFpbGluZyBsaXN0DQo+ IGh0dHA6Ly9saXN0cy5mcmVlYnNkLm9yZy9tYWlsbWFuL2xpc3RpbmZvL2ZyZWVic2QtY3VycmVu dA0KPiBUbyB1bnN1YnNjcmliZSwgc2VuZCBhbnkgbWFpbCB0byAiZnJlZWJzZC1jdXJyZW50LXVu c3Vic2NyaWJlQGZyZWVic2Qub3JnIg0KDQpUaGUgY3VscHJpdCBpcyB0aGUgb3B0aW9uDQoNCkNY WEZMQUdTKz0gICAgICAgICAgICAgLXN0ZD1jKysxMQ0KDQppbiAvZXRjL3NyYy5jb25mDQotLS0t LUJFR0lOIFBHUCBTSUdOQVRVUkUtLS0tLQ0KVmVyc2lvbjogR251UEcgdjINCg0KaVFFY0JBRUJD QUFHQlFKVTl1eitBQW9KRU9nQmNEN0EvNU44cGMwSUFNK0RDaG1SejlEL05Xek5tTFJzeTErcA0K R0MySmIraGE2bjliKy9JMHJhMlFWZ1kxOTNoMkx2UkFqOXlreW04Vmp6M01kMGduckdJeCtTNVp1 bTlkOHpjNA0KMndNbWQrckoxQ1FmL2NNTzA1alZpdnV2TkNXVEYxYTNpa2dkNnNlRzY3TEVodStI dHFobEtzcVNNTUpJWFF2bQ0KZXVKNHV1M0Y4MVY1S1Vsd3YzdXcxdjhDL1BxTndXZ1NrVW5VaXRD cGxQK0gzWTZSb2lWZjZuWStyZUpxaERKMg0KZjRZUEY5bTFWUHpnWlRvL3BrWFRqbmhHcUlIMFh0 ZytQRW9kV3NtYUh6MVdpWjVzYU1sMmNocDVKRVVlS3hVQQ0KSVpNem1vL2gvZnhIcWNlTmxKUXdC SHRROHdISjVLVHNqRDYwYmlBRTM2UW83bjBUb2JPU0plaFBmbTlNOVZ3PQ0KPVV4dkYNCi0tLS0t RU5EIFBHUCBTSUdOQVRVUkUtLS0tLQ0K From owner-freebsd-current@FreeBSD.ORG Wed Mar 4 13:10:19 2015 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 99537BE8 for ; Wed, 4 Mar 2015 13:10:19 +0000 (UTC) Received: from tensor.andric.com (tensor.andric.com [87.251.56.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "tensor.andric.com", Issuer "COMODO RSA Domain Validation Secure Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 52898FF6 for ; Wed, 4 Mar 2015 13:10:18 +0000 (UTC) Received: from coleburn.avinity.tv (unknown [77.243.161.229]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by tensor.andric.com (Postfix) with ESMTPSA id 104A35C47; Wed, 4 Mar 2015 14:10:08 +0100 (CET) Subject: Re: r279514: buildworld failure: /usr/src/lib/libnv/tests/dnv_tests.cc:453:2: error: use of overloaded operator '<<' is ambiguous Mime-Version: 1.0 (Mac OS X Mail 8.2 \(2070.6\)) Content-Type: multipart/signed; boundary="Apple-Mail=_FDFFEC3E-F77B-4ECB-BCBA-53AA0FB9BA53"; protocol="application/pgp-signature"; micalg=pgp-sha1 X-Pgp-Agent: GPGMail 2.5b5 From: Dimitry Andric In-Reply-To: <20150304123110.48aa4abb@prometheus> Date: Wed, 4 Mar 2015 14:10:00 +0100 Message-Id: <9DE59FA1-D495-4796-B3F5-F96D1472F66C@andric.com> References: <20150302115057.62d2c74c@prometheus> <20150304123110.48aa4abb@prometheus> To: "O. Hartmann" X-Mailer: Apple Mail (2.2070.6) Cc: freebsd-current , Ryan Stone X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 04 Mar 2015 13:10:19 -0000 --Apple-Mail=_FDFFEC3E-F77B-4ECB-BCBA-53AA0FB9BA53 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii On 04 Mar 2015, at 12:31, O. Hartmann = wrote: > On Mon, 2 Mar 2015 08:58:05 -0500 > Ryan Stone wrote: >=20 > > Can you post the contents of your make.conf and src.conf? I didn't > > see this in any of my "make tinderbox" runs > > _______________________________________________ > > freebsd-current@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-current > > To unsubscribe, send any mail to = "freebsd-current-unsubscribe@freebsd.org" >=20 > The culprit is the option >=20 > CXXFLAGS+=3D -std=3Dc++11 >=20 > in /etc/src.conf Right, it would be nice to have libnv compiling for C++11 though. I'll = have a look later today. -Dimitry --Apple-Mail=_FDFFEC3E-F77B-4ECB-BCBA-53AA0FB9BA53 Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- Version: GnuPG/MacGPG2 v2.0.26 iEYEARECAAYFAlT3BC8ACgkQsF6jCi4glqMIXQCgxd1AzkNYCm9/I8XDaYPTwUv+ RwIAoJys5EazMTdoCSWRAfs14NOdDiLz =NQbQ -----END PGP SIGNATURE----- --Apple-Mail=_FDFFEC3E-F77B-4ECB-BCBA-53AA0FB9BA53-- From owner-freebsd-current@FreeBSD.ORG Wed Mar 4 13:18:32 2015 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 968D8E45; Wed, 4 Mar 2015 13:18:32 +0000 (UTC) Received: from oslo.ath.cx (oslo.ath.cx [IPv6:2a01:4f8:200:42e4::1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "oslo.ath.cx", Issuer "oslo.ath.cx" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id BEE55164; Wed, 4 Mar 2015 13:18:31 +0000 (UTC) Received: by oslo.ath.cx (OpenSMTPD) with ESMTP id b1a26a87; Wed, 4 Mar 2015 14:18:29 +0100 (CET) Date: Wed, 4 Mar 2015 14:18:29 +0100 From: "Herbert J. Skuhra" To: R0B_ROD Subject: Re: ports/audio/alsa-lib Error code 74 Message-ID: <20150304131829.GA550@oslo.ath.cx> References: <20150304111701.GA42872@k8-bsd.hsd1.ga.comcast.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20150304111701.GA42872@k8-bsd.hsd1.ga.comcast.net> User-Agent: Mutt/1.5.23 (2014-03-12) Cc: freebsd-current@freebsd.org, jbeich@FreeBSD.org X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 04 Mar 2015 13:18:32 -0000 On Wed, Mar 04, 2015 at 06:17:01AM -0500, R0B_ROD wrote: > Hello, > First ld could not find python. What was the error? Python bindings are optional and off by default. Run 'make config' in audio/alsa-lib to disable it and try again. > I removed a $. This change is wrong. It breaks staging! install -o root -g wheel -m 0644 alsa.pc '/usr/ports/audio/alsa-lib/work/stage\/libdata/pkgconfig' vs. install -o root -g wheel -m 0644 alsa.pc '/usr/ports/audio/alsa-lib/work/stage/usr/local/libdata/pkgconfig' Revert it and try again! -- Herbert From owner-freebsd-current@FreeBSD.ORG Wed Mar 4 13:56:14 2015 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 009E8473 for ; Wed, 4 Mar 2015 13:56:13 +0000 (UTC) Received: from mail-ig0-x233.google.com (mail-ig0-x233.google.com [IPv6:2607:f8b0:4001:c05::233]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B8B377A9 for ; Wed, 4 Mar 2015 13:56:13 +0000 (UTC) Received: by igjz20 with SMTP id z20so36676241igj.4 for ; Wed, 04 Mar 2015 05:56:13 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=ItbajXbcJUZrbtutzg2kc6/4yEAoGXlxEAjg/u3jpLc=; b=wnID4JoLCfGr5WJnJTwpM6G+wlpBIc3z+qKquwY/xFRYRyyrW5rl0LKtBqArnt9Yo9 naRBGyrRMSeeayUTfCvXccTBgVU9d2uPXIazjwZTT5gawOjzCtkmcDWJexq7Cmz2Cb7O xvYErxglspMq03vJUkS3dJ+zKorVYYntRXiHblFrGeS9OtyFQcP1nmjGwdBN2DvieZMJ HSZlPRun84rE0rDCoK3xaBRWc3hdmflWDTpDaCTWG425yYc7TatmZIhXctyDYi1lNOZ8 CaBiU8pDXxgTX4kXFOcUb+iugepjAlA3B0hM9J1tPriHQ1dXRwnN4cqLJHRobfYgcr3D J4Zg== MIME-Version: 1.0 X-Received: by 10.107.168.5 with SMTP id r5mr10946260ioe.87.1425477373149; Wed, 04 Mar 2015 05:56:13 -0800 (PST) Received: by 10.107.156.75 with HTTP; Wed, 4 Mar 2015 05:56:13 -0800 (PST) In-Reply-To: <9DE59FA1-D495-4796-B3F5-F96D1472F66C@andric.com> References: <20150302115057.62d2c74c@prometheus> <20150304123110.48aa4abb@prometheus> <9DE59FA1-D495-4796-B3F5-F96D1472F66C@andric.com> Date: Wed, 4 Mar 2015 08:56:13 -0500 Message-ID: Subject: Re: r279514: buildworld failure: /usr/src/lib/libnv/tests/dnv_tests.cc:453:2: error: use of overloaded operator '<<' is ambiguous From: Ryan Stone To: Dimitry Andric Content-Type: text/plain; charset=UTF-8 Cc: freebsd-current , "O. Hartmann" X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 04 Mar 2015 13:56:14 -0000 I don't think that libnv is the problem. It looks like a problem with atf-c++ to me. From owner-freebsd-current@FreeBSD.ORG Wed Mar 4 16:22:58 2015 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 54526F9F; Wed, 4 Mar 2015 16:22:58 +0000 (UTC) Received: from mail-ie0-x234.google.com (mail-ie0-x234.google.com [IPv6:2607:f8b0:4001:c03::234]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 1A305BA1; Wed, 4 Mar 2015 16:22:58 +0000 (UTC) Received: by iery20 with SMTP id y20so15135113ier.12; Wed, 04 Mar 2015 08:22:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:from:date:message-id:subject:to:cc:content-type; bh=xBmUea42GV/SyVsbhF8jzLeRV8MGHZYMCf1pEP1V0Ig=; b=Uit+KSgn0anDkxvP82QhUYcEBwAz5rkZiPBBHuWzPdR8mYDpM5Upo+ED0K/oV6omkF MvIrbTVcWkCF1yaVnLrxAVr8iWXQwuLkbtTnqVh0kd/cnOto1CUbYJNbPKQHu9rLc9Kk D+Ml1/SF3rPwpLiUMk88B+3xuLUdd4FhmroCI4sNSGWOjCLWsENhz7orYN+9eqpesKhr Rk6lohV00Gwb71dfROXvodxPiSoXoRCMnjmv8Xag8zwUJUnCbqg5MNOByuP0d2ssrjG4 Nz+mcBxPMnL/TYd/RzkZzj4dbE/xYyYjH2+FVPBWPc5AmdPcJ3flx3GDJZWXsYJRxC1i Y6+g== X-Received: by 10.50.164.227 with SMTP id yt3mr12610037igb.32.1425486177410; Wed, 04 Mar 2015 08:22:57 -0800 (PST) MIME-Version: 1.0 Received: by 10.64.107.138 with HTTP; Wed, 4 Mar 2015 08:22:37 -0800 (PST) From: Luca Pizzamiglio Date: Wed, 4 Mar 2015 17:22:37 +0100 Message-ID: Subject: [SOLVED] Re: pcie Realtek 8168G issues (re driver)(minnowboard) To: "O. Hartmann" Content-Type: text/plain; charset=UTF-8 Cc: Ben Perrault , freebsd-net@freebsd.org, FreeBSD Current , "freebsd-hardware@freebsd.org" , FreeBSD Hackers X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 04 Mar 2015 16:22:58 -0000 Hi all, I've managed to get the Realtek 8168g running. It's actually a driver bug, the command register enables rx and tx too early. Apparently, it's OK for many Realtek chips, but not for 3 kind of them, as stated by the Realtek developer, who submitted this patch to Linux (https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=d6e572911a4cb2b9fcd1c26a38d5317a3971f2fd) I updated the Bugzilla entry https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=197535 The patch (moving one line, sich!) works on this hardware, but I don't know how much portable it is. Thanks for all your tips. Best regards, Luca Pizzamiglio On Wed, Feb 25, 2015 at 3:59 PM, Luca Pizzamiglio wrote: > Hi, > thanks you all for the replies. > > Unfortunately, the network chip is still not working and I updated the > PR (https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=197535) with the > last tests. > It seems that received packets are not transferred to mbuf or they are > transferred, but later, after the mbuf is already freed; moreover, the > ring entries are written without looping, overwriting and messing up > the whole kernel memory. It looks like a DMA issues, but > > Apparently it seems a hardware error, but using a Linux distro, it works :( > > Has someone maybe any other ideas? In the meanwhile I'll get another > board with the same chip :O > > Best regards, > Luca > > > On Tue, Feb 17, 2015 at 7:31 PM, O. Hartmann > wrote: >> Am Tue, 17 Feb 2015 18:32:22 +0100 >> Luca Pizzamiglio schrieb: >> >>> Hi Ben, >>> thanks for the tip! tso was already disabled. >>> I tried anyway and unfortunately it crashes as before. >>> >>> I filled a bug report >>> (https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=197535) and marius@ >>> is giving me a big help on it. >>> >>> Best regards, >>> Luca >>> >>> On Fri, Feb 13, 2015 at 7:34 PM, Ben Perrault wrote: >>> > Luca, >>> > >>> > I've had the same issue with this interface on both PCIe boards and embedded in a >>> > handful of Lenovo products. The one, fairly ugly workaround I've found that makes it >>> > work well enough is disable tso ( i.e. ifconfig re0 down && ifconfig re0 -tso && >>> > ifconfig re0 up ). This also seems to stop the panics under current. >>> > >>> > I'm not sure it will work for you - but it has on everyone of those interfaces I've >>> > dealt with. >>> > >>> > Good luck, >>> > -bp >>> > >>> >> On Feb 13, 2015, at 8:06 AM, Luca Pizzamiglio wrote: >>> >> >>> >> Hi, I'm Luca, >>> >> >>> >> I've some issues using a PCIe Realtek Ethernet board: >>> >> re0@pci0:3:0:0: class=0x020000 card=0x012310ec chip=0x816810ec rev=0x0c hdr=0x00 >>> >> vendor = 'Realtek Semiconductor Co., Ltd.' >>> >> device = 'RTL8111/8168B PCI Express Gigabit Ethernet controller' >>> >> class = network >>> >> subclass = ethernet >>> >> bar [10] = type I/O Port, range 32, base 0x1000, size 256, enabled >>> >> bar [18] = type Memory, range 64, base 0x90500000, size 4096, enabled >>> >> bar [20] = type Prefetchable Memory, range 64, base 0x90400000, >>> >> size 16384, enabled >>> >> cap 01[40] = powerspec 3 supports D0 D1 D2 D3 current D0 >>> >> cap 05[50] = MSI supports 1 message, 64 bit >>> >> cap 10[70] = PCI-Express 2 endpoint IRQ 1 max data 128(128) link x1(x1) >>> >> speed 2.5(2.5) ASPM disabled(L0s/L1) >>> >> cap 11[b0] = MSI-X supports 4 messages >>> >> Table in map 0x20[0x0], PBA in map 0x20[0x800] >>> >> cap 03[d0] = VPD >>> >> ecap 0001[100] = AER 1 0 fatal 0 non-fatal 0 corrected >>> >> ecap 0002[140] = VC 1 max VC0 >>> >> ecap 0003[160] = Serial 1 01000000684ce000 >>> >> ecap 0018[170] = LTR 1 >>> >> >>> >> Rx and Tx don't work. After some minutes the interface is activated I >>> >> get kernel panic. >>> >> I've already tried to disable MSIx and MSI. >>> >> It seems a DMA problem, rx fill the 256 descriptors and the nothing >>> >> else until the panic. netstat -s shows now new packets. >>> >> >>> >> I filled a bug report with more infos: >>> >> https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=197535 >>> >> >>> >> could someone kindly pointing some ideas? >>> >> >>> >> Best regards, >>> >> Luca >>> >> _______________________________________________ >>> >> freebsd-current@freebsd.org mailing list >>> >> http://lists.freebsd.org/mailman/listinfo/freebsd-current >>> >> To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org" >>> _______________________________________________ >>> freebsd-current@freebsd.org mailing list >>> http://lists.freebsd.org/mailman/listinfo/freebsd-current >>> To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org" >> >> In September 2014 I filed allready a bug acoording to strange behaviour with a Lenovo >> ThinkPad E540 with a Realtek chip: >> >> >> Bug 193743 - RTL8111/8168B PCI Express Gigabit Ethernet >> controller: doesn't work properly, problems getting UP automatically From owner-freebsd-current@FreeBSD.ORG Wed Mar 4 16:23:26 2015 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C012B2BE; Wed, 4 Mar 2015 16:23:26 +0000 (UTC) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 96749BBB; Wed, 4 Mar 2015 16:23:26 +0000 (UTC) Received: from ralph.baldwin.cx (pool-173-54-116-245.nwrknj.fios.verizon.net [173.54.116.245]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id EDABAB91F; Wed, 4 Mar 2015 11:23:24 -0500 (EST) From: John Baldwin To: freebsd-current@freebsd.org Subject: Re: Massive libxo-zation that breaks everything Date: Wed, 04 Mar 2015 11:21:27 -0500 Message-ID: <15905806.StXgP74c8j@ralph.baldwin.cx> User-Agent: KMail/4.14.2 (FreeBSD/10.1-STABLE; KDE/4.14.2; amd64; ; ) In-Reply-To: References: <54F31510.7050607@hot.ee> <54F50F15.4050300@freebsd.org> MIME-Version: 1.0 Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="us-ascii" X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Wed, 04 Mar 2015 11:23:25 -0500 (EST) Cc: David Chisnall , Allan Jude X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 04 Mar 2015 16:23:26 -0000 On Tuesday, March 03, 2015 09:09:43 AM David Chisnall wrote: > Hopefully there's a lesson here that we can learn from: human-readable > formats do not make good intermediate representations when communicating > between tools. I think this is actually an argument against libxo-ification in the one case where I've cringed a bit at the diffs: pciconf. The current pciconf code is tailored to outputting something human readable. For non-human output I would probably generate different output (not just put tags on the human output) because I would want the non-human output to be both more verbose and more raw. I think some other cases like 'netstat -s' are far more straightforward as the current output maps fairly well to the backing structure, but in general I would want machine-readable output that is closer to the structures than to the human-readable formatting of them. For example, for something like 'mfiutil show drives', I would want the human readable format to stay as it is (it only highlights certain fields in the PD structures) but I would want the machine-readable format to basically output tagged versions of the backing structures from sys/dev/mfi/mfireg.h. That way the machine-readable format has all of the data instead of only the subset that is presented in the human-readable output. So while I am in general a big fan of having machine-readable output from tools (and I think it belongs in the base system, and I don't think you want a post-processing tool), I think there is a bit of a flawed assumption that says that I want the same data in the human-readable format that I want in the machine-readable format. I, for one, don't. I want the human-readable form more condensed. > If your argument is about maintainability of these changes, then please > point to concrete instances where the changes are complex and difficult to > maintain. When I've looked at the xo diffs for pciconf, my reaction has been "ugh, I guess I'm not going to work on pciconf again in the future because that's super ugly". I don't object to the idea, I think I would just rather have a very different schema for machine-readable output. I would probably want pciconf -l in that case to dump the entire PCI header (right now the human- readable pciconf -l only dumps a subset), and I would want it to dump fields in capabilities that we don't currently bother printing (and that I don't think the human-readable output should print due to it being too obscure, etc.) -- John Baldwin From owner-freebsd-current@FreeBSD.ORG Wed Mar 4 16:28:33 2015 Return-Path: Delivered-To: freebsd-current@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id DD980717; Wed, 4 Mar 2015 16:28:32 +0000 (UTC) Received: from pmta2.delivery3.ore.mailhop.org (pmta2.delivery3.ore.mailhop.org [54.213.22.21]) by mx1.freebsd.org (Postfix) with ESMTP id AF63CC3A; Wed, 4 Mar 2015 16:28:32 +0000 (UTC) Received: from smtp1.ore.mailhop.org (172.31.18.134) by pmta2.delivery1.ore.mailhop.org id huspcc20u50j; Wed, 4 Mar 2015 16:28:47 +0000 (envelope-from ) Received: from pool-71-255-170-26.bstnma.east.verizon.net ([71.255.170.26] helo=homobox.opal.com) by smtp1.ore.mailhop.org with esmtpsa (TLSv1.2:DHE-RSA-AES256-GCM-SHA384:256) (Exim 4.82) (envelope-from ) id 1YTC9p-0006V9-Cr; Wed, 04 Mar 2015 16:28:25 +0000 Received: from shibato (ip51cd975c.adsl-surfen.hetnet.nl [81.205.151.92]) (authenticated bits=0) by homobox.opal.com (8.14.9/8.14.9) with ESMTP id t24GSIuB049528 (version=TLSv1/SSLv3 cipher=AES128-GCM-SHA256 bits=128 verify=NO); Wed, 4 Mar 2015 11:28:19 -0500 (EST) (envelope-from fbsd@opal.com) X-Mail-Handler: DuoCircle Outbound SMTP X-Originating-IP: 71.255.170.26 X-Report-Abuse-To: abuse@duocircle.com (see https://support.duocircle.com/support/solutions/articles/5000540958-duocircle-standard-smtp-abuse-information for abuse reporting information) X-MHO-User: U2FsdGVkX1/FYTcpF2gPFE7uT30PyVS9 Date: Wed, 4 Mar 2015 17:28:02 +0100 From: "J.R. Oldroyd" To: =?UTF-8?B?SmVhbi1Tw6liYXN0aWVuIFDDqWRyb24=?= Subject: Re: [Call for testers] DRM device-independent code update to Linux 3.8 (take #2) Message-ID: <20150304172802.0b285585@shibato> In-Reply-To: <54F636B3.90701@FreeBSD.org> References: <54F636B3.90701@FreeBSD.org> X-Mailer: Claws Mail 3.11.1 (GTK+ 2.24.25; amd64-portbld-freebsd10.0) MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; boundary="Sig_/gieHqjGF_8F4qJdiTq5hhuf"; protocol="application/pgp-signature" X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.4.3 (homobox.opal.com [71.255.170.26]); Wed, 04 Mar 2015 11:28:21 -0500 (EST) X-Spam-Status: No, score=3.1 required=5.0 tests=AWL,BAYES_00, FSL_HELO_NON_FQDN_1,OPAL_URI_COUNT_5_9,RCVD_IN_PBL shortcircuit=no autolearn=no autolearn_force=no version=3.4.0 X-Spam-Level: *** X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on homobox.opal.com Cc: Hans Petter Selasky , freebsd-x11@FreeBSD.org, freebsd-current@FreeBSD.org, Ed Maste X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 04 Mar 2015 16:28:33 -0000 --Sig_/gieHqjGF_8F4qJdiTq5hhuf Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable On Tue, 03 Mar 2015 23:33:23 +0100 Jean-S=C3=A9bastien P=C3=A9dron wrote: > > Hi! >=20 > Here is a new patch to based on HEAD r279508: > https://people.freebsd.org/~dumbbell/graphics/drm-update-38.i.patch >=20 > You can apply it to a Subversion checkout using the following command: > svn patch drm-update-38.i.patch >=20 > There are few changes: > o The panic reported by J.R. Oldroyd is fixed, but not the CP init > problem. > o A lock assert was added, suggested by Konstantin Belousov >=20 Tested as requested: svn update to r279508 and applied the 38i patch. Mixed results. The first boot, done as a warm "reboot" command while running 10-stable shows that the CP init failed again and I am back to the mtx_lock panic. I then let it auto-reboot back into 10-stable (this is an old 10-stable at the time of r271969 10.1-BETA2 but it is one in which the ring CP init still works). That shows it boots fine and ring CP inits OK - even as a warm reboot after the previous failure. I then thought to try a power-off cold reboot into r279508. This works, both ring CP inits OK and no mtx_lock panic. In fact, I am using it now to send this report. So, perhaps the previous fix for the mtx_lock panic wasn't right, but just seemed to work because I happened to cold-boot those times. The pattern that's emerging here is that older (mid-2014) kernels boot and init fine, warm or cold. The CP init problem started somewhere mid-2014. The mtx_lock panic is new, so I am guessing related to the drm2-38 update. BOTH CP init and mtx_lock problems go away when cold booting. I've uploaded info here: http://opal.com/jr/freebsd/panic/r279508M/dmesg.txt http://opal.com/jr/freebsd/panic/r279508M/core.txt.7 The dmesg shows the three boot sequences with annotations to make it clear which is which. Above is system with ATI Radeon RS690 X1270 IGP, RS690. BTW, I now also have the ring CP init failure on a second system that was just updated to 10.1-RELEASE-p6. It has ATI Radeon HD 4200 which is RS780 and it also shows the ring init failure at CAFEDEAD but in r600_ting_test(). Unfortunately, this system is a production one so I can't easily play with it. Note, no mtx_lock panic here. Fortunately, the graphics not working isn't a problem on this system. -jr --Sig_/gieHqjGF_8F4qJdiTq5hhuf Content-Type: application/pgp-signature Content-Description: OpenPGP digital signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iEYEARECAAYFAlT3MpoACgkQls33urr0k4m5/wCfQyQVzh4EDNupOpX1D5/CRWtN DnwAnR72NiQ2VMElT6LQhHUtAOMW1S2N =MWKE -----END PGP SIGNATURE----- --Sig_/gieHqjGF_8F4qJdiTq5hhuf-- From owner-freebsd-current@FreeBSD.ORG Wed Mar 4 16:48:05 2015 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 197ADFF8 for ; Wed, 4 Mar 2015 16:48:05 +0000 (UTC) Received: from mail-wi0-x236.google.com (mail-wi0-x236.google.com [IPv6:2a00:1450:400c:c05::236]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 9EEFFEC4 for ; Wed, 4 Mar 2015 16:48:04 +0000 (UTC) Received: by widex7 with SMTP id ex7so32261927wid.4 for ; Wed, 04 Mar 2015 08:48:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=berentweb.com; s=google; h=date:from:to:subject:message-id:in-reply-to:references:mime-version :content-type:content-transfer-encoding; bh=HIhdFoJLTGRq/K0U2FoeIsH6z/pH/RAz4CA1pLPzkIo=; b=IHZEYqlq2IRJxnM4TK5P4/lcIUYcCcCKLHaay+lKykv+Gyv3+QHwYV6a+BFMny8dHh niXpeDGDT8LLSb0nRh6faYf+IAPeYLlcffaHQivVf1pEvhZjoVmtZKNW7++KjBhXTz2K ZvZR5OLXVPVh1Xh9IVK14BsHQ++l+9cl2geyE= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:date:from:to:subject:message-id:in-reply-to :references:mime-version:content-type:content-transfer-encoding; bh=HIhdFoJLTGRq/K0U2FoeIsH6z/pH/RAz4CA1pLPzkIo=; b=QlMGbCf9zWs9tA/91fAXgEMCxJ+Z/iviKsy0ibrP8Jt0Wcyk9j4QXuHtPIzAVB/sG3 9FDcFr84zxGj78iDmwvHeQn6agi0ymm8AAgIUoBBfCLNFcvoYAi9C55fs2y2MIcZcrE4 QlE7QMqQblDh8OGJa9JgICJJjoSB0Y7TOmiwcoVb/ThL0wqlKGZKNTYEv+4AMbcKwf59 AhuBM7Y67zQBJJagL5d84dSM+jZJrW/0d6uqRtCckzmyOd4p2zdEIAq0keUzISw2pnju o00oewxj2RBKHj5KB02893ycSnh8fYbgtzS45fzs5s01AtiFN4aWA1m2fSY8dGKC2ST3 pasg== X-Gm-Message-State: ALoCoQlf8ZyIdTPHhRiYDqgJQmkyod71iTKfHie+UhVQVkg9Oa5Z19UCswgghPIhemLndVWcc3Bb X-Received: by 10.194.175.164 with SMTP id cb4mr10430723wjc.14.1425487682769; Wed, 04 Mar 2015 08:48:02 -0800 (PST) Received: from rsbsd.rsb ([31.200.21.33]) by mx.google.com with ESMTPSA id yy9sm6713276wjc.20.2015.03.04.08.47.59 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 04 Mar 2015 08:48:01 -0800 (PST) Date: Wed, 4 Mar 2015 18:47:57 +0200 From: Beeblebrox To: freebsd-current@freebsd.org Subject: Re: system failure: cannot kill webkit-gtk3 if using dumbell/radeon Message-ID: <20150304184757.3679a5b5@rsbsd.rsb> In-Reply-To: <20150227195346.78dbbca9@rsbsd.rsb> References: <20150227183143.708a0656@rsbsd.rsb> <20150227195346.78dbbca9@rsbsd.rsb> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 04 Mar 2015 16:48:05 -0000 Hi. > Sorry for the delay to get back to you (including your private emails). Not a problem, those were various reports left to your judgment as to their= significance. Better to provide more information rather than less. slim failure: Traced to dbus start failure at bootup. Slim now starts, afte= r manual start of dbus. GDM failure: Starts, but after taking a long time. I'll send Koop the log f= iles from my recent attempt. > You mentioned "ghost text" with nano in vt(4). Could you please file > a bug in Bugzilla with a picture of what you get? Please join a dmesg too= .=20 I can confirm this is still an issue. I'll try to get to it at the soonest. > the unkillable processes are interesting. If you can reproduce I really don't know how it came about, so no promises. The most interesting= part of that error however, was the behaviour of vt(4). The problem is not= only unkillable processes, but TTY freeze when doing an unrelated action t= o the locked process. I mean, nano or top have nothing to do with webkit-gt= k3, yet caused TTY freeze (due to GPU lock-up I ignorantly presume) > the flickering you see with some applications/desktop environments This is really serious. It has lessened, but not gone away. Additionally, I started to get a "sticky mouse" problem some time ago and I'm inclined to agree with HPS' assessment that "drm2.ko graphics driver has some hard spinning loops" (http://freebsd.1045724.n5.nabble.com/= Some-unresolved-but-important-X-org-problems-td5987904.html) Have not had the hardware to test with different GPU unfortunately. Regards. --=20 FreeBSD_amd64_11-Current_RadeonKMS Please include my email when responding-I may have unabled auto-delivery. From owner-freebsd-current@FreeBSD.ORG Wed Mar 4 17:20:53 2015 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 35C2DC95 for ; Wed, 4 Mar 2015 17:20:53 +0000 (UTC) Received: from outpost1.zedat.fu-berlin.de (outpost1.zedat.fu-berlin.de [130.133.4.66]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id DD0932FC for ; Wed, 4 Mar 2015 17:20:52 +0000 (UTC) Received: from inpost2.zedat.fu-berlin.de ([130.133.4.69]) by outpost.zedat.fu-berlin.de (Exim 4.85) with esmtp (envelope-from ) id <1YTCyV-000VXh-SI>; Wed, 04 Mar 2015 18:20:47 +0100 Received: from g226180182.adsl.alicedsl.de ([92.226.180.182] helo=thor.walstatt.dynvpn.de) by inpost2.zedat.fu-berlin.de (Exim 4.85) with esmtpsa (envelope-from ) id <1YTCyV-0007Eu-PL>; Wed, 04 Mar 2015 18:20:47 +0100 Date: Wed, 4 Mar 2015 18:18:52 +0100 From: "O. Hartmann" To: Dimitry Andric Subject: Re: r279514: buildworld failure: /usr/src/lib/libnv/tests/dnv_tests.cc:453:2: error: use of overloaded operator '<<' is ambiguous Message-ID: <20150304181852.7bae1df4.ohartman@zedat.fu-berlin.de> In-Reply-To: <9DE59FA1-D495-4796-B3F5-F96D1472F66C@andric.com> References: <20150302115057.62d2c74c@prometheus> <20150304123110.48aa4abb@prometheus> <9DE59FA1-D495-4796-B3F5-F96D1472F66C@andric.com> Organization: FU Berlin X-Mailer: Claws Mail 3.11.1 (GTK+ 2.24.25; amd64-portbld-freebsd11.0) MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; boundary="Sig_/x_f1p9WL1VWVXDP+M+oRbfk"; protocol="application/pgp-signature" X-Originating-IP: 92.226.180.182 X-ZEDAT-Hint: A Cc: freebsd-current , Ryan Stone X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 04 Mar 2015 17:20:53 -0000 --Sig_/x_f1p9WL1VWVXDP+M+oRbfk Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: quoted-printable Am Wed, 4 Mar 2015 14:10:00 +0100 Dimitry Andric schrieb: > On 04 Mar 2015, at 12:31, O. Hartmann wrote: > > On Mon, 2 Mar 2015 08:58:05 -0500 > > Ryan Stone wrote: > >=20 > > > Can you post the contents of your make.conf and src.conf? I didn't > > > see this in any of my "make tinderbox" runs > > > _______________________________________________ > > > freebsd-current@freebsd.org mailing list > > > http://lists.freebsd.org/mailman/listinfo/freebsd-current > > > To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd= .org" > >=20 > > The culprit is the option > >=20 > > CXXFLAGS+=3D -std=3Dc++11 > >=20 > > in /etc/src.conf >=20 > Right, it would be nice to have libnv compiling for C++11 though. I'll h= ave a look > later today. >=20 > -Dimitry >=20 Ah ... thank you very much. oh --Sig_/x_f1p9WL1VWVXDP+M+oRbfk Content-Type: application/pgp-signature Content-Description: OpenPGP digital signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQEcBAEBCAAGBQJU9z58AAoJEOgBcD7A/5N8e20IALNk+djecYqHkojKzrRSZyIG Gcm+2ZqfE2eoyfiPaqJtKVg7gh5BJmfkUfA3LOS6zBu8zbc+xJ8xMONbN2DfPy39 Jj05LyBQAgJx2s4sjXR97wDMR5jRsiCaSIw3X3NOXalXyPKGk+rN1IPPH90CpIg0 rq1jczyRGTqv7qwz0UQN6HP+VSsVahb0h/XS+GP3eV0RBpjDvV4a6f2jeEhjEmNH hZ3OhFiv3L3ZrJ8DN723QiwCt4CQxbHZnHfWgWH7f68Uw7grS+KGs6l1iF6Q23Sc 0GRZdpqe6KGd99MNQBLnCWZsTSup4d5GNecN8MAdbA2+zRFfjZefGrKY7/nZFkY= =ea/R -----END PGP SIGNATURE----- --Sig_/x_f1p9WL1VWVXDP+M+oRbfk-- From owner-freebsd-current@FreeBSD.ORG Wed Mar 4 17:37:38 2015 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 10AAA159 for ; Wed, 4 Mar 2015 17:37:38 +0000 (UTC) Received: from outpost1.zedat.fu-berlin.de (outpost1.zedat.fu-berlin.de [130.133.4.66]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id B7EF174B for ; Wed, 4 Mar 2015 17:37:37 +0000 (UTC) Received: from inpost2.zedat.fu-berlin.de ([130.133.4.69]) by outpost.zedat.fu-berlin.de (Exim 4.85) with esmtp (envelope-from ) id <1YTDEk-000aex-SD>; Wed, 04 Mar 2015 18:37:34 +0100 Received: from g226180182.adsl.alicedsl.de ([92.226.180.182] helo=thor.walstatt.dynvpn.de) by inpost2.zedat.fu-berlin.de (Exim 4.85) with esmtpsa (envelope-from ) id <1YTDEk-0008iU-PG>; Wed, 04 Mar 2015 18:37:34 +0100 Date: Wed, 4 Mar 2015 18:37:33 +0100 From: "O. Hartmann" To: Dimitry Andric Subject: Re: r279514: buildworld failure: /usr/src/lib/libnv/tests/dnv_tests.cc:453:2: error: use of overloaded operator '<<' is ambiguous Message-ID: <20150304183733.4cc67970.ohartman@zedat.fu-berlin.de> In-Reply-To: <9DE59FA1-D495-4796-B3F5-F96D1472F66C@andric.com> References: <20150302115057.62d2c74c@prometheus> <20150304123110.48aa4abb@prometheus> <9DE59FA1-D495-4796-B3F5-F96D1472F66C@andric.com> Organization: FU Berlin X-Mailer: Claws Mail 3.11.1 (GTK+ 2.24.25; amd64-portbld-freebsd11.0) MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; boundary="Sig_/ZVqtQnhfY7DYbTO_4+xuYR_"; protocol="application/pgp-signature" X-Originating-IP: 92.226.180.182 X-ZEDAT-Hint: A Cc: freebsd-current , Ryan Stone X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 04 Mar 2015 17:37:38 -0000 --Sig_/ZVqtQnhfY7DYbTO_4+xuYR_ Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: quoted-printable Am Wed, 4 Mar 2015 14:10:00 +0100 Dimitry Andric schrieb: > On 04 Mar 2015, at 12:31, O. Hartmann wrote: > > On Mon, 2 Mar 2015 08:58:05 -0500 > > Ryan Stone wrote: > >=20 > > > Can you post the contents of your make.conf and src.conf? I didn't > > > see this in any of my "make tinderbox" runs > > > _______________________________________________ > > > freebsd-current@freebsd.org mailing list > > > http://lists.freebsd.org/mailman/listinfo/freebsd-current > > > To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd= .org" > >=20 > > The culprit is the option > >=20 > > CXXFLAGS+=3D -std=3Dc++11 > >=20 > > in /etc/src.conf >=20 > Right, it would be nice to have libnv compiling for C++11 though. I'll h= ave a look > later today. >=20 > -Dimitry >=20 Ah ... thank=20 --Sig_/ZVqtQnhfY7DYbTO_4+xuYR_ Content-Type: application/pgp-signature Content-Description: OpenPGP digital signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQEcBAEBCAAGBQJU90LeAAoJEOgBcD7A/5N8YBgIANHgvJQIrLL3YAzV0aNL9peF 3j8sQ1SQX5ETYmw2jYLSNeHCV9aOuQ15kQk3+zwJGBWkHH4fAyYBUntN5DZ6tQNZ WP8JvJm2wlN7woj/B6sv3a6yZALmgsItW+An61RD2OU0MmjAWOAKIokD2UxcczfA MDfwfD1Jx19AJQN0KHRXdTwVrK3tjHvstzpPLpuKvUHWV7y+2CEAO+ndY8nxGRpy c+HvUz6EXV1obVwgy3+qqRVCcOyEQ3q6cbXxy3fp+ylqsoMMh8EkLi/fsgdZH/fv 6Z33BmJpyOBkKlnSX1qr470yZ/LUiGdfxsbJPs1kzDaH9uGqCaDPuADDWX+p6Kk= =V9ko -----END PGP SIGNATURE----- --Sig_/ZVqtQnhfY7DYbTO_4+xuYR_-- From owner-freebsd-current@FreeBSD.ORG Wed Mar 4 17:41:16 2015 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 0DAAB321 for ; Wed, 4 Mar 2015 17:41:16 +0000 (UTC) Received: from mail-pd0-f177.google.com (mail-pd0-f177.google.com [209.85.192.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C8050814 for ; Wed, 4 Mar 2015 17:41:15 +0000 (UTC) Received: by pdjy10 with SMTP id y10so58993550pdj.13 for ; Wed, 04 Mar 2015 09:41:08 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:subject:mime-version:content-type:from :in-reply-to:date:cc:message-id:references:to; bh=XjUuZjZPmZPV37iSBNWExrBSBbhz3UBJsJDUu4vi2rE=; b=bRkM+jPznrETbIW0gH+ll0ETbZnYMy69/rVt25CMExhohtS4MQgh0TnTecIQX04LRG 9gE5/7QwaRPypu7nlhc4D9aLuVIkjSC71vkeAgkC60zHTgv1BHBbkO8wEIZIe6729dSZ JAamxL2F341Wo/fOGa1NJ2l9OgAqI3PPwtD1x3miWJN8eOHLkR0TKuSPCTcDEwXwxN4W 1KWOlpqJ2OVcthohxHgSLkc/0dTr5x7VVQ1ClXuZyCJAbQ0vqFBu32pBR6f/3sOZZ+8/ gh/mEHZG3Phj4x/dVez+laaGlVx3fztENN+JQ03dYVsOlaE4S+nNuUXG2uAYsXo2SQbo U6wQ== X-Gm-Message-State: ALoCoQkd/A+8opqepJLPYsq1o/X/FEKH1dxnezs0PjqSLDAbVzhoL7IrflMJe/1fIBguF7ZKXF9H X-Received: by 10.66.136.17 with SMTP id pw17mr8717468pab.33.1425490868853; Wed, 04 Mar 2015 09:41:08 -0800 (PST) Received: from [10.64.24.149] ([69.53.236.236]) by mx.google.com with ESMTPSA id d12sm4679592pdj.2.2015.03.04.09.41.06 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Wed, 04 Mar 2015 09:41:07 -0800 (PST) Sender: Warner Losh Subject: Re: r279278 failed to build (yacc: maximum table size exceeded) Mime-Version: 1.0 (Mac OS X Mail 8.2 \(2070.6\)) Content-Type: multipart/signed; boundary="Apple-Mail=_252DEB80-FFCC-4C8F-AC67-F2E2FFEA7A8C"; protocol="application/pgp-signature"; micalg=pgp-sha512 X-Pgp-Agent: GPGMail 2.5b5 From: Warner Losh In-Reply-To: <95891DFC-786D-4064-A9F9-7A8E434EC568@gmail.com> Date: Wed, 4 Mar 2015 10:41:04 -0700 Message-Id: <32AB9AC2-066A-408E-B38A-655095E802F1@bsdimp.com> References: <20150225154327.GD1161@hub.FreeBSD.org> <20150225182201.216f6fee@nonamehost.local> <54EE05EA.3030509@FreeBSD.org> <8E4B6A07-BEB7-46B9-BFD2-0B3F33162760@gmail.com> <54EE1E38.30106@FreeBSD.org> <95891DFC-786D-4064-A9F9-7A8E434EC568@gmail.com> To: Garrett Cooper X-Mailer: Apple Mail (2.2070.6) Cc: Ivan Klymenko , freebsd-current@freebsd.org, Glen Barber , Arseny Nasokin , Allan Jude , Jung-uk Kim X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 04 Mar 2015 17:41:16 -0000 --Apple-Mail=_252DEB80-FFCC-4C8F-AC67-F2E2FFEA7A8C Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 [[ I know this is a bit stale, but this is a dangerous notion ]] > On Feb 25, 2015, at 1:11 PM, Garrett Cooper = wrote: > I was going to propose something a bit more radical =E2=80=94 I can = remove the BOOTSTRAPPING conditionals and simplify the code on 10-STABLE = / 11-CURRENT. >=20 > Maintaining BOOTSTRAPPING is error prone and it=E2=80=99s not saving = much time in the long run in builds (it's taking longer to diagnose = issues, test them, and commit fixes which will break at a later date). = I=E2=80=99ve been bitten by this once because I don=E2=80=99t run = ancient CURRENT/STABLE (r279198) and here are a couple follow up commits = bumping tools versions in the past (e.g. r278975, r269662, etc). >=20 > Just a thought. It=E2=80=99s a terrible thought. We=E2=80=99ve done the bootstrapping = thing for 15 years with very few bumps and biting. No need to ditch it because lately we=E2=80=99ve been = updating yacc more often w/o bumping the revision. Don=E2=80=99t remove it. There was more blood on the floor before we had = it than after. It documents how far back in time we try to support. Sure, things get = missed, but it isn=E2=80=99t always clear why we have things in the bootstrap tools. = Having them documented this way makes it clear. Warner --Apple-Mail=_252DEB80-FFCC-4C8F-AC67-F2E2FFEA7A8C Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- Comment: GPGTools - https://gpgtools.org iQIcBAEBCgAGBQJU90OxAAoJEGwc0Sh9sBEAn5sP/ijlbqbn11hJsFkTTe+sohAj jqKueo1Rqkr691M7fH3/KrzzmTg/8Gnlul96AC7CBwRK3ERQf9OIiVLpPQZQ/XSk lPhPi2+ivpxeN2B5mF3UT8mP85C+UykWI7rGBQR/nLbTTJHBCwLT2kJRF088P1Am +xTh+8zchRftD+aAEPX1tMoiG1a5ThgyCqHAXXPyGTG0BENYFvCx4H1jd1ZkLXIY nSMZySWVgx9sshmcwnkBtZXvWl2DtDo37Vvvyi/UD8jCx0xfBGjkRNcX8bY2bvgk fVd9A7aT4A4w0HisvUSBNQ+bfB6wNAAfG3O2mABy/v8u78i4+hVeh22pD8WB4jN/ pcta2wLH22RTU1KVrIMRZk3nKQGgbioG3RkCFIvRkexTEevYhi6shotVsc+JroX/ NalEOzlKXdkTnmaatceTHo1HBTg3attgXYLj4W0Gr+/SNgogY7D3lB8S7fasqHi4 7mwpCjgpaG0tW70HvGdQv9HK51uheiPwF+LeX4ZJkdomsTjtzfLP05Q0LOkNNEJz P1te278+0nor8geKue10g+nXG5LL2/Wp6pcPCa2sXgcTd+sOscbK2K/d2iJCRltD 50Hp2WT8qK/Aq0U4VLx3YX4ziKXvsDAqg0mV3TTXIQm9SIla1XXeZCQWf7GjbeLr 26HfEtDKh4hQ6zyXDo9J =s7yu -----END PGP SIGNATURE----- --Apple-Mail=_252DEB80-FFCC-4C8F-AC67-F2E2FFEA7A8C-- From owner-freebsd-current@FreeBSD.ORG Wed Mar 4 18:31:49 2015 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A2E31CC1; Wed, 4 Mar 2015 18:31:49 +0000 (UTC) Received: from elvis.mu.org (elvis.mu.org [IPv6:2001:470:1f05:b76::196]) by mx1.freebsd.org (Postfix) with ESMTP id 8D266DA1; Wed, 4 Mar 2015 18:31:49 +0000 (UTC) Received: from AlfredMacbookAir.local (unknown [216.205.235.226]) by elvis.mu.org (Postfix) with ESMTPSA id BED5B341F861; Wed, 4 Mar 2015 10:31:48 -0800 (PST) Message-ID: <54F75076.8060102@mu.org> Date: Wed, 04 Mar 2015 10:35:34 -0800 From: Alfred Perlstein User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:31.0) Gecko/20100101 Thunderbird/31.5.0 MIME-Version: 1.0 To: John Baldwin , freebsd-current@freebsd.org Subject: Re: Massive libxo-zation that breaks everything References: <54F31510.7050607@hot.ee> <54F50F15.4050300@freebsd.org> <15905806.StXgP74c8j@ralph.baldwin.cx> In-Reply-To: <15905806.StXgP74c8j@ralph.baldwin.cx> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit Cc: David Chisnall , Allan Jude X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 04 Mar 2015 18:31:49 -0000 On 3/4/15 8:21 AM, John Baldwin wrote: > On Tuesday, March 03, 2015 09:09:43 AM David Chisnall wrote: >> Hopefully there's a lesson here that we can learn from: human-readable >> formats do not make good intermediate representations when communicating >> between tools. > > I think this is actually an argument against libxo-ification in the one case > where I've cringed a bit at the diffs: pciconf. The current pciconf code is > tailored to outputting something human readable. For non-human output I would > probably generate different output (not just put tags on the human output) > because I would want the non-human output to be both more verbose and more > raw. I think some other cases like 'netstat -s' are far more straightforward > as the current output maps fairly well to the backing structure, but in > general I would want machine-readable output that is closer to the structures > than to the human-readable formatting of them. > > For example, for something like 'mfiutil show drives', I would want the human > readable format to stay as it is (it only highlights certain fields in the PD > structures) but I would want the machine-readable format to basically output > tagged versions of the backing structures from sys/dev/mfi/mfireg.h. That way > the machine-readable format has all of the data instead of only the subset > that is presented in the human-readable output. > > So while I am in general a big fan of having machine-readable output from > tools (and I think it belongs in the base system, and I don't think you want a > post-processing tool), I think there is a bit of a flawed assumption that says > that I want the same data in the human-readable format that I want in the > machine-readable format. I, for one, don't. I want the human-readable form > more condensed. > >> If your argument is about maintainability of these changes, then please >> point to concrete instances where the changes are complex and difficult to >> maintain. > > When I've looked at the xo diffs for pciconf, my reaction has been "ugh, I > guess I'm not going to work on pciconf again in the future because that's > super ugly". I don't object to the idea, I think I would just rather have a > very different schema for machine-readable output. +1 > I would probably want > pciconf -l in that case to dump the entire PCI header (right now the human- > readable pciconf -l only dumps a subset), and I would want it to dump fields > in capabilities that we don't currently bother printing (and that I don't > think the human-readable output should print due to it being too obscure, > etc.) > I actually agree on this and this is why I was upset that the more straightforward GSoC code was shot down in favor of this. That said don't we all need to look at the greater good when making software and we have some consensus on this so its worth going forward imo. We can agree on something even though it might make hairs stand up or we just stagnate. From owner-freebsd-current@FreeBSD.ORG Wed Mar 4 19:55:49 2015 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 760DDFC7 for ; Wed, 4 Mar 2015 19:55:49 +0000 (UTC) Received: from mail-la0-f49.google.com (mail-la0-f49.google.com [209.85.215.49]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id EC428903 for ; Wed, 4 Mar 2015 19:55:48 +0000 (UTC) Received: by labgf13 with SMTP id gf13so22365584lab.10 for ; Wed, 04 Mar 2015 11:55:40 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:message-id:date:from:user-agent:mime-version:to :cc:subject:references:in-reply-to:content-type :content-transfer-encoding; bh=qJCOWzK7PvSxFKH+O+I44NVa0MzQqrImGbxhBlW6EbA=; b=g7cB0HW5mAt7ZO7m6MegtyyeSLKiG+umLEPynN/AU0BMoyiqZHIZCN3a3CKdk5cfxt WoJ+Ac6m80UqZIAGok+a4Uy/HE9M4znCJ8yUBeMEQxGqDl8VtjgBycCR3HfNhdUPK5Lb j4JFMNQus+BctcP7oGcpGa/Q+siHT71rCpePGngLRkLIPrzwHkZ5L4Sf8ViEnQeyv6Sr NlheeUr2xhshdTGcjQjHxyT3VNtMiQ1X8O9zVhhjyX2jVDpXqGyUEeab1EqBsv/kYE1i A67IM81f0RAB7i7aIwQFdPvA4cOz2FZ+dSibjnBoDnVZQJwHHG6w13bGIeG7kG5qsXnd MU3w== X-Gm-Message-State: ALoCoQnUCkvsiiTOi6ON5fSQQq3a9KgrmyIfHfbbolMDZ8QfRZIER+btsqqqkCDrN1Olp7JFF17h X-Received: by 10.152.6.34 with SMTP id x2mr475551lax.47.1425498940715; Wed, 04 Mar 2015 11:55:40 -0800 (PST) Received: from [192.168.1.2] ([89.169.173.68]) by mx.google.com with ESMTPSA id w9sm926166laj.24.2015.03.04.11.55.39 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 04 Mar 2015 11:55:39 -0800 (PST) Message-ID: <54F7633B.3070909@freebsd.org> Date: Wed, 04 Mar 2015 22:55:39 +0300 From: Andrey Chernov User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:31.0) Gecko/20100101 Thunderbird/31.5.0 MIME-Version: 1.0 To: John Baldwin , freebsd-current@freebsd.org Subject: Re: Massive libxo-zation that breaks everything References: <54F31510.7050607@hot.ee> <54F50F15.4050300@freebsd.org> <15905806.StXgP74c8j@ralph.baldwin.cx> In-Reply-To: <15905806.StXgP74c8j@ralph.baldwin.cx> Content-Type: text/plain; charset=koi8-r Content-Transfer-Encoding: 7bit Cc: David Chisnall , Allan Jude X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 04 Mar 2015 19:55:49 -0000 On 04.03.2015 19:21, John Baldwin wrote: > On Tuesday, March 03, 2015 09:09:43 AM David Chisnall wrote: >> Hopefully there's a lesson here that we can learn from: human-readable >> formats do not make good intermediate representations when communicating >> between tools. > > I think this is actually an argument against libxo-ification in the one case > where I've cringed a bit at the diffs: pciconf. The current pciconf code is > tailored to outputting something human readable. For non-human output I would > probably generate different output (not just put tags on the human output) > because I would want the non-human output to be both more verbose and more > raw. I think some other cases like 'netstat -s' are far more straightforward > as the current output maps fairly well to the backing structure, but in > general I would want machine-readable output that is closer to the structures > than to the human-readable formatting of them. > > For example, for something like 'mfiutil show drives', I would want the human > readable format to stay as it is (it only highlights certain fields in the PD > structures) but I would want the machine-readable format to basically output > tagged versions of the backing structures from sys/dev/mfi/mfireg.h. That way > the machine-readable format has all of the data instead of only the subset > that is presented in the human-readable output. > > So while I am in general a big fan of having machine-readable output from > tools (and I think it belongs in the base system, and I don't think you want a > post-processing tool), I think there is a bit of a flawed assumption that says > that I want the same data in the human-readable format that I want in the > machine-readable format. I, for one, don't. I want the human-readable form > more condensed. > >> If your argument is about maintainability of these changes, then please >> point to concrete instances where the changes are complex and difficult to >> maintain. > > When I've looked at the xo diffs for pciconf, my reaction has been "ugh, I > guess I'm not going to work on pciconf again in the future because that's > super ugly". I don't object to the idea, I think I would just rather have a > very different schema for machine-readable output. I would probably want > pciconf -l in that case to dump the entire PCI header (right now the human- > readable pciconf -l only dumps a subset), and I would want it to dump fields > in capabilities that we don't currently bother printing (and that I don't > think the human-readable output should print due to it being too obscure, > etc.) > I agree that adding machine-readable output + separate option on a per-tool basis, when it is really needed, is more clean approach which don't break Unix way of things. In such case we don't intermix structured representation with human readable, so the chances that a bug in one representation code flow will affect other one are minimal unlike in unified libxo approach, which already demonstrate this weakness. -- http://ache.vniz.net/ From owner-freebsd-current@FreeBSD.ORG Wed Mar 4 20:19:59 2015 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B79AC462 for ; Wed, 4 Mar 2015 20:19:59 +0000 (UTC) Received: from tensor.andric.com (tensor.andric.com [87.251.56.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "tensor.andric.com", Issuer "COMODO RSA Domain Validation Secure Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 42955B4F for ; Wed, 4 Mar 2015 20:19:59 +0000 (UTC) Received: from [IPv6:2001:7b8:3a7::ad8c:3fcc:9718:e182] (unknown [IPv6:2001:7b8:3a7:0:ad8c:3fcc:9718:e182]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by tensor.andric.com (Postfix) with ESMTPSA id 6B8695C49; Wed, 4 Mar 2015 21:19:55 +0100 (CET) Subject: Re: r279514: buildworld failure: /usr/src/lib/libnv/tests/dnv_tests.cc:453:2: error: use of overloaded operator '<<' is ambiguous Mime-Version: 1.0 (Mac OS X Mail 8.2 \(2070.6\)) Content-Type: multipart/signed; boundary="Apple-Mail=_2B7170FA-C029-400A-B905-12DDCAAD69A4"; protocol="application/pgp-signature"; micalg=pgp-sha1 X-Pgp-Agent: GPGMail 2.5b5 From: Dimitry Andric In-Reply-To: <20150304181852.7bae1df4.ohartman@zedat.fu-berlin.de> Date: Wed, 4 Mar 2015 21:19:43 +0100 Message-Id: <6CB3E761-8EFA-4DB4-8395-E15151F4F547@FreeBSD.org> References: <20150302115057.62d2c74c@prometheus> <20150304123110.48aa4abb@prometheus> <9DE59FA1-D495-4796-B3F5-F96D1472F66C@andric.com> <20150304181852.7bae1df4.ohartman@zedat.fu-berlin.de> To: "O. Hartmann" X-Mailer: Apple Mail (2.2070.6) Cc: freebsd-current , Ryan Stone X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 04 Mar 2015 20:19:59 -0000 --Apple-Mail=_2B7170FA-C029-400A-B905-12DDCAAD69A4 Content-Type: multipart/mixed; boundary="Apple-Mail=_815E300D-235D-4BB8-904F-B7F6C7430420" --Apple-Mail=_815E300D-235D-4BB8-904F-B7F6C7430420 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii On 04 Mar 2015, at 18:18, O. Hartmann = wrote: >=20 > Am Wed, 4 Mar 2015 14:10:00 +0100 > Dimitry Andric schrieb: >=20 >> On 04 Mar 2015, at 12:31, O. Hartmann = wrote: >>> On Mon, 2 Mar 2015 08:58:05 -0500 >>> Ryan Stone wrote: >>>=20 >>>> Can you post the contents of your make.conf and src.conf? I didn't >>>> see this in any of my "make tinderbox" runs >>>> _______________________________________________ >>>> freebsd-current@freebsd.org mailing list >>>> http://lists.freebsd.org/mailman/listinfo/freebsd-current >>>> To unsubscribe, send any mail to = "freebsd-current-unsubscribe@freebsd.org" >>>=20 >>> The culprit is the option >>>=20 >>> CXXFLAGS+=3D -std=3Dc++11 >>>=20 >>> in /etc/src.conf >>=20 >> Right, it would be nice to have libnv compiling for C++11 though. = I'll have a look >> later today. ... It is caused by the following test in lib/libnv/tests/dnv_tests.cc: ATF_REQUIRE_EQ(actual_val, NULL); In C++ mode, ATF_REQUIRE_EQ will attempt to output the value of NULL onto a std::ostringstream, but this has become ambiguous in C++11. [1] The fix is to cast the NULL value to the specific pointer type ATF is testing against, 'nvlist_t *' in this case. See the attached diff. -Dimitry [1] Before C++11, NULL is defined as 0, e.g. plain zero, and it will therefore be printed as an integer "0". In C++11 and later, NULL is defined as the keyword nullptr, e.g. the null pointer literal. Since the null pointer can be converted to any other pointer type, and there are many different operators defined in C++ to output those operators, the call is ambiguous, and must be resolved by the developer. --Apple-Mail=_815E300D-235D-4BB8-904F-B7F6C7430420 Content-Disposition: attachment; filename=libnv-fix-tests-cxx11-1.diff Content-Type: application/octet-stream; name="libnv-fix-tests-cxx11-1.diff" Content-Transfer-Encoding: 7bit Index: lib/libnv/tests/dnv_tests.cc =================================================================== --- lib/libnv/tests/dnv_tests.cc (revision 279596) +++ lib/libnv/tests/dnv_tests.cc (working copy) @@ -450,7 +450,7 @@ nvl = nvlist_create(0); actual_val = dnvlist_take_nvlist(nvl, "123", NULL); - ATF_REQUIRE_EQ(actual_val, NULL); + ATF_REQUIRE_EQ(actual_val, static_cast(NULL)); free(actual_val); nvlist_destroy(nvl); --Apple-Mail=_815E300D-235D-4BB8-904F-B7F6C7430420-- --Apple-Mail=_2B7170FA-C029-400A-B905-12DDCAAD69A4 Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- Version: GnuPG/MacGPG2 v2.0.26 iEYEARECAAYFAlT3aOoACgkQsF6jCi4glqOGnwCgp9QO0/+dFWRqY0oJZg0Z4hKX ph4An2T9ZTSd7vHhhegOOCBhATieypyF =51Q+ -----END PGP SIGNATURE----- --Apple-Mail=_2B7170FA-C029-400A-B905-12DDCAAD69A4-- From owner-freebsd-current@FreeBSD.ORG Wed Mar 4 21:11:28 2015 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 02CF9287; Wed, 4 Mar 2015 21:11:28 +0000 (UTC) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id CE1EC12B; Wed, 4 Mar 2015 21:11:27 +0000 (UTC) Received: from ralph.baldwin.cx (pool-173-54-116-245.nwrknj.fios.verizon.net [173.54.116.245]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 3CAF8B913; Wed, 4 Mar 2015 16:11:26 -0500 (EST) From: John Baldwin To: Alfred Perlstein Subject: Re: Massive libxo-zation that breaks everything Date: Wed, 04 Mar 2015 16:11:09 -0500 Message-ID: <2442911.VSALjZ0gTW@ralph.baldwin.cx> User-Agent: KMail/4.14.2 (FreeBSD/10.1-STABLE; KDE/4.14.2; amd64; ; ) In-Reply-To: <54F75076.8060102@mu.org> References: <54F31510.7050607@hot.ee> <15905806.StXgP74c8j@ralph.baldwin.cx> <54F75076.8060102@mu.org> MIME-Version: 1.0 Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="us-ascii" X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Wed, 04 Mar 2015 16:11:26 -0500 (EST) Cc: freebsd-current@freebsd.org, David Chisnall , Allan Jude X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 04 Mar 2015 21:11:28 -0000 On Wednesday, March 04, 2015 10:35:34 AM Alfred Perlstein wrote: > On 3/4/15 8:21 AM, John Baldwin wrote: > > I would probably want > > pciconf -l in that case to dump the entire PCI header (right now the > > human-readable pciconf -l only dumps a subset), and I would want it to dump > > fields in capabilities that we don't currently bother printing (and that > > I don't think the human-readable output should print due to it being too > > obscure, etc.) > > I actually agree on this and this is why I was upset that the more > straightforward GSoC code was shot down in favor of this. That said > don't we all need to look at the greater good when making software and > we have some consensus on this so its worth going forward imo. > > We can agree on something even though it might make hairs stand up or we > just stagnate. I think there might some cases where converting the existing output directly is fine (netstat -s comes to mind), but I think we should not be opposed to the idea that some utilities should output a different set of data for machine -readable. Put another way, in my mind something like pciconf -l is a presentation layer, and it's just one way of representing the data involved. Ideally, something like pciconf -l could be rewritten as a consumer of the raw, machine-readable data (I'm not sure I would rewrite it, but in theory the data flow should be something like "raw data in kernel" -> "machine-readable format" -> "presentation".) For example, pciconf -l currently has the misfeature that it is a flat listing and doesn't properly communicate the hierarchy that you can see in devinfo (and often times that hiearchy does matter). I would want a machine-readable schema for PCI devices to describe the hierarcy so that you could implement multiple "views" of the data (think lspci -t vs pciconf -l). However, I think that requires designing the schema in terms of the data you are describing, not based on one extant view/presentation. To expend this further, suppose that pciconf supported multiple views like lspci, would you want that to translate into multiple machine-readable schemas? -- John Baldwin From owner-freebsd-current@FreeBSD.ORG Wed Mar 4 21:31:39 2015 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 72AA06DD; Wed, 4 Mar 2015 21:31:39 +0000 (UTC) Received: from tensor.andric.com (tensor.andric.com [87.251.56.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "tensor.andric.com", Issuer "COMODO RSA Domain Validation Secure Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 17D603F7; Wed, 4 Mar 2015 21:31:38 +0000 (UTC) Received: from [IPv6:2001:7b8:3a7::ad8c:3fcc:9718:e182] (unknown [IPv6:2001:7b8:3a7:0:ad8c:3fcc:9718:e182]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by tensor.andric.com (Postfix) with ESMTPSA id 914C65C47; Wed, 4 Mar 2015 22:31:35 +0100 (CET) Subject: Re: r279514: buildworld failure: /usr/src/lib/libnv/tests/dnv_tests.cc:453:2: error: use of overloaded operator '<<' is ambiguous Mime-Version: 1.0 (Mac OS X Mail 8.2 \(2070.6\)) Content-Type: multipart/signed; boundary="Apple-Mail=_6D2FEF0D-D0A8-421E-9CBD-FDB0F9CBD540"; protocol="application/pgp-signature"; micalg=pgp-sha1 X-Pgp-Agent: GPGMail 2.5b5 From: Dimitry Andric In-Reply-To: <6CB3E761-8EFA-4DB4-8395-E15151F4F547@FreeBSD.org> Date: Wed, 4 Mar 2015 22:31:25 +0100 Message-Id: <05BE1405-E939-46E4-B31B-EA8025C53CEA@FreeBSD.org> References: <20150302115057.62d2c74c@prometheus> <20150304123110.48aa4abb@prometheus> <9DE59FA1-D495-4796-B3F5-F96D1472F66C@andric.com> <20150304181852.7bae1df4.ohartman@zedat.fu-berlin.de> <6CB3E761-8EFA-4DB4-8395-E15151F4F547@FreeBSD.org> To: "O. Hartmann" X-Mailer: Apple Mail (2.2070.6) Cc: Julio Merino , freebsd-current , Ryan Stone X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 04 Mar 2015 21:31:39 -0000 --Apple-Mail=_6D2FEF0D-D0A8-421E-9CBD-FDB0F9CBD540 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii On 04 Mar 2015, at 21:19, Dimitry Andric wrote: >=20 > On 04 Mar 2015, at 18:18, O. Hartmann = wrote: >>=20 >> Am Wed, 4 Mar 2015 14:10:00 +0100 >> Dimitry Andric schrieb: >>=20 >>> On 04 Mar 2015, at 12:31, O. Hartmann = wrote: >>>> On Mon, 2 Mar 2015 08:58:05 -0500 >>>> Ryan Stone wrote: >>>>=20 >>>>> Can you post the contents of your make.conf and src.conf? I = didn't >>>>> see this in any of my "make tinderbox" runs >>>>> _______________________________________________ >>>>> freebsd-current@freebsd.org mailing list >>>>> http://lists.freebsd.org/mailman/listinfo/freebsd-current >>>>> To unsubscribe, send any mail to = "freebsd-current-unsubscribe@freebsd.org" >>>>=20 >>>> The culprit is the option >>>>=20 >>>> CXXFLAGS+=3D -std=3Dc++11 >>>>=20 >>>> in /etc/src.conf >>>=20 >>> Right, it would be nice to have libnv compiling for C++11 though. = I'll have a look >>> later today. > ... >=20 > It is caused by the following test in lib/libnv/tests/dnv_tests.cc: >=20 > ATF_REQUIRE_EQ(actual_val, NULL); >=20 > In C++ mode, ATF_REQUIRE_EQ will attempt to output the value of NULL > onto a std::ostringstream, but this has become ambiguous in C++11. [1] >=20 > The fix is to cast the NULL value to the specific pointer type ATF is > testing against, 'nvlist_t *' in this case. See the attached diff. Hmm, I've now seen that there many more ATF_REQUIRE_EQ() instance in the tests, which compare against NULL. It is rather cumbersome to fix all those, so maybe it should be fixed in atf-c++ instead. Depending on what may be allowed in the headers, something like this: Index: contrib/atf/atf-c++/macros.hpp =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D --- contrib/atf/atf-c++/macros.hpp (revision 279564) +++ contrib/atf/atf-c++/macros.hpp (working copy) @@ -111,7 +111,8 @@ std::ostringstream atfu_ss; \ atfu_ss << "Line " << __LINE__ << ": " \ << #expected << " !=3D " << #actual \ - << " (" << (expected) << " !=3D " << (actual) << = ")"; \ + << " (" << (expected) << " !=3D " << \ + static_cast<__typeof(expected)>(actual) << ")"; \ atf::tests::tc::fail(atfu_ss.str()); \ } \ } while (false) However, that breaks several ATF_REQUIRE_EQ() instances in atf-c++ itself, like this: /usr/src/contrib/atf/atf-c++/detail/process_test.cpp: In member function = 'virtual void {anonymous}::atfu_tc_argv_array_init_varargs::body() = const': /usr/src/contrib/atf/atf-c++/macros.hpp:115:59: error: invalid = static_cast from type 'std::__1::string {aka = std::__1::basic_string, = std::__1::allocator >}' to type 'const char*' static_cast<__typeof(expected)>(actual) << ")"; \ ^ /usr/src/contrib/atf/atf-c++/detail/process_test.cpp:174:9: note: in = expansion of macro 'ATF_REQUIRE_EQ' ATF_REQUIRE_EQ(argv[0], std::string("arg0")); ^ So in these cases, the 'expected' argument's type does not match the 'actual' argument's type. It would be a bit of churn to fix all of them... I guess the easiest option is to forcibly disable C++11 if you are using anything from atf-c++, until it is made C++11 compatible. -Dimitry --Apple-Mail=_6D2FEF0D-D0A8-421E-9CBD-FDB0F9CBD540 Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- Version: GnuPG/MacGPG2 v2.0.26 iEYEARECAAYFAlT3ebYACgkQsF6jCi4glqODtQCggwFhOkBUuWyVYYhyryTz8zML kegAoOdKzuazvFyplv15h3u9NYtbhjSW =Etm/ -----END PGP SIGNATURE----- --Apple-Mail=_6D2FEF0D-D0A8-421E-9CBD-FDB0F9CBD540-- From owner-freebsd-current@FreeBSD.ORG Wed Mar 4 23:03:35 2015 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 278E3A74; Wed, 4 Mar 2015 23:03:35 +0000 (UTC) Received: from mail-yk0-x22d.google.com (mail-yk0-x22d.google.com [IPv6:2607:f8b0:4002:c07::22d]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D22E0FB0; Wed, 4 Mar 2015 23:03:34 +0000 (UTC) Received: by ykbq9 with SMTP id q9so6272267ykb.13; Wed, 04 Mar 2015 15:03:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=QY2eVmAnd7WcVTJ2cc7UoZVzwJVa7OsIFRozBEHgiFQ=; b=HGTMkl6+5F3L/ajUInOxS3PR06Sbh/MvVHizX943Kn9KQPE3MIHwXuGkifav61/ygB iGFk1N4gtIPOgoFaBtEKOSAejEQyMWU/8qtmmUFYykmtpydWl8FP8gWdOiHY6XrZa2My K+ROMZUxEVCDNv6ULBqFRw9jE4PZ7PJngPMw4OgNWFTgpKQ7SelAD8pAxqhSauo+NFMX hZJrDJgomOOuf/6lLw5V/Tell1a5h0Phi7tlJh4MWG74fNsixjGKCdQBYkNF0JoEEniM udo6H98ARhQ2e0PSyrI1d6kUc6gJYHVub8rKQcQJeidbdQzL27a8xmH1ISVnpfXw+kpU JO8g== MIME-Version: 1.0 X-Received: by 10.170.203.198 with SMTP id u189mr4968319yke.107.1425510213962; Wed, 04 Mar 2015 15:03:33 -0800 (PST) Received: by 10.170.60.85 with HTTP; Wed, 4 Mar 2015 15:03:33 -0800 (PST) In-Reply-To: <2442911.VSALjZ0gTW@ralph.baldwin.cx> References: <54F31510.7050607@hot.ee> <15905806.StXgP74c8j@ralph.baldwin.cx> <54F75076.8060102@mu.org> <2442911.VSALjZ0gTW@ralph.baldwin.cx> Date: Wed, 4 Mar 2015 15:03:33 -0800 Message-ID: Subject: Re: Massive libxo-zation that breaks everything From: Mehmet Erol Sanliturk To: John Baldwin Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 Cc: Alfred Perlstein , David Chisnall , Allan Jude , freebsd-current X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 04 Mar 2015 23:03:35 -0000 On Wed, Mar 4, 2015 at 1:11 PM, John Baldwin wrote: > On Wednesday, March 04, 2015 10:35:34 AM Alfred Perlstein wrote: > > On 3/4/15 8:21 AM, John Baldwin wrote: > > > I would probably want > > > pciconf -l in that case to dump the entire PCI header (right now the > > > human-readable pciconf -l only dumps a subset), and I would want it to > dump > > > fields in capabilities that we don't currently bother printing (and > that > > > I don't think the human-readable output should print due to it being > too > > > obscure, etc.) > > > > I actually agree on this and this is why I was upset that the more > > straightforward GSoC code was shot down in favor of this. That said > > don't we all need to look at the greater good when making software and > > we have some consensus on this so its worth going forward imo. > > > > We can agree on something even though it might make hairs stand up or we > > just stagnate. > > I think there might some cases where converting the existing output > directly > is fine (netstat -s comes to mind), but I think we should not be opposed to > the idea that some utilities should output a different set of data for > machine > -readable. > > Put another way, in my mind something like pciconf -l is a presentation > layer, > and it's just one way of representing the data involved. Ideally, > something > like pciconf -l could be rewritten as a consumer of the raw, > machine-readable > data (I'm not sure I would rewrite it, but in theory the data flow should > be > something like "raw data in kernel" -> "machine-readable format" -> > "presentation".) For example, pciconf -l currently has the misfeature > that it > is a flat listing and doesn't properly communicate the hierarchy that you > can > see in devinfo (and often times that hiearchy does matter). I would want a > machine-readable schema for PCI devices to describe the hierarcy so that > you > could implement multiple "views" of the data (think lspci -t vs pciconf > -l). > However, I think that requires designing the schema in terms of the data > you > are describing, not based on one extant view/presentation. To expend this > further, suppose that pciconf supported multiple views like lspci, would > you > want that to translate into multiple machine-readable schemas? > > -- > John Baldwin > _______________________________________________ > > If I were the sole designer of the XML ( or any other such as JSON , YAML , etc. ) output system I would do the following : If a value is generated , and should be stored to output file as soon as possible ( such as debugging needs ) , the present sample in the Phabricator review is suitable . For other cases , I would define a record to store the variable values to list later , for example , pci_XML . Within related program part , by trying as much as to write assignment statements near to each other , I would assign values to the pci_XML variables . At the end of assigning values , I would call a routine to store the pci_XML record to file ( create file , write values from pci_XML record , close file ) . Within that routine , it is possible to store the values of pci_XML record in any way , without making difficult to read primary related program . Updates to that routine will not affect the calling program parts much . As a companion to writing routine , I would develop a reading routine ( open file , read values into pci_XML , close the file ) to call from where processing of the output file is required . As a third routine to display values by using pci_XML to the user during interactive work . Thank you very much . Mehmet Erol Sanliturk From owner-freebsd-current@FreeBSD.ORG Wed Mar 4 23:16:56 2015 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E40F8E92 for ; Wed, 4 Mar 2015 23:16:56 +0000 (UTC) Received: from mail-ig0-x236.google.com (mail-ig0-x236.google.com [IPv6:2607:f8b0:4001:c05::236]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A4E52187 for ; Wed, 4 Mar 2015 23:16:56 +0000 (UTC) Received: by igdh15 with SMTP id h15so41268287igd.3 for ; Wed, 04 Mar 2015 15:16:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=erhUQNPlvBNrkiqEv7BLM1/lghbX1rPduUn3SW/0bXE=; b=lfXSDVP0E+HuAp1jkn3RWPTxYcsrChh0yqfgSslCZCqs3iXxmvlhCagZ0ikHgLXhw4 FphjRGWQ6Ote0mbm0SN64PvqZwqnBDcFsw/S5r48qYbLMCX2DxF0XDa1WSlyqs7xV1Yc eNcGoKnkxZFLkqRpfMM3djK3NHWWLriCsfs1mgcNCCKfTC/6gJKDqTPdldblRLyGUWFD aOZNUBceAtltDxe8gjsBKcFnygjxMODCZZu201j9J2yDzd5kaOpUoRgc7aqeI4a48Vpu tNh6xegI+kZL/Ne8h1pOroccK6NSn2vb+YOoloHFaSwVW1VNVqU8Wz9XPbD2z1AI2nS3 RiOA== MIME-Version: 1.0 X-Received: by 10.50.30.202 with SMTP id u10mr42189621igh.35.1425511015807; Wed, 04 Mar 2015 15:16:55 -0800 (PST) Sender: kob6558@gmail.com Received: by 10.107.174.86 with HTTP; Wed, 4 Mar 2015 15:16:55 -0800 (PST) In-Reply-To: <20150304184757.3679a5b5@rsbsd.rsb> References: <20150227183143.708a0656@rsbsd.rsb> <20150227195346.78dbbca9@rsbsd.rsb> <20150304184757.3679a5b5@rsbsd.rsb> Date: Wed, 4 Mar 2015 15:16:55 -0800 X-Google-Sender-Auth: L9hZI2svC9nUrPX4fn852zO8Sj8 Message-ID: Subject: Re: system failure: cannot kill webkit-gtk3 if using dumbell/radeon From: Kevin Oberman To: Beeblebrox Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 Cc: FreeBSD Current X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 04 Mar 2015 23:16:57 -0000 On Wed, Mar 4, 2015 at 8:47 AM, Beeblebrox wrote: > Hi. > > Sorry for the delay to get back to you (including your private emails). > Not a problem, those were various reports left to your judgment as to > their significance. Better to provide more information rather than less. > > slim failure: Traced to dbus start failure at bootup. Slim now starts, > after manual start of dbus. > GDM failure: Starts, but after taking a long time. I'll send Koop the log > files from my recent attempt. > > > You mentioned "ghost text" with nano in vt(4). Could you please file > > a bug in Bugzilla with a picture of what you get? Please join a dmesg > too. > I can confirm this is still an issue. I'll try to get to it at the soonest. > > > the unkillable processes are interesting. If you can reproduce > I really don't know how it came about, so no promises. The most > interesting part of that error however, was the behaviour of vt(4). The > problem is not only unkillable processes, but TTY freeze when doing an > unrelated action to the locked process. I mean, nano or top have nothing to > do with webkit-gtk3, yet caused TTY freeze (due to GPU lock-up I ignorantly > presume) > > > the flickering you see with some applications/desktop environments > This is really serious. It has lessened, but not gone away. > Additionally, I started to get a "sticky mouse" problem some time ago > and I'm inclined to agree with HPS' assessment that "drm2.ko graphics > driver has some hard spinning loops" ( > http://freebsd.1045724.n5.nabble.com/Some-unresolved-but-important-X-org-problems-td5987904.html > ) > Have not had the hardware to test with different GPU unfortunately. > > Regards. > > This sounds a lot like another rcorder issue. If you run "rcorder /etc/rc.d/* /usr/locl/rtc/rc.d/* > /dev/null", do you get errors... particularly circular dependencies? I know that webcamd and dbus have a problem. These are very dangerous as the behavior when they occur is undefined, very unstable and can easily result in daemons failing to start. They are also a real pain to track down. Most often just looking at "REQUIRES" and "BEFORE" statements is not very useful. Note that reports of "no providers" are warnings and often normal and expected. -- Kevin Oberman, Network Engineer, Retired E-mail: rkoberman@gmail.com From owner-freebsd-current@FreeBSD.ORG Wed Mar 4 23:23:54 2015 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 2CD36417 for ; Wed, 4 Mar 2015 23:23:54 +0000 (UTC) Received: from mail-ie0-x231.google.com (mail-ie0-x231.google.com [IPv6:2607:f8b0:4001:c03::231]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E6CD02A2 for ; Wed, 4 Mar 2015 23:23:53 +0000 (UTC) Received: by iecat20 with SMTP id at20so18349245iec.6 for ; Wed, 04 Mar 2015 15:23:53 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=lEGnYevcNzSu0cFiwArVVjCwPdDCv1ixRPFKKBkyeM8=; b=Q6+pAhZUJ8ZvHj+liFL0tAhpmUP43JPrPC1aCJcx/7XhQN5ZCsHME3Mq1RGtXvS1N5 ChY7o4yAa4xBfrf9HFf5fXkbRNZx2bMCh/BsO2he+YL2lLZ3P+MpmCBoVmcI1dGL0BoR CbQRUTTipBouqH12UdKX6Ucn4rYMN/3DnmW3GXQoBlEE9oOpZ10o6kiVcOUMxxDdJAkF p2ntWG2Pcox6js6EqbT9KJjwiisG5k8I/WmbsrZ8xiPgDTgFSvSRGii0TRpQLO/PQigB Yh3yyrBgpvV85Xo81SvXxyEG/OiyqkAGeHamFVJTL/lmCsQ8+9IsRG7AKhIikdAR6jRa B1DA== MIME-Version: 1.0 X-Received: by 10.107.8.213 with SMTP id h82mr15455989ioi.89.1425511433291; Wed, 04 Mar 2015 15:23:53 -0800 (PST) Received: by 10.107.156.75 with HTTP; Wed, 4 Mar 2015 15:23:53 -0800 (PST) In-Reply-To: <20150304200739.e8e2bcf2a1d517dac626e601@dec.sakura.ne.jp> References: <20150304200739.e8e2bcf2a1d517dac626e601@dec.sakura.ne.jp> Date: Wed, 4 Mar 2015 18:23:53 -0500 Message-ID: Subject: Re: sysutils/lsof does not build (maybe) after r279433 From: Ryan Stone To: Tomoaki AOKI Content-Type: text/plain; charset=UTF-8 Cc: FreeBSD Current X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 04 Mar 2015 23:23:54 -0000 Would: #undef __BSD_VISIBLE #include #define __BSD_VISIBLE 1 Work in lsof? From owner-freebsd-current@FreeBSD.ORG Thu Mar 5 02:43:20 2015 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id BDDE43BD; Thu, 5 Mar 2015 02:43:20 +0000 (UTC) Received: from thebighonker.lerctr.org (thebighonker.lerctr.org [IPv6:2001:470:1f0f:3ad:223:7dff:fe9e:6e8a]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "thebighonker.lerctr.org", Issuer "COMODO RSA Domain Validation Secure Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7A691A01; Thu, 5 Mar 2015 02:43:20 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lerctr.org; s=lerami; h=Message-ID:References:In-Reply-To:Subject:Cc:To:From:Date:Content-Transfer-Encoding:Content-Type:MIME-Version; bh=YfovXNkQVsyopykDZ0497+VM1l+yzUOCSxfaGQYTdKU=; b=iF4hinZkMt4xCWVZ+p76SXS8112C46/DWKBGbDG4qh31IkioakiPqrtdYRrnxtSmjMstJDGZdPgUNqf7HAMB3B2lNp/JT6XpY5GJpu/pxAsX/rP5MrtLGtpt8JeGskpEXmfZPpsbQWsQl5S0tqvA8ign5QESEguztUHuW44+34A=; Received: from thebighonker.lerctr.org ([2001:470:1f0f:3ad:223:7dff:fe9e:6e8a]:37476 helo=webmail.lerctr.org) by thebighonker.lerctr.org with esmtpsa (TLSv1.2:DHE-RSA-AES128-GCM-SHA256:128) (Exim 4.85 (FreeBSD)) (envelope-from ) id 1YTLkl-0001HJ-HS; Wed, 04 Mar 2015 20:43:15 -0600 Received: from 104-54-221-134.lightspeed.austtx.sbcglobal.net ([104.54.221.134]) by webmail.lerctr.org with HTTP (HTTP/1.1 POST); Wed, 04 Mar 2015 20:43:11 -0600 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed Content-Transfer-Encoding: 7bit Date: Wed, 04 Mar 2015 20:43:11 -0600 From: Larry Rosenman To: Ryan Stone Subject: Re: sysutils/lsof does not build (maybe) after r279433 In-Reply-To: References: <20150304200739.e8e2bcf2a1d517dac626e601@dec.sakura.ne.jp> Message-ID: X-Sender: ler@lerctr.org User-Agent: Roundcube Webmail/1.1.0 X-Spam-Score: -2.9 (--) X-LERCTR-Spam-Score: -2.9 (--) X-Spam-Report: SpamScore (-2.9/5.0) ALL_TRUSTED=-1, BAYES_00=-1.9, T_RP_MATCHES_RCVD=-0.01 X-LERCTR-Spam-Report: SpamScore (-2.9/5.0) ALL_TRUSTED=-1, BAYES_00=-1.9, T_RP_MATCHES_RCVD=-0.01 Cc: Tomoaki AOKI , FreeBSD Current , owner-freebsd-current@freebsd.org X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 05 Mar 2015 02:43:20 -0000 On 2015-03-04 17:23, Ryan Stone wrote: > Would: > > #undef __BSD_VISIBLE > #include > #define __BSD_VISIBLE 1 > > > Work in lsof? > _______________________________________________ > freebsd-current@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-current > To unsubscribe, send any mail to > "freebsd-current-unsubscribe@freebsd.org" I've fired off a note to Vic Abell (lsof author). Can one of you file a Bugzilla ticket for me? Thanks! -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 214-642-9640 E-Mail: ler@lerctr.org US Mail: 108 Turvey Cove, Hutto, TX 78634-5688 From owner-freebsd-current@FreeBSD.ORG Thu Mar 5 13:19:03 2015 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A833B730 for ; Thu, 5 Mar 2015 13:19:03 +0000 (UTC) Received: from dec.sakura.ne.jp (dec.sakura.ne.jp [210.188.226.8]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 5C785638 for ; Thu, 5 Mar 2015 13:19:02 +0000 (UTC) Received: from fortune.joker.local (180-199-46-187.nagoya1.commufa.jp [180.199.46.187]) (authenticated bits=0) by dec.sakura.ne.jp (8.14.3/8.14.2/[SAKURA-WEB]/20080708) with ESMTP id t25DItIL024030 for ; Thu, 5 Mar 2015 22:18:55 +0900 (JST) (envelope-from junchoon@dec.sakura.ne.jp) Date: Thu, 5 Mar 2015 22:18:54 +0900 From: Tomoaki AOKI To: freebsd-current@freebsd.org Subject: Re: sysutils/lsof does not build (maybe) after r279433 Message-Id: <20150305221854.a31763c7b62b41ec53cb8fb6@dec.sakura.ne.jp> In-Reply-To: References: <20150304200739.e8e2bcf2a1d517dac626e601@dec.sakura.ne.jp> Organization: Junchoon corps X-Mailer: Sylpheed 3.4.2 (GTK+ 2.24.25; amd64-portbld-freebsd10.1) Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-2022-JP Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 05 Mar 2015 13:19:03 -0000 Done. Bug 198312 - sysutils/lsof no longer builds after head/r279433 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=198312 On Wed, 04 Mar 2015 20:43:11 -0600 Larry Rosenman wrote: > On 2015-03-04 17:23, Ryan Stone wrote: > > Would: > > > > #undef __BSD_VISIBLE > > #include > > #define __BSD_VISIBLE 1 > > > > > > Work in lsof? > > _______________________________________________ > > freebsd-current@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-current > > To unsubscribe, send any mail to > > "freebsd-current-unsubscribe@freebsd.org" > > > I've fired off a note to Vic Abell (lsof author). > > Can one of you file a Bugzilla ticket for me? > > Thanks! > > -- > Larry Rosenman http://www.lerctr.org/~ler > Phone: +1 214-642-9640 E-Mail: ler@lerctr.org > US Mail: 108 Turvey Cove, Hutto, TX 78634-5688 > > _______________________________________________ > freebsd-current@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-current > To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org" > -- $B@DLZ(B $BCNL@(B [Tomoaki AOKI] junchoon@dec.sakura.ne.jp MXE02273@nifty.com From owner-freebsd-current@FreeBSD.ORG Thu Mar 5 15:44:53 2015 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 94541878; Thu, 5 Mar 2015 15:44:53 +0000 (UTC) Received: from mail-lb0-x22c.google.com (mail-lb0-x22c.google.com [IPv6:2a00:1450:4010:c04::22c]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 0F898A8B; Thu, 5 Mar 2015 15:44:53 +0000 (UTC) Received: by lbvp9 with SMTP id p9so29204802lbv.8; Thu, 05 Mar 2015 07:44:51 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=bs+HEKYfnSdV4QmVf9vRlAHlqEUjmifCWecocQdiUzw=; b=pOPiY5Awl6LEfidAmBg0PuLS8SaFZ4loBgFEG1p4h/YxFlGVvwV3i6RYwDVTAcdeeQ 8GQszMOvTcKPjOSG4pjHamdJHAW1voMcjXcEr4rrQXZtF1YcM1vVBTzPQJQ9pTkXvsqq 3B5Zxt/9N7XHmUEiIpKNFP0wWA2PS9uafzoGr44GKfRDz3j0bGDMJbKF87SoMUvSFuug VkCPXdJQBuGIAS95x/HARHHWZ4WUZR39tMScJrdWEd7Qc4e90/TzO9l/NhADfr9LXMre Ax8OV3XSGI/iZcjfHqA8zCLT3GPXtC5Chl8wsbR7nFk1cyEucz57RcKirdOtOMoBUs9Y wUsQ== MIME-Version: 1.0 X-Received: by 10.112.204.197 with SMTP id la5mr8450969lbc.29.1425570291013; Thu, 05 Mar 2015 07:44:51 -0800 (PST) Sender: crodr001@gmail.com Received: by 10.112.82.164 with HTTP; Thu, 5 Mar 2015 07:44:50 -0800 (PST) In-Reply-To: <32AB9AC2-066A-408E-B38A-655095E802F1@bsdimp.com> References: <20150225154327.GD1161@hub.FreeBSD.org> <20150225182201.216f6fee@nonamehost.local> <54EE05EA.3030509@FreeBSD.org> <8E4B6A07-BEB7-46B9-BFD2-0B3F33162760@gmail.com> <54EE1E38.30106@FreeBSD.org> <95891DFC-786D-4064-A9F9-7A8E434EC568@gmail.com> <32AB9AC2-066A-408E-B38A-655095E802F1@bsdimp.com> Date: Thu, 5 Mar 2015 07:44:50 -0800 X-Google-Sender-Auth: gLitRx3G95H6fLcrtniuhztmZmc Message-ID: Subject: Re: r279278 failed to build (yacc: maximum table size exceeded) From: Craig Rodrigues To: Jung-uk Kim Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 Cc: freebsd-current Current X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 05 Mar 2015 15:44:53 -0000 Hi, I ran into this problem when doing a src upgrade of a HEAD system compiled on Oct. 21, 2014 to HEAD on March 4, 2015: yacc -d -pAslCompiler -oaslcompilerparse.c aslcompiler.y maximum table size exceeded *** [aslcompilerparse.c] Error code 2 make[5]: stopped in /usr/src/usr.sbin/acpi/iasl 1 error Does your fix address the problem in HEAD or just STABLE? -- Craig From owner-freebsd-current@FreeBSD.ORG Thu Mar 5 17:16:53 2015 Return-Path: Delivered-To: freebsd-current@FreeBSD.org Received: from mx2.freebsd.org (mx2.freebsd.org [8.8.178.116]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A16F081D; Thu, 5 Mar 2015 17:16:53 +0000 (UTC) Received: from hammer.pct.niksun.com (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) by mx2.freebsd.org (Postfix) with ESMTP id 651F2283F; Thu, 5 Mar 2015 17:16:53 +0000 (UTC) Message-ID: <54F88F84.3060406@FreeBSD.org> Date: Thu, 05 Mar 2015 12:16:52 -0500 From: Jung-uk Kim User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:31.0) Gecko/20100101 Thunderbird/31.5.0 MIME-Version: 1.0 To: Craig Rodrigues Subject: Re: r279278 failed to build (yacc: maximum table size exceeded) References: <20150225154327.GD1161@hub.FreeBSD.org> <20150225182201.216f6fee@nonamehost.local> <54EE05EA.3030509@FreeBSD.org> <8E4B6A07-BEB7-46B9-BFD2-0B3F33162760@gmail.com> <54EE1E38.30106@FreeBSD.org> <95891DFC-786D-4064-A9F9-7A8E434EC568@gmail.com> <32AB9AC2-066A-408E-B38A-655095E802F1@bsdimp.com> In-Reply-To: Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 8bit Cc: FreeBSD-current X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 05 Mar 2015 17:16:53 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 On 03/05/2015 10:44, Craig Rodrigues wrote: > Hi, > > I ran into this problem when doing a src upgrade of a HEAD system > compiled on Oct. 21, 2014 to HEAD on March 4, 2015: > > yacc -d -pAslCompiler -oaslcompilerparse.c aslcompiler.y maximum > table size exceeded *** [aslcompilerparse.c] Error code 2 make[5]: > stopped in /usr/src/usr.sbin/acpi/iasl 1 error > > > Does your fix address the problem in HEAD or just STABLE? Just stable. http://docs.freebsd.org/cgi/mid.cgi?54EE05EA.3030509 Jung-uk Kim -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQEcBAEBCAAGBQJU+I9/AAoJEHyflib82/FG8XwH/3cYegfWMB0fl2Hsa+Z+VlrB SYIS/opP4NpmhXYbwwwhfA8/QHxhTASGxXqrKKtw3zyD8VTox1/t45Bf6tieN3I0 a3tIsQ9Rjbpm9FbOKy+fTGaC1FVl8pBkO/Sp0o3dXVCP2X7ljiyDSpasMMolz9Od TFD2Rrz1wVNRJeCYod9vxQ3SVUEqX2MKk29JOHWZ4BBxCp4nnvXVowM8Pyz58ene LuBEBW9tNhYp7+GBiUntZYYQ0iFIYlWYzGIyku5dHJxntV56ALVsENoWamsQ3Fwc 6pkxZl8KVCRx9MWhQIU5r6mJhTK3UZBjBEYJpUSKN9CSpH+SuMJXEK/9omKZZ9w= =Z0Gm -----END PGP SIGNATURE----- From owner-freebsd-current@FreeBSD.ORG Thu Mar 5 19:52:30 2015 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 9A82E8E5 for ; Thu, 5 Mar 2015 19:52:30 +0000 (UTC) Received: from mail-ig0-x22a.google.com (mail-ig0-x22a.google.com [IPv6:2607:f8b0:4001:c05::22a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5D6ACE7E for ; Thu, 5 Mar 2015 19:52:30 +0000 (UTC) Received: by igkb16 with SMTP id b16so48837792igk.1 for ; Thu, 05 Mar 2015 11:52:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:from:date:message-id:subject:to:content-type; bh=9djoVSJIfmoeoaHkLhcL9gNtIbqn1EwgEnRaP4KwJGA=; b=sUcr4HRQdAkpqwwi9kmRd1J5MrIrNCgYvCMt0Zh3xubSOH/40ZFHvrCspBgERlZyS5 1TB5eHUzuLU5qd9zIol/F6YeLMT0E7TiVVUSQoOjF/EHN73lvOaSarbZ/h7Etz1Xk63x QF92Wju7yR5ahupSXpWEvYMjtO3a4ob8cYxchwUv2RCfq5TO/iUj88BlbGQ9b1vp3luS ReYWlkBiuHUrI/E7tX2ldkpgI6cdqmFcbomU6lZosIKYHmm4ToXNOAJByHIVptYvxN7P EkLnkpfQykvm8H6U6c2Xdn3iNCLBK6koAUIEEq7v36EM62tk03oFprqYHuYOf3RMRinJ oAQg== X-Received: by 10.107.128.219 with SMTP id k88mr22207835ioi.27.1425585149797; Thu, 05 Mar 2015 11:52:29 -0800 (PST) MIME-Version: 1.0 Received: by 10.64.55.167 with HTTP; Thu, 5 Mar 2015 11:52:09 -0800 (PST) From: Miguel Clara Date: Thu, 5 Mar 2015 19:52:09 +0000 Message-ID: Subject: X11 tray icons not visible (show a 'square') in FreeBSD current To: freebsd-current Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 05 Mar 2015 19:52:30 -0000 This is a issue I've noticed in Lumina last week, but seems to be realted to some change in CURRENT. I'm running an older build on another laptop with no issues: 11.0-CURRENT FreeBSD 11.0-CURRENT #7 b6a4dfa(master)-dirt Not sure which one after that might have introduce it, or if its maybe something else.. Both run the same version of Xorg. Melhores Cumprimentos // Best Regards ----------------------------------------------- *Miguel Clara* *IT - Sys Admin & Developer* *E-mail: *miguelmclara@gmail.com www.linkedin.com/in/miguelmclara/ From owner-freebsd-current@FreeBSD.ORG Thu Mar 5 19:54:26 2015 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7E76EA01 for ; Thu, 5 Mar 2015 19:54:26 +0000 (UTC) Received: from mail-ie0-x236.google.com (mail-ie0-x236.google.com [IPv6:2607:f8b0:4001:c03::236]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 42C40E95 for ; Thu, 5 Mar 2015 19:54:26 +0000 (UTC) Received: by iebtr6 with SMTP id tr6so6503753ieb.2 for ; Thu, 05 Mar 2015 11:54:25 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :content-type; bh=TC0xbAew2rFUxoWlLDVCqMMVgAO9meSJSe9rsvLIM9A=; b=EqxqtNRNj2a58PImzaiYU7fpDmEQ+kqxictl1rIYrimQyV3HNxaDRpV/lIzMWtOLFz n/+cqFfBgmT1o6eDqGk4mVhAWL3zEhcXIyhdlvGHKFVJDLspGGMcdx290xdItMYPhnJ0 xP2HhZK6WdKink7gsOyYA5cH/Cp3FdyXw96g2S9Yp56Ic+v474GYENyxNm7tmyDmI5LX vkNBGggOzsQ83HrJ+qh09GBYf6SzZSkO3fFEXj2CiaIuJI7KK2kj3zzjv9ETdK/5CtS9 HyK7hCOujm9BlEX05ZjhGfbEy5avN2gqTbsawot9xgyswzjzf7lcLG67cjpO2Uy25B10 Hp2A== X-Received: by 10.107.128.219 with SMTP id k88mr22219505ioi.27.1425585265689; Thu, 05 Mar 2015 11:54:25 -0800 (PST) MIME-Version: 1.0 Received: by 10.64.55.167 with HTTP; Thu, 5 Mar 2015 11:54:05 -0800 (PST) In-Reply-To: References: From: Miguel Clara Date: Thu, 5 Mar 2015 19:54:05 +0000 Message-ID: Subject: Re: X11 tray icons not visible (show a 'square') in FreeBSD current To: freebsd-current Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 05 Mar 2015 19:54:26 -0000 On Thu, Mar 5, 2015 at 7:52 PM, Miguel Clara wrote: > > This is a issue I've noticed in Lumina last week, but seems to be realted > to some change in CURRENT. > > I'm running an older build on another laptop with no issues: > 11.0-CURRENT FreeBSD 11.0-CURRENT #7 > b6a4dfa(master)-dirt > > Not sure which one after that might have introduce it, or if its maybe > something else.. Both run the same version of Xorg. > > * And the same version of Lumina! From owner-freebsd-current@FreeBSD.ORG Thu Mar 5 20:12:11 2015 Return-Path: Delivered-To: freebsd-current@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id F2C95E47 for ; Thu, 5 Mar 2015 20:12:11 +0000 (UTC) Received: from bitmail.cc (bitmail.cc [84.201.3.213]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id B8C0D112 for ; Thu, 5 Mar 2015 20:12:11 +0000 (UTC) Message-ID: <54F8B653.606@bitmail.cc> DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bitmail.cc; s=mail; t=1425585747; bh=SBpB4hTmUo0LyUa921A7LhSc0/9J6fR9UhK0GanubJE=; h=Date:From:To:Subject:From; b=XW7ysGeTs97nIiohx/BFHFWmmKdWCiq+m73uIA3dS6/KJC2uCJV9bisLbiaCbk04g Tjay99dd9OZ7YEK9RzF45OG2K5X/RWRdG/RVU6oUumUuCk0WBhP8ud1Rl1G1MVbTlF A4HHbUJl/6Po32DbbJp3GF4kBczqCIUSyb8aWpwk= Date: Thu, 05 Mar 2015 21:02:27 +0100 From: David S MIME-Version: 1.0 To: "freebsd-current@freebsd.org >> FreeBSD Current" Subject: upgrading current -> graphics bug Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 05 Mar 2015 20:12:12 -0000 hello, i've been using -current for awhile. lately when i try to upgrade, i see this screenshot after i type "shutdown now": https://i.imgur.com/58aOGnG.jpg from then on the system is unresponsive. to complete the upgrade i hard-reboot and jump into single user mode to finish the upgrade process. i'm following the instructions from https://www.freebsd.org/doc/en/books/handbook/makeworld.html it's been like this for awhile (a couple of months). it might be caused by the new vt terminal driver? i'm not sure. let me know if you need more information on how to reproduce this. greetings, david From owner-freebsd-current@FreeBSD.ORG Thu Mar 5 20:29:33 2015 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D789A6DF for ; Thu, 5 Mar 2015 20:29:33 +0000 (UTC) Received: from barracuda.ixsystems.com (mail.ixsystems.com [69.198.165.135]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "*.ixsystems.com", Issuer "Go Daddy Secure Certification Authority" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id B02012D3 for ; Thu, 5 Mar 2015 20:29:32 +0000 (UTC) X-ASG-Debug-ID: 1425587355-08ca04364f07970002-XDYc8F Received: from [192.168.0.51] (75-130-56-30.static.kgpt.tn.charter.com [75.130.56.30]) by barracuda.ixsystems.com with ESMTP id gv68KpUy2EthcpTH (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NO) for ; Thu, 05 Mar 2015 12:29:16 -0800 (PST) X-Barracuda-Envelope-From: kris@pcbsd.org X-Barracuda-AUTH-User: kris@pcbsd.org X-Barracuda-Apparent-Source-IP: 75.130.56.30 Message-ID: <54F8BC9B.5060801@pcbsd.org> Date: Thu, 05 Mar 2015 15:29:15 -0500 From: Kris Moore User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:31.0) Gecko/20100101 Thunderbird/31.4.0 MIME-Version: 1.0 To: freebsd-current@freebsd.org Subject: Re: X11 tray icons not visible (show a 'square') in FreeBSD current References: X-ASG-Orig-Subj: Re: X11 tray icons not visible (show a 'square') in FreeBSD current In-Reply-To: Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-Barracuda-Connect: 75-130-56-30.static.kgpt.tn.charter.com[75.130.56.30] X-Barracuda-Start-Time: 1425587356 X-Barracuda-Encrypted: ECDHE-RSA-AES128-GCM-SHA256 X-Barracuda-URL: https://10.2.0.41:443/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at ixsystems.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.00 X-Barracuda-Spam-Status: No, SCORE=0.00 using global scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=8.0 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.16238 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 05 Mar 2015 20:29:33 -0000 On 03/05/2015 14:54, Miguel Clara wrote: > On Thu, Mar 5, 2015 at 7:52 PM, Miguel Clara wrote: > >> This is a issue I've noticed in Lumina last week, but seems to be realted >> to some change in CURRENT. >> >> I'm running an older build on another laptop with no issues: >> 11.0-CURRENT FreeBSD 11.0-CURRENT #7 >> b6a4dfa(master)-dirt >> >> Not sure which one after that might have introduce it, or if its maybe >> something else.. Both run the same version of Xorg. >> >> * And the same version of Lumina! > _______________________________________________ > freebsd-current@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-current > To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org" Miguel, We saw this issue with Qt5 based tray apps in Gnome / Mate as well. We threw this patch into x11-toolkits/qt5-gui to fix it, you are welcome to try it on your end also: https://github.com/pcbsd/freebsd-ports/blob/master/x11-toolkits/qt5-gui/files/patch-src_plugins_platforms_xcb_qxcbwindow.cpp -- Kris Moore PC-BSD Software iXsystems From owner-freebsd-current@FreeBSD.ORG Thu Mar 5 20:35:29 2015 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 087A3997 for ; Thu, 5 Mar 2015 20:35:29 +0000 (UTC) Received: from mail-ie0-x22a.google.com (mail-ie0-x22a.google.com [IPv6:2607:f8b0:4001:c03::22a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id BE4723C8 for ; Thu, 5 Mar 2015 20:35:28 +0000 (UTC) Received: by iecat20 with SMTP id at20so26548346iec.6 for ; Thu, 05 Mar 2015 12:35:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; bh=PcttAmQCKSI5wn7ned3GcupMFf2KJu51JnBYiGvIVVA=; b=aVQ0emTaM58plD3bbkMnVtCBdwNajEL6Y8NKCJFQGxCp+3+qgsOxPbLasZlfnsjMzz Ha6T3oQ/gQZQjead7RXuq/f9dVqW/UlmDNsd+GZZ0J5wjjFQsduNkkkIGSbQqjsFdLRu ucngJTZeA6UxZRA6ORfdZZzBD3PSduKACNvApOB3EX+LbUyGzL2O58tVo2lPyQK8wzq7 LM3Ms4w0kmbZ0UfxR23XlSsBpff/YY8mpMNbD0WVEkZeJmEbjJSOShAXW7aQBbpR6+XR oI0ca7KdN8aTNkR32kpeJfe+h+NAn1CvNXu7zJuaECrMN8/K9H7b9eIqExUQ5Owii6ke YNrQ== X-Received: by 10.50.78.131 with SMTP id b3mr49264996igx.0.1425587728070; Thu, 05 Mar 2015 12:35:28 -0800 (PST) MIME-Version: 1.0 Received: by 10.64.55.167 with HTTP; Thu, 5 Mar 2015 12:35:07 -0800 (PST) In-Reply-To: <54F8BC9B.5060801@pcbsd.org> References: <54F8BC9B.5060801@pcbsd.org> From: Miguel Clara Date: Thu, 5 Mar 2015 20:35:07 +0000 Message-ID: Subject: Re: X11 tray icons not visible (show a 'square') in FreeBSD current To: Kris Moore Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 Cc: freebsd-current X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 05 Mar 2015 20:35:29 -0000 On Thu, Mar 5, 2015 at 8:29 PM, Kris Moore wrote: > On 03/05/2015 14:54, Miguel Clara wrote: > > On Thu, Mar 5, 2015 at 7:52 PM, Miguel Clara > wrote: > > > >> This is a issue I've noticed in Lumina last week, but seems to be > realted > >> to some change in CURRENT. > >> > >> I'm running an older build on another laptop with no issues: > >> 11.0-CURRENT FreeBSD 11.0-CURRENT #7 > >> b6a4dfa(master)-dirt > >> > >> Not sure which one after that might have introduce it, or if its maybe > >> something else.. Both run the same version of Xorg. > >> > >> * And the same version of Lumina! > > _______________________________________________ > > freebsd-current@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-current > > To unsubscribe, send any mail to " > freebsd-current-unsubscribe@freebsd.org" > Miguel, > > We saw this issue with Qt5 based tray apps in Gnome / Mate as well. We > threw this patch into x11-toolkits/qt5-gui to fix it, you are welcome to > try it on your end also: > > > https://github.com/pcbsd/freebsd-ports/blob/master/x11-toolkits/qt5-gui/files/patch-src_plugins_platforms_xcb_qxcbwindow.cpp > > > Hum. it seems that patch is in the ports three already... I'll try a qt5-gui rebuild and see if it does the trick. > -- > Kris Moore > PC-BSD Software > iXsystems > > _______________________________________________ > freebsd-current@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-current > To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org" > From owner-freebsd-current@FreeBSD.ORG Thu Mar 5 20:38:09 2015 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id AB8E5B48 for ; Thu, 5 Mar 2015 20:38:09 +0000 (UTC) Received: from mail-yk0-x229.google.com (mail-yk0-x229.google.com [IPv6:2607:f8b0:4002:c07::229]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 62E413F8 for ; Thu, 5 Mar 2015 20:38:09 +0000 (UTC) Received: by ykq19 with SMTP id 19so24352929ykq.4 for ; Thu, 05 Mar 2015 12:38:08 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=zqZPS5kSfjOfIjDWw4BnKnhLA77mizJ4sVqftZVJgg8=; b=yAjXivW/g6/ssOa6aga9/fjH62Rg/l4A5eu+fv1aaGuhtRWIfMjUr9vlchutO+igDM AP+4513J9YV1RYByMvXC+80AYpgrzV2hZKOI1ueosvTZSPp7j7ERTynfAMqHwJfVSlk2 qOulH5ZiBTo4yz5meLa6W/4/yug9hK5Z7dKI1CdXB0n8nYCeSefnJgzS5h5e92XQ65SJ g803EmMon1U3ti1W6xZ5H+zfdTRKfDa56kEu8iHmgc9FMFOvXyZ//YUEHhJM9vd3g4zl FwfTgPpakkmx61BjaOKgiIHt5yO/8aOtEbbCZRlwAuWTErFGD2Bq49f8Str7MRknjnrl Nzhg== X-Received: by 10.170.110.138 with SMTP id c132mr10395602ykb.80.1425587888566; Thu, 05 Mar 2015 12:38:08 -0800 (PST) Received: from k8-bsd.hsd1.ga.comcast.net ([2601:0:4600:611:2ed0:5aff:fe78:91ca]) by mx.google.com with ESMTPSA id q13sm6426880yhb.4.2015.03.05.12.38.07 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 05 Mar 2015 12:38:08 -0800 (PST) Date: Thu, 5 Mar 2015 15:38:58 -0500 From: "Roberto Rodriguez Jr." To: David S Subject: Re: upgrading current -> graphics bug Message-ID: <20150305203858.GA80059@k8-bsd.hsd1.ga.comcast.net> References: <54F8B653.606@bitmail.cc> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <54F8B653.606@bitmail.cc> User-Agent: Mutt/1.5.23 (2014-03-12) Cc: freebsd-current@freebsd.org X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 05 Mar 2015 20:38:09 -0000 On Thu, Mar 05, 2015 at 09:02:27PM +0100, David S wrote: > hello, > > i've been using -current for awhile. lately when i try to upgrade, i see > this screenshot after i type "shutdown now": https://i.imgur.com/58aOGnG.jpg > from then on the system is unresponsive. to complete the upgrade i > hard-reboot and jump into single user mode to finish the upgrade process. > > i'm following the instructions from > https://www.freebsd.org/doc/en/books/handbook/makeworld.html > > it's been like this for awhile (a couple of months). it might be caused > by the new vt terminal driver? i'm not sure. let me know if you need > more information on how to reproduce this. > > greetings, > david Hello, david 1-Is it a laptop, server or desktop? intel or amd? 2-Are you building xorg as a port or installed a pre-compiled pkg? 3-Are you loading drm modules in /boot/loader.conf or do you let xorg auto load the kernel modules for your card? 4-What graphics card is it? ATI or NVIDIA? 5-Are you configuring xorg with xorg.conf(deprecated) or starting xorg with 'startx'(xinit) or are you using a display manager(xdm)? 6-Have you read 24.6.1 #13 in the Handbook? Roberto Rodriguez Jr. Mach1ne Defense Foundati0n From owner-freebsd-current@FreeBSD.ORG Fri Mar 6 00:19:02 2015 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id AD48FEEB for ; Fri, 6 Mar 2015 00:19:02 +0000 (UTC) Received: from mail-ig0-x22b.google.com (mail-ig0-x22b.google.com [IPv6:2607:f8b0:4001:c05::22b]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6D182E67 for ; Fri, 6 Mar 2015 00:19:02 +0000 (UTC) Received: by igbhl2 with SMTP id hl2so50596787igb.3 for ; Thu, 05 Mar 2015 16:19:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; bh=Owni33MGwJQLDTHjx8zTR/Lq4/ceYjZa0OJN2StRf14=; b=uUggvcYBwwq9i4eLRQXh7DGpwMJ1xs/bCntr5Qg4JmSMlg1vijV6CdO1VhbaprvvI1 gcok9zT2UYOgHKYslHe/awkkPegRI4ic6r5RqsrDsFqxaCO5KRFOJ2ejKpWvpo0e81Wi 7iMbhFujE+C/xpp8gQVEq2ehQAsgNZ0S+pvZALz3vAANnTunFFMS0w7KXw4pJNuVjvY8 BmiQoN7Q2xKnXY+MQjM41m5dpv8//D2il6SF6sfQ+GMSGczlZRU/5Y0JE9xVf7+826Zn K5oqrNYwnE4cDCGZNWwSuIsRtbThgLQA6hwapttAuAzJX0T6etQyprfvnzU5ycHtkGzw xhAw== X-Received: by 10.42.109.12 with SMTP id j12mr6956478icp.22.1425601141867; Thu, 05 Mar 2015 16:19:01 -0800 (PST) MIME-Version: 1.0 Received: by 10.64.55.167 with HTTP; Thu, 5 Mar 2015 16:18:41 -0800 (PST) In-Reply-To: References: <54F8BC9B.5060801@pcbsd.org> From: Miguel Clara Date: Fri, 6 Mar 2015 00:18:41 +0000 Message-ID: Subject: Re: X11 tray icons not visible (show a 'square') in FreeBSD current To: Kris Moore Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 Cc: freebsd-current X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Mar 2015 00:19:02 -0000 On Thu, Mar 5, 2015 at 8:35 PM, Miguel Clara wrote: > > On Thu, Mar 5, 2015 at 8:29 PM, Kris Moore wrote: > >> On 03/05/2015 14:54, Miguel Clara wrote: >> > On Thu, Mar 5, 2015 at 7:52 PM, Miguel Clara >> wrote: >> > >> >> This is a issue I've noticed in Lumina last week, but seems to be >> realted >> >> to some change in CURRENT. >> >> >> >> I'm running an older build on another laptop with no issues: >> >> 11.0-CURRENT FreeBSD 11.0-CURRENT #7 >> >> b6a4dfa(master)-dirt >> >> >> >> Not sure which one after that might have introduce it, or if its maybe >> >> something else.. Both run the same version of Xorg. >> >> >> >> * And the same version of Lumina! >> > _______________________________________________ >> > freebsd-current@freebsd.org mailing list >> > http://lists.freebsd.org/mailman/listinfo/freebsd-current >> > To unsubscribe, send any mail to " >> freebsd-current-unsubscribe@freebsd.org" >> Miguel, >> >> We saw this issue with Qt5 based tray apps in Gnome / Mate as well. We >> threw this patch into x11-toolkits/qt5-gui to fix it, you are welcome to >> try it on your end also: >> >> >> https://github.com/pcbsd/freebsd-ports/blob/master/x11-toolkits/qt5-gui/files/patch-src_plugins_platforms_xcb_qxcbwindow.cpp >> >> >> Hum. it seems that patch is in the ports three already... I'll try a > qt5-gui rebuild and see if it does the trick. > Actually I was already running the version with the patch, there was an update to the port today, I'm now running qt5-gui-5.3.2_2, and the other laptop run qt5-gui-5.3.2_1 (which includes the patch) So the issue doesn't seem to be the same, but thanks. > >> -- >> Kris Moore >> PC-BSD Software >> iXsystems >> >> _______________________________________________ >> freebsd-current@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-current >> To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org >> " >> > > From owner-freebsd-current@FreeBSD.ORG Fri Mar 6 16:20:19 2015 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 8ED74822 for ; Fri, 6 Mar 2015 16:20:19 +0000 (UTC) Received: from bitmail.cc (bitmail.cc [84.201.3.213]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 8121DDC2 for ; Fri, 6 Mar 2015 16:20:18 +0000 (UTC) Message-ID: <54F9D3B8.9050308@bitmail.cc> DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bitmail.cc; s=mail; t=1425658808; bh=nRSly+BxpLwpM/Iy5gqImSSdA0mqBSsGdzt+vO/MBHI=; h=Date:From:To:Subject:References:In-Reply-To:From; b=JlglOjUODs1aralGlr5QzM51ls1Z7r8DU+SiKHIoa8OzMe7EcQHntFFHNSuIF3NJ1 LB7Fe6K5L2om8fLE0F344VL/Buwf40jf7/RdS858bZHwv3S77iQcGK0oNWCFIlXdAi ncSeCiC2zcSbrzDUOkfZtz0Aze6UxJv6j4UvlMJI= Date: Fri, 06 Mar 2015 17:20:08 +0100 From: David S MIME-Version: 1.0 To: freebsd-current@freebsd.org Subject: Re: upgrading current -> graphics bug References: <54F8B653.606@bitmail.cc> <20150305203858.GA80059@k8-bsd.hsd1.ga.comcast.net> In-Reply-To: <20150305203858.GA80059@k8-bsd.hsd1.ga.comcast.net> Content-Type: multipart/mixed; boundary="------------050600040709040904020008" X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Mar 2015 16:20:19 -0000 This is a multi-part message in MIME format. --------------050600040709040904020008 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Hi Roberto, i just discovered, that when i stop xorg and unload the nvidia driver (kldunload nvidia) the problem goes away. > 1-Is it a laptop, server or desktop? intel or amd? It's a desktop with an Intel chipset (Core i5-2500K cpu) > 2-Are you building xorg as a port or installed a pre-compiled pkg? pre-compiled xorg pkg > 3-Are you loading drm modules in /boot/loader.conf or do you let > xorg auto load the kernel modules for your card? /boot/loader.conf loads the propietary NVidia driver: nvidia_load=YES and my xorg.conf also loads a nvidia driver: Driver "nvidia" > 4-What graphics card is it? ATI or NVIDIA? it's a GeForce GTX 760 PCIe card > 5-Are you configuring xorg with xorg.conf(deprecated) or starting > xorg with 'startx'(xinit) or are you using a display manager(xdm)? i started with a barebones xorg.conf and had to put in the monitor h-sync and v-sync values. i've attached the file. > 6-Have you read 24.6.1 #13 in the Handbook? yes i've been doing that. cheers, david --------------050600040709040904020008 Content-Type: text/plain; charset=UTF-8; name="xorg.conf" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="xorg.conf" U2VjdGlvbiAiU2VydmVyTGF5b3V0IgoJSWRlbnRpZmllciAgICAgIlgub3JnIENvbmZpZ3Vy ZWQiCglTY3JlZW4gICAgICAwICAiU2NyZWVuMCIgMCAwCglJbnB1dERldmljZSAgICAiTW91 c2UwIiAiQ29yZVBvaW50ZXIiCglJbnB1dERldmljZSAgICAiS2V5Ym9hcmQwIiAiQ29yZUtl eWJvYXJkIgoKCSMgZml4OiBtb3VzZSBzdG9wcyB3b3JraW5nIC4uLgoJT3B0aW9uICJBdXRv QWRkRGV2aWNlcyIgIkZhbHNlIgoJT3B0aW9uICJBbGxvd0VtcHR5SW5wdXQiICJGYWxzZSIK RW5kU2VjdGlvbgoKU2VjdGlvbiAiRmlsZXMiCglNb2R1bGVQYXRoICAgIi91c3IvbG9jYWwv bGliL3hvcmcvbW9kdWxlcyIKCUZvbnRQYXRoICAgICAiL3Vzci9sb2NhbC9saWIvWDExL2Zv bnRzL21pc2MvIgoJRm9udFBhdGggICAgICIvdXNyL2xvY2FsL2xpYi9YMTEvZm9udHMvVFRG LyIKCUZvbnRQYXRoICAgICAiL3Vzci9sb2NhbC9saWIvWDExL2ZvbnRzL09URi8iCglGb250 UGF0aCAgICAgIi91c3IvbG9jYWwvbGliL1gxMS9mb250cy9UeXBlMS8iCglGb250UGF0aCAg ICAgIi91c3IvbG9jYWwvbGliL1gxMS9mb250cy8xMDBkcGkvIgoJRm9udFBhdGggICAgICIv dXNyL2xvY2FsL2xpYi9YMTEvZm9udHMvNzVkcGkvIgoKCUZvbnRQYXRoICAgICAiL3Vzci9s b2NhbC9saWIvWDExL2ZvbnRzL0xpbkxpYmVydGluZUcvIgoJRm9udFBhdGggICAgICIvdXNy L2xvY2FsL2xpYi9YMTEvZm9udHMvR2VudGl1bUJhc2ljLyIKCUZvbnRQYXRoICAgICAiL3Vz ci9sb2NhbC9saWIvWDExL2ZvbnRzL0NhcmxpdG8vIgoJRm9udFBhdGggICAgICIvdXNyL2xv Y2FsL2xpYi9YMTEvZm9udHMvQ2FsYWRlYS8iCglGb250UGF0aCAgICAgIi91c3IvbG9jYWwv bGliL1gxMS9mb250cy9iaXRzdHJlYW0tdmVyYS8iCkVuZFNlY3Rpb24KClNlY3Rpb24gIk1v ZHVsZSIKCUxvYWQgICJkcmkyIgoJTG9hZCAgImV4dG1vZCIKCUxvYWQgICJyZWNvcmQiCglM b2FkICAiZHJpIgoJTG9hZCAgImRiZSIKCUxvYWQgICJnbHgiCkVuZFNlY3Rpb24KClNlY3Rp b24gIklucHV0RGV2aWNlIgoJSWRlbnRpZmllciAgIktleWJvYXJkMCIKCURyaXZlciAgICAg ICJrYmQiCkVuZFNlY3Rpb24KClNlY3Rpb24gIklucHV0RGV2aWNlIgoJSWRlbnRpZmllciAg Ik1vdXNlMCIKCURyaXZlciAgICAgICJtb3VzZSIKCU9wdGlvbgkgICAgIlByb3RvY29sIiAi YXV0byIKCU9wdGlvbgkgICAgIkRldmljZSIgIi9kZXYvc3lzbW91c2UiCglPcHRpb24JICAg ICJaQXhpc01hcHBpbmciICI0IDUgNiA3IgpFbmRTZWN0aW9uCgpTZWN0aW9uICJNb25pdG9y IgoJSWRlbnRpZmllciAgICJNb25pdG9yMCIKCVZlbmRvck5hbWUgICAiTW9uaXRvciBWZW5k b3IiCglNb2RlbE5hbWUgICAgIk1vbml0b3IgTW9kZWwiCglIb3JpelN5bmMgICAgMjQgLSA4 MAoJVmVydFJlZnJlc2ggIDI0IC0gNjAKCU9wdGlvbiAgICAgICAiRFBNUyIKRW5kU2VjdGlv bgoKU2VjdGlvbiAiRGV2aWNlIgoJSWRlbnRpZmllciAgIkNhcmQwIgoJRHJpdmVyICAgICAg Im52aWRpYSIKCUJ1c0lEICAgICAgICJQQ0k6MTowOjAiCkVuZFNlY3Rpb24KClNlY3Rpb24g IlNjcmVlbiIKCUlkZW50aWZpZXIgIlNjcmVlbjAiCglEZXZpY2UgICAgICJDYXJkMCIKCU1v bml0b3IgICAgIk1vbml0b3IwIgoJRGVmYXVsdERlcHRoIDI0CglTdWJTZWN0aW9uICJEaXNw bGF5IgoJCVZpZXdwb3J0ICAgMCAwCgkJRGVwdGggICAgIDI0CgkJTW9kZXMgIjE5MjB4MTIw MCIKCUVuZFN1YlNlY3Rpb24KRW5kU2VjdGlvbgoKCg== --------------050600040709040904020008-- From owner-freebsd-current@FreeBSD.ORG Fri Mar 6 17:55:56 2015 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id EC02C1C5 for ; Fri, 6 Mar 2015 17:55:55 +0000 (UTC) Received: from mail-ig0-x22f.google.com (mail-ig0-x22f.google.com [IPv6:2607:f8b0:4001:c05::22f]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id ADD5BAA8 for ; Fri, 6 Mar 2015 17:55:55 +0000 (UTC) Received: by igbhl2 with SMTP id hl2so5698685igb.3 for ; Fri, 06 Mar 2015 09:55:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=roBBHS4IlubnlU2WR3HGKbctAd4tY3gHtkfGxOKOEqI=; b=rgEbKlMp3Gaj56iob0u4JexOmAdwTLv/VYPCU+B8tlPYWXLN1z3cRjrF1BQEsGROu4 G6eGSnnrmmbLeDDgwu2MU9XRpeCceQgOQS5IzrKBvXgglhvbbaLUHhB637nMryq0QpEm TZpoAxndYh1B3IGSXCtk7z5MhUPNx4KM3RPi2VuakL47KNwxZABt6ZCOr7nOMI3H9+A/ pY5F8sdbJWBEqYea9lPDbo4ToRg5d4HwNeOIUp4fBs8krhovNI58eX3+zPmPV/pmahT0 OUL1piG+gIYnifzqiMZ2L2xPrUEZH+x+Ls74J+ut/zVbq12xI58gzDTEJNBPIA6wLEnU IMtg== MIME-Version: 1.0 X-Received: by 10.107.27.143 with SMTP id b137mr29724264iob.76.1425664554955; Fri, 06 Mar 2015 09:55:54 -0800 (PST) Sender: kob6558@gmail.com Received: by 10.107.174.86 with HTTP; Fri, 6 Mar 2015 09:55:54 -0800 (PST) In-Reply-To: <20150305203858.GA80059@k8-bsd.hsd1.ga.comcast.net> References: <54F8B653.606@bitmail.cc> <20150305203858.GA80059@k8-bsd.hsd1.ga.comcast.net> Date: Fri, 6 Mar 2015 09:55:54 -0800 X-Google-Sender-Auth: 3SRo2YDjzpuK08JyqoDZ8fNIVuk Message-ID: Subject: Re: upgrading current -> graphics bug From: Kevin Oberman To: "Roberto Rodriguez Jr." Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 Cc: FreeBSD Current , David S X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Mar 2015 17:55:56 -0000 On Thu, Mar 5, 2015 at 12:38 PM, Roberto Rodriguez Jr. < witchdoctor.mdf@gmail.com> wrote: > > 5-Are you configuring xorg with xorg.conf(deprecated) or starting > xorg with 'startx'(xinit) or are you using a display manager(xdm)? > Huh? Since when is xorg.conf deprecated? While it is generally best to start out with no xorg.conf file, reality is that you will most likely need a small xorg.conf file with things like additional modules, additional fonts, setup of multiple displays, etc. As far as I know, the use of the nvidia driver requires an xorg.conf file in all cases. (I have not had an nVidia card for a few years, so this may have changed.) It is certainly a good idea to keep the xorg.conf file as small as possible. I am also confused by the series of "or"s in that xorg.conf is entirely orthogonal to whether X is started with "startx" or a display manager. -- Kevin Oberman, Network Engineer, Retired E-mail: rkoberman@gmail.com From owner-freebsd-current@FreeBSD.ORG Fri Mar 6 18:24:54 2015 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 09C3E7AD for ; Fri, 6 Mar 2015 18:24:54 +0000 (UTC) Received: from mail-pd0-x22d.google.com (mail-pd0-x22d.google.com [IPv6:2607:f8b0:400e:c02::22d]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id CA3ACE06 for ; Fri, 6 Mar 2015 18:24:53 +0000 (UTC) Received: by pdbfp1 with SMTP id fp1so25956250pdb.2 for ; Fri, 06 Mar 2015 10:24:53 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; bh=iMf9GC77LeqbA749jiuUFiuEPzEzCLVN58CyOLp3zzs=; b=X6pb+zFBvQHfOGuI4EwqAyBjEcRFQNTodkuUQBcDJzZ2sQq5Jo7GKb1XukJI3RkmtK z9tMHpQTsy5Q2dWwhcModbcfPwGL8z+4h17KNqIhxDNWu7D/s8okczGBvqvgGGCXclxd 92DvLHwgzXi9a/Joiq/i0k7SUXNYd7KK7lS4fa6gohdvPWgl0EZO49rpt5IjqjrUsA09 NGUdX+hleZrgjjTsdUMWyqp9N2B0NgbKCY6sB1NY1uBHff1AyfO3e9yZ+byv5QaX5bCb iB+zckV5mtVnG+tsUYPh5zuWEOe9JL5/JdA8O+vCucN55xjl7wiRYYJhdCHhgdO5sO65 lYuw== X-Received: by 10.68.242.200 with SMTP id ws8mr28141496pbc.138.1425666293323; Fri, 06 Mar 2015 10:24:53 -0800 (PST) MIME-Version: 1.0 Received: by 10.66.237.39 with HTTP; Fri, 6 Mar 2015 10:24:33 -0800 (PST) In-Reply-To: <54F8B653.606@bitmail.cc> References: <54F8B653.606@bitmail.cc> From: Arseny Nasokin Date: Fri, 6 Mar 2015 21:24:33 +0300 Message-ID: Subject: Re: upgrading current -> graphics bug To: David S Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 Cc: "freebsd-current@freebsd.org >> FreeBSD Current" X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Mar 2015 18:24:54 -0000 On 5 March 2015 at 23:02, David S wrote: > hello, > > i've been using -current for awhile. lately when i try to upgrade, i see > this screenshot after i type "shutdown now": https://i.imgur.com/58aOGnG. > jpg > from then on the system is unresponsive. to complete the upgrade i > hard-reboot and jump into single user mode to finish the upgrade process. > > i'm following the instructions from https://www.freebsd.org/doc/ > en/books/handbook/makeworld.html > > it's been like this for awhile (a couple of months). it might be caused by > the new vt terminal driver? i'm not sure. let me know if you need more > information on how to reproduce this. > > greetings, > david > _______________________________________________ > freebsd-current@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-current > To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org" > Which SVN revision do you use? `uname -a` should tell it. -- Eir Nym From owner-freebsd-current@FreeBSD.ORG Fri Mar 6 19:27:48 2015 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 066952F9 for ; Fri, 6 Mar 2015 19:27:48 +0000 (UTC) Received: from mail-yk0-x22f.google.com (mail-yk0-x22f.google.com [IPv6:2607:f8b0:4002:c07::22f]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B0597753 for ; Fri, 6 Mar 2015 19:27:47 +0000 (UTC) Received: by ykq142 with SMTP id 142so6905437ykq.2 for ; Fri, 06 Mar 2015 11:27:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=ydE08cumvteygbW7OCSEodWVc4IDEqjEEXqVCHiV23g=; b=e9urliDfgS+J1kg4y6UrA1iranjCzTIUDignc2KWRzHdcK05p7f09/SMGv5c5rswJY gPcyWSPeR1lDKM3Ohlgp5rLN2BOt2hBnLKF4I4EaTHPm0Fkm3g7JAobJLDO86kE/IlR3 hFQVhan1gMRM2966FIwsJtgOXp6ctZlhi//QAPIXlOOcoz3xa0mZ9KltGuq3sTJTbyHe JiND1WzIjehppWRsAMAFwJyxmAiuIealrHaVue2kM3zNt+xbGTkEPhYLbaBWdyc2sjIJ hwyDMKYLDYm8GNOffSYM5E8ILmE7Qd4SI8NJ4yf913/tU2CVeduVswsrhblCezwFFONM PbMw== X-Received: by 10.170.159.196 with SMTP id a187mr15830960ykd.118.1425670066881; Fri, 06 Mar 2015 11:27:46 -0800 (PST) Received: from k8-bsd.hsd1.ga.comcast.net (c-71-226-13-107.hsd1.ga.comcast.net. [71.226.13.107]) by mx.google.com with ESMTPSA id c26sm8844462yhb.20.2015.03.06.11.27.45 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 06 Mar 2015 11:27:46 -0800 (PST) Date: Fri, 6 Mar 2015 14:28:35 -0500 From: "Roberto Rodriguez Jr." To: Kevin Oberman Subject: Re: upgrading current -> graphics bug Message-ID: <20150306192835.GA36835@k8-bsd.hsd1.ga.comcast.net> References: <54F8B653.606@bitmail.cc> <20150305203858.GA80059@k8-bsd.hsd1.ga.comcast.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.23 (2014-03-12) Cc: freebsd-current@freebsd.org X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Mar 2015 19:27:48 -0000 On Fri, Mar 06, 2015 at 09:55:54AM -0800, Kevin Oberman wrote: > On Thu, Mar 5, 2015 at 12:38 PM, Roberto Rodriguez Jr. < > witchdoctor.mdf@gmail.com> wrote: > > > > > 5-Are you configuring xorg with xorg.conf(deprecated) or starting > > xorg with 'startx'(xinit) or are you using a display manager(xdm)? > > > > Huh? Since when is xorg.conf deprecated? While it is generally best to OK, thanks. My mistake. I read such thing in wiki.archlinux.org > start out with no xorg.conf file, reality is that you will most likely > need a small xorg.conf file with things like additional modules, > additional fonts, setup of multiple displays, etc. As far as I know, the Didn't know I needed to use the config file for those things. I use .xinitrc and other files. > use of the nvidia driver requires an xorg.conf file in all cases. (I have > not had an nVidia card for a few years, so this may have changed.) It is > certainly a good idea to keep the xorg.conf file as small as possible. Good point keeping UNIX tradition, small and simple. I use AMD architecture, Im sure there is a difference with Intel in terms of process, files and commands. > I am also confused by the series of "or"s in that xorg.conf is entirely > orthogonal to whether X is started with "startx" or a display manager. Honestly I won't define that word now cuz I'm busy with family, hence my bad english. I am not completely accurate with the technology so I do not know if there actually is a difference between xdm and xinit. Did not mean to confuse any further. -Roberto Rodriguez Jr. Wannabe UNIX enthusiast unemployed 404-474-3997 From owner-freebsd-current@FreeBSD.ORG Fri Mar 6 21:18:10 2015 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 27517958 for ; Fri, 6 Mar 2015 21:18:10 +0000 (UTC) Received: from udns.ultimatedns.net (unknown [IPv6:2602:d1:b4d6:e600:4261:86ff:fef6:aa2a]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id F0159330 for ; Fri, 6 Mar 2015 21:18:09 +0000 (UTC) Received: from ultimatedns.net (localhost [127.0.0.1]) by udns.ultimatedns.net (8.14.9/8.14.9) with ESMTP id t26LJMPi014040; Fri, 6 Mar 2015 13:19:22 -0800 (PST) (envelope-from bsd-lists@bsdforge.com) To: freebsd-current@freebsd.org, David S In-Reply-To: <54F9D3B8.9050308@bitmail.cc> References: <54F8B653.606@bitmail.cc> <20150305203858.GA80059@k8-bsd.hsd1.ga.comcast.net>, <54F9D3B8.9050308@bitmail.cc> From: "Chris H" Subject: Re: upgrading current -> graphics bug Date: Fri, 06 Mar 2015 13:19:24 -0800 Content-Type: text/plain; charset=UTF-8; format=fixed MIME-Version: 1.0 Message-id: <13a7356d67f767eca9f50ad962d12d42@ultimatedns.net> Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Mar 2015 21:18:10 -0000 On Fri, 06 Mar 2015 17:20:08 +0100 David S wrote > Hi Roberto, > > i just discovered, that when i stop xorg and unload the nvidia driver > (kldunload nvidia) the problem goes away. > > > > 1-Is it a laptop, server or desktop? intel or amd? > > It's a desktop with an Intel chipset (Core i5-2500K cpu) > > > 2-Are you building xorg as a port or installed a pre-compiled pkg? > > pre-compiled xorg pkg > > > 3-Are you loading drm modules in /boot/loader.conf or do you let > > xorg auto load the kernel modules for your card? > > /boot/loader.conf loads the propietary NVidia driver: nvidia_load=YES > and my xorg.conf also loads a nvidia driver: Driver "nvidia" > > > 4-What graphics card is it? ATI or NVIDIA? > > it's a GeForce GTX 760 PCIe card > > > 5-Are you configuring xorg with xorg.conf(deprecated) or starting > > xorg with 'startx'(xinit) or are you using a display manager(xdm)? > > i started with a barebones xorg.conf and had to put in the monitor > h-sync and v-sync values. i've attached the file. FWIW the nVidia driver docs indicate that you should comment, or remove the reference to dri in xorg.conf # Load "dri" but that dri2 is fine. Don't know that that has anything to do with your issue. But thought I'd mention it FWIW. --Chris > > > 6-Have you read 24.6.1 #13 in the Handbook? > > yes i've been doing that. > > > cheers, > david From owner-freebsd-current@FreeBSD.ORG Sat Mar 7 03:28:01 2015 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 2A14281D for ; Sat, 7 Mar 2015 03:28:01 +0000 (UTC) Received: from mail-lb0-x235.google.com (mail-lb0-x235.google.com [IPv6:2a00:1450:4010:c04::235]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 96894EC9 for ; Sat, 7 Mar 2015 03:28:00 +0000 (UTC) Received: by lbiw7 with SMTP id w7so14528232lbi.6 for ; Fri, 06 Mar 2015 19:27:58 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=ejGIXEzWPmv/3CO3gfGMhTb7rbq8elnwMwQj7hkTUUk=; b=SuQG2uWxEt44O7obMnW3AaU2dkDh/M0HE9J4KbuRBzUlSACP5v/Uj0BWA5w9pP+YQU sk2Kkvn4tYrrxfxrU+Ef1M4AGqWK/faqH4u9lE8+sVtZy+LxoUZLuL68bKj2KoXt/Hi9 Lnak4ZEYKwG+bax1R9G2QfyF/USSsqlIbGjwpvUZQixa2uvuyeA3DZmUAVm8XY/BXgxs bhMkH2HRXMe89DjtRoJ7DwyTBU22qxaol7+AEyWaDGOVa/R6DpTWHYW2Lh/BMg7v9Qea LkvSCFayFQi2maNpeJnw6boUucq4e2SYhyB3W82ZZL0VE1HNJPn8w2UO8z27PpF/V/+I UyYg== MIME-Version: 1.0 X-Received: by 10.112.26.209 with SMTP id n17mr10655817lbg.84.1425698878472; Fri, 06 Mar 2015 19:27:58 -0800 (PST) Sender: crodr001@gmail.com Received: by 10.112.82.164 with HTTP; Fri, 6 Mar 2015 19:27:58 -0800 (PST) In-Reply-To: References: <54F31510.7050607@hot.ee> <54F34B6E.2040809@astrodoggroup.com> <54F35F29.4000603@astrodoggroup.com> <54F429EF.5050400@freebsd.org> <54F46536.8040607@mu.org> <54F4C03F.7030704@freebsd.org> <54F4FECB.90501@freebsd.org> <197AA5DC-0591-4F71-BF10-51A5D8104C11@mu.org> <54F5270D.1010009@freebsd.org> Date: Fri, 6 Mar 2015 19:27:58 -0800 X-Google-Sender-Auth: GyhVI8tJNq1dhXzZzZvIhJXI78o Message-ID: Subject: Re: Massive libxo-zation that breaks everything From: Craig Rodrigues To: Alfred Perlstein Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 Cc: "freebsd-current@freebsd.org" X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 07 Mar 2015 03:28:01 -0000 On Mon, Mar 2, 2015 at 7:33 PM, Alfred Perlstein wrote: > > > Actually I want to shame third party ports into adopting libxo (or at least providing machine readable output). Well if upstream third party ports adopt libxo will remain to be seen. However, one thing we can do is to make it super-duper easy for them to incorporate. Then it will be a no-brainer for them to use libxo. I would like to see a Debian package of libxo made, so it can be incorporated into Debian and Ubuntu derived Linux distributions. I have done similar Debian packaging efforts for kyua, atf, and lutok: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=773277 https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=773278 https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=773279 Does anyone have connections in the Debian community to push these kinds of reviews forward? I've asked on the debian-bsd mailing list, but got no help. I basically need a "Debian Developer" who has privileges to upload packages to the Debian distribution to take the packages that I have made and upload them (after making sure they pass lintian checks). I can take a whack and do something similar for libxo. I was actually very surprised at how hard/bureaucratic it is to get a new package into Debian. FreeBSD ports and Mac Homebrew are actually way easier to work with in terms of adding a new package. However, Debian and Ubuntu have a lot of users, so it is worth the effort to push this kind of stuff through. -- Craig From owner-freebsd-current@FreeBSD.ORG Sat Mar 7 13:30:39 2015 Return-Path: Delivered-To: current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 18033D2A; Sat, 7 Mar 2015 13:30:39 +0000 (UTC) Received: from mx0.gentlemail.de (mx0.gentlemail.de [IPv6:2a00:e10:2800::a130]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 7A8B7D13; Sat, 7 Mar 2015 13:30:38 +0000 (UTC) Received: from mh0.gentlemail.de (ezra.dcm1.omnilan.net [78.138.80.135]) by mx0.gentlemail.de (8.14.5/8.14.5) with ESMTP id t27DUY6g041576; Sat, 7 Mar 2015 14:30:34 +0100 (CET) (envelope-from h.schmalzbauer@omnilan.de) Received: from titan.inop.mo1.omnilan.net (titan.inop.mo1.omnilan.net [IPv6:2001:a60:f0bb:1::3:1]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mh0.gentlemail.de (Postfix) with ESMTPSA id AE9739B; Sat, 7 Mar 2015 14:30:33 +0100 (CET) Message-ID: <54FAFD72.4080704@omnilan.de> Date: Sat, 07 Mar 2015 14:30:26 +0100 From: Harald Schmalzbauer Organization: OmniLAN User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; de-DE; rv:1.9.2.8) Gecko/20100906 Lightning/1.0b2 Thunderbird/3.1.2 MIME-Version: 1.0 To: "Kenneth D. Merry" Subject: Re: sa(4) driver changes available for test References: <20150214003232.GA63990@mithlond.kdm.org> <20150219001347.GA57416@mithlond.kdm.org> In-Reply-To: <20150219001347.GA57416@mithlond.kdm.org> X-Enigmail-Version: 1.1.2 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enig8E6262DF2F82D76D7986ED99" X-Greylist: ACL 119 matched, not delayed by milter-greylist-4.2.7 (mx0.gentlemail.de [78.138.80.130]); Sat, 07 Mar 2015 14:30:34 +0100 (CET) X-Milter: Spamilter (Reciever: mx0.gentlemail.de; Sender-ip: 78.138.80.135; Sender-helo: mh0.gentlemail.de; ) Cc: current@freebsd.org, scsi@freebsd.org X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 07 Mar 2015 13:30:39 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enig8E6262DF2F82D76D7986ED99 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Bez=FCglich Kenneth D. Merry's Nachricht vom 19.02.2015 01:13 (localtime= ): > I have updated the patches. > > I have removed the XPT_DEV_ADVINFO changes from the patches to head, si= nce > I committed those separately. > > I have (hopefully) fixed the build for the stable/10 patches by MFCing > dependencies. (One of them mav did for me, thanks!) > > Rough draft commit message: > > http://people.freebsd.org/~ken/sa_changes_commitmsg.20150218.1.txt > > The patches against FreeBSD/head as of SVN revision 278975: > > http://people.freebsd.org/~ken/sa_changes.20150218.1.txt > > And (untested) patches against FreeBSD stable/10 as of SVN revision 278= 974: > > http://people.freebsd.org/~ken/sa_changes.stable_10.20150218.1.txt Hello, on 26/02/2105, r278964 seems to be part from the sa_changes patchset. Do you have a sa_changes.stable_10.20150226 ready? Or is it just a matter of exluding all parts, comitted with r278964=20 from the patchset? I've done so in the mean while: ftp://ftp.omnilan.de/pub/FreeBSD/OmniLAN/misc/sa_changes.stable_10.201502= 26.fudge.patch Noticed that in sys/dev/mps/mps_sas.c 'cdai.flags' gets conditionally (#if __FreeBSD_version >=3D 1100061) the new "CDAI_FLAG_NONE", while in sbin/camcontrol/camcontrol.c, this is unconditionally used. Haven't really looked at the code, mostly because my skills wouldN#t allow me to answer this qusteion myself, but is that versioncheck in mps_sas.c still vaild with the rest of the sa_driver-changes? Thanks, -Harry --------------enig8E6262DF2F82D76D7986ED99 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.18 (FreeBSD) iEYEARECAAYFAlT6/XgACgkQLDqVQ9VXb8hYdQCgqeV5CmoPFyz60K3Nio7gVXSM 0boAn0U8uT2dR93gN3i0KVwqVAtD1+Np =8H3a -----END PGP SIGNATURE----- --------------enig8E6262DF2F82D76D7986ED99-- From owner-freebsd-current@FreeBSD.ORG Sat Mar 7 16:06:51 2015 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id AEB8A3DF for ; Sat, 7 Mar 2015 16:06:51 +0000 (UTC) Received: from bitmail.cc (bitmail.cc [84.201.3.213]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 71C051F3 for ; Sat, 7 Mar 2015 16:06:50 +0000 (UTC) Message-ID: <54FB220F.60200@bitmail.cc> DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bitmail.cc; s=mail; t=1425744400; bh=UpP6AJfz6awHFr3s3KY5TO67JD4xjTQiqEDvT/WsPxU=; h=Date:From:To:CC:Subject:References:In-Reply-To:From; b=SR0vv0re3rJbPjHTC6TTXz3vk2yCu+SLfDxKTHMq2z2DN4ojo2qwDyvbBPjo3+JS2 jNmGFSN5fYdnftKcKfMbo0ktB9U/UMnuJMCxUFnOBYGhgvyhNHlXvDQsIi8jZmCTbM w8QXVIXbYJcjHdFyfVJSv6PKR9EI7J38DSr+8cK0= Date: Sat, 07 Mar 2015 17:06:39 +0100 From: David S MIME-Version: 1.0 To: Arseny Nasokin Subject: Re: upgrading current -> graphics bug References: <54F8B653.606@bitmail.cc> In-Reply-To: Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit Cc: "freebsd-current@freebsd.org >> FreeBSD Current" X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 07 Mar 2015 16:06:51 -0000 > Which SVN revision do you use? `uname -a` should tell it. r279687 compiled on Fri Mar 6 16:47:15 CET 2015 From owner-freebsd-current@FreeBSD.ORG Sat Mar 7 16:07:38 2015 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 9570B4DD for ; Sat, 7 Mar 2015 16:07:38 +0000 (UTC) Received: from bitmail.cc (bitmail.cc [84.201.3.213]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 594251FD for ; Sat, 7 Mar 2015 16:07:38 +0000 (UTC) Message-ID: <54FB2248.3080300@bitmail.cc> DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bitmail.cc; s=mail; t=1425744456; bh=cnPSWthONW3wJPwqH8C7/Ga9XRHheJ6YiCnb57i1bQA=; h=Date:From:To:Subject:References:In-Reply-To:From; b=J3IjbksyU8ttZN1YB4Hl7OLEzKeOEtI2QjRUMRU5beQjZB6pujFkNliLLcLJkTohz 0HU+HfaMhEG9XkpkLcQ9rJMZ5N/OEHKwy7WfUW4zsr7ZTteY/S7DayjBglHLVSQxKQ Pdqj7UgXYN/x+b/JTISxTdFi8vLuOhKmd+IBr0LM= Date: Sat, 07 Mar 2015 17:07:36 +0100 From: David S MIME-Version: 1.0 To: Chris H , freebsd-current@freebsd.org Subject: Re: upgrading current -> graphics bug References: <54F8B653.606@bitmail.cc> <20150305203858.GA80059@k8-bsd.hsd1.ga.comcast.net>, <54F9D3B8.9050308@bitmail.cc> <13a7356d67f767eca9f50ad962d12d42@ultimatedns.net> In-Reply-To: <13a7356d67f767eca9f50ad962d12d42@ultimatedns.net> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 07 Mar 2015 16:07:38 -0000 > FWIW the nVidia driver docs indicate that you should > comment, or remove the reference to dri in xorg.conf > # Load "dri" > but that dri2 is fine. Don't know that that has anything to > do with your issue. But thought I'd mention it FWIW. thanks, i missed that part. i took it out of xorg.conf and will see next time i upgrade if it helps. From owner-freebsd-current@FreeBSD.ORG Sat Mar 7 16:13:33 2015 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 958F760C; Sat, 7 Mar 2015 16:13:33 +0000 (UTC) Received: from mail-qc0-x233.google.com (mail-qc0-x233.google.com [IPv6:2607:f8b0:400d:c01::233]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46A922DE; Sat, 7 Mar 2015 16:13:33 +0000 (UTC) Received: by qcxr5 with SMTP id r5so57676328qcx.10; Sat, 07 Mar 2015 08:13:32 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=date:from:to:cc:subject:message-id:in-reply-to:references :mime-version:content-type; bh=utGUAhu9ygBcnN3FyJkFTkbhluPwJAHWhEYG4lryuO8=; b=Ck6E0h3iqG5YWdfBhKeq9dklHzuwfar3x8W+bL5J1c1tkWjtRmbj13PWO5KZwP37Os ROcj2xf0nPxxMBngF73L2D8jZDCivirYOfK3vexCnAm4ZhwB1lhsROSfJ6G/Xc8nDLNH 9CBp4SBrKFFP5+lSXrX1KpslFxk1riAXqK7BI/oB+tpj3qU4vu9Kj81SmtGKm56SYNB8 9ochrutuUZTSRbwpuyuiYcy91AQrgAS6M3789C3VR8AuQTS6h73+uAfupgljBuP43T37 bW4Ju/GouwhuHrZf/+xOJiL+kBuy40LUhxYR02c9t0Ka5Xlr6Qy0CpdjyLZCACkHTfmC cVHg== X-Received: by 10.140.151.74 with SMTP id 71mr26498595qhx.15.1425744812107; Sat, 07 Mar 2015 08:13:32 -0800 (PST) Received: from kan ([2601:6:6780:7e0:226:18ff:fe00:232e]) by mx.google.com with ESMTPSA id y69sm7996150qgy.6.2015.03.07.08.13.30 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sat, 07 Mar 2015 08:13:31 -0800 (PST) Date: Sat, 7 Mar 2015 11:13:05 -0500 From: Alexander Kabaev To: =?KOI8-R?Q?Jean-S'ebastien_P'edron?= Subject: Re: [Call for testers] DRM device-independent code update to Linux 3.8 (take #2) Message-ID: <20150307111305.10d7678d@kan> In-Reply-To: <54F636B3.90701@FreeBSD.org> References: <54F636B3.90701@FreeBSD.org> X-Mailer: Claws Mail 3.11.1 (GTK+ 2.24.25; amd64-portbld-freebsd11.0) MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; boundary="Sig_/v4bwYYgoWMpVU4b8bvEXJ/2"; protocol="application/pgp-signature" Cc: Hans Petter Selasky , freebsd-x11@FreeBSD.org, freebsd-current@FreeBSD.org, Ed Maste X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 07 Mar 2015 16:13:33 -0000 --Sig_/v4bwYYgoWMpVU4b8bvEXJ/2 Content-Type: text/plain; charset=KOI8-R Content-Transfer-Encoding: quoted-printable On Tue, 03 Mar 2015 23:33:23 +0100 Jean-S'ebastien P'edron wrote: > Hi! >=20 > Here is a new patch to based on HEAD r279508: > https://people.freebsd.org/~dumbbell/graphics/drm-update-38.i.patch >=20 > You can apply it to a Subversion checkout using the following command: > svn patch drm-update-38.i.patch >=20 > There are few changes: > o The panic reported by J.R. Oldroyd is fixed, but not the CP > init problem. > o A lock assert was added, suggested by Konstantin Belousov >=20 > I had several panics ("Stray timeout") with a taskqueue used by TTM, > but it didn't occur in the past 6 days. Maybe it was a problem > outside of DRM. >=20 > I would like people to test again and report :) If something fails, > please post a full dmesg, taken after loading the relevant *kms > module, or the core.txt.$N file if it's a panic. >=20 > Hans Petter and Ed, I couldn't reproduce neither of your problems > (HDMI hotplug events storm and VT-switch misbehaviour). However, they > both involve callouts, like the "Stray timeout" panic I had with TTM. > I suspect there was a transient problem with callouts in HEAD at the > same time. Could you please test again with this patch and a very > recent HEAD? >=20 > Thank you to everyone! >=20 > --=20 > Jean-S'ebastien P'edron >=20 Just as a data point, 'stray timeout' happens in clean -current without new patch as well. So whatever that is, it is not caused by the patch. --=20 Alexander Kabaev --Sig_/v4bwYYgoWMpVU4b8bvEXJ/2 Content-Type: application/pgp-signature Content-Description: OpenPGP digital signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iEYEARECAAYFAlT7I5IACgkQQ6z1jMm+XZZgCACcC3+fHhPOuv9OG4Mg+Hf0Itz0 ZHoAoIkRP41U111b/+kS2WBZauCM/SC/ =v9HE -----END PGP SIGNATURE----- --Sig_/v4bwYYgoWMpVU4b8bvEXJ/2-- From owner-freebsd-current@FreeBSD.ORG Sat Mar 7 16:16:39 2015 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 6368A751; Sat, 7 Mar 2015 16:16:39 +0000 (UTC) Received: from mail-wg0-x22d.google.com (mail-wg0-x22d.google.com [IPv6:2a00:1450:400c:c00::22d]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E5DC82F9; Sat, 7 Mar 2015 16:16:38 +0000 (UTC) Received: by wghn12 with SMTP id n12so11830751wgh.6; Sat, 07 Mar 2015 08:16:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=8iDu3cJhYzU5XvXs3bwTClwAImBfFgTD13bokCjubPQ=; b=C++Vt74qaWNi6Rz5gad+zOBHrgdv8QLQV+4yxOiyDaJyW+5Xa4tLhQoeZTDq69gqr6 flChdRFr8p/qM3F2i2bLM/OoJmxX9L/M2YuM7FVEAdfLjaTXe1BN0kKtCSwhglHCSax5 fzDPpFbD1nCsZrYzQ1/6Xcq2NSMpw/ZObF33lUZayz1d7AEqo6d5o9aRupOY7v8Pw1ZY T7wJDrP+1YHJDim0ZsUskxWBFVdfvKBiJvY4h71yOjBqku4NhqxrAwH/JBpSosBK/Ekm d62rVxMXNQLvvMTreZwm60vGPTVOQpYgUp/M8W2HcEmFe6lc4CAmlGOTGPtE5w3giwNG ymHw== MIME-Version: 1.0 X-Received: by 10.180.102.199 with SMTP id fq7mr42857907wib.89.1425744997278; Sat, 07 Mar 2015 08:16:37 -0800 (PST) Received: by 10.27.136.67 with HTTP; Sat, 7 Mar 2015 08:16:37 -0800 (PST) In-Reply-To: <20150307111305.10d7678d@kan> References: <54F636B3.90701@FreeBSD.org> <20150307111305.10d7678d@kan> Date: Sat, 7 Mar 2015 10:16:37 -0600 Message-ID: Subject: Re: [Call for testers] DRM device-independent code update to Linux 3.8 (take #2) From: "Sam Fourman Jr." To: Alexander Kabaev Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 Cc: Hans Petter Selasky , freebsd-x11 , FreeBSD Current , Ed Maste , Jean-S'ebastien P'edron X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 07 Mar 2015 16:16:39 -0000 > Just as a data point, 'stray timeout' happens in clean -current without > new patch as well. So whatever that is, it is not caused by the patch. > > -- > Alexander Kabaev > I can confirm this, however, when it happens is RARE... it seems like it happens when I am typing in the address bar of chromium.. im using HEAD with a RAEDON on amd64 -- Sam Fourman Jr. From owner-freebsd-current@FreeBSD.ORG Sat Mar 7 20:56:47 2015 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B7CDC853; Sat, 7 Mar 2015 20:56:47 +0000 (UTC) Received: from mail.turbocat.net (mail.turbocat.net [IPv6:2a01:4f8:d16:4514::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 766D8329; Sat, 7 Mar 2015 20:56:47 +0000 (UTC) Received: from laptop015.home.selasky.org (cm-176.74.213.204.customer.telag.net [176.74.213.204]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.turbocat.net (Postfix) with ESMTPSA id 883EF1FE022; Sat, 7 Mar 2015 21:56:44 +0100 (CET) Message-ID: <54FB663B.9080803@selasky.org> Date: Sat, 07 Mar 2015 21:57:31 +0100 From: Hans Petter Selasky User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:31.0) Gecko/20100101 Thunderbird/31.4.0 MIME-Version: 1.0 To: Shawn Webb Subject: Re: Pluggable frame buffer devices References: <54E11A57.3030105@selasky.org> <1963872.pLReSBKNjx@shawnwebb-laptop> <54E6F060.2010301@selasky.org> <4020865.FJlWBPXrRZ@shawnwebb-laptop> <54E99CCE.5030401@selasky.org> In-Reply-To: <54E99CCE.5030401@selasky.org> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit Cc: Aleksandr Rybalko , freebsd-current@freebsd.org, Bruce Simpson X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 07 Mar 2015 20:56:47 -0000 Hi Shawn, I've just put some more pieces into the kernel. https://svnweb.freebsd.org/changeset/base/279752 https://svnweb.freebsd.org/changeset/base/279753 Can you give the current code a spin? To get X.org/X11 working simply create a config file under /usr/local/etc/X11/xorg.conf.d which contains the device section given by "man xf86-video-scfb". Open issues: - Restoring the frame buffer does not always work. I looks like the restored text console size is 1x1 afterwards. At least there is no panic. And you can re-plug the UDL device again. - You need to unplug the UDL device before rebooting. - UDL device must be connected to a monitor before you plug it to USB, or you can load/unload the driver to re-probe the video format. --HPS From owner-freebsd-current@FreeBSD.ORG Sat Mar 7 23:20:11 2015 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id EA6216DC; Sat, 7 Mar 2015 23:20:11 +0000 (UTC) Received: from tensor.andric.com (tensor.andric.com [IPv6:2001:7b8:3a7:1:2d0:b7ff:fea0:8c26]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "tensor.andric.com", Issuer "COMODO RSA Domain Validation Secure Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 9D842295; Sat, 7 Mar 2015 23:20:11 +0000 (UTC) Received: from [IPv6:2001:7b8:3a7::4866:e5b9:91ad:dbf1] (unknown [IPv6:2001:7b8:3a7:0:4866:e5b9:91ad:dbf1]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by tensor.andric.com (Postfix) with ESMTPSA id A1C3B5C48; Sun, 8 Mar 2015 00:20:07 +0100 (CET) Subject: Re: r279514: buildworld failure: /usr/src/lib/libnv/tests/dnv_tests.cc:453:2: error: use of overloaded operator '<<' is ambiguous Mime-Version: 1.0 (Mac OS X Mail 8.2 \(2070.6\)) Content-Type: multipart/signed; boundary="Apple-Mail=_11E81FE9-51F4-437E-A008-FCC3FE5D0ECB"; protocol="application/pgp-signature"; micalg=pgp-sha1 X-Pgp-Agent: GPGMail 2.5b5 From: Dimitry Andric In-Reply-To: <05BE1405-E939-46E4-B31B-EA8025C53CEA@FreeBSD.org> Date: Sun, 8 Mar 2015 00:20:01 +0100 Message-Id: References: <20150302115057.62d2c74c@prometheus> <20150304123110.48aa4abb@prometheus> <9DE59FA1-D495-4796-B3F5-F96D1472F66C@andric.com> <20150304181852.7bae1df4.ohartman@zedat.fu-berlin.de> <6CB3E761-8EFA-4DB4-8395-E15151F4F547@FreeBSD.org> <05BE1405-E939-46E4-B31B-EA8025C53CEA@FreeBSD.org> To: "O. Hartmann" X-Mailer: Apple Mail (2.2070.6) Cc: Julio Merino , Ryan Stone , freebsd-current X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 07 Mar 2015 23:20:12 -0000 --Apple-Mail=_11E81FE9-51F4-437E-A008-FCC3FE5D0ECB Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=us-ascii On 04 Mar 2015, at 22:31, Dimitry Andric wrote: ... > Hmm, I've now seen that there many more ATF_REQUIRE_EQ() instance in the > tests, which compare against NULL. It is rather cumbersome to fix all > those, so maybe it should be fixed in atf-c++ instead. It turns out there aren't so many after all. See this review: https://reviews.freebsd.org/D2027 -Dimitry --Apple-Mail=_11E81FE9-51F4-437E-A008-FCC3FE5D0ECB Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- Version: GnuPG/MacGPG2 v2.0.26 iEYEARECAAYFAlT7h6QACgkQsF6jCi4glqPi4ACdGRNVd1w4h++r+4/PoxsK61wG CMwAoPGUn8rLcJCadZBPKufPxikqN+Xy =t8yx -----END PGP SIGNATURE----- --Apple-Mail=_11E81FE9-51F4-437E-A008-FCC3FE5D0ECB--