Date: Fri, 18 Feb 2022 00:12:49 -0800 From: Dan Mahoney <freebsd@gushi.org> To: Christoph Moench-Tegeder <cmt@burggraben.net> Cc: ports@freebsd.org Subject: Re: p5-dbd-pg Message-ID: <75FF81DA-9B77-46B2-AA54-2715DAAAC88F@gushi.org> In-Reply-To: <AD8BE06E-7964-48CC-9E85-7ECE04EB8C25@gushi.org> References: <760559B7-EE1A-421D-BB8A-8402A9305FF6@gushi.org> <YemeA8A5ihwEeQZ1@elch.exwg.net> <AD8BE06E-7964-48CC-9E85-7ECE04EB8C25@gushi.org>
next in thread | previous in thread | raw e-mail | index | archive | help
--Apple-Mail=_006587CE-578B-4CAE-BD7F-7449AD95A743
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;
charset=us-ascii
Just to touch on this again, this is the output of the pkg upgrade that =
messed this up:
New packages to be INSTALLED:
postgresql13-client: 13.5
Installed packages to be UPGRADED:
bash: 5.1.8 -> 5.1.12
bash-completion: 2.11,2 -> 2.11_1,2
bind-tools: 9.16.22 -> 9.16.25
boost-libs: 1.72.0_5 -> 1.72.0_6
ca_root_nss: 3.69_1 -> 3.71
db5: 5.3.28_7 -> 5.3.28_8
expat: 2.4.3 -> 2.4.4
freetype2: 2.11.0 -> 2.11.1
fribidi: 1.0.10 -> 1.0.11
gdbm: 1.21 -> 1.22
glib: 2.70.0_1,2 -> 2.70.3,2
gnupg: 2.3.2 -> 2.3.3_2
graphviz: 2.44.1_15 -> 2.44.1_18
harfbuzz: 3.0.0 -> 3.2.0
icu: 69.1,1 -> 70.1_1,1
jpeg-turbo: 2.1.1 -> 2.1.1_1
libXfixes: 5.0.3_2 -> 6.0.0
libXft: 2.3.3 -> 2.3.4
libdrm: 2.4.107_1,1 -> 2.4.109,1
libglvnd: 1.3.4 -> 1.4.0
libgpg-error: 1.42 -> 1.43
libpaper: 1.1.24.4 -> 1.1.28
libssh2: 1.9.0_3,3 -> 1.10.0,3
libtasn1: 4.17.0 -> 4.18.0
llvm11: 11.0.1_2 -> 11.0.1_4
mesa-libs: 21.1.8 -> 21.3.4
nano: 5.8 -> 5.9
open-vm-tools-nox11: 11.3.5,2 -> 11.3.5_1,2
openldap24-client: 2.4.59_2 -> 2.4.59_4
p5-Apache-LogFormat-Compiler: 0.35 -> 0.36
p5-B-Hooks-EndOfScope: 0.24 -> 0.25
p5-Cookie-Baker: 0.09 -> 0.11
p5-Cpanel-JSON-XS: 4.26 -> 4.27
p5-Date-Manip: 6.83 -> 6.85
p5-DateTime-Locale: 1.32 -> 1.33
p5-DateTime-TimeZone: 2.47,1 -> 2.51,1
p5-Encode: 3.12 -> 3.16
p5-Filesys-Notify-Simple: 0.12_1 -> 0.14
p5-HTTP-Headers-Fast: 0.21 -> 0.22
p5-HTTP-Message: 6.33 -> 6.35
p5-HTTP-Tiny: 0.078 -> 0.080
p5-IO-Socket-SSL: 2.072 -> 2.073
p5-JavaScript-Minifier-XS: 0.14 -> 0.15
p5-Log-Any: 1.707 -> 1.710
p5-MIME-Types: 2.21 -> 2.22
p5-Mozilla-CA: 20200520 -> 20211001
p5-POSIX-strftime-Compiler: 0.41 -> 0.44
p5-Plack: 1.0047 -> 1.0048
p5-Scalar-List-Utils: 1.59,1 -> 1.60,1
p5-Test-Simple: 1.302187 -> 1.302188
p5-Try-Tiny: 0.30 -> 0.31
p5-URI: 5.09 -> 5.10
p5-libwww: 6.57 -> 6.59
pango: 1.48.7 -> 1.48.11
pciids: 20210829 -> 20211124
postfix: 3.6.2_1,1 -> 3.6.3,1
postgresql12-client: 12.8 -> 12.9
postgresql12-server: 12.8 -> 12.9
python37: 3.7.12 -> 3.7.12_1
python38: 3.8.12 -> 3.8.12_1
rubygem-json_pure: 2.5.1 -> 2.6.1
sqlite3: 3.35.5_3,1 -> 3.35.5_4,1
vim: 8.2.3458 -> 8.2.3745
wayland: 1.19.0_1 -> 1.20.0
xorgproto: 2021.4 -> 2021.5
Installed packages to be REINSTALLED:
p5-DBD-Pg-3.15.0 (direct dependency changed: =
postgresql13-client)
p5-Heap-0.80_1 (ABI changed: 'freebsd:12:x86:64' -> =
'freebsd:12:*')
p5-Pg-2.1.1_6,1 (direct dependency changed: postgresql13-client)
Number of packages to be installed: 1
Number of packages to be upgraded: 65
Number of packages to be reinstalled: 3
The process will require 13 MiB more space.
236 MiB to be downloaded.
Proceed with this action? [y/N]: y
(Why did the pkg dependency solver not cause an upgrade to =
postgresql13-server?)
-Dan
> On Jan 20, 2022, at 10:13 AM, Dan Mahoney <freebsd@gushi.org> wrote:
>=20
>=20
>=20
>> On Jan 20, 2022, at 9:38 AM, Christoph Moench-Tegeder =
<cmt@burggraben.net> wrote:
>>=20
>> ## Dan Mahoney (freebsd@gushi.org):
>>=20
>>> On doing a routine pkg upgrade at the dayjob, the perl module =
p5-DBD-pg
>>> forced an upgrade from postgresql12-client to postgresql13-client =
(which
>>> uninstalled postgresql12-server).
>>=20
>> p5-DBD-Pg uses the "default" postgresql version, which switched from
>> 12 to 13 in Octobe r(or with the Quarterly switch, depending on what
>> you use).
>> If you build your own packages, you could pin the default postgresql
>> version according to your needs via the make.conf mechanism, that is
>> DEFAULT_VERSIONS+=3D pgsql=3D12 as an example.
>=20
> The whole point of using pkg is that I should not have to build my own =
packages.
>=20
> My ticket system was down for several hours while I reinstalled =
postgresql12 (which uininstalled p5-dbd-pg), dumped a 3.5G db, and then =
reinstalled postgresql13-server, and reimported, because that's what you =
have to do when you upgrade postgres. (I hate this about postgres, but =
it is what it is).
>=20
> Worse still, the initial install of p5-dbd-pg pulled in =
postgres13-client, but DID NOT DELETE postgres12-client or =
postgres12-server, so I assumed the whole "these things can coexist" =
problem had been fixed. As people had said, the actual client is stable =
-- and besides which, a 13 client can talk to a 12 DB (as it might do =
over the internet). The SECOND pkg upgrade then wiped out the =
postgresql12 stuff.
>=20
> For a perl module that could be flavored so it works with whatever =
thing you have installed. (It doesn't call for a specific version, it =
just has uses =3D pgsql).
>=20
> Had I seen this level of fail coming, I would have deployed on a =
parallel VM, or done the db dump beforehand, but the initial pkg upgrade =
did not show it was deleting postgres 12.
>=20
> Luckily, this ticket system is internal, and we have a process to =
cache inbound tickets on our border MXes, so it wasn't the end of the =
world, but it was still annoying.
>=20
> (We have backups, of course, but they're not down-to-the-transaction).
>=20
> -Dan
--Apple-Mail=_006587CE-578B-4CAE-BD7F-7449AD95A743
Content-Transfer-Encoding: quoted-printable
Content-Type: text/html;
charset=us-ascii
<html><head><meta http-equiv=3D"Content-Type" content=3D"text/html; =
charset=3Dus-ascii"></head><body style=3D"word-wrap: break-word; =
-webkit-nbsp-mode: space; line-break: after-white-space;" class=3D"">Just =
to touch on this again, this is the output of the pkg upgrade that =
messed this up:<div class=3D""><br class=3D""></div><div class=3D""><div =
class=3D"">New packages to be INSTALLED:</div><div class=3D""> =
postgresql13-client: 13.5</div><div class=3D""><br =
class=3D""></div><div class=3D"">Installed packages to be =
UPGRADED:</div><div class=3D""> bash: 5.1.8 =
-> 5.1.12</div><div class=3D""> =
bash-completion: 2.11,2 -> 2.11_1,2</div><div class=3D""> =
bind-tools: 9.16.22 -> 9.16.25</div><div =
class=3D""> boost-libs: 1.72.0_5 -> =
1.72.0_6</div><div class=3D""> ca_root_nss: =
3.69_1 -> 3.71</div><div class=3D""> db5: =
5.3.28_7 -> 5.3.28_8</div><div class=3D""> =
expat: 2.4.3 -> 2.4.4</div><div class=3D""> =
freetype2: 2.11.0 -> 2.11.1</div><div class=3D""> =
fribidi: 1.0.10 -> 1.0.11</div><div class=3D""> =
gdbm: 1.21 -> 1.22</div><div class=3D""> =
glib: 2.70.0_1,2 -> 2.70.3,2</div><div =
class=3D""> gnupg: 2.3.2 -> =
2.3.3_2</div><div class=3D""> graphviz: =
2.44.1_15 -> 2.44.1_18</div><div class=3D""> =
harfbuzz: 3.0.0 -> 3.2.0</div><div class=3D""> =
icu: 69.1,1 -> 70.1_1,1</div><div class=3D""> =
jpeg-turbo: 2.1.1 -> 2.1.1_1</div><div =
class=3D""> libXfixes: 5.0.3_2 -> =
6.0.0</div><div class=3D""> libXft: 2.3.3 =
-> 2.3.4</div><div class=3D""> libdrm: =
2.4.107_1,1 -> 2.4.109,1</div><div class=3D""> =
libglvnd: 1.3.4 -> 1.4.0</div><div class=3D""> =
libgpg-error: 1.42 -> 1.43</div><div class=3D""> =
libpaper: 1.1.24.4 -> 1.1.28</div><div =
class=3D""> libssh2: 1.9.0_3,3 -> =
1.10.0,3</div><div class=3D""> libtasn1: =
4.17.0 -> 4.18.0</div><div class=3D""> =
llvm11: 11.0.1_2 -> 11.0.1_4</div><div class=3D""> =
mesa-libs: 21.1.8 -> 21.3.4</div><div class=3D""> =
nano: 5.8 -> 5.9</div><div class=3D""> =
open-vm-tools-nox11: 11.3.5,2 -> =
11.3.5_1,2</div><div class=3D""> =
openldap24-client: 2.4.59_2 -> 2.4.59_4</div><div class=3D""> =
p5-Apache-LogFormat-Compiler: 0.35 -> =
0.36</div><div class=3D""> =
p5-B-Hooks-EndOfScope: 0.24 -> 0.25</div><div class=3D""> =
p5-Cookie-Baker: 0.09 -> 0.11</div><div =
class=3D""> p5-Cpanel-JSON-XS: 4.26 -> =
4.27</div><div class=3D""> p5-Date-Manip: =
6.83 -> 6.85</div><div class=3D""> =
p5-DateTime-Locale: 1.32 -> 1.33</div><div class=3D""> =
p5-DateTime-TimeZone: 2.47,1 -> 2.51,1</div><div =
class=3D""> p5-Encode: 3.12 -> =
3.16</div><div class=3D""> =
p5-Filesys-Notify-Simple: 0.12_1 -> 0.14</div><div class=3D""> =
p5-HTTP-Headers-Fast: 0.21 -> 0.22</div><div =
class=3D""> p5-HTTP-Message: 6.33 -> =
6.35</div><div class=3D""> p5-HTTP-Tiny: =
0.078 -> 0.080</div><div class=3D""> =
p5-IO-Socket-SSL: 2.072 -> 2.073</div><div class=3D""> =
p5-JavaScript-Minifier-XS: 0.14 -> 0.15</div><div =
class=3D""> p5-Log-Any: 1.707 -> =
1.710</div><div class=3D""> p5-MIME-Types: =
2.21 -> 2.22</div><div class=3D""> =
p5-Mozilla-CA: 20200520 -> 20211001</div><div class=3D""> =
p5-POSIX-strftime-Compiler: 0.41 -> =
0.44</div><div class=3D""> p5-Plack: 1.0047 =
-> 1.0048</div><div class=3D""> =
p5-Scalar-List-Utils: 1.59,1 -> 1.60,1</div><div class=3D""> =
p5-Test-Simple: 1.302187 -> 1.302188</div><div =
class=3D""> p5-Try-Tiny: 0.30 -> =
0.31</div><div class=3D""> p5-URI: 5.09 -> =
5.10</div><div class=3D""> p5-libwww: 6.57 =
-> 6.59</div><div class=3D""> pango: =
1.48.7 -> 1.48.11</div><div class=3D""> =
pciids: 20210829 -> 20211124</div><div class=3D""> =
postfix: 3.6.2_1,1 -> 3.6.3,1</div><div class=3D""> =
postgresql12-client: 12.8 -> 12.9</div><div =
class=3D""> postgresql12-server: 12.8 -> =
12.9</div><div class=3D""> python37: 3.7.12 =
-> 3.7.12_1</div><div class=3D""> =
python38: 3.8.12 -> 3.8.12_1</div><div class=3D""> =
rubygem-json_pure: 2.5.1 -> 2.6.1</div><div =
class=3D""> sqlite3: 3.35.5_3,1 -> =
3.35.5_4,1</div><div class=3D""> vim: =
8.2.3458 -> 8.2.3745</div><div class=3D""> =
wayland: 1.19.0_1 -> 1.20.0</div><div class=3D""> =
xorgproto: 2021.4 -> 2021.5</div><div class=3D""><br =
class=3D""></div><div class=3D"">Installed packages to be =
REINSTALLED:</div><div class=3D""> =
p5-DBD-Pg-3.15.0 (direct dependency changed: =
postgresql13-client)</div><div class=3D""> =
p5-Heap-0.80_1 (ABI changed: 'freebsd:12:x86:64' -> =
'freebsd:12:*')</div><div class=3D""> =
p5-Pg-2.1.1_6,1 (direct dependency changed: =
postgresql13-client)</div><div class=3D""><br class=3D""></div><div =
class=3D"">Number of packages to be installed: 1</div><div =
class=3D"">Number of packages to be upgraded: 65</div><div =
class=3D"">Number of packages to be reinstalled: 3</div><div =
class=3D""><br class=3D""></div><div class=3D"">The process will require =
13 MiB more space.</div><div class=3D"">236 MiB to be =
downloaded.</div><div class=3D""><br class=3D""></div><div =
class=3D"">Proceed with this action? [y/N]: y</div><div class=3D""><br =
class=3D""></div><div class=3D"">(Why did the pkg dependency solver not =
cause an upgrade to postgresql13-server?)</div><div class=3D""><br =
class=3D""></div><div class=3D"">-Dan</div><div><br class=3D""><blockquote=
type=3D"cite" class=3D""><div class=3D"">On Jan 20, 2022, at 10:13 AM, =
Dan Mahoney <<a href=3D"mailto:freebsd@gushi.org" =
class=3D"">freebsd@gushi.org</a>> wrote:</div><br =
class=3D"Apple-interchange-newline"><div class=3D""><meta =
charset=3D"UTF-8" class=3D""><br style=3D"caret-color: rgb(0, 0, 0); =
font-family: Helvetica; font-size: 12px; font-style: normal; =
font-variant-caps: normal; font-weight: normal; letter-spacing: normal; =
text-align: start; text-indent: 0px; text-transform: none; white-space: =
normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; =
text-decoration: none;" class=3D""><br style=3D"caret-color: rgb(0, 0, =
0); font-family: Helvetica; font-size: 12px; font-style: normal; =
font-variant-caps: normal; font-weight: normal; letter-spacing: normal; =
text-align: start; text-indent: 0px; text-transform: none; white-space: =
normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; =
text-decoration: none;" class=3D""><blockquote type=3D"cite" =
style=3D"font-family: Helvetica; font-size: 12px; font-style: normal; =
font-variant-caps: normal; font-weight: normal; letter-spacing: normal; =
orphans: auto; text-align: start; text-indent: 0px; text-transform: =
none; white-space: normal; widows: auto; word-spacing: 0px; =
-webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; =
text-decoration: none;" class=3D"">On Jan 20, 2022, at 9:38 AM, =
Christoph Moench-Tegeder <<a href=3D"mailto:cmt@burggraben.net" =
class=3D"">cmt@burggraben.net</a>> wrote:<br class=3D""><br =
class=3D"">## Dan Mahoney (<a href=3D"mailto:freebsd@gushi.org" =
class=3D"">freebsd@gushi.org</a>):<br class=3D""><br =
class=3D""><blockquote type=3D"cite" class=3D"">On doing a routine pkg =
upgrade at the dayjob, the perl module p5-DBD-pg<br class=3D"">forced an =
upgrade from postgresql12-client to postgresql13-client (which<br =
class=3D"">uninstalled postgresql12-server).<br =
class=3D""></blockquote><br class=3D"">p5-DBD-Pg uses the "default" =
postgresql version, which switched from<br class=3D"">12 to 13 in Octobe =
r(or with the Quarterly switch, depending on what<br class=3D"">you =
use).<br class=3D"">If you build your own packages, you could pin the =
default postgresql<br class=3D"">version according to your needs via the =
make.conf mechanism, that is<br class=3D"">DEFAULT_VERSIONS+=3D pgsql=3D12=
as an example.<br class=3D""></blockquote><br style=3D"caret-color: =
rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: =
normal; font-variant-caps: normal; font-weight: normal; letter-spacing: =
normal; text-align: start; text-indent: 0px; text-transform: none; =
white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; =
text-decoration: none;" class=3D""><span style=3D"caret-color: rgb(0, 0, =
0); font-family: Helvetica; font-size: 12px; font-style: normal; =
font-variant-caps: normal; font-weight: normal; letter-spacing: normal; =
text-align: start; text-indent: 0px; text-transform: none; white-space: =
normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; =
text-decoration: none; float: none; display: inline !important;" =
class=3D"">The whole point of using pkg is that I should not have to =
build my own packages.</span><br style=3D"caret-color: rgb(0, 0, 0); =
font-family: Helvetica; font-size: 12px; font-style: normal; =
font-variant-caps: normal; font-weight: normal; letter-spacing: normal; =
text-align: start; text-indent: 0px; text-transform: none; white-space: =
normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; =
text-decoration: none;" class=3D""><br style=3D"caret-color: rgb(0, 0, =
0); font-family: Helvetica; font-size: 12px; font-style: normal; =
font-variant-caps: normal; font-weight: normal; letter-spacing: normal; =
text-align: start; text-indent: 0px; text-transform: none; white-space: =
normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; =
text-decoration: none;" class=3D""><span style=3D"caret-color: rgb(0, 0, =
0); font-family: Helvetica; font-size: 12px; font-style: normal; =
font-variant-caps: normal; font-weight: normal; letter-spacing: normal; =
text-align: start; text-indent: 0px; text-transform: none; white-space: =
normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; =
text-decoration: none; float: none; display: inline !important;" =
class=3D"">My ticket system was down for several hours while I =
reinstalled postgresql12 (which uininstalled p5-dbd-pg), dumped a 3.5G =
db, and then reinstalled postgresql13-server, and reimported, because =
that's what you have to do when you upgrade postgres. (I hate this =
about postgres, but it is what it is).</span><br style=3D"caret-color: =
rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: =
normal; font-variant-caps: normal; font-weight: normal; letter-spacing: =
normal; text-align: start; text-indent: 0px; text-transform: none; =
white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; =
text-decoration: none;" class=3D""><br style=3D"caret-color: rgb(0, 0, =
0); font-family: Helvetica; font-size: 12px; font-style: normal; =
font-variant-caps: normal; font-weight: normal; letter-spacing: normal; =
text-align: start; text-indent: 0px; text-transform: none; white-space: =
normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; =
text-decoration: none;" class=3D""><span style=3D"caret-color: rgb(0, 0, =
0); font-family: Helvetica; font-size: 12px; font-style: normal; =
font-variant-caps: normal; font-weight: normal; letter-spacing: normal; =
text-align: start; text-indent: 0px; text-transform: none; white-space: =
normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; =
text-decoration: none; float: none; display: inline !important;" =
class=3D"">Worse still, the initial install of p5-dbd-pg pulled in =
postgres13-client, but DID NOT DELETE postgres12-client or =
postgres12-server, so I assumed the whole "these things can coexist" =
problem had been fixed. As people had said, the actual client is =
stable -- and besides which, a 13 client can talk to a 12 DB (as it =
might do over the internet). The SECOND pkg upgrade then wiped out =
the postgresql12 stuff.</span><br style=3D"caret-color: rgb(0, 0, 0); =
font-family: Helvetica; font-size: 12px; font-style: normal; =
font-variant-caps: normal; font-weight: normal; letter-spacing: normal; =
text-align: start; text-indent: 0px; text-transform: none; white-space: =
normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; =
text-decoration: none;" class=3D""><br style=3D"caret-color: rgb(0, 0, =
0); font-family: Helvetica; font-size: 12px; font-style: normal; =
font-variant-caps: normal; font-weight: normal; letter-spacing: normal; =
text-align: start; text-indent: 0px; text-transform: none; white-space: =
normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; =
text-decoration: none;" class=3D""><span style=3D"caret-color: rgb(0, 0, =
0); font-family: Helvetica; font-size: 12px; font-style: normal; =
font-variant-caps: normal; font-weight: normal; letter-spacing: normal; =
text-align: start; text-indent: 0px; text-transform: none; white-space: =
normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; =
text-decoration: none; float: none; display: inline !important;" =
class=3D"">For a perl module that could be flavored so it works with =
whatever thing you have installed. (It doesn't call for a specific =
version, it just has uses =3D pgsql).</span><br style=3D"caret-color: =
rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: =
normal; font-variant-caps: normal; font-weight: normal; letter-spacing: =
normal; text-align: start; text-indent: 0px; text-transform: none; =
white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; =
text-decoration: none;" class=3D""><br style=3D"caret-color: rgb(0, 0, =
0); font-family: Helvetica; font-size: 12px; font-style: normal; =
font-variant-caps: normal; font-weight: normal; letter-spacing: normal; =
text-align: start; text-indent: 0px; text-transform: none; white-space: =
normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; =
text-decoration: none;" class=3D""><span style=3D"caret-color: rgb(0, 0, =
0); font-family: Helvetica; font-size: 12px; font-style: normal; =
font-variant-caps: normal; font-weight: normal; letter-spacing: normal; =
text-align: start; text-indent: 0px; text-transform: none; white-space: =
normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; =
text-decoration: none; float: none; display: inline !important;" =
class=3D"">Had I seen this level of fail coming, I would have deployed =
on a parallel VM, or done the db dump beforehand, but the initial pkg =
upgrade did not show it was deleting postgres 12.</span><br =
style=3D"caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: =
12px; font-style: normal; font-variant-caps: normal; font-weight: =
normal; letter-spacing: normal; text-align: start; text-indent: 0px; =
text-transform: none; white-space: normal; word-spacing: 0px; =
-webkit-text-stroke-width: 0px; text-decoration: none;" class=3D""><br =
style=3D"caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: =
12px; font-style: normal; font-variant-caps: normal; font-weight: =
normal; letter-spacing: normal; text-align: start; text-indent: 0px; =
text-transform: none; white-space: normal; word-spacing: 0px; =
-webkit-text-stroke-width: 0px; text-decoration: none;" class=3D""><span =
style=3D"caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: =
12px; font-style: normal; font-variant-caps: normal; font-weight: =
normal; letter-spacing: normal; text-align: start; text-indent: 0px; =
text-transform: none; white-space: normal; word-spacing: 0px; =
-webkit-text-stroke-width: 0px; text-decoration: none; float: none; =
display: inline !important;" class=3D"">Luckily, this ticket system is =
internal, and we have a process to cache inbound tickets on our border =
MXes, so it wasn't the end of the world, but it was still =
annoying.</span><br style=3D"caret-color: rgb(0, 0, 0); font-family: =
Helvetica; font-size: 12px; font-style: normal; font-variant-caps: =
normal; font-weight: normal; letter-spacing: normal; text-align: start; =
text-indent: 0px; text-transform: none; white-space: normal; =
word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: =
none;" class=3D""><br style=3D"caret-color: rgb(0, 0, 0); font-family: =
Helvetica; font-size: 12px; font-style: normal; font-variant-caps: =
normal; font-weight: normal; letter-spacing: normal; text-align: start; =
text-indent: 0px; text-transform: none; white-space: normal; =
word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: =
none;" class=3D""><span style=3D"caret-color: rgb(0, 0, 0); font-family: =
Helvetica; font-size: 12px; font-style: normal; font-variant-caps: =
normal; font-weight: normal; letter-spacing: normal; text-align: start; =
text-indent: 0px; text-transform: none; white-space: normal; =
word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: =
none; float: none; display: inline !important;" class=3D"">(We have =
backups, of course, but they're not down-to-the-transaction).</span><br =
style=3D"caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: =
12px; font-style: normal; font-variant-caps: normal; font-weight: =
normal; letter-spacing: normal; text-align: start; text-indent: 0px; =
text-transform: none; white-space: normal; word-spacing: 0px; =
-webkit-text-stroke-width: 0px; text-decoration: none;" class=3D""><br =
style=3D"caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: =
12px; font-style: normal; font-variant-caps: normal; font-weight: =
normal; letter-spacing: normal; text-align: start; text-indent: 0px; =
text-transform: none; white-space: normal; word-spacing: 0px; =
-webkit-text-stroke-width: 0px; text-decoration: none;" class=3D""><span =
style=3D"caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: =
12px; font-style: normal; font-variant-caps: normal; font-weight: =
normal; letter-spacing: normal; text-align: start; text-indent: 0px; =
text-transform: none; white-space: normal; word-spacing: 0px; =
-webkit-text-stroke-width: 0px; text-decoration: none; float: none; =
display: inline !important;" =
class=3D"">-Dan</span></div></blockquote></div><br =
class=3D""></div></body></html>=
--Apple-Mail=_006587CE-578B-4CAE-BD7F-7449AD95A743--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?75FF81DA-9B77-46B2-AA54-2715DAAAC88F>