From owner-freebsd-net@freebsd.org Fri Jul 17 18:21:31 2020 Return-Path: Delivered-To: freebsd-net@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id F2EAA36B2E8 for ; Fri, 17 Jul 2020 18:21:31 +0000 (UTC) (envelope-from ticso@cicely7.cicely.de) Received: from raven.bwct.de (raven.bwct.de [IPv6:2a02:21e0:16e0:fe::101:1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "raven.bwct.de", Issuer "raven.bwct.de" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4B7fZy3XD7z4MF8 for ; Fri, 17 Jul 2020 18:21:30 +0000 (UTC) (envelope-from ticso@cicely7.cicely.de) Received: from mail.cicely.de ([10.1.1.37]) by raven.bwct.de (8.15.2/8.15.2) with ESMTPS id 06HILRbV091475 (version=TLSv1 cipher=DHE-RSA-AES256-SHA bits=256 verify=FAIL); Fri, 17 Jul 2020 20:21:28 +0200 (CEST) (envelope-from ticso@cicely7.cicely.de) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cicely.de; s=default; t=1595010088; bh=uYIep2JZGA7J9RDd7Ek0riEYm5wRdgkdVrlli2K+rSI=; h=Date:From:To:Cc:Subject:Reply-To:References:In-Reply-To; b=wFH0SlhDS0/B89alghhBLjLYWoYLk74yWQPgydX+pdEWfl36GV0B7YmHq8FWYrNcJ FUUniAyMsSV0HTVlk64WNXSDTL1jPpMXH4csKm//+IhGS+ArelPk0invgafDUXkTXA 931kysLYhFQMxlXIVOffb6jOoIIbwsju5NHtF0Pc= Received: from cicely7.cicely.de (cicely7.cicely.de [10.1.1.9]) by mail.cicely.de (8.14.5/8.14.4) with ESMTP id 06HILGd1065451 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Fri, 17 Jul 2020 20:21:17 +0200 (CEST) (envelope-from ticso@cicely7.cicely.de) Received: from cicely7.cicely.de (localhost [127.0.0.1]) by cicely7.cicely.de (8.15.2/8.15.2) with ESMTP id 06HILBrv081467; Fri, 17 Jul 2020 20:21:11 +0200 (CEST) (envelope-from ticso@cicely7.cicely.de) Received: (from ticso@localhost) by cicely7.cicely.de (8.15.2/8.15.2/Submit) id 06HIL6ni081466; Fri, 17 Jul 2020 20:21:06 +0200 (CEST) (envelope-from ticso) Date: Fri, 17 Jul 2020 20:21:06 +0200 From: Bernd Walter To: Michael Tuexen Cc: ticso@cicely.de, freebsd-net@freebsd.org, Bernd Walter Subject: Re: SCTP problem, how to debug? Message-ID: <20200717182106.GC79604@cicely7.cicely.de> Reply-To: ticso@cicely.de References: <20200717160739.GA79604@cicely7.cicely.de> <20200717175055.GB79604@cicely7.cicely.de> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20200717175055.GB79604@cicely7.cicely.de> X-Operating-System: FreeBSD cicely7.cicely.de 12.0-STABLE amd64 User-Agent: Mutt/1.5.11 X-Spam-Status: No, score=-2.9 required=4.5 tests=ALL_TRUSTED=-1, BAYES_00=-1.9 autolearn=ham version=3.3.0 X-Spam-Checker-Version: SpamAssassin 3.3.0 (2010-01-18) on spamd.cicely.de X-Rspamd-Queue-Id: 4B7fZy3XD7z4MF8 X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; dkim=pass header.d=cicely.de header.s=default header.b=wFH0SlhD; dmarc=none; spf=none (mx1.freebsd.org: domain of ticso@cicely7.cicely.de has no SPF policy when checking 2a02:21e0:16e0:fe::101:1) smtp.mailfrom=ticso@cicely7.cicely.de X-Spamd-Result: default: False [-0.95 / 15.00]; HAS_REPLYTO(0.00)[ticso@cicely.de]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[cicely.de:s=default]; NEURAL_HAM_MEDIUM(-0.95)[-0.947]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; MV_CASE(0.50)[]; NEURAL_HAM_LONG(-0.98)[-0.983]; MIME_GOOD(-0.10)[text/plain]; REPLYTO_DOM_NEQ_FROM_DOM(0.00)[]; DMARC_NA(0.00)[cicely.de]; TO_DN_SOME(0.00)[]; RCVD_COUNT_THREE(0.00)[4]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[cicely.de:+]; NEURAL_HAM_SHORT(-0.22)[-0.222]; R_SPF_NA(0.00)[no SPF record]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; SUBJECT_ENDS_QUESTION(1.00)[]; ASN(0.00)[asn:21461, ipnet:2a02:21e0::/32, country:DE]; RCVD_TLS_LAST(0.00)[]; MID_RHS_MATCH_FROM(0.00)[] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 17 Jul 2020 18:21:32 -0000 On Fri, Jul 17, 2020 at 07:50:55PM +0200, Bernd Walter wrote: > On Fri, Jul 17, 2020 at 07:27:00PM +0200, Michael Tuexen wrote: > > > > > > > On 17. Jul 2020, at 18:07, Bernd Walter wrote: > > > > > > I'm running an LED matrix with SCTP. > > > The matrix consists from 24 raspberry pi running NFS-root FreeBSD > > > 12.0-RELEASE (they have an SD card for u-boot and loader). > > > A client system is running FreeBSD 12.1-RELEASE. > > I fixed iterator related bugs and this was MFCed to stable/12 recently. > > The problem was that the iterator stopped sending. > > The client side should be easy to update. > The modules can be a bit more tricky, but I will try. > > > > > > > The matrix modules have a one to many service socket. > > > The daemon regularily sends status informations (temperature, etc.) to > > > each association and has a second thread to receive. > > > > > > The client system has two deamons running. > > > > > > One daemon is to control power output related to temperature states. > > > It has one thread reestablishing associations via sctp_connectx() to > > > each of the 24 modules using a single one to many socket. > > > Another thread collects all regular received data and updates stored > > > values. > > > Yet another thread sends power control data via SCTP_SENDALL, so that all modules know > > > the maxed allowed brightness rating. > > > > > > The other daemon uses the same threads to reconnect and receive. > > > It connects to the very same sockets on the modules. > > > Another thread updates picture data and wanted power rating. > > > That is sending picture data to the given matrix module and then > > > SCTP_SENDALL an update trigger to all modules. > > > That is reduced brightness at night times, ... > > > > > > All SCTP_SENDALL are just trigger with 0 length and different ppid values. > > Are you really sending messages of length 0? That shouldn't work... > > No - I was wrong. > Just checked and I do send a dummy byte: > void > apa_push_leds() > { > // send dummy content, because we wouldn't send anything without > char message = 0; > send_message(&message, 1, 0, 0x00000002, SCTP_SENDALL); > } > > send_message(const void* data, size_t len, uint32_t stream, uint32_t ppid, uint32_t flags, const String& dest = "") > { > Mutex::Guard mtx(sctp_mtx); > ssize_t remain = len; > > if (!dest.empty()) { > struct addrinfo ai; > struct addrinfo *lips; > bzero(&ai, sizeof(ai)); > ai.ai_flags = AI_ADDRCONFIG | AI_NUMERICSERV; > ai.ai_family = AF_INET6; > ai.ai_protocol = IPPROTO_SCTP; > ai.ai_socktype = SOCK_SEQPACKET; > int res = 0; > String sport = port; > String addr = dest; > res = getaddrinfo(addr.c_str(), sport.c_str(), &ai, &lips); > if (res != 0) { > throw Error("failed to resolve local ips"); > } > struct addrinfo *lip; > for (lip = lips; lip && remain > 0; lip = lip->ai_next) { > while (remain > 0) { > ssize_t res; > res = sctp_sendmsgx(sctp_socket, ((uint8_t*)data) + (len - remain), remain, lip->ai_addr, 1, ppid, flags | SCTP_EOR, stream, 0, 0); I know that I could use all resolved addresses in one call. In this case it is just a single IP address. Might change that later nonethelesss to have better example code for other needs. > if (res > 0) { > remain -= res; > } else { > if (errno != EAGAIN && errno != ENOBUFS) { Just fixed that posssible addrinfo leak. Don't think that is the issue here though. > return; > } else { > int res; > do { > struct pollfd pfd; > pfd.fd = sctp_socket; > pfd.events = POLLOUT; > pfd.revents = 0; > res = poll(&pfd, 1, 5000); > } while (res == 0); > } > } > } > } > freeaddrinfo(lips); > } else { > while (remain > 0) { > ssize_t res; > res = sctp_sendmsg(sctp_socket, ((uint8_t*)data) + (len - remain), remain, NULL, 0, ppid, flags | SCTP_EOR, stream, 0, 0); > if (res > 0) { > remain -= res; > } else { > if (errno != EAGAIN && errno != ENOBUFS) { > return; > } else { > int res; > do { > struct pollfd pfd; > pfd.fd = sctp_socket; > pfd.events = POLLOUT; > pfd.revents = 0; > res = poll(&pfd, 1, 5000); > } while (res == 0); > } > } > } > } > } > -- B.Walter http://www.bwct.de Modbus/TCP Ethernet I/O Baugruppen, ARM basierte FreeBSD Rechner uvm.