From owner-freebsd-amd64@FreeBSD.ORG Tue Aug 10 05:10:04 2010 Return-Path: Delivered-To: freebsd-amd64@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 91FBA1065679 for ; Tue, 10 Aug 2010 05:10:04 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 6DD388FC16 for ; Tue, 10 Aug 2010 05:10:04 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.4/8.14.4) with ESMTP id o7A5A4B7017950 for ; Tue, 10 Aug 2010 05:10:04 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.4/8.14.4/Submit) id o7A5A4w7017949; Tue, 10 Aug 2010 05:10:04 GMT (envelope-from gnats) Resent-Date: Tue, 10 Aug 2010 05:10:04 GMT Resent-Message-Id: <201008100510.o7A5A4w7017949@freefall.freebsd.org> Resent-From: FreeBSD-gnats-submit@FreeBSD.org (GNATS Filer) Resent-To: freebsd-amd64@FreeBSD.org Resent-Reply-To: FreeBSD-gnats-submit@FreeBSD.org, Sebastien Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 88F4F1065674 for ; Tue, 10 Aug 2010 05:00:17 +0000 (UTC) (envelope-from nobody@FreeBSD.org) Received: from www.freebsd.org (www.freebsd.org [IPv6:2001:4f8:fff6::21]) by mx1.freebsd.org (Postfix) with ESMTP id 689B88FC13 for ; Tue, 10 Aug 2010 05:00:17 +0000 (UTC) Received: from www.freebsd.org (localhost [127.0.0.1]) by www.freebsd.org (8.14.3/8.14.3) with ESMTP id o7A50H2i070712 for ; Tue, 10 Aug 2010 05:00:17 GMT (envelope-from nobody@www.freebsd.org) Received: (from nobody@localhost) by www.freebsd.org (8.14.3/8.14.3/Submit) id o7A50HgI070711; Tue, 10 Aug 2010 05:00:17 GMT (envelope-from nobody) Message-Id: <201008100500.o7A50HgI070711@www.freebsd.org> Date: Tue, 10 Aug 2010 05:00:17 GMT From: Sebastien To: freebsd-gnats-submit@FreeBSD.org X-Send-Pr-Version: www-3.1 X-Mailman-Approved-At: Tue, 10 Aug 2010 05:19:14 +0000 Cc: Subject: amd64/149488: SCTP streams not working on AMD64 platform X-BeenThere: freebsd-amd64@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Porting FreeBSD to the AMD64 platform List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Aug 2010 05:10:04 -0000 >Number: 149488 >Category: amd64 >Synopsis: SCTP streams not working on AMD64 platform >Confidential: no >Severity: serious >Priority: low >Responsible: freebsd-amd64 >State: open >Quarter: >Keywords: >Date-Required: >Class: sw-bug >Submitter-Id: current-users >Arrival-Date: Tue Aug 10 05:10:04 UTC 2010 >Closed-Date: >Last-Modified: >Originator: Sebastien >Release: 8.1R >Organization: freediameter >Environment: FreeBSD bsd64.testbed.freediameter.net 8.1-RELEASE FreeBSD 8.1-RELEASE #0: Mon Jul 19 02:36:49 UTC 2010 root@mason.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC amd64 >Description: The SCTP stack seems unable to handle the stream identifier correctly on amd64 platform (works properly on i386, not tested on other platforms). More specifically, when sending a message on stream different that 0, it is received as coming on stream 0. Since both sides of the connection are on the same machine, I am not sure if sending or receiving is faulty. This feature is critical in order to implement TLS over SCTP, since each pair of streams has a different TLS state in that case. Thank you, Best regards, Sebastien. >How-To-Repeat: I have attached a sample test program to reproduce the problem: $ gcc -o sctp -Wall sctp.c $ ./sctp Server socket: 3 Client socket: 4 Connection established : Initiator socket: # streams in = 10 # streams out = 10 Receiver socket: # streams in = 10 # streams out = 10 Data received on stream 0 while expected on 1! >Fix: Patch attached with submission follows: /* Test to show problem on FreeBSD 64bit with SCTP */ #include #include #include #include #include #include #include #include #define TEST_PORT 3868 #define CMSG_BUF_LEN 1024 #define ASSERT(x) assert(x) /* Set options on the socket to request 10 streams */ void set_use_of_streams(int sock) { int ret; /* Set the number of streams */ #ifdef SCTP_INITMSG { struct sctp_initmsg init; memset(&init, 0, sizeof(init)); init.sinit_num_ostreams = 10; /* desired number of outgoing streams */ /* Set the option to the socket */ ret = setsockopt(sock, IPPROTO_SCTP, SCTP_INITMSG, &init, sizeof(init)); if (ret < 0) { perror("setsockopt(SCTP_INITMSG)"); exit (1); } } #else /* SCTP_INITMSG */ printf("SCTP_INITMSG unsupported\n"); #endif /* SCTP_INITMSG */ #ifdef SCTP_EVENTS { struct sctp_event_subscribe event; memset(&event, 0, sizeof(event)); event.sctp_data_io_event = 1; /* to receive the stream ID in SCTP_SNDRCV ancilliary data on message reception */ event.sctp_association_event = 0; /* new or closed associations (mostly for one-to-many style sockets) */ event.sctp_address_event = 0; /* address changes */ event.sctp_send_failure_event = 0; /* delivery failures */ event.sctp_peer_error_event = 0; /* remote peer sends an error */ event.sctp_shutdown_event = 0; /* peer has sent a SHUTDOWN */ event.sctp_partial_delivery_event = 0; /* a partial delivery is aborted, probably indicating the connection is being shutdown */ /* Set the option to the socket */ ret = setsockopt(sock, IPPROTO_SCTP, SCTP_EVENTS, &event, sizeof(event)); if (ret < 0) { perror("setsockopt(SCTP_EVENTS)"); exit (1); } } #else /* SCTP_EVENTS */ printf("SCTP_EVENTS unsupported\n"); #endif /* SCTP_EVENTS */ } /* Create, bind, listen for a server socket */ int create_server(void) { int sock; struct sockaddr_in sin; /* Create the server socket */ sock = socket(AF_INET, SOCK_STREAM, IPPROTO_SCTP); if (sock < 0) { perror("socket"); exit (1); } printf("Server socket: %d\n", sock); set_use_of_streams(sock); /* Bind the socket to default wildcard address */ memset(&sin, 0, sizeof(sin)); sin.sin_family = AF_INET; sin.sin_port = htons(TEST_PORT); if ( bind(sock, (struct sockaddr *)&sin, sizeof(sin)) < 0) { perror("bind"); exit (1); } /* Now, accept incoming clients */ if ( listen(sock, 5) < 0) { perror("listen"); exit (1); } return sock; } /* Connect to the bound socket */ int create_client(void) { int sock; struct sockaddr_in sin; /* Create the new socket */ sock = socket(AF_INET, SOCK_STREAM, IPPROTO_SCTP); if (sock < 0) { perror("socket"); exit (1); } printf("Client socket: %d\n", sock); set_use_of_streams(sock); /* Connect to the server */ memset(&sin, 0, sizeof(sin)); sin.sin_family = AF_INET; sin.sin_port = htons(TEST_PORT); sin.sin_addr.s_addr = inet_addr("127.0.0.1"); if ( connect(sock, (struct sockaddr *)&sin, sizeof(sin)) < 0 ) { perror("sctp_connectx"); exit (1); } return sock; } /* Send a message on a socket and a particular stream */ void send_on_stream(int sock, unsigned int strid, unsigned char * msg, size_t sz) { struct msghdr mhdr; struct iovec iov; struct { struct cmsghdr hdr; struct sctp_sndrcvinfo sndrcv; } anci; ssize_t ret; memset(&mhdr, 0, sizeof(mhdr)); memset(&iov, 0, sizeof(iov)); memset(&anci, 0, sizeof(anci)); /* IO Vector: message data */ iov.iov_base = msg; iov.iov_len = sz; /* Anciliary data: specify SCTP stream */ anci.hdr.cmsg_len = sizeof(anci); anci.hdr.cmsg_level = IPPROTO_SCTP; anci.hdr.cmsg_type = SCTP_SNDRCV; anci.sndrcv.sinfo_stream = strid; mhdr.msg_iov = &iov; mhdr.msg_iovlen = 1; mhdr.msg_control = &anci; mhdr.msg_controllen = sizeof(anci); if ( (ret = sendmsg(sock, &mhdr, 0)) < 0) { perror("sendmsg"); exit (1); } ASSERT( ret == sz ); /* There should not be partial delivery with sendmsg... */ return; } /* Receive the next message from a socket and check it is on the expected stream */ void receive_with_stream(int sock, unsigned int strid, unsigned char * buf, size_t sz) { ssize_t ret = 0; struct msghdr mhdr; char ancidata[ CMSG_BUF_LEN ]; struct iovec iov; int verified_str = 0; /* Prepare header for receiving message */ memset(&mhdr, 0, sizeof(mhdr)); mhdr.msg_iov = &iov; mhdr.msg_iovlen = 1; mhdr.msg_control = &ancidata; mhdr.msg_controllen = sizeof(ancidata); iov.iov_base = buf; iov.iov_len = sz; /* Receive data from the socket */ if ((ret = recvmsg(sock, &mhdr, 0)) < 0) { perror("recvmsg"); exit (1); } if (ret != sz) { printf("Received %zdbytes of data, expected %zdbytes...\n", ret, sz); } /* SCTP provides an indication when we received a full record */ if ( ! (mhdr.msg_flags & MSG_EOR) ) { printf("Received data does not contain End-Of-Record flag\n"); } /* Handle the case where the data received is a notification */ if (mhdr.msg_flags & MSG_NOTIFICATION) { printf("Received data is a notification, exiting.\n"); exit (1); } /* Get the stream information */ { struct cmsghdr *hdr; struct sctp_sndrcvinfo *sndrcv; /* Handle the anciliary data */ for (hdr = CMSG_FIRSTHDR(&mhdr); hdr; hdr = CMSG_NXTHDR(&mhdr, hdr)) { /* We deal only with anciliary data at SCTP level */ if (hdr->cmsg_level != IPPROTO_SCTP) { continue; } /* Also only interested in SCTP_SNDRCV message for the moment */ if (hdr->cmsg_type != SCTP_SNDRCV) { continue; } sndrcv = (struct sctp_sndrcvinfo *) CMSG_DATA(hdr); #if 0 { printf( "Anciliary block IPPROTO_SCTP / SCTP_SNDRCV\n"); printf( " sinfo_stream : %hu\n", sndrcv->sinfo_stream); printf( " sinfo_ssn : %hu\n", sndrcv->sinfo_ssn); printf( " sinfo_flags : %hu\n", sndrcv->sinfo_flags); /* printf( " sinfo_pr_policy : %hu\n", sndrcv->sinfo_pr_policy); */ printf( " sinfo_ppid : %u\n" , sndrcv->sinfo_ppid); printf( " sinfo_context : %u\n" , sndrcv->sinfo_context); /* printf( " sinfo_pr_value : %u\n" , sndrcv->sinfo_pr_value); */ printf( " sinfo_tsn : %u\n" , sndrcv->sinfo_tsn); printf( " sinfo_cumtsn : %u\n" , sndrcv->sinfo_cumtsn); } #endif /* 0 */ if (strid != sndrcv->sinfo_stream ) { printf("Data received on stream %hu while expected on %hu!\n", sndrcv->sinfo_stream, strid); exit (1); } else { verified_str = 1; } } } if (!verified_str) { printf("Data received without the stream information! Unable to verify...\n"); exit (1); } return; } /* Main test */ int main(int argc, char *argv[]) { int servsock, inisock, rcvsock; unsigned char msg[]="abcdef"; unsigned char buf[sizeof(msg)]; /* Bind a server socket */ servsock = create_server(); /* Connect a client socket to this server */ inisock = create_client(); /* And accept this client */ rcvsock = accept(servsock, NULL, NULL); if (rcvsock < 0) { perror("accept"); exit (1); } /* Informations */ printf("Connection established :\n"); { struct sctp_status status; socklen_t sz = sizeof(status); /* Read the association parameters */ memset(&status, 0, sizeof(status)); if ( (getsockopt(inisock, IPPROTO_SCTP, SCTP_STATUS, &status, &sz) < 0 ) || (sz != sizeof(status))) { perror("getsockopt(SCTP_STATUS)"); exit (1); } printf(" Initiator socket: # streams in = %hu\n", status.sstat_instrms); printf(" # streams out = %hu\n", status.sstat_outstrms); memset(&status, 0, sizeof(status)); if ( (getsockopt(rcvsock, IPPROTO_SCTP, SCTP_STATUS, &status, &sz) < 0 ) || (sz != sizeof(status))) { perror("getsockopt(SCTP_STATUS)"); exit (1); } printf(" Receiver socket: # streams in = %hu\n", status.sstat_instrms); printf(" # streams out = %hu\n", status.sstat_outstrms); } /* Send a message on stream 1 */ send_on_stream(inisock, 1, msg, sizeof(msg)); /* Receive this message */ receive_with_stream(rcvsock, 1, buf, sizeof(buf)); /* Done! */ shutdown(inisock, SHUT_RDWR); shutdown(rcvsock, SHUT_RDWR); close(inisock); close(rcvsock); close(servsock); printf("Test passed\n"); return 0; } >Release-Note: >Audit-Trail: >Unformatted: