From owner-svn-soc-all@FreeBSD.ORG Sun Aug 18 00:00:02 2013 Return-Path: Delivered-To: svn-soc-all@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 21500649 for ; Sun, 18 Aug 2013 00:00:02 +0000 (UTC) (envelope-from peter@FreeBSD.org) Received: from socsvn.freebsd.org (socsvn.freebsd.org [IPv6:2001:1900:2254:206a::50:2]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id EAB792358 for ; Sun, 18 Aug 2013 00:00:01 +0000 (UTC) Received: from socsvn.freebsd.org ([127.0.1.124]) by socsvn.freebsd.org (8.14.7/8.14.7) with ESMTP id r7I001sS087802 for ; Sun, 18 Aug 2013 00:00:01 GMT (envelope-from peter@FreeBSD.org) Received: (from socsvn-import@localhost) by socsvn.freebsd.org (8.14.7/8.14.6/Submit) id r7I001h3087797 for svn-soc-all@FreeBSD.org; Sun, 18 Aug 2013 00:00:01 GMT (envelope-from peter@FreeBSD.org) Date: Sun, 18 Aug 2013 00:00:01 GMT Message-Id: <201308180000.r7I001h3087797@socsvn.freebsd.org> X-Authentication-Warning: socsvn.freebsd.org: socsvn-import set sender to peter@FreeBSD.org using -f From: peter@FreeBSD.org To: svn-soc-all@FreeBSD.org Subject: socsvn commit: r256090 - mirror/FreeBSD MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-soc-all@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: SVN commit messages for the entire Summer of Code repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 18 Aug 2013 00:00:02 -0000 Author: peter Date: Sun Aug 18 00:00:01 2013 New Revision: 256090 URL: http://svnweb.FreeBSD.org/socsvn/?view=rev&rev=256090 Log: Enable auto-props for the repository. Modified: mirror/FreeBSD/ (props changed) From owner-svn-soc-all@FreeBSD.ORG Sun Aug 18 02:00:01 2013 Return-Path: Delivered-To: svn-soc-all@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 3D1A2388 for ; Sun, 18 Aug 2013 02:00:01 +0000 (UTC) (envelope-from peter@FreeBSD.org) Received: from socsvn.freebsd.org (socsvn.freebsd.org [IPv6:2001:1900:2254:206a::50:2]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 106082786 for ; Sun, 18 Aug 2013 02:00:01 +0000 (UTC) Received: from socsvn.freebsd.org ([127.0.1.124]) by socsvn.freebsd.org (8.14.7/8.14.7) with ESMTP id r7I200e5021732 for ; Sun, 18 Aug 2013 02:00:00 GMT (envelope-from peter@FreeBSD.org) Received: (from socsvn-import@localhost) by socsvn.freebsd.org (8.14.7/8.14.6/Submit) id r7I200Ul021724 for svn-soc-all@FreeBSD.org; Sun, 18 Aug 2013 02:00:00 GMT (envelope-from peter@FreeBSD.org) Date: Sun, 18 Aug 2013 02:00:00 GMT Message-Id: <201308180200.r7I200Ul021724@socsvn.freebsd.org> X-Authentication-Warning: socsvn.freebsd.org: socsvn-import set sender to peter@FreeBSD.org using -f From: peter@FreeBSD.org To: svn-soc-all@FreeBSD.org Subject: socsvn commit: r256091 - mirror/FreeBSD MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-soc-all@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: SVN commit messages for the entire Summer of Code repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 18 Aug 2013 02:00:01 -0000 Author: peter Date: Sun Aug 18 02:00:00 2013 New Revision: 256091 URL: http://svnweb.FreeBSD.org/socsvn/?view=rev&rev=256091 Log: Add *.dts, *.dtsi, bsd.*.mk Submitted by: rpaulo Modified: mirror/FreeBSD/ (props changed) From owner-svn-soc-all@FreeBSD.ORG Sun Aug 18 14:44:56 2013 Return-Path: Delivered-To: svn-soc-all@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 31741F25 for ; Sun, 18 Aug 2013 14:44:56 +0000 (UTC) (envelope-from mattbw@FreeBSD.org) Received: from socsvn.freebsd.org (socsvn.freebsd.org [IPv6:2001:1900:2254:206a::50:2]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 0FBAD2A7E for ; Sun, 18 Aug 2013 14:44:56 +0000 (UTC) Received: from socsvn.freebsd.org ([127.0.1.124]) by socsvn.freebsd.org (8.14.7/8.14.7) with ESMTP id r7IEitOq027095 for ; Sun, 18 Aug 2013 14:44:55 GMT (envelope-from mattbw@FreeBSD.org) Received: (from www@localhost) by socsvn.freebsd.org (8.14.7/8.14.6/Submit) id r7IEittJ027088 for svn-soc-all@FreeBSD.org; Sun, 18 Aug 2013 14:44:55 GMT (envelope-from mattbw@FreeBSD.org) Date: Sun, 18 Aug 2013 14:44:55 GMT Message-Id: <201308181444.r7IEittJ027088@socsvn.freebsd.org> X-Authentication-Warning: socsvn.freebsd.org: www set sender to mattbw@FreeBSD.org using -f From: mattbw@FreeBSD.org To: svn-soc-all@FreeBSD.org Subject: socsvn commit: r256110 - in soc2013/mattbw/backend: actions query MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-soc-all@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: SVN commit messages for the entire Summer of Code repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 18 Aug 2013 14:44:56 -0000 Author: mattbw Date: Sun Aug 18 14:44:55 2013 New Revision: 256110 URL: http://svnweb.FreeBSD.org/socsvn/?view=rev&rev=256110 Log: (BROKEN) Dump of initial query refactoring. The query code is being rewritten in places to be more modular and allow easier usage outside of actions. The main body of work at the moment is writing code to massage the various different query inputs (PackageIDs, package names, name-versions, bundles of name, version, arch and repo) into one single format that can be used to check matches. I've decided to go with a combination of namever (as in "package-1.2.34", arch and repo, which is referred to internally as a 'query ID'. Currently I'm midway through writing the converters from other formats to this one. This code will NOT compile; please use a previous revision. It's intended as a quick dump of the state of my working copy. Added: soc2013/mattbw/backend/query/id.c Modified: soc2013/mattbw/backend/actions/resolve.c soc2013/mattbw/backend/query/core.c soc2013/mattbw/backend/query/core.h soc2013/mattbw/backend/query/depends.c soc2013/mattbw/backend/query/match.c Modified: soc2013/mattbw/backend/actions/resolve.c ============================================================================== --- soc2013/mattbw/backend/actions/resolve.c Sun Aug 18 13:38:39 2013 (r256109) +++ soc2013/mattbw/backend/actions/resolve.c Sun Aug 18 14:44:55 2013 (r256110) @@ -43,6 +43,7 @@ s.type = QUERY_BACKEND_MIXED; t.load_flags = PKG_LOAD_BASIC; t.f = emit; + t.error_if_not_found = true; success = query_do(backend, &s, &t); Modified: soc2013/mattbw/backend/query/core.c ============================================================================== --- soc2013/mattbw/backend/query/core.c Sun Aug 18 13:38:39 2013 (r256109) +++ soc2013/mattbw/backend/query/core.c Sun Aug 18 14:44:55 2013 (r256110) @@ -55,6 +55,154 @@ static struct pkgdb_it *local_query(struct query *q); static struct pkgdb_it *remote_query(struct query *q); + +struct query_split_id { + gchar *namever; + gchar *arch; + gchar *repo; +} + +/* Given an array of PackageIDs, returns an array of matching packages. + * + * Returns NULL if any of the packages did not match. + */ +struct pkg ** +query_by_ids(PkBackend *backend, unsigned int load_flags, + gchar **package_ids, unsigned int package_count) +{ + struct pkg **packages; + struct query_split_id **package_split_ids; + + assert(backend != NULL); + assert(package_ids != NULL); + + packages = NULL; + + package_split_ids = split_package_ids(package_ids, count); + if (package_split_ids != NULL) { + packages = query_by_split_ids(backend, load_flags, + package_split_ids, package_count); + free_split_package_ids(&package_split_ids); + } + + return packages; +} + + +struct pkg ** +query_by_names(PkBackend *backend, unsigned int load_flags, + const char **package_names, unsigned int package_count) +{ + struct pkg **packages; + struct query_split_id **package_split_ids; + + assert(backend != NULL); + assert(package_names != NULL); + + packages = NULL; + + package_split_ids = names_to_split_ids(package_names, count); + if (package_split_ids != NULL) { + packages = query_by_split_ids(backend, load_flags, + package_split_ids, package_count); + free_split_package_ids(&package_split_ids); + } + + return packages; +} + +struct pkg ** +query_by_split_ids(PkBackend *backend, unsigned int load_flags, + struct query_split_id *package_split_ids, unsigned int package_count) +{ + struct pkg **packages; + + packages = +} + +/* Converts an array of PackageIDs to split IDs. */ +static struct query_split_id * +split_package_ids(gchar **package_ids, unsigned int package_count) +{ + unsigned int i; + struct query_split_id *split_ids; + + split_ids = alloc_split_ids(package_count); + for () + + return split_ids; +} + +/* Converts an array of names (or name-version strings) to split IDs. */ +static struct query_split_id * +names_to_split(gchar **package_names, unsigned int package_count) +{ + unsigned int i; + gchar *split_package_id; + struct query_split_id *split_ids; + + split_ids = alloc_split_ids(package_count); + if (split_ids != NULL) { + bool error; + unsigned int i; + + error = false; + for (i = 0; i < package_count; i++) { + error = name_to_split(package_names[i], split_ids + i); + if (error) { + free_split_ids(&split_ids, package_count); + break; + } + } + } + + return split_ids; +} + +bool +name_to_split(const char *name, struct query_split_id *split_id) +{ + bool success; + + assert(name != NULL); + assert(split_id != NULL); + + split_ids->namever = strdup(name); + split_ids->arch = NULL; + split_ids->repo = NULL; + + return (split_ids->namever == NULL); +} + +bool +package_id_to_split(gchar *package_id, struct query_split_id *split_id) +{ + bool success; + gchar **split_package_id; + + assert(name != NULL); + assert(split_id != NULL); + + split_package_id = pk_package_id_split(package_id); + if (split_package_id == NULL) { + error = true; + } else if () { + + split_ids[i].namever = join_namever(name, version); + g_strfreev(split_package_id); + } + } + } + + return split_ids; +} + +static struct query_split_id * +alloc_split_ids(unsigned int count) +{ + return calloc(package_count, sizeof(struct query_split_id)); +} + /* * Returns the backend stored inside the given query. */ Modified: soc2013/mattbw/backend/query/core.h ============================================================================== --- soc2013/mattbw/backend/query/core.h Sun Aug 18 13:38:39 2013 (r256109) +++ soc2013/mattbw/backend/query/core.h Sun Aug 18 14:44:55 2013 (r256110) @@ -57,6 +57,7 @@ struct query_target { unsigned int load_flags; emit_ptr f; + bool error_if_not_found; }; PkBackend *query_backend(struct query *q); Modified: soc2013/mattbw/backend/query/depends.c ============================================================================== --- soc2013/mattbw/backend/query/depends.c Sun Aug 18 13:38:39 2013 (r256109) +++ soc2013/mattbw/backend/query/depends.c Sun Aug 18 14:44:55 2013 (r256110) @@ -30,6 +30,7 @@ query_depends_emit(PkBackend *backend, struct pkg *pkg, depends_get_ptr get) { bool success; + gchar *package_id; struct pkg_dep *dep; assert(backend != NULL); @@ -40,7 +41,10 @@ dep = NULL; while (get(pkg, &dep) == EPKG_OK) { - ; /* Do something here */ + package_id = dep_to_package_id(dep); + assert(package_id != NULL); + + g_free(package_id); } ERR(backend, PK_ERROR_ENUM_NOT_SUPPORTED, "soon"); Added: soc2013/mattbw/backend/query/id.c ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ soc2013/mattbw/backend/query/id.c Sun Aug 18 14:44:55 2013 (r256110) @@ -0,0 +1,147 @@ +typedef bool (*id_from_string_ptr) (const char *, struct query_id *); + +/* Converts an array of PackageIDs to query IDs. */ +static struct query_id * +query_id_array_from_package_ids(gchar **package_ids, + unsigned int package_count) +{ + + return query_id_array_from_strings(package_names, package_count, + query_id_from_package_id); +} + +/* Converts an array of names (or name-version strings) to query IDs. */ +static struct query_id * +query_id_array_from_names(gchar **package_names, unsigned int package_count) +{ + + return query_id_array_from_strings(package_names, package_count, + query_id_from_name); +} + +/* + * Converts an array of strings to query IDs using the given transforming + * function. + */ +static struct query_id * +query_id_array_from_strings(char **strings, unsigned int package_count, + id_from_string_ptr id_from_string) +{ + unsigned int i; + gchar *split_package_id; + struct query_id *query_ids; + + assert(strings != NULL); + assert(id_from_string != NULL); + + query_ids = query_id_alloc_array(package_count); + if (query_ids != NULL) { + bool error; + unsigned int i; + + error = false; + for (i = 0; i < package_count; i++) { + error = id_from_string(strings[i], query_ids + i); + if (error) { + free_split_ids(&query_ids, package_count); + break; + } + } + } + + return split_ids; +} + +/* + * Converts a package name or name-version string to a query ID. + * Overwrites the contents of *query_id with said ID. + */ +static bool +query_id_from_name(const char *name, struct query_id *query_id) +{ + + assert(name != NULL); + assert(query_id != NULL); + + query_id->namever = strdup(name); + query_id->arch = NULL; + query_id->repo = NULL; + + return (query_id->namever == NULL); +} + +/* + * Converts a PackageKit package ID to a query ID. + * Overwrites the contents of *query_id with said ID. + */ +bool +query_id_from_package_id(const gchar *package_id, struct query_id *split_id) +{ + bool success; + gchar **split_package_id; + + assert(name != NULL); + assert(split_id != NULL); + + success = false; + + split_package_id = pk_package_id_split(package_id); + if (split_package_id == NULL) { + error = true; + } else { + bool have_name; + bool have_version; + gchar *name; + gchar *version; + + /* We're not allowed to have an empty name or version. */ + + name = split_package_id[PK_PACKAGE_ID_NAME]; + have_name = !(name != NULL && name[0] != '\0'); + + version = split_package_id[PK_PACKAGE_ID_VERSION]; + have_version = (version != NULL && version[0] != '\0'); + + if (have_name && have_version) { + split_ids[1].namever = join_namever(name, version); + } + + if (*split_package_id[PK_PACKAGE_ID_VERSION] == '\0') { + + + g_strfreev(split_package_id); + } + i + o + } + } + + return split_ids; +} + +/* + * Joins a name and version together to create a 'namever'. + * The returned string is newly allocated and must be freed using free(3). + */ +static char * +join_namever(const char *name, const char *version) { + char *namever; + + /* + * The calling code should have checked to make sure the name and + * version are present. + */ + assert(name != NULL); + assert(version != NULL); + + namever = NULL; + asprintf(&namever, "%s-%s", name, version); + +} + +/* Allocates an array of query IDs. */ +static struct query_id * +query_id_alloc_array(unsigned int count) +{ + return calloc(count, sizeof(struct query_id)); +} Modified: soc2013/mattbw/backend/query/match.c ============================================================================== --- soc2013/mattbw/backend/query/match.c Sun Aug 18 13:38:39 2013 (r256109) +++ soc2013/mattbw/backend/query/match.c Sun Aug 18 14:44:55 2013 (r256110) @@ -26,6 +26,8 @@ #include "match.h" /* query_match_... */ + + /* * Runs a query over the PackageIDs selected in the backend that sends the * first match to an emitting function. @@ -44,6 +46,7 @@ t.f = emitter; t.load_flags = load_flags; + t.error_if_not_found = true; return query_do(backend, &s, &t); } From owner-svn-soc-all@FreeBSD.ORG Mon Aug 19 12:00:15 2013 Return-Path: Delivered-To: svn-soc-all@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 86661BEE for ; Mon, 19 Aug 2013 12:00:15 +0000 (UTC) (envelope-from mattbw@FreeBSD.org) Received: from socsvn.freebsd.org (socsvn.freebsd.org [IPv6:2001:1900:2254:206a::50:2]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 722A52957 for ; Mon, 19 Aug 2013 12:00:15 +0000 (UTC) Received: from socsvn.freebsd.org ([127.0.1.124]) by socsvn.freebsd.org (8.14.7/8.14.7) with ESMTP id r7JC0FbY018753 for ; Mon, 19 Aug 2013 12:00:15 GMT (envelope-from mattbw@FreeBSD.org) Received: (from www@localhost) by socsvn.freebsd.org (8.14.7/8.14.6/Submit) id r7JC0Fkg018722 for svn-soc-all@FreeBSD.org; Mon, 19 Aug 2013 12:00:15 GMT (envelope-from mattbw@FreeBSD.org) Date: Mon, 19 Aug 2013 12:00:15 GMT Message-Id: <201308191200.r7JC0Fkg018722@socsvn.freebsd.org> X-Authentication-Warning: socsvn.freebsd.org: www set sender to mattbw@FreeBSD.org using -f From: mattbw@FreeBSD.org To: svn-soc-all@FreeBSD.org Subject: socsvn commit: r256139 - soc2013/mattbw/backend/query MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-soc-all@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: SVN commit messages for the entire Summer of Code repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 Aug 2013 12:00:15 -0000 Author: mattbw Date: Mon Aug 19 12:00:15 2013 New Revision: 256139 URL: http://svnweb.FreeBSD.org/socsvn/?view=rev&rev=256139 Log: (STILL BROKEN) More work on the ID conversion functions for query. I'm experimenting with using ATF/kyua to run tests for the backend, with this part of the backend being the testbed. Currently the tests report that my code segfaults, so the next commit will hopefully fix that. Added: soc2013/mattbw/backend/query/Atffile soc2013/mattbw/backend/query/id.h soc2013/mattbw/backend/query/id_test.c Modified: soc2013/mattbw/backend/query/id.c Added: soc2013/mattbw/backend/query/Atffile ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ soc2013/mattbw/backend/query/Atffile Mon Aug 19 12:00:15 2013 (r256139) @@ -0,0 +1,5 @@ +Content-Type: application/X-atf-atffile; version="1" + +prop: test-suite = "query" + +tp: id_test Modified: soc2013/mattbw/backend/query/id.c ============================================================================== --- soc2013/mattbw/backend/query/id.c Mon Aug 19 11:16:53 2013 (r256138) +++ soc2013/mattbw/backend/query/id.c Mon Aug 19 12:00:15 2013 (r256139) @@ -1,17 +1,52 @@ +/*- + * Copyright (C) 2013 Matt Windsor + * + * Licensed under the GNU General Public License Version 2 + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA. + */ +#include /* assert */ +#include /* g... */ +#include /* calloc */ +#include /* strdup */ +#include "../pk-backend.h" /* pk_... */ + +#include "id.h" /* query_id_... */ + + typedef bool (*id_from_string_ptr) (const char *, struct query_id *); +static char *join_namever(const char *name, const char *version); +static char *null_if_empty(char *in); +static struct query_id *query_id_array_alloc(unsigned int count); +static struct query_id *query_id_array_from_strings(char **strings, unsigned int package_count, id_from_string_ptr id_from_string); +static void query_id_array_free(struct query_id **query_ids_p, unsigned int count); +static void query_id_free_contents(struct query_id *query_id); + /* Converts an array of PackageIDs to query IDs. */ -static struct query_id * +struct query_id * query_id_array_from_package_ids(gchar **package_ids, unsigned int package_count) { - return query_id_array_from_strings(package_names, package_count, + return query_id_array_from_strings(package_ids, package_count, query_id_from_package_id); } /* Converts an array of names (or name-version strings) to query IDs. */ -static struct query_id * +struct query_id * query_id_array_from_names(gchar **package_names, unsigned int package_count) { @@ -27,14 +62,12 @@ query_id_array_from_strings(char **strings, unsigned int package_count, id_from_string_ptr id_from_string) { - unsigned int i; - gchar *split_package_id; struct query_id *query_ids; assert(strings != NULL); assert(id_from_string != NULL); - query_ids = query_id_alloc_array(package_count); + query_ids = query_id_array_alloc(package_count); if (query_ids != NULL) { bool error; unsigned int i; @@ -43,20 +76,24 @@ for (i = 0; i < package_count; i++) { error = id_from_string(strings[i], query_ids + i); if (error) { - free_split_ids(&query_ids, package_count); + query_id_array_free(&query_ids, package_count); break; } } + + if (error) { + query_id_array_free(&query_ids, package_count); + } } - return split_ids; + return query_ids; } /* * Converts a package name or name-version string to a query ID. * Overwrites the contents of *query_id with said ID. */ -static bool +bool query_id_from_name(const char *name, struct query_id *query_id) { @@ -75,24 +112,24 @@ * Overwrites the contents of *query_id with said ID. */ bool -query_id_from_package_id(const gchar *package_id, struct query_id *split_id) +query_id_from_package_id(const gchar *package_id, struct query_id *query_id) { bool success; gchar **split_package_id; - assert(name != NULL); - assert(split_id != NULL); + assert(package_id != NULL); + assert(query_id != NULL); success = false; split_package_id = pk_package_id_split(package_id); - if (split_package_id == NULL) { - error = true; - } else { + if (split_package_id != NULL) { bool have_name; bool have_version; gchar *name; gchar *version; + gchar *arch; + gchar *repo; /* We're not allowed to have an empty name or version. */ @@ -102,21 +139,26 @@ version = split_package_id[PK_PACKAGE_ID_VERSION]; have_version = (version != NULL && version[0] != '\0'); + /* + * Names and versions are mandatory. + * Anything else is optional. + */ if (have_name && have_version) { - split_ids[1].namever = join_namever(name, version); - } + query_id->namever = join_namever(name, version); - if (*split_package_id[PK_PACKAGE_ID_VERSION] == '\0') { + arch = split_package_id[PK_PACKAGE_ID_ARCH]; + query_id->arch = null_if_empty(arch); + repo = split_package_id[PK_PACKAGE_ID_DATA]; + query_id->repo = null_if_empty(repo); - g_strfreev(split_package_id); - } - i - o + success = true; } + + g_strfreev(split_package_id); } - return split_ids; + return success; } /* @@ -135,13 +177,51 @@ assert(version != NULL); namever = NULL; - asprintf(&namever, "%s-%s", name, version); + (void)asprintf(&namever, "%s-%s", name, version); + return namever; } /* Allocates an array of query IDs. */ static struct query_id * -query_id_alloc_array(unsigned int count) +query_id_array_alloc(unsigned int count) { + return calloc(count, sizeof(struct query_id)); } + +/* + * Returns NULL if the given string is NULL or empty, and the string otherwise. + */ +static char * +null_if_empty(char *in) +{ + + return (in == NULL || in[0] == '\0' ? NULL : in); +} + +static void +query_id_array_free(struct query_id **query_ids_p, unsigned int count) +{ + unsigned int i; + + assert(query_ids_p != NULL); + assert(*query_ids_p != NULL); + + for (i = 0; i < count; i++) { + query_id_free_contents((*query_ids_p) + i); + } + + free(*query_ids_p); + *query_ids_p = NULL; +} + +static void +query_id_free_contents(struct query_id *query_id) +{ + + assert(query_id != NULL); + free(query_id->namever); + free(query_id->arch); + free(query_id->repo); +} Added: soc2013/mattbw/backend/query/id.h ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ soc2013/mattbw/backend/query/id.h Mon Aug 19 12:00:15 2013 (r256139) @@ -0,0 +1,39 @@ +/*- + * Copyright (C) 2013 Matt Windsor + * + * Licensed under the GNU General Public License Version 2 + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA. + */ + +#ifndef _PKGNG_BACKEND_QUERY_ID_H_ +#define _PKGNG_BACKEND_QUERY_ID_H_ + +#include +#include "../pk-backend.h" +#include "pkg.h" + +struct query_id { + char *namever; + char *arch; + char *repo; +}; + +bool query_id_from_name(const char *name, struct query_id *query_id); +bool query_id_from_package_id(const gchar *package_id, struct query_id *query_id); +struct query_id *query_id_array_from_names(gchar **package_names, unsigned int package_count); +struct query_id *query_id_array_from_package_ids(gchar **package_ids, unsigned int package_count); + +#endif /* !_PKGNG_BACKEND_QUERY_ID_H_ */ Added: soc2013/mattbw/backend/query/id_test.c ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ soc2013/mattbw/backend/query/id_test.c Mon Aug 19 12:00:15 2013 (r256139) @@ -0,0 +1,116 @@ +/*- + * Copyright (C) 2013 Matt Windsor + * + * Licensed under the GNU General Public License Version 2 + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA. + */ +#include +#include /* free */ +#include /* strcmp */ + +#include "id.h" /* query_id_... */ + + +/* ATF/kyua tests for 'id.c'. */ + +ATF_TC(query_id_from_package_id_single_valid); +ATF_TC_HEAD(query_id_from_package_id_single_valid, tc) +{ + + atf_tc_set_md_var(tc, "descr", + "Ensure single PackageIDs properly convert to query IDs."); +} +ATF_TC_BODY(query_id_from_package_id_single_valid, tc) +{ + bool success; + struct query_id result; + const char *id = "pkg;1.1.4;freebsd:9:x86:32;packagesite"; + + result.namever = NULL; + result.arch = NULL; + result.repo = NULL; + + success = query_id_from_package_id(id, &result); + ATF_CHECK(success); + + ATF_CHECK_STREQ(result.namever, "pkg-1.1.4"); + ATF_CHECK_STREQ(result.arch, "freebsd:9:x86:32"); + ATF_CHECK_STREQ(result.repo, "packagesite"); + + free(result.namever); + free(result.arch); + free(result.repo); +} + +ATF_TC(query_id_from_package_id_single_no_name); +ATF_TC_HEAD(query_id_from_package_id_single_no_name, tc) +{ + + atf_tc_set_md_var(tc, "descr", + "Ensure PackageIDs with no name fail to convert to query IDs."); +} +ATF_TC_BODY(query_id_from_package_id_single_no_name, tc) +{ + bool success; + struct query_id result; + const char *no_name = ";1.1.4;freebsd:9:x86:32;packagesite"; + + result.namever = NULL; + result.arch = NULL; + result.repo = NULL; + + success = query_id_from_package_id(no_name, &result); + ATF_CHECK(!success); + + ATF_CHECK_EQ(result.namever, NULL); + ATF_CHECK_EQ(result.arch, NULL); + ATF_CHECK_EQ(result.repo, NULL); +} + +ATF_TC(query_id_from_package_id_single_no_version); +ATF_TC_HEAD(query_id_from_package_id_single_no_version, tc) +{ + + atf_tc_set_md_var(tc, "descr", + "Ensure PackageIDs with no version fail to convert to query IDs."); +} +ATF_TC_BODY(query_id_from_package_id_single_no_version, tc) +{ + bool success; + struct query_id result; + const char *no_version = "pkg;;freebsd:9:x86:32;packagesite"; + + result.namever = NULL; + result.arch = NULL; + result.repo = NULL; + + success = query_id_from_package_id(no_version, &result); + ATF_CHECK(!success); + + ATF_CHECK_EQ(result.namever, NULL); + ATF_CHECK_EQ(result.arch, NULL); + ATF_CHECK_EQ(result.repo, NULL); +} + +ATF_TP_ADD_TCS(tp) +{ + + ATF_TP_ADD_TC(tp, query_id_from_package_id_single_valid); + ATF_TP_ADD_TC(tp, query_id_from_package_id_single_no_name); + ATF_TP_ADD_TC(tp, query_id_from_package_id_single_no_version); + + return atf_no_error(); +} From owner-svn-soc-all@FreeBSD.ORG Mon Aug 19 12:08:30 2013 Return-Path: Delivered-To: svn-soc-all@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 6DA5ED96 for ; Mon, 19 Aug 2013 12:08:30 +0000 (UTC) (envelope-from mattbw@FreeBSD.org) Received: from socsvn.freebsd.org (socsvn.freebsd.org [IPv6:2001:1900:2254:206a::50:2]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 406002A16 for ; Mon, 19 Aug 2013 12:08:30 +0000 (UTC) Received: from socsvn.freebsd.org ([127.0.1.124]) by socsvn.freebsd.org (8.14.7/8.14.7) with ESMTP id r7JC8UAD084572 for ; Mon, 19 Aug 2013 12:08:30 GMT (envelope-from mattbw@FreeBSD.org) Received: (from www@localhost) by socsvn.freebsd.org (8.14.7/8.14.6/Submit) id r7JC8UtC084557 for svn-soc-all@FreeBSD.org; Mon, 19 Aug 2013 12:08:30 GMT (envelope-from mattbw@FreeBSD.org) Date: Mon, 19 Aug 2013 12:08:30 GMT Message-Id: <201308191208.r7JC8UtC084557@socsvn.freebsd.org> X-Authentication-Warning: socsvn.freebsd.org: www set sender to mattbw@FreeBSD.org using -f From: mattbw@FreeBSD.org To: svn-soc-all@FreeBSD.org Subject: socsvn commit: r256140 - soc2013/mattbw/backend/query MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-soc-all@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: SVN commit messages for the entire Summer of Code repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 Aug 2013 12:08:30 -0000 Author: mattbw Date: Mon Aug 19 12:08:29 2013 New Revision: 256140 URL: http://svnweb.FreeBSD.org/socsvn/?view=rev&rev=256140 Log: Fixed test failure. The code for checking whether or not there is a valid name was incorrectly inverted, so the valid IDs were being rejected, the query ID was full of nulls, and the test code then tried to strcmp them. Also strdup the arch and repo, as we free the split package ID later. Modified: soc2013/mattbw/backend/query/id.c Modified: soc2013/mattbw/backend/query/id.c ============================================================================== --- soc2013/mattbw/backend/query/id.c Mon Aug 19 12:00:15 2013 (r256139) +++ soc2013/mattbw/backend/query/id.c Mon Aug 19 12:08:29 2013 (r256140) @@ -29,7 +29,7 @@ typedef bool (*id_from_string_ptr) (const char *, struct query_id *); static char *join_namever(const char *name, const char *version); -static char *null_if_empty(char *in); +static char *strdup_null_if_empty(char *in); static struct query_id *query_id_array_alloc(unsigned int count); static struct query_id *query_id_array_from_strings(char **strings, unsigned int package_count, id_from_string_ptr id_from_string); static void query_id_array_free(struct query_id **query_ids_p, unsigned int count); @@ -134,7 +134,7 @@ /* We're not allowed to have an empty name or version. */ name = split_package_id[PK_PACKAGE_ID_NAME]; - have_name = !(name != NULL && name[0] != '\0'); + have_name = (name != NULL && name[0] != '\0'); version = split_package_id[PK_PACKAGE_ID_VERSION]; have_version = (version != NULL && version[0] != '\0'); @@ -147,10 +147,10 @@ query_id->namever = join_namever(name, version); arch = split_package_id[PK_PACKAGE_ID_ARCH]; - query_id->arch = null_if_empty(arch); + query_id->arch = strdup_null_if_empty(arch); repo = split_package_id[PK_PACKAGE_ID_DATA]; - query_id->repo = null_if_empty(repo); + query_id->repo = strdup_null_if_empty(repo); success = true; } @@ -191,13 +191,14 @@ } /* - * Returns NULL if the given string is NULL or empty, and the string otherwise. + * Returns NULL if the given string is NULL or empty, and a duplicate of the + * string otherwise. */ static char * -null_if_empty(char *in) +strdup_null_if_empty(char *in) { - return (in == NULL || in[0] == '\0' ? NULL : in); + return (in == NULL || in[0] == '\0') ? NULL : strdup(in); } static void From owner-svn-soc-all@FreeBSD.ORG Mon Aug 19 13:50:00 2013 Return-Path: Delivered-To: svn-soc-all@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 48381877 for ; Mon, 19 Aug 2013 13:50:00 +0000 (UTC) (envelope-from mattbw@FreeBSD.org) Received: from socsvn.freebsd.org (socsvn.freebsd.org [IPv6:2001:1900:2254:206a::50:2]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 24D932FA6 for ; Mon, 19 Aug 2013 13:50:00 +0000 (UTC) Received: from socsvn.freebsd.org ([127.0.1.124]) by socsvn.freebsd.org (8.14.7/8.14.7) with ESMTP id r7JDo0DN006649 for ; Mon, 19 Aug 2013 13:50:00 GMT (envelope-from mattbw@FreeBSD.org) Received: (from www@localhost) by socsvn.freebsd.org (8.14.7/8.14.6/Submit) id r7JDnx64006640 for svn-soc-all@FreeBSD.org; Mon, 19 Aug 2013 13:49:59 GMT (envelope-from mattbw@FreeBSD.org) Date: Mon, 19 Aug 2013 13:49:59 GMT Message-Id: <201308191349.r7JDnx64006640@socsvn.freebsd.org> X-Authentication-Warning: socsvn.freebsd.org: www set sender to mattbw@FreeBSD.org using -f From: mattbw@FreeBSD.org To: svn-soc-all@FreeBSD.org Subject: socsvn commit: r256143 - soc2013/mattbw/backend/query MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-soc-all@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: SVN commit messages for the entire Summer of Code repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 Aug 2013 13:50:00 -0000 Author: mattbw Date: Mon Aug 19 13:49:59 2013 New Revision: 256143 URL: http://svnweb.FreeBSD.org/socsvn/?view=rev&rev=256143 Log: Added more tests and fixed a logical error. Query IDs from names would fail, due to the return value of 'query_id_from_name' being inverted. In addition, there was no testing to ensure that empty names would be assigned to NULL. Modified: soc2013/mattbw/backend/query/id.c soc2013/mattbw/backend/query/id_test.c Modified: soc2013/mattbw/backend/query/id.c ============================================================================== --- soc2013/mattbw/backend/query/id.c Mon Aug 19 12:37:13 2013 (r256142) +++ soc2013/mattbw/backend/query/id.c Mon Aug 19 13:49:59 2013 (r256143) @@ -29,7 +29,7 @@ typedef bool (*id_from_string_ptr) (const char *, struct query_id *); static char *join_namever(const char *name, const char *version); -static char *strdup_null_if_empty(char *in); +static char *strdup_null_if_empty(const char *in); static struct query_id *query_id_array_alloc(unsigned int count); static struct query_id *query_id_array_from_strings(char **strings, unsigned int package_count, id_from_string_ptr id_from_string); static void query_id_array_free(struct query_id **query_ids_p, unsigned int count); @@ -100,11 +100,11 @@ assert(name != NULL); assert(query_id != NULL); - query_id->namever = strdup(name); + query_id->namever = strdup_null_if_empty(name); query_id->arch = NULL; query_id->repo = NULL; - return (query_id->namever == NULL); + return (query_id->namever != NULL); } /* @@ -195,7 +195,7 @@ * string otherwise. */ static char * -strdup_null_if_empty(char *in) +strdup_null_if_empty(const char *in) { return (in == NULL || in[0] == '\0') ? NULL : strdup(in); Modified: soc2013/mattbw/backend/query/id_test.c ============================================================================== --- soc2013/mattbw/backend/query/id_test.c Mon Aug 19 12:37:13 2013 (r256142) +++ soc2013/mattbw/backend/query/id_test.c Mon Aug 19 13:49:59 2013 (r256143) @@ -26,6 +26,10 @@ /* ATF/kyua tests for 'id.c'. */ +/* + * SINGLE PACKAGE IDS + */ + ATF_TC(query_id_from_package_id_single_valid); ATF_TC_HEAD(query_id_from_package_id_single_valid, tc) { @@ -105,12 +109,121 @@ ATF_CHECK_EQ(result.repo, NULL); } +ATF_TC(query_id_from_package_id_single_no_repo); +ATF_TC_HEAD(query_id_from_package_id_single_no_repo, tc) +{ + + atf_tc_set_md_var(tc, "descr", + "Ensure PackageIDs with no repo properly convert to query IDs."); +} +ATF_TC_BODY(query_id_from_package_id_single_no_repo, tc) +{ + bool success; + struct query_id result; + const char *no_version = "pkg;1.1.4;freebsd:9:x86:32;"; + + result.namever = NULL; + result.arch = NULL; + result.repo = NULL; + + success = query_id_from_package_id(no_version, &result); + ATF_CHECK(success); + + ATF_CHECK_STREQ(result.namever, "pkg-1.1.4"); + ATF_CHECK_STREQ(result.arch, "freebsd:9:x86:32"); + ATF_CHECK_EQ(result.repo, NULL); +} + +ATF_TC(query_id_from_package_id_single_no_arch); +ATF_TC_HEAD(query_id_from_package_id_single_no_arch, tc) +{ + + atf_tc_set_md_var(tc, "descr", + "Ensure PackageIDs with no arch properly convert to query IDs."); +} +ATF_TC_BODY(query_id_from_package_id_single_no_arch, tc) +{ + bool success; + struct query_id result; + const char *no_version = "pkg;1.1.4;;packagesite"; + + result.namever = NULL; + result.arch = NULL; + result.repo = NULL; + + success = query_id_from_package_id(no_version, &result); + ATF_CHECK(success); + + ATF_CHECK_STREQ(result.namever, "pkg-1.1.4"); + ATF_CHECK_EQ(result.arch, NULL); + ATF_CHECK_STREQ(result.repo, "packagesite"); +} + +ATF_TC(query_id_from_name_single_valid); +ATF_TC_HEAD(query_id_from_name_single_valid, tc) +{ + + atf_tc_set_md_var(tc, "descr", + "Ensure valid package names properly convert to query IDs."); +} +ATF_TC_BODY(query_id_from_name_single_valid, tc) +{ + bool success; + struct query_id result; + const char *valid_name = "pkg-1.1.4"; + + result.namever = NULL; + result.arch = NULL; + result.repo = NULL; + + success = query_id_from_name(valid_name, &result); + ATF_CHECK(success); + + ATF_CHECK_STREQ(result.namever, "pkg-1.1.4"); + ATF_CHECK_EQ(result.arch, NULL); + ATF_CHECK_EQ(result.repo, NULL); +} + +ATF_TC(query_id_from_name_single_empty); +ATF_TC_HEAD(query_id_from_name_single_empty, tc) +{ + + atf_tc_set_md_var(tc, "descr", + "Ensure empty package names fail to convert to query IDs."); +} +ATF_TC_BODY(query_id_from_name_single_empty, tc) +{ + bool success; + struct query_id result; + const char *empty_name = ""; + + result.namever = NULL; + result.arch = NULL; + result.repo = NULL; + + success = query_id_from_name(empty_name, &result); + ATF_CHECK(!success); + + ATF_CHECK_EQ(result.namever, NULL); + ATF_CHECK_EQ(result.arch, NULL); + ATF_CHECK_EQ(result.repo, NULL); +} + +/* + * SINGLE NAMES + */ + ATF_TP_ADD_TCS(tp) { ATF_TP_ADD_TC(tp, query_id_from_package_id_single_valid); ATF_TP_ADD_TC(tp, query_id_from_package_id_single_no_name); ATF_TP_ADD_TC(tp, query_id_from_package_id_single_no_version); + ATF_TP_ADD_TC(tp, query_id_from_package_id_single_no_repo); + ATF_TP_ADD_TC(tp, query_id_from_package_id_single_no_arch); + + ATF_TP_ADD_TC(tp, query_id_from_name_single_valid); + ATF_TP_ADD_TC(tp, query_id_from_name_single_empty); return atf_no_error(); } From owner-svn-soc-all@FreeBSD.ORG Mon Aug 19 16:40:11 2013 Return-Path: Delivered-To: svn-soc-all@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 2F92F20E for ; Mon, 19 Aug 2013 16:40:11 +0000 (UTC) (envelope-from dpl@FreeBSD.org) Received: from socsvn.freebsd.org (socsvn.freebsd.org [IPv6:2001:1900:2254:206a::50:2]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 1BAF629BD for ; Mon, 19 Aug 2013 16:40:11 +0000 (UTC) Received: from socsvn.freebsd.org ([127.0.1.124]) by socsvn.freebsd.org (8.14.7/8.14.7) with ESMTP id r7JGeBkb043689 for ; Mon, 19 Aug 2013 16:40:11 GMT (envelope-from dpl@FreeBSD.org) Received: (from www@localhost) by socsvn.freebsd.org (8.14.7/8.14.6/Submit) id r7JGeANP043650 for svn-soc-all@FreeBSD.org; Mon, 19 Aug 2013 16:40:10 GMT (envelope-from dpl@FreeBSD.org) Date: Mon, 19 Aug 2013 16:40:10 GMT Message-Id: <201308191640.r7JGeANP043650@socsvn.freebsd.org> X-Authentication-Warning: socsvn.freebsd.org: www set sender to dpl@FreeBSD.org using -f From: dpl@FreeBSD.org To: svn-soc-all@FreeBSD.org Subject: socsvn commit: r256157 - in soc2013/dpl/head/lib/libzcap: . test zlibworker MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-soc-all@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: SVN commit messages for the entire Summer of Code repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 Aug 2013 16:40:11 -0000 Author: dpl Date: Mon Aug 19 16:40:10 2013 New Revision: 256157 URL: http://svnweb.FreeBSD.org/socsvn/?view=rev&rev=256157 Log: Updated version. Things start to work. Added: soc2013/dpl/head/lib/libzcap/test/zcaplibtest.c Deleted: soc2013/dpl/head/lib/libzcap/test/functionstest.c soc2013/dpl/head/lib/libzcap/zlib.3 Modified: soc2013/dpl/head/lib/libzcap/capsicum.c soc2013/dpl/head/lib/libzcap/commands.c soc2013/dpl/head/lib/libzcap/deflate.c soc2013/dpl/head/lib/libzcap/gzlib.c soc2013/dpl/head/lib/libzcap/inflate.c soc2013/dpl/head/lib/libzcap/zlibworker/commands.c soc2013/dpl/head/lib/libzcap/zlibworker/zlibworker.c Modified: soc2013/dpl/head/lib/libzcap/capsicum.c ============================================================================== --- soc2013/dpl/head/lib/libzcap/capsicum.c Mon Aug 19 15:58:39 2013 (r256156) +++ soc2013/dpl/head/lib/libzcap/capsicum.c Mon Aug 19 16:40:10 2013 (r256157) @@ -12,7 +12,7 @@ #define SUM 0 -pid_t pid = 0; +pid_t pid = 0; int sv[2]; int startChild(void); @@ -23,28 +23,32 @@ startChild(void) { if (socketpair(PF_LOCAL, SOCK_STREAM, 0, sv) < 0 ) - perror("socketpair"); + perror("zcaplib: socketpair"); if( (pid = fork()) == 0 ){ - close(STDIN_FILENO); + cap_rights_limit(STDIN_FILENO, CAP_WRITE); cap_rights_limit(STDOUT_FILENO, CAP_READ); cap_rights_limit(STDERR_FILENO, CAP_WRITE); - dup(sv[1]); /* 3 is now the socket */ close(sv[0]); - close(sv[1]); + dup2(sv[1] , 3); /* 3 is now the socket */ + close(4); - /* open and fexec() listener */ + /* open and execl() listener */ if (access("/usr/libexec/zlibworker", X_OK) < 0) { - fprintf(stderr, "zlibworker doesn't exist, or can't be executed.\n"); - fprintf(stderr, "You should compile and install it.\n"); + perror("zcaplib: zlibworker:"); exit (-1); } - execl("/usr/libexec/zlibworker", ""); + if ( execl("/usr/libexec/zlibworker", "zlibworker", NULL) <0 ) { + perror("zcaplib: zlibworker:"); + exit (-1); + } exit(0); - } else + } else { + close(sv[1]); atexit(killChild); + } return pid; } @@ -53,21 +57,19 @@ sendCommand( nvlist_t *nvl ) { if( nvlist_send(sv[0], nvl) != 0 ) { - perror("nvlist_send()"); + perror("zcaplib: nvlist_send()"); return NULL; } - nvlist_destroy(nvl); /* recv results */ - if ((nvl = nvlist_recv(sv[0])) != NULL) { - perror("nvlist_send()"); + if ((nvl = nvlist_recv(sv[0])) == NULL) { + perror("zcaplib: nvlist_recv()"); return NULL; } - printf("done sending/recving\n"); return (nvl); } -void killChild(void){ +void killChild(void) { kill(pid, SIGKILL); } \ No newline at end of file Modified: soc2013/dpl/head/lib/libzcap/commands.c ============================================================================== --- soc2013/dpl/head/lib/libzcap/commands.c Mon Aug 19 15:58:39 2013 (r256156) +++ soc2013/dpl/head/lib/libzcap/commands.c Mon Aug 19 16:40:10 2013 (r256157) @@ -113,7 +113,7 @@ if( (args = nvlist_create(0)) == NULL || (nvl = nvlist_create(0)) == NULL ) { - perror("nvlist_create"); + perror("zcaplib: nvlist_create"); return(0); } nvlist_add_number(nvl, "command", ZCAPCMD_DEFLATEINIT); @@ -154,7 +154,7 @@ if( (args = nvlist_create(0)) == NULL || (nvl = nvlist_create(0)) == NULL ) { - perror("nvlist_create"); + perror("zcaplib: nvlist_create"); return(0); } nvlist_add_number(nvl, "command", ZCAPCMD_DEFLATEINIT); @@ -181,7 +181,7 @@ if( (args = nvlist_create(0)) == NULL || (nvl = nvlist_create(0)) == NULL ) { - perror("nvlist_create"); + perror("zcaplib: nvlist_create"); return(0); } nvlist_add_number(nvl, "command", ZCAPCMD_DEFLATEINIT); @@ -208,7 +208,7 @@ if( (args = nvlist_create(0)) == NULL || (nvl = nvlist_create(0)) == NULL ) { - perror("nvlist_create"); + perror("zcaplib: nvlist_create"); return(0); } nvlist_add_number(nvl, "command", ZCAPCMD_DEFLATEINIT); @@ -239,7 +239,7 @@ if( (args = nvlist_create(0)) == NULL || (nvl = nvlist_create(0)) == NULL ) { - perror("nvlist_create"); + perror("zcaplib: nvlist_create"); return(0); } nvlist_add_number(nvl, "command", ZCAPCMD_DEFLATEINIT); @@ -265,7 +265,7 @@ if( (args = nvlist_create(0)) == NULL || (nvl = nvlist_create(0)) == NULL ) { - perror("nvlist_create"); + perror("zcaplib: nvlist_create"); return(0); } nvlist_add_number(nvl, "command", ZCAPCMD_DEFLATEINIT); @@ -293,7 +293,7 @@ if( (args = nvlist_create(0)) == NULL || (nvl = nvlist_create(0)) == NULL ) { - perror("nvlist_create"); + perror("zcaplib: nvlist_create"); return(0); } nvlist_add_number(nvl, "command", ZCAPCMD_DEFLATESETDICTIONARY); @@ -319,7 +319,7 @@ if( (args = nvlist_create(0)) == NULL || (nvl = nvlist_create(0)) == NULL ) { - perror("nvlist_create"); + perror("zcaplib: nvlist_create"); return(0); } nvlist_add_number(nvl, "command", ZCAPCMD_DEFLATECOPY); @@ -346,7 +346,7 @@ if( (args = nvlist_create(0)) == NULL || (nvl = nvlist_create(0)) == NULL ) { - perror("nvlist_create"); + perror("zcaplib: nvlist_create"); return(0); } nvlist_add_number(nvl, "command", ZCAPCMD_DEFLATERESET); @@ -374,7 +374,7 @@ if( (args = nvlist_create(0)) == NULL || (nvl = nvlist_create(0)) == NULL ) { - perror("nvlist_create"); + perror("zcaplib: nvlist_create"); return(0); } nvlist_add_number(nvl, "command", ZCAPCMD_DEFLATEPARAMS); @@ -403,7 +403,7 @@ if( (args = nvlist_create(0)) == NULL || (nvl = nvlist_create(0)) == NULL ) { - perror("nvlist_create"); + perror("zcaplib: nvlist_create"); return(0); } nvlist_add_number(nvl, "command", ZCAPCMD_DEFLATETUNE); @@ -432,7 +432,7 @@ if( (args = nvlist_create(0)) == NULL || (nvl = nvlist_create(0)) == NULL ) { - perror("nvlist_create"); + perror("zcaplib: nvlist_create"); return(0); } nvlist_add_number(nvl, "command", ZCAPCMD_DEFLATEBOUND); @@ -458,7 +458,7 @@ if( (args = nvlist_create(0)) == NULL || (nvl = nvlist_create(0)) == NULL ) { - perror("nvlist_create"); + perror("zcaplib: nvlist_create"); return(0); } nvlist_add_number(nvl, "command", ZCAPCMD_DEFLATEPENDING); @@ -486,7 +486,7 @@ if( (args = nvlist_create(0)) == NULL || (nvl = nvlist_create(0)) == NULL ) { - perror("nvlist_create"); + perror("zcaplib: nvlist_create"); return(0); } nvlist_add_number(nvl, "command", ZCAPCMD_DEFLATEPRIME); @@ -515,7 +515,7 @@ if( (args = nvlist_create(0)) == NULL || (nvl = nvlist_create(0)) == NULL ) { - perror("nvlist_create"); + perror("zcaplib: nvlist_create"); return(0); } nvlist_add_number(nvl, "command", ZCAPCMD_DEFLATESETHEADER); @@ -544,7 +544,7 @@ if( (args = nvlist_create(0)) == NULL || (nvl = nvlist_create(0)) == NULL ) { - perror("nvlist_create"); + perror("zcaplib: nvlist_create"); return(0); } nvlist_add_number(nvl, "command", ZCAPCMD_INFLATESETDICTIONARY); @@ -570,7 +570,7 @@ if( (args = nvlist_create(0)) == NULL || (nvl = nvlist_create(0)) == NULL ) { - perror("nvlist_create"); + perror("zcaplib: nvlist_create"); return(0); } nvlist_add_number(nvl, "command", ZCAPCMD_INFLATEGETDICTIONARY); @@ -595,7 +595,7 @@ if( (args = nvlist_create(0)) == NULL || (nvl = nvlist_create(0)) == NULL ) { - perror("nvlist_create"); + perror("zcaplib: nvlist_create"); return(0); } nvlist_add_number(nvl, "command", ZCAPCMD_INFLATESYNC); @@ -620,7 +620,7 @@ if( (args = nvlist_create(0)) == NULL || (nvl = nvlist_create(0)) == NULL ) { - perror("nvlist_create"); + perror("zcaplib: nvlist_create"); return(0); } nvlist_add_number(nvl, "command", ZCAPCMD_INFLATECOPY); @@ -647,7 +647,7 @@ if( (args = nvlist_create(0)) == NULL || (nvl = nvlist_create(0)) == NULL ) { - perror("nvlist_create"); + perror("zcaplib: nvlist_create"); return(0); } nvlist_add_number(nvl, "command", ZCAPCMD_INFLATERESET); @@ -674,7 +674,7 @@ if( (args = nvlist_create(0)) == NULL || (nvl = nvlist_create(0)) == NULL ) { - perror("nvlist_create"); + perror("zcaplib: nvlist_create"); return(0); } nvlist_add_number(nvl, "command", ZCAPCMD_INFLATERESET2); @@ -702,7 +702,7 @@ if( (args = nvlist_create(0)) == NULL || (nvl = nvlist_create(0)) == NULL ) { - perror("nvlist_create"); + perror("zcaplib: nvlist_create"); return(0); } nvlist_add_number(nvl, "command", ZCAPCMD_INFLATEPRIME); @@ -729,7 +729,7 @@ if( (args = nvlist_create(0)) == NULL || (nvl = nvlist_create(0)) == NULL ) { - perror("nvlist_create"); + perror("zcaplib: nvlist_create"); return(0); } nvlist_add_number(nvl, "command", ZCAPCMD_INFLATEMARK); @@ -755,7 +755,7 @@ if( (args = nvlist_create(0)) == NULL || (nvl = nvlist_create(0)) == NULL ) { - perror("nvlist_create"); + perror("zcaplib: nvlist_create"); return(0); } nvlist_add_number(nvl, "command", ZCAPCMD_INFLATEGETHEADER); @@ -785,7 +785,7 @@ if( (args = nvlist_create(0)) == NULL || (nvl = nvlist_create(0)) == NULL ) { - perror("nvlist_create"); + perror("zcaplib: nvlist_create"); return(0); } nvlist_add_number(nvl, "command", ZCAPCMD_INFLATEBACKINIT); @@ -815,7 +815,7 @@ if( (args = nvlist_create(0)) == NULL || (nvl = nvlist_create(0)) == NULL ) { - perror("nvlist_create"); + perror("zcaplib: nvlist_create"); return(0); } nvlist_add_number(nvl, "command", ZCAPCMD_INFLATEBACK); @@ -840,7 +840,7 @@ if( (args = nvlist_create(0)) == NULL || (nvl = nvlist_create(0)) == NULL ) { - perror("nvlist_create"); + perror("zcaplib: nvlist_create"); return(0); } nvlist_add_number(nvl, "command", ZCAPCMD_INFLATEBACKEND); @@ -865,12 +865,13 @@ if( (args = nvlist_create(0)) == NULL || (nvl = nvlist_create(0)) == NULL ) { - perror("nvlist_create"); + perror("zcaplib: nvlist_create"); return(0); } nvlist_add_number(nvl, "command", ZCAPCMD_ZLIBCOMPILEFLAGS); result = sendCommand(nvl); + printf("after sendCommand()\n"); ret = nvlist_take_number(result, "result"); destroy(); @@ -887,7 +888,7 @@ if( (args = nvlist_create(0)) == NULL || (nvl = nvlist_create(0)) == NULL ) { - perror("nvlist_create"); + perror("zcaplib: nvlist_create"); return(0); } nvlist_add_number(nvl, "command", ZCAPCMD_COMPRESSBOUND); @@ -910,7 +911,7 @@ if( (args = nvlist_create(0)) == NULL || (nvl = nvlist_create(0)) == NULL ) { - perror("nvlist_create"); + perror("zcaplib: nvlist_create"); return (NULL); } nvlist_add_number(nvl, "command", ZCAPCMD_GZOPEN); @@ -937,7 +938,7 @@ if( (args = nvlist_create(0)) == NULL || (nvl = nvlist_create(0)) == NULL ) { - perror("nvlist_create"); + perror("zcaplib: nvlist_create"); return(0); } nvlist_add_number(nvl, "command", ZCAPCMD_GZBUFFER); @@ -960,7 +961,7 @@ if( (args = nvlist_create(0)) == NULL || (nvl = nvlist_create(0)) == NULL ) { - perror("nvlist_create"); + perror("zcaplib: nvlist_create"); return(0); } nvlist_add_number(nvl, "command", ZCAPCMD_GZSETPARAMS); @@ -985,7 +986,7 @@ if( (args = nvlist_create(0)) == NULL || (nvl = nvlist_create(0)) == NULL ) { - perror("nvlist_create"); + perror("zcaplib: nvlist_create"); return(0); } nvlist_add_number(nvl, "command", ZCAPCMD_GZREAD); @@ -1009,7 +1010,7 @@ if( (args = nvlist_create(0)) == NULL || (nvl = nvlist_create(0)) == NULL ) { - perror("nvlist_create"); + perror("zcaplib: nvlist_create"); return(0); } nvlist_add_number(nvl, "command", ZCAPCMD_GZWRITE); @@ -1032,7 +1033,7 @@ if( (args = nvlist_create(0)) == NULL || (nvl = nvlist_create(0)) == NULL ) { - perror("nvlist_create"); + perror("zcaplib: nvlist_create"); return(0); } nvlist_add_number(nvl, "command", ZCAPCMD_GZPRINTF); @@ -1055,7 +1056,7 @@ if( (args = nvlist_create(0)) == NULL || (nvl = nvlist_create(0)) == NULL ) { - perror("nvlist_create"); + perror("zcaplib: nvlist_create"); return(0); } nvlist_add_number(nvl, "command", ZCAPCMD_GZPUTS); @@ -1078,7 +1079,7 @@ if( (args = nvlist_create(0)) == NULL || (nvl = nvlist_create(0)) == NULL ) { - perror("nvlist_create"); + perror("zcaplib: nvlist_create"); return (NULL); } nvlist_add_number(nvl, "command", ZCAPCMD_GZGETS); @@ -1105,7 +1106,7 @@ if( (args = nvlist_create(0)) == NULL || (nvl = nvlist_create(0)) == NULL ) { - perror("nvlist_create"); + perror("zcaplib: nvlist_create"); return(0); } nvlist_add_number(nvl, "command", ZCAPCMD_GZPUTC); @@ -1128,7 +1129,7 @@ if( (args = nvlist_create(0)) == NULL || (nvl = nvlist_create(0)) == NULL ) { - perror("nvlist_create"); + perror("zcaplib: nvlist_create"); return(0); } nvlist_add_number(nvl, "command", ZCAPCMD_GZUNGETC); @@ -1151,7 +1152,7 @@ if( (args = nvlist_create(0)) == NULL || (nvl = nvlist_create(0)) == NULL ) { - perror("nvlist_create"); + perror("zcaplib: nvlist_create"); return(0); } nvlist_add_number(nvl, "command", ZCAPCMD_GZFLUSH); @@ -1174,7 +1175,7 @@ if( (args = nvlist_create(0)) == NULL || (nvl = nvlist_create(0)) == NULL ) { - perror("nvlist_create"); + perror("zcaplib: nvlist_create"); return(0); } nvlist_add_number(nvl, "command", ZCAPCMD_GZSEEK); @@ -1202,7 +1203,7 @@ if( (args = nvlist_create(0)) == NULL || (nvl = nvlist_create(0)) == NULL ) { - perror("nvlist_create"); + perror("zcaplib: nvlist_create"); return(0); } nvlist_add_number(nvl, "command", command); @@ -1224,7 +1225,7 @@ if( (args = nvlist_create(0)) == NULL || (nvl = nvlist_create(0)) == NULL ) { - perror("nvlist_create"); + perror("zcaplib: nvlist_create"); return(0); } nvlist_add_number(nvl, "command", ZCAPCMD_GZPUTC); @@ -1250,7 +1251,7 @@ if( (args = nvlist_create(0)) == NULL || (nvl = nvlist_create(0)) == NULL ) { - perror("nvlist_create"); + perror("zcaplib: nvlist_create"); return(0); } nvlist_add_number(nvl, "command", ZCAPCMD_ADLER32); @@ -1274,7 +1275,7 @@ if( (args = nvlist_create(0)) == NULL || (nvl = nvlist_create(0)) == NULL ) { - perror("nvlist_create"); + perror("zcaplib: nvlist_create"); return(0); } nvlist_add_number(nvl, "command", ZCAPCMD_ADLER32_COMBINE); @@ -1298,7 +1299,7 @@ checkChild(); if( (args = nvlist_create(0)) == NULL || (nvl = nvlist_create(0)) == NULL ) { - perror("nvlist_create"); + perror("zcaplib: nvlist_create"); return(0); } nvlist_add_number(nvl, "command", ZCAPCMD_CRC32); @@ -1323,7 +1324,7 @@ if( (args = nvlist_create(0)) == NULL || (nvl = nvlist_create(0)) == NULL ) { - perror("nvlist_create"); + perror("zcaplib: nvlist_create"); return(0); } nvlist_add_number(nvl, "command", ZCAPCMD_CRC32_COMBINE); Modified: soc2013/dpl/head/lib/libzcap/deflate.c ============================================================================== --- soc2013/dpl/head/lib/libzcap/deflate.c Mon Aug 19 15:58:39 2013 (r256156) +++ soc2013/dpl/head/lib/libzcap/deflate.c Mon Aug 19 16:40:10 2013 (r256157) @@ -3,50 +3,6 @@ * For conditions of distribution and use, see copyright notice in zlib.h */ -/* - * ALGORITHM - * - * The "deflation" process depends on being able to identify portions - * of the input text which are identical to earlier input (within a - * sliding window trailing behind the input currently being processed). - * - * The most straightforward technique turns out to be the fastest for - * most input files: try all possible matches and select the longest. - * The key feature of this algorithm is that insertions into the string - * dictionary are very simple and thus fast, and deletions are avoided - * completely. Insertions are performed at each input character, whereas - * string matches are performed only when the previous match ends. So it - * is preferable to spend more time in matches to allow very fast string - * insertions and avoid deletions. The matching algorithm for small - * strings is inspired from that of Rabin & Karp. A brute force approach - * is used to find longer strings when a small match has been found. - * A similar algorithm is used in comic (by Jan-Mark Wams) and freeze - * (by Leonid Broukhis). - * A previous version of this file used a more sophisticated algorithm - * (by Fiala and Greene) which is guaranteed to run in linear amortized - * time, but has a larger average cost, uses more memory and is patented. - * However the F&G algorithm may be faster for some highly redundant - * files if the parameter max_chain_length (described below) is too large. - * - * ACKNOWLEDGEMENTS - * - * The idea of lazy evaluation of matches is due to Jan-Mark Wams, and - * I found it in 'freeze' written by Leonid Broukhis. - * Thanks to many people for bug reports and testing. - * - * REFERENCES - * - * Deutsch, L.P.,"DEFLATE Compressed Data Format Specification". - * Available in http://tools.ietf.org/html/rfc1951 - * - * A description of the Rabin and Karp algorithm is given in the book - * "Algorithms" by R. Sedgewick, Addison-Wesley, p252. - * - * Fiala,E.R., and Greene,D.H. - * Data Compression with Finite Windows, Comm.ACM, 32,4 (1989) 490-595 - * - */ - /* @(#) $Id$ */ #include "deflate.h" @@ -79,6 +35,10 @@ const char *version; int stream_size; { + /* The other process can't use this process functions. */ + strm->zalloc = Z_NULL; + strm->zfree = Z_NULL; + strm->opaque = Z_NULL; return zcapcmd_deflateInit(strm, level, method, windowBits, memLevel, strategy, version, stream_size); } @@ -90,6 +50,9 @@ const Bytef *dictionary; uInt dictLength; { + strm->zalloc = Z_NULL; + strm->zfree = Z_NULL; + strm->opaque = Z_NULL; int ret = 0; /* We can send all the dictionary data, but it doesn't seem a good idea. */ /* We also can send the last 5kb, but that would make the lib slower. */ @@ -110,6 +73,9 @@ int ZEXPORT deflateReset (strm) z_streamp strm; { + strm->zalloc = Z_NULL; + strm->zfree = Z_NULL; + strm->opaque = Z_NULL; return zcapcmd_deflateReset(strm); } @@ -119,6 +85,9 @@ z_streamp strm; gz_headerp head; { + strm->zalloc = Z_NULL; + strm->zfree = Z_NULL; + strm->opaque = Z_NULL; return zcapcmd_deflateSetHeader(strm, head); } @@ -129,6 +98,9 @@ unsigned *pending; int *bits; { + strm->zalloc = Z_NULL; + strm->zfree = Z_NULL; + strm->opaque = Z_NULL; return zcapcmd_deflatePending(strm, pending, bits); } @@ -139,6 +111,9 @@ int bits; int value; { + strm->zalloc = Z_NULL; + strm->zfree = Z_NULL; + strm->opaque = Z_NULL; return zcapcmd_deflatePrime(strm, bits, value); } @@ -149,6 +124,9 @@ int level; int strategy; { + strm->zalloc = Z_NULL; + strm->zfree = Z_NULL; + strm->opaque = Z_NULL; return zcapcmd_deflateParams(strm, level, strategy); } @@ -161,6 +139,9 @@ int nice_length; int max_chain; { + strm->zalloc = Z_NULL; + strm->zfree = Z_NULL; + strm->opaque = Z_NULL; return zcapcmd_deflateTune(strm, good_length, max_lazy, nice_length, max_chain); } @@ -170,6 +151,9 @@ z_streamp strm; uLong sourceLen; { + strm->zalloc = Z_NULL; + strm->zfree = Z_NULL; + strm->opaque = Z_NULL; return zcapcmd_deflateBound(strm, sourceLen); } @@ -179,6 +163,9 @@ z_streamp strm; int flush; { + strm->zalloc = Z_NULL; + strm->zfree = Z_NULL; + strm->opaque = Z_NULL; return zcapcmd_deflate(strm, flush); } @@ -187,7 +174,9 @@ int ZEXPORT deflateEnd (strm) z_streamp strm; { - if (strm == Z_NULL || strm->state == Z_NULL) return Z_STREAM_ERROR; + strm->zalloc = Z_NULL; + strm->zfree = Z_NULL; + strm->opaque = Z_NULL; return zcapcmd_deflateEnd(strm); } @@ -197,6 +186,12 @@ z_streamp dest; z_streamp source; { + dest->zalloc = Z_NULL; + dest->zfree = Z_NULL; + dest->opaque = Z_NULL; + source->zalloc = Z_NULL; + source->zfree = Z_NULL; + source->opaque = Z_NULL; #ifdef MAXSEG_64K return Z_STREAM_ERROR; #else Modified: soc2013/dpl/head/lib/libzcap/gzlib.c ============================================================================== --- soc2013/dpl/head/lib/libzcap/gzlib.c Mon Aug 19 15:58:39 2013 (r256156) +++ soc2013/dpl/head/lib/libzcap/gzlib.c Mon Aug 19 16:40:10 2013 (r256157) @@ -68,7 +68,7 @@ #endif if ((fd = open(path, oflag)) < 0) { - perror("Couldn't open"); + perror("zcaplib: Couldn't open"); abort(); } Modified: soc2013/dpl/head/lib/libzcap/inflate.c ============================================================================== --- soc2013/dpl/head/lib/libzcap/inflate.c Mon Aug 19 15:58:39 2013 (r256156) +++ soc2013/dpl/head/lib/libzcap/inflate.c Mon Aug 19 16:40:10 2013 (r256157) @@ -29,6 +29,9 @@ const char *version; int stream_size; { + strm->zalloc = Z_NULL; + strm->zfree = Z_NULL; + strm->opaque = Z_NULL; return zcapcmd_inflateInit(strm, windowBits, version, stream_size); } @@ -36,6 +39,9 @@ int ZEXPORT inflateEnd(strm) z_streamp strm; { + strm->zalloc = Z_NULL; + strm->zfree = Z_NULL; + strm->opaque = Z_NULL; return zcapcmd_inflateEnd(strm); } @@ -46,6 +52,9 @@ uInt dictLength; { /* XXX This ain't working yet. */ + strm->zalloc = Z_NULL; + strm->zfree = Z_NULL; + strm->opaque = Z_NULL; return zcapcmd_inflateSetDictionary(strm, dictionary, dictLength); } @@ -56,6 +65,9 @@ uInt *dictLength; { /* XXX This ain't working yet. */ + strm->zalloc = Z_NULL; + strm->zfree = Z_NULL; + strm->opaque = Z_NULL; return zcapcmd_inflateGetDictionary(strm, dictionary, dictLength); } @@ -63,6 +75,9 @@ int ZEXPORT inflateSync(strm) z_streamp strm; { + strm->zalloc = Z_NULL; + strm->zfree = Z_NULL; + strm->opaque = Z_NULL; return zcapcmd_inflateSync(strm); } @@ -71,6 +86,12 @@ z_streamp dest; z_streamp source; { + dest->zalloc = Z_NULL; + dest->zfree = Z_NULL; + dest->opaque = Z_NULL; + source->zalloc = Z_NULL; + source->zfree = Z_NULL; + source->opaque = Z_NULL; return zcapcmd_inflateCopy(dest, source); } @@ -78,6 +99,9 @@ int ZEXPORT inflateReset(strm) z_streamp strm; { + strm->zalloc = Z_NULL; + strm->zfree = Z_NULL; + strm->opaque = Z_NULL; return zcapcmd_inflateReset(strm); } @@ -86,6 +110,9 @@ z_streamp strm; int windowBits; { + strm->zalloc = Z_NULL; + strm->zfree = Z_NULL; + strm->opaque = Z_NULL; return zcapcmd_inflateReset2(strm, windowBits); } @@ -95,6 +122,9 @@ int bits; int value; { + strm->zalloc = Z_NULL; + strm->zfree = Z_NULL; + strm->opaque = Z_NULL; return zcapcmd_inflatePrime(strm, bits, value); } @@ -102,6 +132,9 @@ long ZEXPORT inflateMark(strm) z_streamp strm; { + strm->zalloc = Z_NULL; + strm->zfree = Z_NULL; + strm->opaque = Z_NULL; return zcapcmd_inflateMark(strm); } @@ -111,7 +144,10 @@ gz_headerp head; { /* XXX Do I really have to compartmentalize this??? */ - struct inflate_state FAR *state; + strm->zalloc = Z_NULL; + strm->zfree = Z_NULL; + strm->opaque = Z_NULL; + struct inflate_state FAR *state; /* check state */ if (strm == Z_NULL || strm->state == Z_NULL) return Z_STREAM_ERROR; @@ -128,5 +164,8 @@ z_streamp strm; int flush; { + strm->zalloc = Z_NULL; + strm->zfree = Z_NULL; + strm->opaque = Z_NULL; return -1; } Added: soc2013/dpl/head/lib/libzcap/test/zcaplibtest.c ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ soc2013/dpl/head/lib/libzcap/test/zcaplibtest.c Mon Aug 19 16:40:10 2013 (r256157) @@ -0,0 +1,149 @@ +#include + +#include +#include + +void testzlibVersion(void); +void testdeflateInit(z_streamp strm); +void testdeflateEnd(z_streamp strm); +void testinflateInit(z_streamp strm); +void testinflateEnd(z_streamp strm); + +void testzlibCompileFlags(void); + +void testCompressBound(void); + +void testchecksums(void); + +int i; +char *data; +z_streamp strm; + +int +main() { + + if ( (strm = malloc(sizeof (z_stream))) == NULL) { + printf("zcaplibtest: malloc()"); + abort(); + } + + if ( (data = malloc(10*1024)) == NULL) { + printf("zcaplibtest: malloc()"); + abort(); + } + + for (i=0; i<(10*1024); i++) { + data[i] = rand(); + } + + printf("This is a zcaplib program tester.\n"); + printf("I will now test all the zcaplib functions.\n\n"); + +/* testdeflateInit(strm);*/ +/* testdeflateEnd(strm);*/ +/* testinflateInit(strm);*/ +/* testinflateEnd(strm);*/ + + testzlibCompileFlags(); +/* testCompressBound();*/ + + testchecksums(); + return 0; +} + +void +testdeflateInit(z_streamp strm) +{ + printf("deflateInit()\t...\t"); + + return; +} + +void +testdeflateEnd(z_streamp strm) +{ + printf("deflateEnd()\t...\t"); + + return; +} + +void +testinflateInit(z_streamp strm) +{ + printf("inflateInit()\t...\t"); + + return; +} + +void +testinflateEnd(z_streamp strm) +{ + printf("inflateEnd()\t...\t"); + + return; +} + +void +testzlibCompileFlags(void) +{ + uLong ret = 0; + printf("zlibCompileFlags()\t...\t"); + + if ( (ret = zlibCompileFlags()) > 0) + printf("OK\n"); + else + printf("Error\n"); + printf("zlibCompileFlags: %lu\n", ret); + + return; +} + +void +testCompressBound(void) +{ + printf("compressBound()\t...\t"); + return; +} + +void +testchecksums(void) +{ + long data[2]; + + uLong first = 0; + uLong second = 0; + uLong all = 0; + uLong combined = 0; + int lsize = sizeof(long); + + data[0] = random(); + data[1] = random(); + + first = adler32(0, (void *)data, sizeof(long)); + printf("adler32(0, %ld, %d): %ld\n", data[0], sizeof(long), first ); + + printf("adler32\t...\t"); + if (first != 0) + printf("OK\n"); + else + printf("Error\n"); + + second = adler32(0, (void *)(data+lsize), sizeof(long)); + printf("adler32(0, %ld, %d): %ld\n", data[1], sizeof(long), second ); + + combined = adler32_combine(first, second, sizeof(long)); + printf("adler32_combine(%ld, %ld, %d): %ld\n", first, second, sizeof(long), combined ); + + all = adler32(0L, (void *)data, 2*sizeof(long)); + printf("adler32(0, %ld, %d): %ld\n", data[0], 2*sizeof(long), all ); + + + printf("adler32_combine()\t...\t"); + if (all == combined) + printf("OK\n"); + else + printf("Error\n"); + + + return; +} \ No newline at end of file Modified: soc2013/dpl/head/lib/libzcap/zlibworker/commands.c ============================================================================== --- soc2013/dpl/head/lib/libzcap/zlibworker/commands.c Mon Aug 19 15:58:39 2013 (r256156) +++ soc2013/dpl/head/lib/libzcap/zlibworker/commands.c Mon Aug 19 16:40:10 2013 (r256157) @@ -19,7 +19,7 @@ if ((args = nvlist_take_nvlist(nvl, "args")) == NULL) { memcpy(stream, - nvlist_take_binary(args, "strm", &zstreamsize), zstreamsize); + nvlist_take_binary(args, "strm", &zstreamsize), zstreamsize); ret = deflateInit2_(stream, nvlist_take_number(args, "level"), nvlist_take_number(args, "method"), @@ -255,9 +255,9 @@ { int ret = -1; int len = 0; - void * data; + void * data = NULL; - if ((args = nvlist_take_nvlist(nvl, "args")) != NULL) + if ((args = nvlist_take_nvlist(nvl, "args")) != NULL) { len = nvlist_take_number(args, "len"); *** DIFF OUTPUT TRUNCATED AT 1000 LINES *** From owner-svn-soc-all@FreeBSD.ORG Mon Aug 19 20:31:58 2013 Return-Path: Delivered-To: svn-soc-all@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id E4975A71 for ; Mon, 19 Aug 2013 20:31:57 +0000 (UTC) (envelope-from ambarisha@FreeBSD.org) Received: from socsvn.freebsd.org (socsvn.freebsd.org [IPv6:2001:1900:2254:206a::50:2]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id C3B08273A for ; Mon, 19 Aug 2013 20:31:57 +0000 (UTC) Received: from socsvn.freebsd.org ([127.0.1.124]) by socsvn.freebsd.org (8.14.7/8.14.7) with ESMTP id r7JKVvQ8092654 for ; Mon, 19 Aug 2013 20:31:57 GMT (envelope-from ambarisha@FreeBSD.org) Received: (from www@localhost) by socsvn.freebsd.org (8.14.7/8.14.6/Submit) id r7JKVv0R092644 for svn-soc-all@FreeBSD.org; Mon, 19 Aug 2013 20:31:57 GMT (envelope-from ambarisha@FreeBSD.org) Date: Mon, 19 Aug 2013 20:31:57 GMT Message-Id: <201308192031.r7JKVv0R092644@socsvn.freebsd.org> X-Authentication-Warning: socsvn.freebsd.org: www set sender to ambarisha@FreeBSD.org using -f From: ambarisha@FreeBSD.org To: svn-soc-all@FreeBSD.org Subject: socsvn commit: r256163 - in soc2013/ambarisha/head/usr.bin: dmget dms MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-soc-all@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: SVN commit messages for the entire Summer of Code repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 Aug 2013 20:31:58 -0000 Author: ambarisha Date: Mon Aug 19 20:31:57 2013 New Revision: 256163 URL: http://svnweb.FreeBSD.org/socsvn/?view=rev&rev=256163 Log: DMS is now threaded. Each worker is a new thread. Modified: soc2013/ambarisha/head/usr.bin/dmget/dmget.c soc2013/ambarisha/head/usr.bin/dmget/dmget.h soc2013/ambarisha/head/usr.bin/dmget/fetch.c soc2013/ambarisha/head/usr.bin/dmget/utils.c soc2013/ambarisha/head/usr.bin/dms/Makefile soc2013/ambarisha/head/usr.bin/dms/dms.c soc2013/ambarisha/head/usr.bin/dms/dms.h soc2013/ambarisha/head/usr.bin/dms/utils.c soc2013/ambarisha/head/usr.bin/dms/worker.c Modified: soc2013/ambarisha/head/usr.bin/dmget/dmget.c ============================================================================== --- soc2013/ambarisha/head/usr.bin/dmget/dmget.c Mon Aug 19 18:57:58 2013 (r256162) +++ soc2013/ambarisha/head/usr.bin/dmget/dmget.c Mon Aug 19 20:31:57 2013 (r256163) @@ -13,7 +13,7 @@ #include "dm.h" #include "dmget.h" -auth_t dmAuthMethod; +dm_auth_t dmAuthMethod; stat_display_t dmStatDisplayMethod; int dmTimeout; int dmRestartCalls; @@ -37,7 +37,7 @@ return ptr; } -void dm_sighandler(int signal) +void dmSigHandler(int signal) { switch(signal) { case SIGINT: @@ -242,12 +242,16 @@ send_request(int sock, struct dmreq dmreq) { char *reqbuf; - int bufsize, err; + int bufsize, err, fd; bufsize = mk_reqbuf(dmreq, &reqbuf, DMREQ); err = sigsafe_write(sock, reqbuf, bufsize); - int fd = open(dmreq.path, O_CREAT|O_RDWR|O_TRUNC, S_IRUSR|S_IWUSR); + if (dmreq.flags & O_STDOUT) + fd = STDOUT_FILENO; + else + fd = open(dmreq.path, O_CREAT|O_RDWR|O_TRUNC, S_IRUSR|S_IWUSR); + Write_fd(sock, fd); close(fd); @@ -255,56 +259,62 @@ return(err); } -static void -free_msg(struct dmmsg **msg) +struct dmauth * +mk_dmauth(char *buf, int bufsize) { - free((*msg)->buf); - free(*msg); - *msg = NULL; -} + int i = 0, len; + struct dmauth *dmauth = (struct dmauth *) Malloc(sizeof(struct dmauth)); -static struct dmmsg * -recv_msg(int sock) -{ - int err; - fd_set fds; - sigset_t sm; - struct dmmsg *msg; + len = strlen(buf + i); + dmauth->scheme = (char *) Malloc(len + 1); + strncpy(dmauth->scheme, buf + i, len); + i += len + 1; - msg = (struct dmmsg *) Malloc(sizeof(struct dmmsg)); + len = strlen(buf + i); + dmauth->host = (char *) Malloc(len + 1); + strncpy(dmauth->host, buf + i, len); + i += len + 1; - FD_ZERO(&fds); - FD_SET(sock, &fds); + dmauth->port = *(int *)(buf + i); + i += sizeof(int); - err = Select(sock + 1, &fds, NULL, NULL, NULL); - if (err == -1) - return(-1); + return dmauth; +} - sigemptyset(&sm); - sigaddset(&sm, SIGINT); - sigaddset(&sm, SIGINFO); +void +rm_dmauth(struct dmauth **dmauth) +{ + free((*dmauth)->scheme); + free((*dmauth)->host); + free(*dmauth); + *dmauth = NULL; +} - sigprocmask(SIG_BLOCK, &sm, NULL); - err = Peel(sock, msg); - sigprocmask(SIG_UNBLOCK, &sm, NULL); +static int +send_dmauth(int sock, struct dmauth *dmauth) +{ + int ulen = strlen(dmauth->user) + 1; + int bufsize = ulen + strlen(dmauth->pwd) + 1; + char *buf = (char *) Malloc(bufsize); - if (err == -1) { - /* Set dmg_err* */ - free_msg(&msg); - return NULL; - } - - return msg; + strcpy(buf, dmauth->user); + strcpy(buf + ulen, dmauth->user); + + struct dmmsg msg; + msg.op = DMAUTHRESP; + msg.buf = buf; + msg.len = bufsize; + send_msg(sock, msg); } int dmget(struct dmreq dmreq) { - int sock, err, ret, force; + int sock, err, ret, force, i; struct sockaddr_un dms_addr; struct dmres *dmres; struct xferstat xs; - + struct dmauth *dmauth; sock = Socket(AF_UNIX, SOCK_STREAM, 0); dms_addr.sun_family = AF_UNIX; @@ -347,6 +357,13 @@ dmStatDisplayMethod(&xs, force); break; case DMAUTHREQ: + dmauth = mk_dmauth(msg->buf, msg->len); + if (dmAuthMethod(dmauth) == -1) { + + } + send_dmauth(sock, dmauth); + rm_dmauth(&dmauth); + break; default: break; } Modified: soc2013/ambarisha/head/usr.bin/dmget/dmget.h ============================================================================== --- soc2013/ambarisha/head/usr.bin/dmget/dmget.h Mon Aug 19 18:57:58 2013 (r256162) +++ soc2013/ambarisha/head/usr.bin/dmget/dmget.h Mon Aug 19 20:31:57 2013 (r256163) @@ -5,20 +5,31 @@ #include #include -#include #include "dm.h" +#define AUTH_USERLEN 256 +#define AUTH_PWDLEN 256 + + +struct dmauth { + int port; + char *scheme; + char *host; + char user[AUTH_USERLEN+1]; + char pwd[AUTH_PWDLEN+1] +}; + extern int dmLastErrCode; extern int dmRestartCalls; extern char dmLastErrString[]; -typedef int (*auth_t)(struct url *); -extern auth_t dmAuthMethod; +typedef int (*dm_auth_t)(struct dmauth *); +extern dm_auth_t dmAuthMethod; typedef void (*stat_display_t) (struct xferstat *, int); extern stat_display_t dmStatDisplayMethod; -int dm_request(struct dmreq); -void dm_sighandler(int sig); +int dmget(struct dmreq); +void dmSigHandler(int sig); #endif /* _DMCLIENT_H */ Modified: soc2013/ambarisha/head/usr.bin/dmget/fetch.c ============================================================================== --- soc2013/ambarisha/head/usr.bin/dmget/fetch.c Mon Aug 19 18:57:58 2013 (r256162) +++ soc2013/ambarisha/head/usr.bin/dmget/fetch.c Mon Aug 19 20:31:57 2013 (r256163) @@ -207,21 +207,21 @@ * Ask the user for authentication details */ static int -query_auth(struct url *URL) +query_auth(struct dmauth *auth) { struct termios tios; tcflag_t saved_flags; int i, nopwd; fprintf(stderr, "Authentication required for <%s://%s:%d/>!\n", - URL->scheme, URL->host, URL->port); + auth->scheme, auth->host, auth->port); fprintf(stderr, "Login: "); - if (fgets(URL->user, sizeof URL->user, stdin) == NULL) + if (fgets(auth->user, sizeof auth->user, stdin) == NULL) return (-1); - for (i = strlen(URL->user); i >= 0; --i) - if (URL->user[i] == '\r' || URL->user[i] == '\n') - URL->user[i] = '\0'; + for (i = strlen(auth->user); i >= 0; --i) + if (auth->user[i] == '\r' || auth->user[i] == '\n') + auth->user[i] = '\0'; fprintf(stderr, "Password: "); if (tcgetattr(STDIN_FILENO, &tios) == 0) { @@ -229,17 +229,17 @@ tios.c_lflag &= ~ECHO; tios.c_lflag |= ECHONL|ICANON; tcsetattr(STDIN_FILENO, TCSAFLUSH|TCSASOFT, &tios); - nopwd = (fgets(URL->pwd, sizeof URL->pwd, stdin) == NULL); + nopwd = (fgets(auth->pwd, sizeof auth->pwd, stdin) == NULL); tios.c_lflag = saved_flags; tcsetattr(STDIN_FILENO, TCSANOW|TCSASOFT, &tios); } else { - nopwd = (fgets(URL->pwd, sizeof URL->pwd, stdin) == NULL); + nopwd = (fgets(auth->pwd, sizeof auth->pwd, stdin) == NULL); } if (nopwd) return (-1); - for (i = strlen(URL->pwd); i >= 0; --i) - if (URL->pwd[i] == '\r' || URL->pwd[i] == '\n') - URL->pwd[i] = '\0'; + for (i = strlen(auth->pwd); i >= 0; --i) + if (auth->pwd[i] == '\r' || auth->pwd[i] == '\n') + auth->pwd[i] = '\0'; return (0); } @@ -473,7 +473,7 @@ /* signal handling */ sa.sa_flags = 0; - sa.sa_handler = dm_sighandler; + sa.sa_handler = dmSigHandler; sigemptyset(&sa.sa_mask); sigaction(SIGALRM, &sa, NULL); sa.sa_flags = SA_RESETHAND; Modified: soc2013/ambarisha/head/usr.bin/dmget/utils.c ============================================================================== --- soc2013/ambarisha/head/usr.bin/dmget/utils.c Mon Aug 19 18:57:58 2013 (r256162) +++ soc2013/ambarisha/head/usr.bin/dmget/utils.c Mon Aug 19 20:31:57 2013 (r256163) @@ -122,25 +122,24 @@ int err; err = Read(sock, &bufsize, sizeof(bufsize)); if (err == 0) - return (-1); + return (err); bufsize -= sizeof(bufsize); err = Read(sock, &(msg->op), sizeof(msg->op)); if (err == 0) - return (-1); + return (err); bufsize -= sizeof(msg->op); msg->buf = (char *) Malloc(bufsize); msg->len = bufsize; - Read(sock, msg->buf, bufsize); + err = Read(sock, msg->buf, bufsize); if (err == 0) { free(msg->buf); msg->len = 0; - return (-1); } - return 1; + return bufsize; } int @@ -181,3 +180,45 @@ return (nbytes); } +struct dmmsg * +recv_msg(int sock) +{ + int bufsize = 0; + int err; + struct dmmsg *msg; + err = Read(sock, &bufsize, sizeof(bufsize)); + if (err == 0) { + /* set dms_error */ + return (NULL); + } + + bufsize -= sizeof(bufsize); + + err = Read(sock, &(msg->op), sizeof(msg->op)); + if (err == 0) { + /* set dms_error */ + return (NULL); + } + bufsize -= sizeof(msg->op); + + msg->buf = (char *) Malloc(bufsize); + msg->len = bufsize; + + err = Read(sock, msg->buf, bufsize); + if (err == 0) { + free(msg->buf); + msg->len = 0; + /* set dms_error */ + return (NULL); + } + + return msg; +} + +void +free_msg(struct dmmsg **msg) +{ + free((*msg)->buf); + free(*msg); + *msg = NULL; +} Modified: soc2013/ambarisha/head/usr.bin/dms/Makefile ============================================================================== --- soc2013/ambarisha/head/usr.bin/dms/Makefile Mon Aug 19 18:57:58 2013 (r256162) +++ soc2013/ambarisha/head/usr.bin/dms/Makefile Mon Aug 19 20:31:57 2013 (r256163) @@ -1,12 +1,12 @@ # $FreeBSD$ .include -SRCS= utils.c list.c dms.c worker.c +SRCS= utils.c dms.c worker.c PROG= dms CSTD?= c99 .if ${MK_OPENSSL} != "no" DPADD= ${LIBFETCH} ${LIBSSL} ${LIBCRYPTO} -LDADD= -lfetch -lssl -lcrypto +LDADD= -lfetch -lssl -lcrypto -lpthread .else DPADD= ${LIBFETCH} LDADD= -lfetch Modified: soc2013/ambarisha/head/usr.bin/dms/dms.c ============================================================================== --- soc2013/ambarisha/head/usr.bin/dms/dms.c Mon Aug 19 18:57:58 2013 (r256162) +++ soc2013/ambarisha/head/usr.bin/dms/dms.c Mon Aug 19 20:31:57 2013 (r256163) @@ -9,12 +9,49 @@ #include #include #include +#include #include "dm.h" -#include "list.h" #include "dms.h" -int stop; +int stop; +struct dmjob *jobs; + +void *run_worker(struct dmjob *job); + +static struct dmjob * +add_job(struct dmjob *head, struct dmjob *new) +{ + new->prev = NULL; + new->next = NULL; + + if (head == NULL) + return new; + + head->prev = new; + new->next = head; +} + +static struct dmjob * +rm_job(struct dmjob *head, struct dmjob *job) +{ + if (head == NULL) + return NULL; + + if (job == NULL) + return head; + + if (job->next != NULL) + job->next->prev = job->prev; + + if (job->prev != NULL) + job->prev->next = job->next; + + if (job == head) + return job->next; + + return head; +} static int read_fd(int sock) @@ -61,80 +98,35 @@ return newfd; } -static int -Read_fd(int sock) -{ - int ret = read_fd(sock); - if (ret == -1) { - perror("Read_fd():"); - } else { - printf("Read_fd(): Success\n"); - } - return(ret); -} - static struct dmjob * -mk_dmjob(int sock, struct dmreq dmreq) +mk_dmjob(struct dmreq *dmreq, int client) { struct dmjob *dmjob = (struct dmjob *) Malloc(sizeof(struct dmjob)); - - /* Right now dmjob and dmreq are same */ - dmjob->v_level = dmreq.v_level; - dmjob->family = dmreq.family; - dmjob->ftp_timeout = dmreq.ftp_timeout; - dmjob->http_timeout = dmreq.http_timeout; - dmjob->B_size = dmreq.B_size; - - dmjob->S_size = dmreq.S_size; - dmjob->T_secs = dmreq.T_secs; - dmjob->flags = dmreq.flags; - - dmjob->i_filename = (char *) Malloc(strlen(dmreq.i_filename) + 1); - strcpy(dmjob->i_filename, dmreq.i_filename); - - dmjob->URL = (char *) Malloc(strlen(dmreq.URL) + 1); - strcpy(dmjob->URL, dmreq.URL); - - dmjob->path = (char *) Malloc(strlen(dmreq.path) + 1); - strcpy(dmjob->path, dmreq.path); - - dmjob->fd = Read_fd(sock); - dmjob->csock = sock; - + dmjob->request = dmreq; + dmjob->ofd = read_fd(client); + if (dmjob->ofd == -1) { + /* Handle error */ + free(dmjob); + return NULL; + } + dmjob->client = client; + dmjob->sigint = 0; + dmjob->sigalrm = 0; + dmjob->siginfo = 0; + dmjob->siginfo_en = 0; + dmjob->state = RUNNING; + dmjob->url = NULL; return dmjob; } -static struct dmjob * -Mk_dmjob(int sock, struct dmreq dmreq) -{ - struct dmjob *dmjob = mk_dmjob(sock, dmreq); - if (dmjob == NULL) { - perror("mk_dmjob():"); -#if DEBUG - } else { - printf("mk_dmjob(): Success\n"); -#endif - } - return dmjob; -} - -static void -rm_dmjob(struct dmjob **dmjob) -{ - free((*dmjob)->i_filename); - free((*dmjob)->path); - free((*dmjob)->URL); - free(*dmjob); - *dmjob = NULL; -} - -static int -parse_request(char *rcvbuf, int bufsize) +static struct dmreq * +mk_dmreq(char *rcvbuf, int bufsize) { int i = 0; struct dmreq *dmreq = (struct dmreq *) Malloc(sizeof(struct dmreq)); - + if (dmreq == NULL) + return NULL; memcpy(&(dmreq->v_level), rcvbuf + i, sizeof(dmreq->v_level)); i += sizeof(dmreq->v_level); @@ -157,7 +149,6 @@ i += sizeof(dmreq->T_secs); memcpy(&(dmreq->flags), rcvbuf + i, sizeof(dmreq->flags)); - i += sizeof(dmreq->flags); int sz = strlen(rcvbuf+i); @@ -178,22 +169,8 @@ return dmreq; } -static int -Parse_request(char *rcvbuf, int bufsize) -{ - struct dmreq *dmreq = parse_request(rcvbuf, bufsize); - if (dmreq == NULL) { - perror("Parse_request():"); -#if DEBUG - } else { - printf("Parse_reqeust(): Success\n"); -#endif - } - return dmreq; -} - static void -Free_request(struct dmreq **dmreq) +rm_dmreq(struct dmreq **dmreq) { free((*dmreq)->i_filename); free((*dmreq)->URL); @@ -202,61 +179,40 @@ *dmreq = NULL; } -static void -send_report(int sock, struct dmrep report, char op) -{ - char *buf; - int bufsize = sizeof(report) - sizeof(report.errstr); - int errlen = strlen(report.errstr); - bufsize += errlen; - - buf = (char *) Malloc(bufsize); - int i = 0; - - memcpy(buf + i, &(report.status), sizeof(report.status)); - i += sizeof(report.status); - - memcpy(buf + i, &(report.errcode), sizeof(report.errcode)); - i += sizeof(report.errcode); - - strcpy(buf + i, report.errstr); - i += errlen; - - struct dmmsg msg; - msg.op = op; - msg.buf = buf; - msg.len = bufsize; - send_msg(sock, msg); - - free(buf); -} - static int -handle_request(int csock, struct conn **conns) +handle_request(int csock) { - struct dmjob *dmjob; - struct dmreq *dmreq; - struct dmmsg msg; - struct dmrep report; - int err; + struct dmreq *dmreq; + struct dmmsg *msg; + struct dmjob *dmjob; + int ret; pid_t pid; - Peel(csock, &msg); + msg = recv_msg(csock); + if (msg == NULL) { + /* set dms_error */ + return -1; + } - switch (msg.op) { + switch (msg->op) { case DMREQ: - dmreq = Parse_request(msg.buf, msg.len); - dmjob = Mk_dmjob(csock, *dmreq); - Free_request(&dmreq); - do_job(*dmjob, &report); - send_report(csock, report, DMRESP); - default: - /* Unknown opcode recieved */ - return -1; + dmreq = mk_dmreq(msg->buf, msg->len); + dmjob = mk_dmjob(dmreq, csock); + jobs = add_job(jobs, dmjob); + pthread_create(&(dmjob->worker), NULL, run_worker, dmjob); + break; + default: + goto error; break; } - - return 1; +success: + ret = 0; + goto done; +error: + ret = -1; +done: + free_msg(msg); + return ret; } void @@ -267,93 +223,38 @@ } static int -handle_client_msg(struct conn *conn) -{ - struct dmmsg msg; - int ret = Peel(conn->client, &msg); - if (ret == 0) - return(1); - - switch(msg.op) { - case DMSIG: - send_msg(conn->worker, msg); - break; - case DMAUTHRESP: - /* TODO: Implement these */ - break; - default: - /* Unrecognized opcode */ - break; - } - return(0); -} - -static int -handle_worker_msg(struct conn *conn) -{ - struct dmmsg msg; - - int ret = Peel(conn->worker, &msg); - if (ret == 0) /* Worker closed the socket !! */ - return(1); - - switch(msg.op) { - case DMRESP: - send_msg(conn->client, msg); - ret = 1; - break; - case DMAUTHREQ: - /* TODO: Implement these */ - break; - default: - /* Unrecoginized opcode */ - break; - } - return (0); -} - -static int -service_conn(struct conn *conn, fd_set *fdset) +service_job(struct dmjob *job, fd_set *fdset) { int ret = 0; - if (FD_ISSET(conn->client, fdset)) { - ret = handle_client_msg(conn); - } - - if (FD_ISSET(conn->worker, fdset)) { - ret |= handle_worker_msg(conn); - /* TODO: Do this better */ - } + if (FD_ISSET(job->client, fdset)) + pthread_kill(job->worker, SIGUSR1); + if (job->state == DONE) + ret = 1; return (ret); } static void run_event_loop(int socket) { - int i, maxfd = socket; - - struct conn *conns = NULL, *cur; - + int i, ret, maxfd = socket; + struct dmjob *cur; + void *retptr; + jobs = NULL; fd_set fdset; signal(SIGINT, sigint_handler); - while (!stop) { /* Prepare fdset and make select call */ FD_ZERO(&fdset); FD_SET(socket, &fdset); - cur = conns; + cur = jobs; while (cur != NULL) { FD_SET(cur->client, &fdset); - FD_SET(cur->worker, &fdset); - if (cur->client > maxfd) maxfd = cur->client; - if (cur->worker > maxfd) - maxfd = cur->worker; cur = cur->next; } @@ -364,30 +265,29 @@ size_t cliaddrlen = sizeof(cliaddr); int csock = Accept(socket, (struct sockaddr *) &cliaddr, &cliaddrlen); - handle_request(csock, &conns); + handle_request(csock); } - cur = conns; + cur = jobs; while (cur != NULL) { - int ret = service_conn(cur, &fdset); + ret = service_job(cur, &fdset); if (ret == 1) { close(cur->client); - close(cur->worker); - /* What should happen to the worker */ - conns = rm_conn(conns, cur); + pthread_join(cur->worker, &retptr); + jobs = rm_job(jobs, cur); } cur = cur->next; } } - cur = conns; + cur = jobs; while (cur != NULL) { close(cur->client); - close(cur->worker); + ret = service_job(cur, &fdset); /* TODO: Force the worker to quit as well */ - conns = rm_conn(conns, cur); - cur = conns; + jobs = rm_job(jobs, cur); + cur = jobs; } } Modified: soc2013/ambarisha/head/usr.bin/dms/dms.h ============================================================================== --- soc2013/ambarisha/head/usr.bin/dms/dms.h Mon Aug 19 18:57:58 2013 (r256162) +++ soc2013/ambarisha/head/usr.bin/dms/dms.h Mon Aug 19 20:31:57 2013 (r256163) @@ -1,34 +1,24 @@ +#ifndef _DMS_H +#define _DMS_H + #include +typedef enum {RUNNING=0, DONE=1} state_t; + struct dmjob { - int v_level; - int family; - long ftp_timeout; - long http_timeout; - off_t B_size; - off_t S_size; - long T_secs; - long flags; - int fd; - int csock; - -#define A_FLAG (1 << 0) -#define F_FLAG (1 << 1) -#define O_STDOUT (1 << 2) -#define R_FLAG (1 << 3) -#define U_FLAG (1 << 4) -#define d_FLAG (1 << 5) -#define i_FLAG (1 << 6) -#define l_FLAG (1 << 7) -#define m_FLAG (1 << 8) -#define n_FLAG (1 << 9) -#define p_FLAG (1 << 10) -#define r_FLAG (1 << 11) -#define s_FLAG (1 << 12) - - char *i_filename; - char *URL; - char *path; + int ofd; + int client; + state_t state; + int sigint; + int sigalrm; + int siginfo; + int siginfo_en; + struct dmreq *request; + struct url *url; + pthread_t *worker; + + struct dmjob *next; + struct dmjob *prev; }; struct dmrep { @@ -41,3 +31,5 @@ #define MAX_LISTEN_QUEUE 5 #define MINBUFSIZE 4096 + +#endif Modified: soc2013/ambarisha/head/usr.bin/dms/utils.c ============================================================================== --- soc2013/ambarisha/head/usr.bin/dms/utils.c Mon Aug 19 18:57:58 2013 (r256162) +++ soc2013/ambarisha/head/usr.bin/dms/utils.c Mon Aug 19 20:31:57 2013 (r256163) @@ -133,16 +133,12 @@ msg->buf = (char *) Malloc(bufsize); msg->len = bufsize; - Read(sock, msg->buf, bufsize); + err = Read(sock, msg->buf, bufsize); if (err == 0) { free(msg->buf); msg->len = 0; } -#if DEBUG - printf("{ msg->op = %d; msg->len = %d }\n", msg->op, msg->len); -#endif - return bufsize; } @@ -184,3 +180,58 @@ return (nbytes); } +struct dmmsg * +recv_msg(int sock) +{ + printf("in recv_msg\n"); + int bufsize = 0; + int err; + struct dmmsg *msg; + err = Read(sock, &bufsize, sizeof(bufsize)); + if (err == 0) { + /* set dms_error */ +#if DEBUG + fprintf(stderr, "recv_msg: remote end closed connection\n"); +#endif + return (NULL); + } + + printf("bufsize = %d\n", bufsize); + + bufsize -= sizeof(bufsize); + + printf("sock = %d\n", sock); + err = Read(sock, &(msg->op), sizeof(msg->op)); + if (err == 0) { + /* set dms_error */ +#if DEBUG + fprintf(stderr, "recv_msg: remote end closed connection\n"); +#endif + return (NULL); + } + bufsize -= sizeof(msg->op); + + msg->buf = (char *) Malloc(bufsize); + msg->len = bufsize; + + err = Read(sock, msg->buf, bufsize); + if (err == 0) { + free(msg->buf); + msg->len = 0; + /* set dms_error */ +#if DEBUG + fprintf(stderr, "recv_msg: remote end closed connection\n"); +#endif + return (NULL); + } + + return msg; +} + +void +free_msg(struct dmmsg **msg) +{ + free((*msg)->buf); + free(*msg); + *msg = NULL; +} Modified: soc2013/ambarisha/head/usr.bin/dms/worker.c ============================================================================== --- soc2013/ambarisha/head/usr.bin/dms/worker.c Mon Aug 19 18:57:58 2013 (r256162) +++ soc2013/ambarisha/head/usr.bin/dms/worker.c Mon Aug 19 20:31:57 2013 (r256163) @@ -10,10 +10,48 @@ #include "dms.h" #include "dm.h" -static int sigalrm; /* SIGALRM received by client */ -static int siginfo; /* SIGINFO received by client */ -static int sigint; /* SIGINT received by client */ -static int handle_siginfo; /* Yes or No */ +extern struct dmjob *jobs; + +static int +authenticate(struct url *url) +{ + struct dmmsg msg; + struct dmjob *cur = jobs; + while (cur != NULL) { + if (cur->url == url) + break; + cur = cur->next; + } + + if (cur == NULL) + return -1; // Todo: Verify this + + int bufsize = 0, i = 0, schlen, hlen; + schlen = strlen(url->scheme) + 1; + hlen = strlen(url->host) + 1; + bufsize += schlen + hlen + sizeof(url->port); + + msg.buf = (char *) Malloc(bufsize); + + strcpy(msg.buf, url->scheme); + i += schlen; + + strcpy(msg.buf + i, url->host); + i += hlen; + + *(int *) (msg.buf + i) = url->port; + + msg.op = DMAUTHREQ; + msg.len = bufsize; + send_msg(cur->client, msg); + + struct dmmsg *rcvmsg; + rcvmsg = recv_msg(cur->client); + + strncpy(url->user, rcvmsg->buf, sizeof(url->user)); + strncpy(url->pwd, rcvmsg->buf + strlen(rcvmsg->buf) + 1, sizeof(url->pwd)); + free_msg(&rcvmsg); +} static void stat_send(int csock, struct xferstat *xs, int force) @@ -68,7 +106,7 @@ static void stat_start(struct xferstat *xs, const char *name, off_t size, - off_t offset, struct dmjob dmjob) + off_t offset, struct dmjob *dmjob) { snprintf(xs->name, sizeof xs->name, "%s", name); gettimeofday(&xs->start, NULL); @@ -77,44 +115,36 @@ xs->offset = offset; xs->rcvd = offset; xs->lastrcvd = offset; - if ((dmjob.flags & V_TTY) && dmjob.v_level > 0) - stat_send(dmjob.csock, xs, 1); - else if (dmjob.v_level > 0) + if ((dmjob->request->flags & V_TTY) && dmjob->request->v_level > 0) + stat_send(dmjob->client, xs, 1); + else if (dmjob->request->v_level > 0) fprintf(stderr, "%-46s", xs->name); } static void -stat_end(struct xferstat *xs, struct dmjob dmjob) +stat_end(struct xferstat *xs, struct dmjob *dmjob) { gettimeofday(&xs->last, NULL); - if ((dmjob.flags & V_TTY) && dmjob.v_level > 0) { - stat_send(dmjob.csock, xs, 2); + if ((dmjob->request->flags & V_TTY) && dmjob->request->v_level > 0) { + stat_send(dmjob->client, xs, 2); putc('\n', stderr); - } else if (dmjob.v_level > 0) { + } else if (dmjob->request->v_level > 0) { fprintf(stderr, " %s %s\n", stat_bytes(xs->size), stat_bps(xs)); } } static void -stat_update(struct xferstat *xs, off_t rcvd, struct dmjob dmjob) *** DIFF OUTPUT TRUNCATED AT 1000 LINES *** From owner-svn-soc-all@FreeBSD.ORG Mon Aug 19 20:39:15 2013 Return-Path: Delivered-To: svn-soc-all@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id B4930C05 for ; Mon, 19 Aug 2013 20:39:15 +0000 (UTC) (envelope-from ambarisha@FreeBSD.org) Received: from socsvn.freebsd.org (socsvn.freebsd.org [IPv6:2001:1900:2254:206a::50:2]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 9FD232773 for ; Mon, 19 Aug 2013 20:39:15 +0000 (UTC) Received: from socsvn.freebsd.org ([127.0.1.124]) by socsvn.freebsd.org (8.14.7/8.14.7) with ESMTP id r7JKdFiM078351 for ; Mon, 19 Aug 2013 20:39:15 GMT (envelope-from ambarisha@FreeBSD.org) Received: (from www@localhost) by socsvn.freebsd.org (8.14.7/8.14.6/Submit) id r7JKdFjV078331 for svn-soc-all@FreeBSD.org; Mon, 19 Aug 2013 20:39:15 GMT (envelope-from ambarisha@FreeBSD.org) Date: Mon, 19 Aug 2013 20:39:15 GMT Message-Id: <201308192039.r7JKdFjV078331@socsvn.freebsd.org> X-Authentication-Warning: socsvn.freebsd.org: www set sender to ambarisha@FreeBSD.org using -f From: ambarisha@FreeBSD.org To: svn-soc-all@FreeBSD.org Subject: socsvn commit: r256164 - in soc2013/ambarisha/head/usr.bin: dmget dms MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-soc-all@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: SVN commit messages for the entire Summer of Code repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 Aug 2013 20:39:15 -0000 Author: ambarisha Date: Mon Aug 19 20:39:15 2013 New Revision: 256164 URL: http://svnweb.FreeBSD.org/socsvn/?view=rev&rev=256164 Log: Duplicate request handing. When it receives a request, DMS downloads the requested file into a temporary file. It then uses this file to serve both the original request that started this download and any duplicate requests received in the mean time. Modified: soc2013/ambarisha/head/usr.bin/dmget/utils.c soc2013/ambarisha/head/usr.bin/dms/Makefile soc2013/ambarisha/head/usr.bin/dms/dms.c soc2013/ambarisha/head/usr.bin/dms/dms.h soc2013/ambarisha/head/usr.bin/dms/utils.c soc2013/ambarisha/head/usr.bin/dms/worker.c Modified: soc2013/ambarisha/head/usr.bin/dmget/utils.c ============================================================================== --- soc2013/ambarisha/head/usr.bin/dmget/utils.c Mon Aug 19 20:31:57 2013 (r256163) +++ soc2013/ambarisha/head/usr.bin/dmget/utils.c Mon Aug 19 20:39:15 2013 (r256164) @@ -185,18 +185,27 @@ { int bufsize = 0; int err; - struct dmmsg *msg; + struct dmmsg *msg = (struct dmmsg *) Malloc(sizeof(struct dmmsg)); err = Read(sock, &bufsize, sizeof(bufsize)); if (err == 0) { /* set dms_error */ +#if DEBUG + fprintf(stderr, "recv_msg: remote end closed connection\n"); +#endif + free(msg); return (NULL); } + bufsize -= sizeof(bufsize); err = Read(sock, &(msg->op), sizeof(msg->op)); if (err == 0) { /* set dms_error */ +#if DEBUG + fprintf(stderr, "recv_msg: remote end closed connection\n"); +#endif + free(msg); return (NULL); } bufsize -= sizeof(msg->op); @@ -209,6 +218,10 @@ free(msg->buf); msg->len = 0; /* set dms_error */ +#if DEBUG + fprintf(stderr, "recv_msg: remote end closed connection\n"); +#endif + free(msg); return (NULL); } @@ -218,6 +231,7 @@ void free_msg(struct dmmsg **msg) { + free((*msg)->buf); free(*msg); *msg = NULL; Modified: soc2013/ambarisha/head/usr.bin/dms/Makefile ============================================================================== --- soc2013/ambarisha/head/usr.bin/dms/Makefile Mon Aug 19 20:31:57 2013 (r256163) +++ soc2013/ambarisha/head/usr.bin/dms/Makefile Mon Aug 19 20:39:15 2013 (r256164) @@ -11,5 +11,6 @@ DPADD= ${LIBFETCH} LDADD= -lfetch .endif +CFLAGS+= -g .include Modified: soc2013/ambarisha/head/usr.bin/dms/dms.c ============================================================================== --- soc2013/ambarisha/head/usr.bin/dms/dms.c Mon Aug 19 20:31:57 2013 (r256163) +++ soc2013/ambarisha/head/usr.bin/dms/dms.c Mon Aug 19 20:39:15 2013 (r256164) @@ -200,6 +200,7 @@ dmjob = mk_dmjob(dmreq, csock); jobs = add_job(jobs, dmjob); pthread_create(&(dmjob->worker), NULL, run_worker, dmjob); + pthread_detach(dmjob->worker); break; default: goto error; @@ -211,7 +212,7 @@ error: ret = -1; done: - free_msg(msg); + free_msg(&msg); return ret; } @@ -222,16 +223,15 @@ exit(1); // Temporary } -static int +static state_t service_job(struct dmjob *job, fd_set *fdset) { int ret = 0; - if (FD_ISSET(job->client, fdset)) - pthread_kill(job->worker, SIGUSR1); - - if (job->state == DONE) - ret = 1; - return (ret); + if (FD_ISSET(job->client, fdset)) { + /* TODO: Worker can't handle this signal yet */ + //pthread_kill(job->worker, SIGUSR1); + } + return (job->state); } static void @@ -248,6 +248,7 @@ /* Prepare fdset and make select call */ FD_ZERO(&fdset); + maxfd = socket; FD_SET(socket, &fdset); cur = jobs; @@ -257,37 +258,37 @@ maxfd = cur->client; cur = cur->next; } - + Select(maxfd + 1, &fdset, NULL, NULL, NULL); - if (FD_ISSET(socket, &fdset)) { - struct sockaddr_un cliaddr; - size_t cliaddrlen = sizeof(cliaddr); - int csock = Accept(socket, (struct sockaddr *) &cliaddr, - &cliaddrlen); - handle_request(csock); - } - cur = jobs; while (cur != NULL) { ret = service_job(cur, &fdset); - if (ret == 1) { + if (ret == DONE) { close(cur->client); - pthread_join(cur->worker, &retptr); jobs = rm_job(jobs, cur); } cur = cur->next; } - + + if (FD_ISSET(socket, &fdset)) { + struct sockaddr_un cliaddr; + size_t cliaddrlen = sizeof(cliaddr); + int csock = Accept(socket, (struct sockaddr *) &cliaddr, + &cliaddrlen); + handle_request(csock); + } } + /* Notify all running workers that we've to wrap up */ cur = jobs; - while (cur != NULL) { - close(cur->client); - ret = service_job(cur, &fdset); - /* TODO: Force the worker to quit as well */ + while (cur != NULL) { + if (cur->state == RUNNING) + pthread_kill(cur->worker, SIGINT); + + rm_dmreq(&(cur->request)); jobs = rm_job(jobs, cur); - cur = jobs; + cur = cur->next; } } Modified: soc2013/ambarisha/head/usr.bin/dms/dms.h ============================================================================== --- soc2013/ambarisha/head/usr.bin/dms/dms.h Mon Aug 19 20:31:57 2013 (r256163) +++ soc2013/ambarisha/head/usr.bin/dms/dms.h Mon Aug 19 20:39:15 2013 (r256164) @@ -3,7 +3,7 @@ #include -typedef enum {RUNNING=0, DONE=1} state_t; +typedef enum {RUNNING=0, DONE, DUPLICATE} state_t; struct dmjob { int ofd; @@ -13,9 +13,10 @@ int sigalrm; int siginfo; int siginfo_en; + unsigned timeout; + pthread_t worker; struct dmreq *request; struct url *url; - pthread_t *worker; struct dmjob *next; struct dmjob *prev; Modified: soc2013/ambarisha/head/usr.bin/dms/utils.c ============================================================================== --- soc2013/ambarisha/head/usr.bin/dms/utils.c Mon Aug 19 20:31:57 2013 (r256163) +++ soc2013/ambarisha/head/usr.bin/dms/utils.c Mon Aug 19 20:39:15 2013 (r256164) @@ -183,30 +183,29 @@ struct dmmsg * recv_msg(int sock) { - printf("in recv_msg\n"); int bufsize = 0; int err; - struct dmmsg *msg; + struct dmmsg *msg = (struct dmmsg *) Malloc(sizeof(struct dmmsg)); err = Read(sock, &bufsize, sizeof(bufsize)); if (err == 0) { /* set dms_error */ #if DEBUG fprintf(stderr, "recv_msg: remote end closed connection\n"); #endif + free(msg); return (NULL); } - printf("bufsize = %d\n", bufsize); bufsize -= sizeof(bufsize); - printf("sock = %d\n", sock); err = Read(sock, &(msg->op), sizeof(msg->op)); if (err == 0) { /* set dms_error */ #if DEBUG fprintf(stderr, "recv_msg: remote end closed connection\n"); #endif + free(msg); return (NULL); } bufsize -= sizeof(msg->op); @@ -222,6 +221,7 @@ #if DEBUG fprintf(stderr, "recv_msg: remote end closed connection\n"); #endif + free(msg); return (NULL); } @@ -231,6 +231,7 @@ void free_msg(struct dmmsg **msg) { + free((*msg)->buf); free(*msg); *msg = NULL; Modified: soc2013/ambarisha/head/usr.bin/dms/worker.c ============================================================================== --- soc2013/ambarisha/head/usr.bin/dms/worker.c Mon Aug 19 20:31:57 2013 (r256163) +++ soc2013/ambarisha/head/usr.bin/dms/worker.c Mon Aug 19 20:39:15 2013 (r256164) @@ -6,12 +6,17 @@ #include #include #include +#include +#include #include "dms.h" #include "dm.h" + extern struct dmjob *jobs; +#define TMP_EXT ".tmp" + static int authenticate(struct url *url) { @@ -53,6 +58,12 @@ free_msg(&rcvmsg); } +static int +compare_jobs(struct dmjob *j1, struct dmjob *j2) +{ + return strcmp(j1->request->URL, j2->request->URL); +} + static void stat_send(int csock, struct xferstat *xs, int force) { @@ -143,29 +154,14 @@ } static int -fetch(struct dmjob *dmjob) +mk_url(struct dmjob *dmjob, char *flags) { - struct url_stat us; - struct stat sb, nsb; - struct xferstat xs; - FILE *f, *of; - size_t size, readcnt, wr; - off_t count; - char flags[8]; - const char *slash; - char *tmppath; - int r; - unsigned timeout; - char *ptr; - char *buf; struct dmreq *dmreq = dmjob->request; + struct stat sb; + int r; - f = of = NULL; - tmppath = NULL; - - timeout = 0; - *flags = 0; - count = 0; + if (dmjob->url != NULL) + return 0; /* set verbosity level */ if (dmreq->v_level > 1) @@ -174,7 +170,6 @@ fetchDebug = 1; /* parse URL */ - dmjob->url = NULL; if (*dmreq->URL == '\0') { warnx("empty URL"); goto failure; @@ -212,7 +207,7 @@ strcat(flags, "d"); if (dmreq->flags & U_FLAG) strcat(flags, "l"); - timeout = dmreq->T_secs ? dmreq->T_secs : dmreq->ftp_timeout; + dmjob->timeout = dmreq->T_secs ? dmreq->T_secs : dmreq->ftp_timeout; } /* HTTP specific flags */ @@ -222,7 +217,7 @@ strcat(flags, "d"); if ((dmreq->flags & A_FLAG)) strcat(flags, "A"); - timeout = dmreq->T_secs ? dmreq->T_secs : dmreq->http_timeout; + dmjob->timeout = dmreq->T_secs ? dmreq->T_secs : dmreq->http_timeout; if (dmreq->flags & i_FLAG) { if (stat(dmreq->i_filename, &sb)) { warn("%s: stat()", dmreq->i_filename); @@ -234,27 +229,58 @@ } /* set the protocol timeout. */ - fetchTimeout = timeout; + fetchTimeout = dmjob->timeout; + goto success; - /* just print size */ - if (dmreq->flags & s_FLAG) { - // if (timeout) - // alarm(timeout); - r = fetchStat(dmjob->url, &us, flags); - if (timeout) - alarm(0); - if (dmjob->sigalrm || dmjob->sigint) - goto signal; - if (r == -1) { - warnx("%s", fetchLastErrString); - goto failure; - } - if (us.size == -1) - printf("Unknown\n"); - else - printf("%jd\n", (intmax_t)us.size); - goto success; - } +signal: + /* report that we were timedout/interrupted */ +failure: + free(dmjob->url->doc); + free(dmjob->url); + dmjob->url = NULL; +success: + return (r); +} + +static int +fetch(struct dmjob *dmjob, FILE *f, struct url_stat us) +{ + struct stat sb, nsb; + struct xferstat xs; + FILE *of; + size_t size, readcnt, wr; + off_t count; + char flags[8]; + const char *slash; + char *tmppath; + int r; + char *ptr; + char *buf; + struct dmreq *dmreq = dmjob->request; + + of = NULL; + tmppath = NULL; + + dmjob->timeout = 0; + *flags = 0; + count = 0; + + r = mk_url(dmjob, flags); + + /* Initialize signal flags */ + dmjob->sigint = 0; + dmjob->sigalrm = 0; + dmjob->siginfo = 0; + + /* Set timeout */ + if (strcmp(dmjob->url->scheme, SCHEME_FTP) == 0) + dmjob->timeout = dmreq->T_secs ? dmreq->T_secs : dmreq->ftp_timeout; + else if (strcmp(dmjob->url->scheme, SCHEME_HTTP) == 0 || + strcmp(dmjob->url->scheme, SCHEME_HTTPS) == 0) + dmjob->timeout = dmreq->T_secs ? dmreq->T_secs : dmreq->http_timeout; + + /* set the protocol timeout. */ + fetchTimeout = dmjob->timeout; /* * If the -r flag was specified, we have to compare the local @@ -289,24 +315,12 @@ } /* start the transfer */ - if (timeout) - alarm(timeout); - f = fetchXGet(dmjob->url, &us, flags); - if (timeout) - alarm(0); + if (dmjob->timeout) + alarm(dmjob->timeout); + if (dmjob->sigalrm || dmjob->sigint) goto signal; - if (f == NULL) { - warnx("%s: %s", dmreq->URL, fetchLastErrString); - if ((dmreq->flags & i_FLAG) && strcmp(dmjob->url->scheme, SCHEME_HTTP) == 0 - && fetchLastErrCode == FETCH_OK - && strcmp(fetchLastErrString, "Not Modified") == 0) { - /* HTTP Not Modified Response, return OK. */ - r = 0; - goto done; - } else - goto failure; - } + if (dmjob->sigint) goto signal; @@ -422,14 +436,11 @@ * from scratch if we want the whole file */ dmjob->url->offset = 0; - if ((f = fetchXGet(dmjob->url, &us, flags)) == NULL) { - warnx("%s: %s", dmreq->URL, fetchLastErrString); - goto failure; - } if (dmjob->sigint) goto signal; } + /* construct a temp file name */ if (sb.st_size != -1 && S_ISREG(sb.st_mode)) { if ((slash = strrchr(dmreq->path, '/')) == NULL) @@ -448,6 +459,7 @@ chmod(tmppath, sb.st_mode & ALLPERMS); } } + if (of == NULL) of = fdopen(dmjob->ofd, "w"); if (of == NULL) { @@ -485,10 +497,12 @@ if ((readcnt = fread(buf, 1, size, f)) < size) { if (ferror(f) && errno == EINTR && !dmjob->sigint) clearerr(f); - else if (readcnt == 0) + else if (readcnt == 0) { break; + } } + stat_update(&xs, count += readcnt, dmjob); for (ptr = buf; readcnt > 0; ptr += wr, readcnt -= wr) if ((wr = fwrite(ptr, 1, readcnt, of)) < readcnt) { @@ -500,6 +514,7 @@ if (readcnt != 0) break; } + if (!dmjob->sigalrm) dmjob->sigalrm = ferror(f) && errno == ETIMEDOUT; dmjob->siginfo_en = 0; @@ -589,9 +604,89 @@ fetchFreeURL(dmjob->url); if (tmppath != NULL) free(tmppath); + return (r); } +FILE * +dmXGet(struct dmjob *dmjob, struct url_stat *us) +{ + char flags[8]; + int ret; + struct dmjob tmpjob; + struct dmreq tmpreq; + struct dmreq *dmreq = dmjob->request; + + /* populate tmpjob */ + + /* TODO : Modify stat_* to udpate jobs of progress, + * right now we just put the msgs on stderr + * */ + tmpjob.client = STDERR_FILENO; + + tmpjob.request = &tmpreq; + tmpreq.v_level = dmreq->v_level; + tmpreq.ftp_timeout = dmjob->request->ftp_timeout; + tmpreq.http_timeout = dmjob->request->http_timeout; + tmpreq.B_size = dmjob->request->B_size; + tmpreq.S_size = dmjob->request->S_size; + tmpreq.T_secs = dmjob->request->T_secs; + tmpreq.flags = dmjob->request->flags; + tmpreq.family = dmjob->request->family; + + tmpreq.i_filename = (char *) Malloc(strlen(dmreq->i_filename)); + strcpy(tmpreq.i_filename, dmreq->i_filename); + + tmpreq.URL = (char *) Malloc(strlen(dmreq->URL)); + strcpy(tmpreq.URL, dmreq->URL); + + tmpjob.url = NULL; + ret = mk_url(&tmpjob, flags); + if (ret <= 0) { + + } + + /* special case : -s flag + if (tmpreq.flags & s_FLAG) { + if (dmjob->timeout) + alarm(dmjob->timeout); + r = fetchStat(dmjob->url, us, flags); + if (dmjob->timeout) + alarm(0); + if (dmjob->sigalrm || dmjob->sigint) + goto signal; + if (r == -1) { + warnx("%s", fetchLastErrString); + goto failure; + } + if (us->size == -1) + printf("Unknown\n"); + else + printf("%jd\n", (intmax_t)us->size); + goto success; + } + */ + tmpreq.path = (char *) Malloc(strlen(dmreq->path) + strlen(TMP_EXT)); + strcpy(tmpreq.path, dmreq->path); + strcat(tmpreq.path, TMP_EXT); + + tmpjob.ofd = open(tmpreq.path, O_RDWR | O_CREAT | O_TRUNC, S_IRUSR | S_IWUSR); + + FILE *f = fetchXGet(tmpjob.url, us, flags); + fetch(&tmpjob, f, *us); + fclose(f); + + f = fopen(tmpreq.path, "r"); + + free(tmpjob.url->doc); + free(tmpjob.url); + free(tmpreq.i_filename); + free(tmpreq.URL); + free(tmpreq.path); + + return f; +} + static void send_report(int sock, struct dmrep report, char op) { @@ -621,6 +716,9 @@ free(buf); } +/* TODO: This handler isn't registered as SIGUSR1 interrupts the download + * Figure out a proper way to handle this + * */ void sig_handler(int sig) { @@ -630,7 +728,7 @@ pthread_t tid = pthread_self(); if (sig == SIGUSR1) { while (tmp != NULL) { - if (pthread_equal(tid, *(tmp->worker)) != 0) + if (pthread_equal(tid, tmp->worker) != 0) break; tmp = tmp->next; } @@ -641,8 +739,9 @@ tmp->sigint = 1; else if (*clisig == SIGINFO) tmp->siginfo = 1; - else if (*clisig == SIGALRM) - tmp->siginfo = 1; + else if (*clisig == SIGALRM) { + tmp->sigalrm = 1; + } } } @@ -650,13 +749,61 @@ run_worker(struct dmjob *dmjob) { struct dmrep report; + struct dmjob *tmp; + struct url_stat us; + int err; + FILE *f; + char *tmppath; + char flags[8]; + + /* check if this is a duplicate */ + mk_url(dmjob, flags); + tmp = jobs; + while (tmp != NULL) { + if (tmp != dmjob && compare_jobs(tmp, dmjob) == 0) { + dmjob->state = DUPLICATE; + dmjob->worker = tmp->worker; + return NULL; + } + tmp = tmp->next; + } + + /* fetch the remote file into a local tmp file */ + f = dmXGet(dmjob, &us); + + /* Serve any outstanding requests from the local tmp file */ + tmp = jobs; + while (tmp != NULL) { + if (compare_jobs(tmp, dmjob) != 0) { + tmp = tmp->next; + continue; + } - int err = fetch(dmjob); - report.status = err; - report.errcode = fetchLastErrCode; - report.errstr = fetchLastErrString; - send_report(dmjob->client, report, DMRESP); - dmjob->state = RUNNING; + if (f == NULL) { + err = -1; + } else { + fseek(f, 0, SEEK_SET); + err = fetch(dmjob, f, us); + } + + report.status = err; + report.errcode = fetchLastErrCode; + report.errstr = fetchLastErrString; + send_report(dmjob->client, report, DMRESP); + + tmp->state = DONE; + tmp = tmp->next; + } + /* remove the local tmp file */ + if (f != NULL) { + tmppath = (char *) Malloc(strlen(dmjob->request->path) + strlen(TMP_EXT)); + strcpy(tmppath, dmjob->request->path); + strcat(tmppath, TMP_EXT); - return NULL; + remove(tmppath); + free(tmppath); + } + + /* TODO : What is this? Yew!! */ + sleep(10); } From owner-svn-soc-all@FreeBSD.ORG Mon Aug 19 20:41:32 2013 Return-Path: Delivered-To: svn-soc-all@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 66318C6A for ; Mon, 19 Aug 2013 20:41:32 +0000 (UTC) (envelope-from ambarisha@FreeBSD.org) Received: from socsvn.freebsd.org (socsvn.freebsd.org [IPv6:2001:1900:2254:206a::50:2]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 51A4527AC for ; Mon, 19 Aug 2013 20:41:32 +0000 (UTC) Received: from socsvn.freebsd.org ([127.0.1.124]) by socsvn.freebsd.org (8.14.7/8.14.7) with ESMTP id r7JKfWeH027488 for ; Mon, 19 Aug 2013 20:41:32 GMT (envelope-from ambarisha@FreeBSD.org) Received: (from www@localhost) by socsvn.freebsd.org (8.14.7/8.14.6/Submit) id r7JKfWKa027484 for svn-soc-all@FreeBSD.org; Mon, 19 Aug 2013 20:41:32 GMT (envelope-from ambarisha@FreeBSD.org) Date: Mon, 19 Aug 2013 20:41:32 GMT Message-Id: <201308192041.r7JKfWKa027484@socsvn.freebsd.org> X-Authentication-Warning: socsvn.freebsd.org: www set sender to ambarisha@FreeBSD.org using -f From: ambarisha@FreeBSD.org To: svn-soc-all@FreeBSD.org Subject: socsvn commit: r256165 - soc2013/ambarisha/head/usr.bin/dms MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-soc-all@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: SVN commit messages for the entire Summer of Code repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 Aug 2013 20:41:32 -0000 Author: ambarisha Date: Mon Aug 19 20:41:32 2013 New Revision: 256165 URL: http://svnweb.FreeBSD.org/socsvn/?view=rev&rev=256165 Log: Cleaner error reporting in worker Modified: soc2013/ambarisha/head/usr.bin/dms/dms.c soc2013/ambarisha/head/usr.bin/dms/utils.c soc2013/ambarisha/head/usr.bin/dms/worker.c Modified: soc2013/ambarisha/head/usr.bin/dms/dms.c ============================================================================== --- soc2013/ambarisha/head/usr.bin/dms/dms.c Mon Aug 19 20:39:15 2013 (r256164) +++ soc2013/ambarisha/head/usr.bin/dms/dms.c Mon Aug 19 20:41:32 2013 (r256165) @@ -14,45 +14,14 @@ #include "dm.h" #include "dms.h" +static int dm_err; +static char dm_errstr[512]; + int stop; struct dmjob *jobs; void *run_worker(struct dmjob *job); -static struct dmjob * -add_job(struct dmjob *head, struct dmjob *new) -{ - new->prev = NULL; - new->next = NULL; - - if (head == NULL) - return new; - - head->prev = new; - new->next = head; -} - -static struct dmjob * -rm_job(struct dmjob *head, struct dmjob *job) -{ - if (head == NULL) - return NULL; - - if (job == NULL) - return head; - - if (job->next != NULL) - job->next->prev = job->prev; - - if (job->prev != NULL) - job->prev->next = job->next; - - if (job == head) - return job->next; - - return head; -} - static int read_fd(int sock) { @@ -79,19 +48,34 @@ msg.msg_iov = iov; msg.msg_iovlen = 1; - if ( (n = recvmsg(sock, &msg, 0)) <= 0) - return (n); + if ( (n = recvmsg(sock, &msg, 0)) <= 0) { + strcpy(dm_errstr, "Couldn't recieve output file descriptor"); + fprintf(stderr, "read_fd: recvmsg: %s\n", strerror(errno)); + return -1; + } if ( (cmptr = CMSG_FIRSTHDR(&msg)) != NULL && cmptr->cmsg_len == CMSG_LEN(sizeof(int))) { - if (cmptr->cmsg_level != SOL_SOCKET) - /* ERROR : control level != SOL_SOCKET */; + if (cmptr->cmsg_level != SOL_SOCKET) { + strcpy(dm_errstr, "Couldn't recieve" + "output file descriptor"); + fprintf(stderr, "read_fd: recvmsg:" + "control level != SOL_SOCKET\n"); + return -1; + } - if (cmptr->cmsg_type != SCM_RIGHTS) - /* ERROR : control type != SCM_RIGHTS */; + if (cmptr->cmsg_type != SCM_RIGHTS) { + strcpy(dm_errstr, "Couldn't recieve" + "output file descriptor"); + fprintf(stderr, "read_fd: recvmsg:" + "control type != SCM_RIGHTS\n"); + return -1; + } newfd = *((int *) CMSG_DATA(cmptr)); } else { + strcpy(dm_errstr, "Couldn't recieve output file descriptor"); + fprintf(stderr, "read_fd: Invalid control message header\n"); newfd = -1; } @@ -99,16 +83,50 @@ } static struct dmjob * +add_job(struct dmjob *head, struct dmjob *new) +{ + new->prev = NULL; + new->next = NULL; + + if (head == NULL) + return new; + + head->prev = new; + new->next = head; +} + +static struct dmjob * +rm_job(struct dmjob *head, struct dmjob *job) +{ + if (job->next != NULL) + job->next->prev = job->prev; + + if (job->prev != NULL) + job->prev->next = job->next; + + if (job == head) + return job->next; + + return head; +} + +static struct dmjob * mk_dmjob(struct dmreq *dmreq, int client) { - struct dmjob *dmjob = (struct dmjob *) Malloc(sizeof(struct dmjob)); + struct dmjob *dmjob = (struct dmjob *) malloc(sizeof(struct dmjob)); + if (dmjob == NULL) { + fprintf(stderr, "mk_dmjob: malloc: insufficient memory\n"); + strcpy(dm_errstr, "Insufficient memory\n"); + } + dmjob->request = dmreq; + dmjob->ofd = read_fd(client); if (dmjob->ofd == -1) { - /* Handle error */ free(dmjob); return NULL; } + dmjob->client = client; dmjob->sigint = 0; dmjob->sigalrm = 0; @@ -124,9 +142,13 @@ { int i = 0; - struct dmreq *dmreq = (struct dmreq *) Malloc(sizeof(struct dmreq)); - if (dmreq == NULL) + struct dmreq *dmreq = (struct dmreq *) malloc(sizeof(struct dmreq)); + if (dmreq == NULL) { + fprintf(stderr, "mk_dmreq: malloc: insufficient memory\n"); + strcpy(dm_errstr, "Insufficient memory"); return NULL; + } + memcpy(&(dmreq->v_level), rcvbuf + i, sizeof(dmreq->v_level)); i += sizeof(dmreq->v_level); @@ -152,17 +174,38 @@ i += sizeof(dmreq->flags); int sz = strlen(rcvbuf+i); - dmreq->i_filename = (char *) Malloc(sz); + dmreq->i_filename = (char *) malloc(sz); + if (dmreq->i_filename == NULL) { + fprintf(stderr, "mk_dmreq: malloc: insufficient memory\n"); + strcpy(dm_errstr, "Insufficient memory"); + free(dmreq); + return NULL; + } strcpy(dmreq->i_filename, rcvbuf+i); i += sz + 1; sz = strlen(rcvbuf+i); - dmreq->URL = (char *) Malloc(sz); + dmreq->URL = (char *) malloc(sz); + if (dmreq->URL == NULL) { + fprintf(stderr, "mk_dmreq: malloc: insufficient memory\n"); + strcpy(dm_errstr, "Insufficient memory"); + free(dmreq->i_filename); + free(dmreq); + return NULL; + } strcpy(dmreq->URL, rcvbuf+i); i += sz + 1; sz = strlen(rcvbuf+i); - dmreq->path = (char *) Malloc(sz); + dmreq->path = (char *) malloc(sz); + if (dmreq->path == NULL) { + fprintf(stderr, "mk_dmreq: malloc: insufficient memory\n"); + strcpy(dm_errstr, "Insufficient memory"); + free(dmreq->i_filename); + free(dmreq->URL); + free(dmreq); + return NULL; + } strcpy(dmreq->path, rcvbuf+i); i += sz + 1; @@ -172,6 +215,9 @@ static void rm_dmreq(struct dmreq **dmreq) { + if (*dmreq == NULL) + return; + free((*dmreq)->i_filename); free((*dmreq)->URL); free((*dmreq)->path); @@ -179,40 +225,62 @@ *dmreq = NULL; } +static void +rm_dmjob(struct dmjob **dmjob) +{ + if (*dmjob == NULL) + return; + rm_dmreq(&((*dmjob)->request)); + free((*dmjob)->url); +} + static int handle_request(int csock) { struct dmreq *dmreq; struct dmmsg *msg; struct dmjob *dmjob; + struct dmrep report; int ret; pid_t pid; - msg = recv_msg(csock); + msg = recv_dmmsg(csock); if (msg == NULL) { - /* set dms_error */ + report.status = -1; + report.errcode = FETCH_UNKNOWN; + report.errstr = dm_errstr; + send_report(csock, report); return -1; } switch (msg->op) { case DMREQ: - dmreq = mk_dmreq(msg->buf, msg->len); - dmjob = mk_dmjob(dmreq, csock); + if ((dmreq = mk_dmreq(msg->buf, msg->len)) == NULL) + goto error; + + if ((dmjob = mk_dmjob(dmreq, csock)) == NULL) + goto error; + jobs = add_job(jobs, dmjob); pthread_create(&(dmjob->worker), NULL, run_worker, dmjob); pthread_detach(dmjob->worker); - break; + goto done; default: + free_dmmsg(&msg); goto error; - break; } -success: - ret = 0; - goto done; + error: + report.status = -1; + report.errcode = FETCH_UNKNOWN; + report.errstr = dm_errstr; + send_report(csock, report); + + rm_dmreq(&dmreq); + rm_dmjob(&dmjob); ret = -1; done: - free_msg(&msg); + free_dmmsg(&msg); return ret; } @@ -238,6 +306,7 @@ run_event_loop(int socket) { int i, ret, maxfd = socket; + state_t state; struct dmjob *cur; void *retptr; jobs = NULL; @@ -259,14 +328,20 @@ cur = cur->next; } - Select(maxfd + 1, &fdset, NULL, NULL, NULL); + ret = select(maxfd + 1, &fdset, NULL, NULL, NULL); + if (ret == -1) { + fprintf(stderr, "run_event_loop: " + "select: %s\n", strerror(errno)); + goto wrap_up; + } cur = jobs; while (cur != NULL) { - ret = service_job(cur, &fdset); - if (ret == DONE) { + state = service_job(cur, &fdset); + if (state == DONE) { close(cur->client); jobs = rm_job(jobs, cur); + rm_dmjob(&cur); } cur = cur->next; } @@ -274,12 +349,18 @@ if (FD_ISSET(socket, &fdset)) { struct sockaddr_un cliaddr; size_t cliaddrlen = sizeof(cliaddr); - int csock = Accept(socket, (struct sockaddr *) &cliaddr, + int csock = accept(socket, (struct sockaddr *) &cliaddr, &cliaddrlen); + if (csock == -1) { + fprintf(stderr, "run_event_loop: " + "select: %s\n", strerror(errno)); + goto wrap_up; + } handle_request(csock); } } +wrap_up: /* Notify all running workers that we've to wrap up */ cur = jobs; while (cur != NULL) { @@ -294,13 +375,30 @@ int main(int argc, char **argv) { - int sock = Socket(AF_UNIX, SOCK_STREAM, 0); + int sock, err; + sock = socket(AF_UNIX, SOCK_STREAM, 0); + if (sock == -1) { + fprintf(stderr, "main: socket: %s\n", strerror(errno)); + exit(1); + } + struct sockaddr_un sunaddr; sunaddr.sun_family = AF_UNIX; strcpy(sunaddr.sun_path, DMS_UDS_PATH); - int err = Bind(sock, (struct sockaddr *)&sunaddr, sizeof(sunaddr)); - err = Listen(sock, MAX_LISTEN_QUEUE); + err = bind(sock, (struct sockaddr *)&sunaddr, sizeof(sunaddr)); + if (err == -1) { + fprintf(stderr, "main: bind: %s\n", strerror(errno)); + close(sock); + exit(1); + } + + err = listen(sock, MAX_LISTEN_QUEUE); + if (err == -1) { + fprintf(stderr, "main: listen: %s\n", strerror(errno)); + close(sock); + exit(1); + } run_event_loop(sock); Modified: soc2013/ambarisha/head/usr.bin/dms/utils.c ============================================================================== --- soc2013/ambarisha/head/usr.bin/dms/utils.c Mon Aug 19 20:39:15 2013 (r256164) +++ soc2013/ambarisha/head/usr.bin/dms/utils.c Mon Aug 19 20:41:32 2013 (r256165) @@ -4,165 +4,22 @@ #include "dm.h" -void * -Malloc(size_t size) -{ - void *ptr = malloc(size); - if (ptr == NULL) { - perror("Malloc():"); - /* Notifiy ENOMEM and exit gracefully */ - } - return ptr; -} - -ssize_t -Read(int fd, void *buf, size_t size) -{ - ssize_t err = read(fd, buf, size); - if (err == -1) { - perror("Read(): "); - } else if (err != size) { - printf("Warning: %d bytes received %d expected", err, size); - } else { - printf("Read() : Success\n"); - } - - return err; -} - -int -Socket(int domain, int type, int flags) -{ - int err = socket(domain, type, flags); - if (err == -1) { - perror("Socket():"); - } else { - printf("Socket(): Success\n"); - } - - return err; -} - -int -Write(int fd, void *buf, size_t size) -{ - int err = write(fd, buf, size); - if (err == -1) { - perror("Write():"); - } else { - printf("Write(): Success\n"); - } - return err; -} - -int -Socketpair(int domain, int type, int protocol, int socket_vector[2]) -{ - int err = socketpair(domain, type, protocol, socket_vector); - if (err == -1) { - perror("Socketpair():"); - } else { - printf("Socketpair() : Success\n"); - } - return err; -} - -int -Bind(int socket, const struct sockaddr *addr, socklen_t addrlen) -{ - int err = bind(socket, addr, addrlen); - if (err == -1) { - perror("Bind():"); - } else { - printf("Bind() : Success\n"); - } - return err; -} - -int -Accept(int socket, struct sockaddr *addr, socklen_t *addrlen) -{ - int err = accept(socket, addr, addrlen); - if (err == -1) { - perror("Accept():"); - } else { - printf("Accept() : Success\n"); - } - return err; -} - -int -Connect(int socket, struct sockaddr *addr, socklen_t addrlen) -{ - int err = connect(socket, addr, addrlen); - if (err == -1) { - perror("Connect():"); - } else { - printf("Connect() : Success\n"); - } - return err; -} - -int -Listen(int socket, int backlog) -{ - int err = listen(socket, backlog); - if (err == -1) { - perror("Listen():"); - } else { - printf("Listen() : Success\n"); - } - return err; -} - -int -Peel(int sock, struct dmmsg *msg) -{ - int bufsize = 0; - int err; - err = Read(sock, &bufsize, sizeof(bufsize)); - if (err == 0) - return (err); - bufsize -= sizeof(bufsize); - - err = Read(sock, &(msg->op), sizeof(msg->op)); - if (err == 0) - return (err); - bufsize -= sizeof(msg->op); +/* Utils for handling messages */ - msg->buf = (char *) Malloc(bufsize); - msg->len = bufsize; - - err = Read(sock, msg->buf, bufsize); - if (err == 0) { - free(msg->buf); - msg->len = 0; - } - - return bufsize; -} - -int -Select(int maxfd, fd_set *rset, fd_set *wset, fd_set *xset, - const struct timeval *timeout) -{ - int err = select(maxfd, rset, wset, xset, timeout); - if (err == -1) { - perror("Select():"); - } else { - printf("Select(): Success\n"); - } - return err; -} +extern char *dm_errstr; int -send_msg(int socket, struct dmmsg msg) +send_dmmsg(int socket, struct dmmsg msg) { int bufsize = sizeof(bufsize); // Buffer size bufsize += 1; // Op bufsize += msg.len; // Signal number - char *sndbuf = (char *) Malloc(bufsize); + char *sndbuf = (char *) malloc(bufsize); + if (sndbuf == NULL) { + fprintf(stderr, "send_dmmsg: malloc: insufficient memory\n"); + return -1; + } int i = 0; memcpy(sndbuf + i, &bufsize, sizeof(bufsize)); @@ -174,64 +31,87 @@ memcpy(sndbuf + i, msg.buf, msg.len); i += msg.len; - int nbytes = Write(socket, sndbuf, bufsize); + int nbytes = write(socket, sndbuf, bufsize); free(sndbuf); + + if (nbytes == -1) { + fprintf(stderr, "send_dmmsg: write: %s\n", + strerror(errno)); + } return (nbytes); } struct dmmsg * -recv_msg(int sock) +recv_dmmsg(int sock) { int bufsize = 0; int err; - struct dmmsg *msg = (struct dmmsg *) Malloc(sizeof(struct dmmsg)); - err = Read(sock, &bufsize, sizeof(bufsize)); + struct dmmsg *msg = (struct dmmsg *) malloc(sizeof(struct dmmsg)); + if (msg == NULL) { + fprintf(stderr, "send_dmmsg: malloc: insufficient memory\n"); + return -1; + } + + err = read(sock, &bufsize, sizeof(bufsize)); if (err == 0) { - /* set dms_error */ -#if DEBUG - fprintf(stderr, "recv_msg: remote end closed connection\n"); -#endif - free(msg); - return (NULL); + fprintf(stderr, "recv_dmmsg: remote end" + " closed connection\n"); + goto error; + } else if (err == -1) { + fprintf(stderr, "recv_dmmsg: %s\n", strerror(errno)); + goto error; } bufsize -= sizeof(bufsize); - err = Read(sock, &(msg->op), sizeof(msg->op)); + err = read(sock, &(msg->op), sizeof(msg->op)); if (err == 0) { - /* set dms_error */ -#if DEBUG - fprintf(stderr, "recv_msg: remote end closed connection\n"); -#endif - free(msg); - return (NULL); + fprintf(stderr,stderr, "recv_dmmsg: remote end" + " closed connection\n"); + goto error; + } else if (err == -1) { + fprintf(stderr, "recv_dmmsg: %s\n", strerror(errno)); + goto error; } + bufsize -= sizeof(msg->op); - msg->buf = (char *) Malloc(bufsize); + msg->buf = (char *) malloc(bufsize); + if (msg == NULL) { + fprintf(stderr, "send_dmmsg: malloc: insufficient memory\n"); + goto error; + } + msg->len = bufsize; - err = Read(sock, msg->buf, bufsize); + err = read(sock, msg->buf, bufsize); if (err == 0) { - free(msg->buf); msg->len = 0; - /* set dms_error */ -#if DEBUG - fprintf(stderr, "recv_msg: remote end closed connection\n"); -#endif + fprintf(stderr,stderr, "recv_dmmsg: remote end" + " closed connection\n"); + free(msg->buf); free(msg); return (NULL); + } else if (err == -1) { + fprintf(stderr, "recv_dmmsg: %s\n", strerror(errno)); + free(msg->buf); + goto error; } + return msg; +error: + free(msg); + return NULL; } void -free_msg(struct dmmsg **msg) +free_dmmsg(struct dmmsg **msg) { - + if (*msg == NULL) + return; free((*msg)->buf); free(*msg); *msg = NULL; Modified: soc2013/ambarisha/head/usr.bin/dms/worker.c ============================================================================== --- soc2013/ambarisha/head/usr.bin/dms/worker.c Mon Aug 19 20:39:15 2013 (r256164) +++ soc2013/ambarisha/head/usr.bin/dms/worker.c Mon Aug 19 20:41:32 2013 (r256165) @@ -13,6 +13,7 @@ #include "dm.h" +static const char *prefixes = " kMGTP"; extern struct dmjob *jobs; #define TMP_EXT ".tmp" @@ -23,6 +24,7 @@ struct dmmsg msg; struct dmjob *cur = jobs; while (cur != NULL) { + /* TODO: May be a more thorough comparison? */ if (cur->url == url) break; cur = cur->next; @@ -36,7 +38,11 @@ hlen = strlen(url->host) + 1; bufsize += schlen + hlen + sizeof(url->port); - msg.buf = (char *) Malloc(bufsize); + msg.buf = (char *) malloc(bufsize); + if (msg.buf == NULL) { + fprintf(stderr, "fetch: authenticate: Insufficient memory\n"); + return -1; + } strcpy(msg.buf, url->scheme); i += schlen; @@ -48,14 +54,24 @@ msg.op = DMAUTHREQ; msg.len = bufsize; - send_msg(cur->client, msg); + ret = send_dmmsg(cur->client, msg); + if (ret == -1) { + free(msg.buf); + return -1; + } struct dmmsg *rcvmsg; - rcvmsg = recv_msg(cur->client); + rcvmsg = recv_dmmsg(cur->client); + if (rcvmsg == NULL) { + free(msg.buf); + return -1; + } strncpy(url->user, rcvmsg->buf, sizeof(url->user)); strncpy(url->pwd, rcvmsg->buf + strlen(rcvmsg->buf) + 1, sizeof(url->pwd)); - free_msg(&rcvmsg); + free_dmmsg(&rcvmsg); + + return 1; // TODO: Verify this } static int @@ -67,7 +83,12 @@ static void stat_send(int csock, struct xferstat *xs, int force) { - char *buf = (char *) Malloc(sizeof(struct xferstat) + sizeof(force)); + char *buf = (char *) malloc(sizeof(struct xferstat) + sizeof(force)); + if (buf == NULL) { + fprintf(stderr, "stat_send: Insufficient memory\n"); + return; + } + *((int *) buf) = force; memcpy(buf + sizeof(force), xs, sizeof(struct xferstat)); @@ -76,13 +97,11 @@ msg.op = DMSTAT; msg.buf = buf; msg.len = sizeof(*xs) + sizeof(force); - send_msg(csock, msg); - - free(msg.buf); + send_dmmsg(csock, msg); + free(buf); return; } -static const char *prefixes = " kMGTP"; static const char * stat_bytes(off_t bytes) { @@ -171,7 +190,7 @@ /* parse URL */ if (*dmreq->URL == '\0') { - warnx("empty URL"); + fprintf(stderr, "warning: mk_url: URL empty\n"); goto failure; } if ((dmjob->url = fetchParseURL(dmreq->URL)) == NULL) { @@ -230,6 +249,7 @@ /* set the protocol timeout. */ fetchTimeout = dmjob->timeout; + r = 0; goto success; signal: @@ -238,6 +258,7 @@ free(dmjob->url->doc); free(dmjob->url); dmjob->url = NULL; + r = -1; success: return (r); } @@ -266,6 +287,8 @@ count = 0; r = mk_url(dmjob, flags); + if (r == -1) + return -1; /* Initialize signal flags */ dmjob->sigint = 0; @@ -476,7 +499,7 @@ if (dmreq->B_size < MINBUFSIZE) dmreq->B_size = MINBUFSIZE; - buf = (char *) Malloc(dmreq->B_size); + buf = (char *) malloc(dmreq->B_size); /* suck in the data */ dmjob->siginfo_en = 1; @@ -634,16 +657,29 @@ tmpreq.flags = dmjob->request->flags; tmpreq.family = dmjob->request->family; - tmpreq.i_filename = (char *) Malloc(strlen(dmreq->i_filename)); + tmpreq.i_filename = (char *) malloc(strlen(dmreq->i_filename)); + if (tmpreq.i_filename == NULL) { + fprintf(stderr, "dmXGet: Insufficient memory\n"); + return NULL; + } + strcpy(tmpreq.i_filename, dmreq->i_filename); - tmpreq.URL = (char *) Malloc(strlen(dmreq->URL)); + tmpreq.URL = (char *) malloc(strlen(dmreq->URL)); + if (tmpreq.URL == NULL) { + fprintf(stderr, "dmXGet: Insufficient memory\n"); + free(tmpreq.i_filename); + return NULL; + } + strcpy(tmpreq.URL, dmreq->URL); tmpjob.url = NULL; ret = mk_url(&tmpjob, flags); - if (ret <= 0) { - + if (ret == -1) { + free(tmpreq.i_filename); + free(tmpreq.URL); + return NULL; } /* special case : -s flag @@ -666,7 +702,14 @@ goto success; } */ - tmpreq.path = (char *) Malloc(strlen(dmreq->path) + strlen(TMP_EXT)); + tmpreq.path = (char *) malloc(strlen(dmreq->path) + strlen(TMP_EXT)); + if (tmpreq.path == NULL) { + fprintf(stderr, "dmXGet: Insufficient memory\n"); + free(tmpreq.i_filename); + free(tmpreq.path); + free(tmpreq.URL); + return NULL; + } strcpy(tmpreq.path, dmreq->path); strcat(tmpreq.path, TMP_EXT); @@ -687,35 +730,6 @@ return f; } -static void -send_report(int sock, struct dmrep report, char op) -{ - char *buf; - int bufsize = sizeof(report) - sizeof(report.errstr); - int errlen = strlen(report.errstr); - bufsize += errlen; - - buf = (char *) Malloc(bufsize); - int i = 0; - - memcpy(buf + i, &(report.status), sizeof(report.status)); - i += sizeof(report.status); - - memcpy(buf + i, &(report.errcode), sizeof(report.errcode)); - i += sizeof(report.errcode); - - strcpy(buf + i, report.errstr); - i += errlen; - - struct dmmsg msg; - msg.op = op; - msg.buf = buf; - msg.len = bufsize; - send_msg(sock, msg); - - free(buf); -} - /* TODO: This handler isn't registered as SIGUSR1 interrupts the download * Figure out a proper way to handle this * */ @@ -733,7 +747,7 @@ tmp = tmp->next; } - msg = recv_msg(tmp->client); + msg = recv_dmmsg(tmp->client); clisig = msg->buf; if (*clisig == SIGINT) tmp->sigint = 1; @@ -745,19 +759,52 @@ } } +int +send_report(int sock, struct dmrep report) +{ + char *buf; + int i = 0; + int bufsize = sizeof(report) - sizeof(report.errstr); + int errlen = strlen(report.errstr); + + bufsize += errlen; + buf = (char *) malloc(bufsize); + if (buf == NULL) { + fprintf(stderr, "send_report: Insufficient memory\n"); + return -1; + } + + memcpy(buf + i, &(report.status), sizeof(report.status)); + i += sizeof(report.status); + + memcpy(buf + i, &(report.errcode), sizeof(report.errcode)); + i += sizeof(report.errcode); + + strcpy(buf + i, report.errstr); + i += errlen; + + struct dmmsg msg; + msg.op = DMRESP; + msg.buf = buf; + msg.len = bufsize; + send_dmmsg(sock, msg); + + free(buf); +} + void * run_worker(struct dmjob *dmjob) { struct dmrep report; struct dmjob *tmp; struct url_stat us; - int err; + int ret; FILE *f; char *tmppath; char flags[8]; /* check if this is a duplicate */ - mk_url(dmjob, flags); + ret = mk_url(dmjob, flags); tmp = jobs; while (tmp != NULL) { if (tmp != dmjob && compare_jobs(tmp, dmjob) == 0) { @@ -780,23 +827,24 @@ } if (f == NULL) { - err = -1; + ret = -1; } else { fseek(f, 0, SEEK_SET); - err = fetch(dmjob, f, us); + ret = fetch(dmjob, f, us); } - report.status = err; + report.status = ret; report.errcode = fetchLastErrCode; report.errstr = fetchLastErrString; - send_report(dmjob->client, report, DMRESP); + send_report(dmjob->client, report); tmp->state = DONE; tmp = tmp->next; } + /* remove the local tmp file */ if (f != NULL) { - tmppath = (char *) Malloc(strlen(dmjob->request->path) + strlen(TMP_EXT)); + tmppath = (char *) malloc(strlen(dmjob->request->path) + strlen(TMP_EXT)); strcpy(tmppath, dmjob->request->path); strcat(tmppath, TMP_EXT); From owner-svn-soc-all@FreeBSD.ORG Mon Aug 19 20:44:03 2013 Return-Path: Delivered-To: svn-soc-all@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 4573BCCE for ; Mon, 19 Aug 2013 20:44:03 +0000 (UTC) (envelope-from ambarisha@FreeBSD.org) Received: from socsvn.freebsd.org (socsvn.freebsd.org [IPv6:2001:1900:2254:206a::50:2]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 335A127C3 for ; Mon, 19 Aug 2013 20:44:03 +0000 (UTC) Received: from socsvn.freebsd.org ([127.0.1.124]) by socsvn.freebsd.org (8.14.7/8.14.7) with ESMTP id r7JKi3U3028115 for ; Mon, 19 Aug 2013 20:44:03 GMT (envelope-from ambarisha@FreeBSD.org) Received: (from www@localhost) by socsvn.freebsd.org (8.14.7/8.14.6/Submit) id r7JKi3qT028113 for svn-soc-all@FreeBSD.org; Mon, 19 Aug 2013 20:44:03 GMT (envelope-from ambarisha@FreeBSD.org) Date: Mon, 19 Aug 2013 20:44:03 GMT Message-Id: <201308192044.r7JKi3qT028113@socsvn.freebsd.org> X-Authentication-Warning: socsvn.freebsd.org: www set sender to ambarisha@FreeBSD.org using -f From: ambarisha@FreeBSD.org To: svn-soc-all@FreeBSD.org Subject: socsvn commit: r256166 - soc2013/ambarisha/head/usr.bin/dms MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-soc-all@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: SVN commit messages for the entire Summer of Code repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 Aug 2013 20:44:03 -0000 Author: ambarisha Date: Mon Aug 19 20:44:02 2013 New Revision: 256166 URL: http://svnweb.FreeBSD.org/socsvn/?view=rev&rev=256166 Log: Worker sends status updates to client Modified: soc2013/ambarisha/head/usr.bin/dms/worker.c Modified: soc2013/ambarisha/head/usr.bin/dms/worker.c ============================================================================== --- soc2013/ambarisha/head/usr.bin/dms/worker.c Mon Aug 19 20:41:32 2013 (r256165) +++ soc2013/ambarisha/head/usr.bin/dms/worker.c Mon Aug 19 20:44:02 2013 (r256166) @@ -23,17 +23,8 @@ { struct dmmsg msg; struct dmjob *cur = jobs; - while (cur != NULL) { - /* TODO: May be a more thorough comparison? */ - if (cur->url == url) - break; - cur = cur->next; - } - - if (cur == NULL) - return -1; // Todo: Verify this - int bufsize = 0, i = 0, schlen, hlen; + schlen = strlen(url->scheme) + 1; hlen = strlen(url->host) + 1; bufsize += schlen + hlen + sizeof(url->port); @@ -54,24 +45,22 @@ msg.op = DMAUTHREQ; msg.len = bufsize; - ret = send_dmmsg(cur->client, msg); - if (ret == -1) { - free(msg.buf); - return -1; - } - struct dmmsg *rcvmsg; - rcvmsg = recv_dmmsg(cur->client); - if (rcvmsg == NULL) { - free(msg.buf); - return -1; - } + while (cur != NULL) { + /* TODO: May be a more thorough comparison? */ + if (cur->url != url) { + cur = cur->next; + continue; + } - strncpy(url->user, rcvmsg->buf, sizeof(url->user)); - strncpy(url->pwd, rcvmsg->buf + strlen(rcvmsg->buf) + 1, sizeof(url->pwd)); - free_dmmsg(&rcvmsg); + + /* TODO: How do we figure out which request's + * authentication credentials to use ??? + * */ - return 1; // TODO: Verify this + } + + return 1; } static int @@ -81,8 +70,12 @@ } static void -stat_send(int csock, struct xferstat *xs, int force) +stat_send(struct xferstat *xs, int force) { + struct dmjob *cur; + struct dmmsg msg; + pthread_t self = pthread_self(); + char *buf = (char *) malloc(sizeof(struct xferstat) + sizeof(force)); if (buf == NULL) { fprintf(stderr, "stat_send: Insufficient memory\n"); @@ -93,11 +86,17 @@ memcpy(buf + sizeof(force), xs, sizeof(struct xferstat)); - struct dmmsg msg; msg.op = DMSTAT; msg.buf = buf; msg.len = sizeof(*xs) + sizeof(force); - send_dmmsg(csock, msg); + cur = jobs; + + while (cur != NULL) { + if (pthread_equal(self, cur->worker) != 0) + send_dmmsg(cur->client, msg); + cur = cur->next; + } + free(buf); return; } @@ -146,7 +145,7 @@ xs->rcvd = offset; xs->lastrcvd = offset; if ((dmjob->request->flags & V_TTY) && dmjob->request->v_level > 0) - stat_send(dmjob->client, xs, 1); + stat_send(xs, 1); else if (dmjob->request->v_level > 0) fprintf(stderr, "%-46s", xs->name); } @@ -156,7 +155,7 @@ { gettimeofday(&xs->last, NULL); if ((dmjob->request->flags & V_TTY) && dmjob->request->v_level > 0) { - stat_send(dmjob->client, xs, 2); + stat_send(xs, 2); putc('\n', stderr); } else if (dmjob->request->v_level > 0) { fprintf(stderr, " %s %s\n", @@ -169,7 +168,7 @@ { xs->rcvd = rcvd; if ((dmjob->request->flags & V_TTY) && dmjob->request->v_level > 0) - stat_send(dmjob->client, xs, 0); + stat_send(xs, 0); } static int @@ -815,6 +814,8 @@ tmp = tmp->next; } + dmjob->worker = pthread_self(); + /* fetch the remote file into a local tmp file */ f = dmXGet(dmjob, &us); From owner-svn-soc-all@FreeBSD.ORG Mon Aug 19 20:45:06 2013 Return-Path: Delivered-To: svn-soc-all@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 516EFCE6 for ; Mon, 19 Aug 2013 20:45:06 +0000 (UTC) (envelope-from ambarisha@FreeBSD.org) Received: from socsvn.freebsd.org (socsvn.freebsd.org [IPv6:2001:1900:2254:206a::50:2]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 3E47327C9 for ; Mon, 19 Aug 2013 20:45:06 +0000 (UTC) Received: from socsvn.freebsd.org ([127.0.1.124]) by socsvn.freebsd.org (8.14.7/8.14.7) with ESMTP id r7JKj6Ig028371 for ; Mon, 19 Aug 2013 20:45:06 GMT (envelope-from ambarisha@FreeBSD.org) Received: (from www@localhost) by socsvn.freebsd.org (8.14.7/8.14.6/Submit) id r7JKj6oW028368 for svn-soc-all@FreeBSD.org; Mon, 19 Aug 2013 20:45:06 GMT (envelope-from ambarisha@FreeBSD.org) Date: Mon, 19 Aug 2013 20:45:06 GMT Message-Id: <201308192045.r7JKj6oW028368@socsvn.freebsd.org> X-Authentication-Warning: socsvn.freebsd.org: www set sender to ambarisha@FreeBSD.org using -f From: ambarisha@FreeBSD.org To: svn-soc-all@FreeBSD.org Subject: socsvn commit: r256167 - soc2013/ambarisha/head/usr.bin/dmget MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-soc-all@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: SVN commit messages for the entire Summer of Code repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 Aug 2013 20:45:06 -0000 Author: ambarisha Date: Mon Aug 19 20:45:05 2013 New Revision: 256167 URL: http://svnweb.FreeBSD.org/socsvn/?view=rev&rev=256167 Log: Cleaner error reporting in client (dmget) Modified: soc2013/ambarisha/head/usr.bin/dmget/dmget.c soc2013/ambarisha/head/usr.bin/dmget/utils.c Modified: soc2013/ambarisha/head/usr.bin/dmget/dmget.c ============================================================================== --- soc2013/ambarisha/head/usr.bin/dmget/dmget.c Mon Aug 19 20:44:02 2013 (r256166) +++ soc2013/ambarisha/head/usr.bin/dmget/dmget.c Mon Aug 19 20:45:05 2013 (r256167) @@ -25,17 +25,7 @@ static int siginfo; static int dmg_error; -static char *dmg_errstr; - -static void * -Malloc(size_t size) -{ - void *ptr = malloc(size); - if (ptr == NULL) { - /* Notifiy ENOMEM and exit gracefully */ - } - return ptr; -} +static char dmg_errstr[512]; void dmSigHandler(int signal) { @@ -59,7 +49,12 @@ sigaddset(&sm, SIGINFO); sigprocmask(SIG_BLOCK, &sm, NULL); - ret = Write(sock, buf, bufsize); + ret = write(sock, buf, bufsize); + if (ret == -1) { + fprintf(stderr, "dmget: Write failed (%s)\n", strerror(errno)); + strcpy(dmg_errstr, "Write failed - "); + strcat(dmg_errstr, strerror(errno)); + } sigprocmask(SIG_UNBLOCK, &sm, NULL); return ret; @@ -73,18 +68,18 @@ /* If the first read was an error return * because that could be because of a signal * */ - ret = Read(sock, buf, bufsize); - if (ret == -1 || ret == 0) + ret = read(sock, buf, bufsize); + if (ret == -1 || ret == 0) { + fprintf(stderr, "dmget: read failed (%s)\n", strerror(errno)); return ret; - + } + /* But if we've already started reading, we keep reading */ while ((ret == -1 && errno == EINTR) || n > 0 && n < bufsize) { ret = read(sock, buf + n, bufsize - n); if (ret == 0) { - /* Read ended prematurely - * Set dmg_error appropriately and return - */ - + fprintf(stderr, "dmget: Remote end closed connection\n"); + strcpy(dmg_errstr, "Remote end closed connection"); break; } @@ -94,6 +89,7 @@ if (ret != -1) return(n); + return(ret); } @@ -109,7 +105,12 @@ bufsize += strlen(dmreq.URL) + 1; bufsize += strlen(dmreq.path) + 1; - *reqbuf = (char *) Malloc(bufsize); + *reqbuf = (char *) malloc(bufsize); + if (*reqbuf == NULL) { + fprintf(stderr, "dmget: Insufficient memory"); + strcpy(dmg_errstr, "Insufficient memory"); + return -1; + } memcpy(*reqbuf, &bufsize, sizeof(bufsize)); i += sizeof(bufsize); @@ -159,7 +160,11 @@ int i = 0, len; struct dmres *dmres; - dmres = (struct dmres*) Malloc(sizeof(struct dmres)); + dmres = (struct dmres*) malloc(sizeof(struct dmres)); + if (dmres == NULL) { + fprintf(stderr, "dmget: mk_dmres: Insufficient memory\n"); + return NULL; + } memcpy(&(dmres->status), buf + i, sizeof(dmres->status)); i += sizeof(dmres->status); @@ -168,7 +173,12 @@ i += sizeof(dmres->errcode); len = strlen(buf + i); - dmres->errstr = (char *) Malloc(len); + dmres->errstr = (char *) malloc(len); + if (dmres->errstr == NULL) { + fprintf(stderr, "dmget: mk_dmres: Insufficient memory\n"); + free(dmres); + return NULL; + } strcpy(dmres->errstr, buf + i); return dmres; @@ -189,7 +199,7 @@ msg.op = DMSIG; msg.buf = &signal; msg.len = sizeof(signal); - return (send_msg(sock, msg)); + return (send_dmmsg(sock, msg)); } static int @@ -221,57 +231,73 @@ *((int *) CMSG_DATA(cmptr)) = fd; ret = sendmsg(sock, &msg, 0); - if (ret == -1) + if (ret == -1) { + fprintf(stderr, "dmget: Sending local file fd to daemon failed\n"); return (-1); - else - return (0); -} + } -static int -Write_fd(int sock, int fd) -{ - int ret = write_fd(sock, fd); - if (ret == -1) { - perror("Write_fd():"); - } else { - printf("Write_fd(): Success\n"); - } + return (0); } static int send_request(int sock, struct dmreq dmreq) { char *reqbuf; - int bufsize, err, fd; + int bufsize, ret, fd; bufsize = mk_reqbuf(dmreq, &reqbuf, DMREQ); - err = sigsafe_write(sock, reqbuf, bufsize); + if (bufsize == -1) + return -1; + + ret = sigsafe_write(sock, reqbuf, bufsize); + free(reqbuf); + + if (ret == -1) + return -1; if (dmreq.flags & O_STDOUT) fd = STDOUT_FILENO; else fd = open(dmreq.path, O_CREAT|O_RDWR|O_TRUNC, S_IRUSR|S_IWUSR); - Write_fd(sock, fd); - close(fd); + ret = write_fd(sock, fd); - free(reqbuf); - return(err); + if (!(dmreq.flags & O_STDOUT)) + close(fd); + + return(ret); } struct dmauth * mk_dmauth(char *buf, int bufsize) { int i = 0, len; - struct dmauth *dmauth = (struct dmauth *) Malloc(sizeof(struct dmauth)); + struct dmauth *dmauth = (struct dmauth *) malloc(sizeof(struct dmauth)); + if (dmauth == NULL) { + fprintf(stderr, "dmget: mk_dmauth: Insufficient memory\n"); + return NULL; + } len = strlen(buf + i); - dmauth->scheme = (char *) Malloc(len + 1); + dmauth->scheme = (char *) malloc(len + 1); + if (dmauth->scheme == NULL) { + fprintf(stderr, "dmget: mk_dmauth: Insufficient memory\n"); + free(dmauth); + return NULL; + } + strncpy(dmauth->scheme, buf + i, len); i += len + 1; len = strlen(buf + i); - dmauth->host = (char *) Malloc(len + 1); + dmauth->host = (char *) malloc(len + 1); + if (dmauth->host == NULL) { + fprintf(stderr, "dmget: mk_dmauth: Insufficient memory\n"); + free(dmauth->scheme); + free(dmauth); + return NULL; + } + strncpy(dmauth->host, buf + i, len); i += len + 1; @@ -293,18 +319,26 @@ static int send_dmauth(int sock, struct dmauth *dmauth) { - int ulen = strlen(dmauth->user) + 1; - int bufsize = ulen + strlen(dmauth->pwd) + 1; - char *buf = (char *) Malloc(bufsize); + int ret, ulen, bufsize; + char *buf; + struct dmmsg msg; + + ulen = strlen(dmauth->user) + 1; + bufsize = ulen + strlen(dmauth->pwd) + 1; + buf = (char *) malloc(bufsize); + if (buf == NULL) { + fprintf(stderr, "dmget: send_dmauth: Insufficient memory\n"); + return -1; + } strcpy(buf, dmauth->user); strcpy(buf + ulen, dmauth->user); - struct dmmsg msg; msg.op = DMAUTHRESP; msg.buf = buf; msg.len = bufsize; - send_msg(sock, msg); + ret = send_dmmsg(sock, msg); + return (ret); } int @@ -315,20 +349,33 @@ struct dmres *dmres; struct xferstat xs; struct dmauth *dmauth; - sock = Socket(AF_UNIX, SOCK_STREAM, 0); + + sock = socket(AF_UNIX, SOCK_STREAM, 0); + if (sock == -1) { + fprintf(stderr, "dmget: Could not create socket" + " (%s)\n", strerror(errno)); + return -1; + } dms_addr.sun_family = AF_UNIX; strncpy(dms_addr.sun_path, DMS_UDS_PATH, sizeof(dms_addr.sun_path)); - err = Connect(sock, (struct sockaddr *) &dms_addr, sizeof(dms_addr)); + ret = connect(sock, (struct sockaddr *) &dms_addr, sizeof(dms_addr)); + if (ret == -1) { + fprintf(stderr, "dmget: Could not connect to daemon" + " (%s)\n", strerror(errno)); + return -1; + } if (siginfo || sigint) goto signal; - send_request(sock, dmreq); + ret = send_request(sock, dmreq); + if (ret == -1) + return -1; while (!sigint) { struct dmmsg *msg; - msg = recv_msg(sock); + msg = recv_dmmsg(sock); if (msg == NULL) { goto failure; } @@ -341,7 +388,7 @@ switch(msg->op) { case DMRESP: dmres = mk_dmres(msg->buf, msg->len); - free_msg(&msg); + free_dmmsg(&msg); if (dmres->status == 0){ /* set dmLastErr* */ rm_dmres(&dmres); @@ -353,16 +400,23 @@ case DMSTAT: force = *((int *)(msg->buf)); memcpy(&xs, (msg->buf) + sizeof(force), sizeof(xs)); - free_msg(&msg); + free_dmmsg(&msg); dmStatDisplayMethod(&xs, force); break; case DMAUTHREQ: dmauth = mk_dmauth(msg->buf, msg->len); - if (dmAuthMethod(dmauth) == -1) { - + ret = dmAuthMethod(dmauth); + if (ret == -1) { + fprintf(stderr, "dmget: Authentication failed\n"); + strcpy(dmauth->user, ""); + strcpy(dmauth->pwd, ""); } + send_dmauth(sock, dmauth); rm_dmauth(&dmauth); + + if (ret == -1) + goto failure; break; default: break; Modified: soc2013/ambarisha/head/usr.bin/dmget/utils.c ============================================================================== --- soc2013/ambarisha/head/usr.bin/dmget/utils.c Mon Aug 19 20:44:02 2013 (r256166) +++ soc2013/ambarisha/head/usr.bin/dmget/utils.c Mon Aug 19 20:45:05 2013 (r256167) @@ -4,165 +4,22 @@ #include "dm.h" -void * -Malloc(size_t size) -{ - void *ptr = malloc(size); - if (ptr == NULL) { - perror("Malloc():"); - /* Notifiy ENOMEM and exit gracefully */ - } - return ptr; -} - -ssize_t -Read(int fd, void *buf, size_t size) -{ - ssize_t err = read(fd, buf, size); - if (err == -1) { - perror("Read(): "); - } else if (err != size) { - printf("Warning: %d bytes received %d expected", err, size); - } else { - printf("Read() : Success\n"); - } - - return err; -} - -int -Socket(int domain, int type, int flags) -{ - int err = socket(domain, type, flags); - if (err == -1) { - perror("Socket():"); - } else { - printf("Socket(): Success\n"); - } - - return err; -} - -int -Write(int fd, void *buf, size_t size) -{ - int err = write(fd, buf, size); - if (err == -1) { - perror("Write():"); - } else { - printf("Write(): Success\n"); - } - return err; -} - -int -Socketpair(int domain, int type, int protocol, int socket_vector[2]) -{ - int err = socketpair(domain, type, protocol, socket_vector); - if (err == -1) { - perror("Socketpair():"); - } else { - printf("Socketpair() : Success\n"); - } - return err; -} - -int -Bind(int socket, const struct sockaddr *addr, socklen_t addrlen) -{ - int err = bind(socket, addr, addrlen); - if (err == -1) { - perror("Bind():"); - } else { - printf("Bind() : Success\n"); - } - return err; -} - -int -Accept(int socket, struct sockaddr *addr, socklen_t *addrlen) -{ - int err = accept(socket, addr, addrlen); - if (err == -1) { - perror("Accept():"); - } else { - printf("Accept() : Success\n"); - } - return err; -} - -int -Connect(int socket, struct sockaddr *addr, socklen_t addrlen) -{ - int err = connect(socket, addr, addrlen); - if (err == -1) { - perror("Connect():"); - } else { - printf("Connect() : Success\n"); - } - return err; -} - -int -Listen(int socket, int backlog) -{ - int err = listen(socket, backlog); - if (err == -1) { - perror("Listen():"); - } else { - printf("Listen() : Success\n"); - } - return err; -} - -int -Peel(int sock, struct dmmsg *msg) -{ - int bufsize = 0; - int err; - err = Read(sock, &bufsize, sizeof(bufsize)); - if (err == 0) - return (err); - bufsize -= sizeof(bufsize); - - err = Read(sock, &(msg->op), sizeof(msg->op)); - if (err == 0) - return (err); - bufsize -= sizeof(msg->op); +/* Utils for handling messages */ - msg->buf = (char *) Malloc(bufsize); - msg->len = bufsize; - - err = Read(sock, msg->buf, bufsize); - if (err == 0) { - free(msg->buf); - msg->len = 0; - } - - return bufsize; -} - -int -Select(int maxfd, fd_set *rset, fd_set *wset, fd_set *xset, - const struct timeval *timeout) -{ - int err = select(maxfd, rset, wset, xset, timeout); - if (err == -1) { - perror("Select():"); - } else { - printf("Select(): Success\n"); - } - return err; -} +extern char *dm_errstr; int -send_msg(int socket, struct dmmsg msg) +send_dmmsg(int socket, struct dmmsg msg) { int bufsize = sizeof(bufsize); // Buffer size bufsize += 1; // Op bufsize += msg.len; // Signal number - char *sndbuf = (char *) Malloc(bufsize); + char *sndbuf = (char *) malloc(bufsize); + if (sndbuf == NULL) { + fprintf(stderr, "send_dmmsg: malloc: insufficient memory\n"); + return -1; + } int i = 0; memcpy(sndbuf + i, &bufsize, sizeof(bufsize)); @@ -174,64 +31,87 @@ memcpy(sndbuf + i, msg.buf, msg.len); i += msg.len; - int nbytes = Write(socket, sndbuf, bufsize); + int nbytes = write(socket, sndbuf, bufsize); free(sndbuf); + + if (nbytes == -1) { + fprintf(stderr, "send_dmmsg: write: %s\n", + strerror(errno)); + } return (nbytes); } struct dmmsg * -recv_msg(int sock) +recv_dmmsg(int sock) { int bufsize = 0; int err; - struct dmmsg *msg = (struct dmmsg *) Malloc(sizeof(struct dmmsg)); - err = Read(sock, &bufsize, sizeof(bufsize)); + struct dmmsg *msg = (struct dmmsg *) malloc(sizeof(struct dmmsg)); + if (msg == NULL) { + fprintf(stderr, "send_dmmsg: malloc: insufficient memory\n"); + return -1; + } + + err = read(sock, &bufsize, sizeof(bufsize)); if (err == 0) { - /* set dms_error */ -#if DEBUG - fprintf(stderr, "recv_msg: remote end closed connection\n"); -#endif - free(msg); - return (NULL); + fprintf(stderr, "recv_dmmsg: remote end" + " closed connection\n"); + goto error; + } else if (err == -1) { + fprintf(stderr, "recv_dmmsg: %s\n", strerror(errno)); + goto error; } bufsize -= sizeof(bufsize); - err = Read(sock, &(msg->op), sizeof(msg->op)); + err = read(sock, &(msg->op), sizeof(msg->op)); if (err == 0) { - /* set dms_error */ -#if DEBUG - fprintf(stderr, "recv_msg: remote end closed connection\n"); -#endif - free(msg); - return (NULL); + fprintf(stderr,stderr, "recv_dmmsg: remote end" + " closed connection\n"); + goto error; + } else if (err == -1) { + fprintf(stderr, "recv_dmmsg: %s\n", strerror(errno)); + goto error; } + bufsize -= sizeof(msg->op); - msg->buf = (char *) Malloc(bufsize); + msg->buf = (char *) malloc(bufsize); + if (msg == NULL) { + fprintf(stderr, "send_dmmsg: malloc: insufficient memory\n"); + goto error; + } + msg->len = bufsize; - err = Read(sock, msg->buf, bufsize); + err = read(sock, msg->buf, bufsize); if (err == 0) { - free(msg->buf); msg->len = 0; - /* set dms_error */ -#if DEBUG - fprintf(stderr, "recv_msg: remote end closed connection\n"); -#endif + fprintf(stderr,stderr, "recv_dmmsg: remote end" + " closed connection\n"); + free(msg->buf); free(msg); return (NULL); + } else if (err == -1) { + fprintf(stderr, "recv_dmmsg: %s\n", strerror(errno)); + free(msg->buf); + goto error; } + return msg; +error: + free(msg); + return NULL; } void -free_msg(struct dmmsg **msg) +free_dmmsg(struct dmmsg **msg) { - + if (*msg == NULL) + return; free((*msg)->buf); free(*msg); *msg = NULL; From owner-svn-soc-all@FreeBSD.ORG Mon Aug 19 20:46:33 2013 Return-Path: Delivered-To: svn-soc-all@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 25ECFEC1 for ; Mon, 19 Aug 2013 20:46:33 +0000 (UTC) (envelope-from ambarisha@FreeBSD.org) Received: from socsvn.freebsd.org (socsvn.freebsd.org [IPv6:2001:1900:2254:206a::50:2]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 0505127DB for ; Mon, 19 Aug 2013 20:46:33 +0000 (UTC) Received: from socsvn.freebsd.org ([127.0.1.124]) by socsvn.freebsd.org (8.14.7/8.14.7) with ESMTP id r7JKkWSt029019 for ; Mon, 19 Aug 2013 20:46:32 GMT (envelope-from ambarisha@FreeBSD.org) Received: (from www@localhost) by socsvn.freebsd.org (8.14.7/8.14.6/Submit) id r7JKkW6c029016 for svn-soc-all@FreeBSD.org; Mon, 19 Aug 2013 20:46:32 GMT (envelope-from ambarisha@FreeBSD.org) Date: Mon, 19 Aug 2013 20:46:32 GMT Message-Id: <201308192046.r7JKkW6c029016@socsvn.freebsd.org> X-Authentication-Warning: socsvn.freebsd.org: www set sender to ambarisha@FreeBSD.org using -f From: ambarisha@FreeBSD.org To: svn-soc-all@FreeBSD.org Subject: socsvn commit: r256168 - soc2013/ambarisha/head/usr.bin/dms MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-soc-all@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: SVN commit messages for the entire Summer of Code repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 Aug 2013 20:46:33 -0000 Author: ambarisha Date: Mon Aug 19 20:46:32 2013 New Revision: 256168 URL: http://svnweb.FreeBSD.org/socsvn/?view=rev&rev=256168 Log: Thread-proofing the globals. The global job queue is now protected by a mutex lock Modified: soc2013/ambarisha/head/usr.bin/dms/dms.c soc2013/ambarisha/head/usr.bin/dms/worker.c Modified: soc2013/ambarisha/head/usr.bin/dms/dms.c ============================================================================== --- soc2013/ambarisha/head/usr.bin/dms/dms.c Mon Aug 19 20:45:05 2013 (r256167) +++ soc2013/ambarisha/head/usr.bin/dms/dms.c Mon Aug 19 20:46:32 2013 (r256168) @@ -17,8 +17,9 @@ static int dm_err; static char dm_errstr[512]; -int stop; -struct dmjob *jobs; +int stop; +struct dmjob *jobs; +pthread_mutex_t job_queue_mutex; void *run_worker(struct dmjob *job); @@ -261,7 +262,25 @@ if ((dmjob = mk_dmjob(dmreq, csock)) == NULL) goto error; + /* Acquire job queue lock */ + ret = pthread_mutex_lock(&job_queue_mutex); + if (ret == -1) { + fprintf(stderr, "handle_request: Attempt to acquire" + " job queue mutex failed\n"); + goto error; + } + jobs = add_job(jobs, dmjob); + + ret = pthread_mutex_unlock(&job_queue_mutex); + if (ret == -1) { + fprintf(stderr, "handle_request: Couldn't release " + "job queue lock\n"); + + goto error; + } + /* Job queue lock released */ + pthread_create(&(dmjob->worker), NULL, run_worker, dmjob); pthread_detach(dmjob->worker); goto done; @@ -310,6 +329,7 @@ struct dmjob *cur; void *retptr; jobs = NULL; + job_queue_mutex = PTHREAD_MUTEX_INITIALIZER; fd_set fdset; signal(SIGINT, sigint_handler); @@ -320,6 +340,14 @@ maxfd = socket; FD_SET(socket, &fdset); + /* Acquire job queue lock */ + ret = pthread_mutex_lock(&job_queue_mutex); + if (ret == -1) { + fprintf(stderr, "handle_request: Attempt to acquire" + " job queue mutex failed\n"); + return -1; + } + cur = jobs; while (cur != NULL) { FD_SET(cur->client, &fdset); @@ -328,6 +356,15 @@ cur = cur->next; } + ret = pthread_mutex_unlock(&job_queue_mutex); + if (ret == -1) { + fprintf(stderr, "handle_request: Couldn't release " + "job queue lock\n"); + + return -1; + } + /* Job queue lock released */ + ret = select(maxfd + 1, &fdset, NULL, NULL, NULL); if (ret == -1) { fprintf(stderr, "run_event_loop: " @@ -335,6 +372,14 @@ goto wrap_up; } + /* Acquire job queue lock */ + ret = pthread_mutex_lock(&job_queue_mutex); + if (ret == -1) { + fprintf(stderr, "handle_request: Attempt to acquire" + " job queue mutex failed\n"); + return -1; + } + cur = jobs; while (cur != NULL) { state = service_job(cur, &fdset); @@ -346,6 +391,15 @@ cur = cur->next; } + ret = pthread_mutex_unlock(&job_queue_mutex); + if (ret == -1) { + fprintf(stderr, "handle_request: Couldn't release " + "job queue lock\n"); + + return -1; + } + /* Job queue lock released */ + if (FD_ISSET(socket, &fdset)) { struct sockaddr_un cliaddr; size_t cliaddrlen = sizeof(cliaddr); @@ -362,6 +416,14 @@ wrap_up: /* Notify all running workers that we've to wrap up */ + /* Acquire job queue lock */ + ret = pthread_mutex_lock(&job_queue_mutex); + if (ret == -1) { + fprintf(stderr, "handle_request: Attempt to acquire" + " job queue mutex failed\n"); + return -1; + } + cur = jobs; while (cur != NULL) { if (cur->state == RUNNING) @@ -371,6 +433,16 @@ jobs = rm_job(jobs, cur); cur = cur->next; } + + ret = pthread_mutex_unlock(&job_queue_mutex); + if (ret == -1) { + fprintf(stderr, "handle_request: Couldn't release " + "job queue lock\n"); + + return -1; + } + /* Job queue lock released */ + } int main(int argc, char **argv) Modified: soc2013/ambarisha/head/usr.bin/dms/worker.c ============================================================================== --- soc2013/ambarisha/head/usr.bin/dms/worker.c Mon Aug 19 20:45:05 2013 (r256167) +++ soc2013/ambarisha/head/usr.bin/dms/worker.c Mon Aug 19 20:46:32 2013 (r256168) @@ -13,8 +13,9 @@ #include "dm.h" -static const char *prefixes = " kMGTP"; +static const char *prefixes = " kMGTP"; extern struct dmjob *jobs; +extern pthread_mutex_t job_queue_mutex; #define TMP_EXT ".tmp" @@ -22,8 +23,8 @@ authenticate(struct url *url) { struct dmmsg msg; - struct dmjob *cur = jobs; - int bufsize = 0, i = 0, schlen, hlen; + struct dmjob *cur; + int bufsize = 0, i = 0, schlen, hlen, ret; schlen = strlen(url->scheme) + 1; hlen = strlen(url->host) + 1; @@ -46,6 +47,15 @@ msg.op = DMAUTHREQ; msg.len = bufsize; + /* Acquire job queue lock */ + ret = pthread_mutex_lock(&job_queue_mutex); + if (ret == -1) { + fprintf(stderr, "handle_request: Attempt to acquire" + " job queue mutex failed\n"); + return -1; + } + + cur = jobs; while (cur != NULL) { /* TODO: May be a more thorough comparison? */ if (cur->url != url) { @@ -53,13 +63,22 @@ continue; } - + /* TODO: How do we figure out which request's * authentication credentials to use ??? * */ } + ret = pthread_mutex_unlock(&job_queue_mutex); + if (ret == -1) { + fprintf(stderr, "handle_request: Couldn't release " + "job queue lock\n"); + + return -1; + } + /* Job queue lock released */ + return 1; } @@ -72,6 +91,7 @@ static void stat_send(struct xferstat *xs, int force) { + int ret; struct dmjob *cur; struct dmmsg msg; pthread_t self = pthread_self(); @@ -89,14 +109,30 @@ msg.op = DMSTAT; msg.buf = buf; msg.len = sizeof(*xs) + sizeof(force); - cur = jobs; + /* Acquire job queue lock */ + ret = pthread_mutex_lock(&job_queue_mutex); + if (ret == -1) { + fprintf(stderr, "handle_request: Attempt to acquire" + " job queue mutex failed\n"); + goto done; + } + + cur = jobs; while (cur != NULL) { if (pthread_equal(self, cur->worker) != 0) send_dmmsg(cur->client, msg); cur = cur->next; } + ret = pthread_mutex_unlock(&job_queue_mutex); + if (ret == -1) { + fprintf(stderr, "handle_request: Couldn't release " + "job queue lock\n"); + } + /* Job queue lock released */ + +done: free(buf); return; } @@ -740,6 +776,7 @@ int *clisig; pthread_t tid = pthread_self(); if (sig == SIGUSR1) { + /* TODO: Umm...Locking? */ while (tmp != NULL) { if (pthread_equal(tid, tmp->worker) != 0) break; @@ -802,8 +839,14 @@ char *tmppath; char flags[8]; - /* check if this is a duplicate */ - ret = mk_url(dmjob, flags); + /* Acquire job queue lock */ + ret = pthread_mutex_lock(&job_queue_mutex); + if (ret == -1) { + fprintf(stderr, "handle_request: Attempt to acquire" + " job queue mutex failed\n"); + return -1; + } + tmp = jobs; while (tmp != NULL) { if (tmp != dmjob && compare_jobs(tmp, dmjob) == 0) { @@ -814,11 +857,30 @@ tmp = tmp->next; } + ret = pthread_mutex_unlock(&job_queue_mutex); + if (ret == -1) { + fprintf(stderr, "handle_request: Couldn't release " + "job queue lock\n"); + + return -1; + } + /* Job queue lock released */ + + /* check if this is a duplicate */ + ret = mk_url(dmjob, flags); dmjob->worker = pthread_self(); /* fetch the remote file into a local tmp file */ f = dmXGet(dmjob, &us); + /* Acquire job queue lock */ + ret = pthread_mutex_lock(&job_queue_mutex); + if (ret == -1) { + fprintf(stderr, "handle_request: Attempt to acquire" + " job queue mutex failed\n"); + return -1; + } + /* Serve any outstanding requests from the local tmp file */ tmp = jobs; while (tmp != NULL) { @@ -843,6 +905,15 @@ tmp = tmp->next; } + ret = pthread_mutex_unlock(&job_queue_mutex); + if (ret == -1) { + fprintf(stderr, "handle_request: Couldn't release " + "job queue lock\n"); + + return -1; + } + /* Job queue lock released */ + /* remove the local tmp file */ if (f != NULL) { tmppath = (char *) malloc(strlen(dmjob->request->path) + strlen(TMP_EXT)); From owner-svn-soc-all@FreeBSD.ORG Mon Aug 19 20:48:16 2013 Return-Path: Delivered-To: svn-soc-all@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 18E46EE2 for ; Mon, 19 Aug 2013 20:48:16 +0000 (UTC) (envelope-from ambarisha@FreeBSD.org) Received: from socsvn.freebsd.org (socsvn.freebsd.org [IPv6:2001:1900:2254:206a::50:2]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 05C3527E2 for ; Mon, 19 Aug 2013 20:48:16 +0000 (UTC) Received: from socsvn.freebsd.org ([127.0.1.124]) by socsvn.freebsd.org (8.14.7/8.14.7) with ESMTP id r7JKmFDM031911 for ; Mon, 19 Aug 2013 20:48:15 GMT (envelope-from ambarisha@FreeBSD.org) Received: (from www@localhost) by socsvn.freebsd.org (8.14.7/8.14.6/Submit) id r7JKmFw2031887 for svn-soc-all@FreeBSD.org; Mon, 19 Aug 2013 20:48:15 GMT (envelope-from ambarisha@FreeBSD.org) Date: Mon, 19 Aug 2013 20:48:15 GMT Message-Id: <201308192048.r7JKmFw2031887@socsvn.freebsd.org> X-Authentication-Warning: socsvn.freebsd.org: www set sender to ambarisha@FreeBSD.org using -f From: ambarisha@FreeBSD.org To: svn-soc-all@FreeBSD.org Subject: socsvn commit: r256169 - in soc2013/ambarisha/head/usr.bin: dmget dms MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-soc-all@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: SVN commit messages for the entire Summer of Code repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 Aug 2013 20:48:16 -0000 Author: ambarisha Date: Mon Aug 19 20:48:15 2013 New Revision: 256169 URL: http://svnweb.FreeBSD.org/socsvn/?view=rev&rev=256169 Log: Added SHA1 and MD5 checksum verification The checksum file has to be passed using the -C switch for dmget Modified: soc2013/ambarisha/head/usr.bin/dmget/dm.h soc2013/ambarisha/head/usr.bin/dmget/dmget.c soc2013/ambarisha/head/usr.bin/dmget/fetch.c soc2013/ambarisha/head/usr.bin/dms/dm.h soc2013/ambarisha/head/usr.bin/dms/dms.c soc2013/ambarisha/head/usr.bin/dms/worker.c Modified: soc2013/ambarisha/head/usr.bin/dmget/dm.h ============================================================================== --- soc2013/ambarisha/head/usr.bin/dmget/dm.h Mon Aug 19 20:46:32 2013 (r256168) +++ soc2013/ambarisha/head/usr.bin/dmget/dm.h Mon Aug 19 20:48:15 2013 (r256169) @@ -6,10 +6,18 @@ #include #include +#include +#include /* TODO : Fix the path, make sure the perms on it are good */ #define DMS_UDS_PATH "/tmp/dms.uds" +#define NO_CHKSUM 0 +#define SHA1_CHKSUM 1 +#define MD5_CHKSUM 2 + +#define MAX_CHKSUM_LEN SHA_DIGEST_LENGTH /* TODO: Any better alternative? */ + struct dmres { int status; int errcode; @@ -24,8 +32,16 @@ off_t B_size; off_t S_size; long T_secs; - long flags; + int sha1; + char sha1sum[SHA_DIGEST_LENGTH]; + int chksum_type; + union { + char sha1sum[SHA_DIGEST_LENGTH]; + char md5sum[MD5_DIGEST_LENGTH]; + } chksum; + + long flags; #define A_FLAG (1 << 0) #define F_FLAG (1 << 1) #define O_STDOUT (1 << 2) Modified: soc2013/ambarisha/head/usr.bin/dmget/dmget.c ============================================================================== --- soc2013/ambarisha/head/usr.bin/dmget/dmget.c Mon Aug 19 20:46:32 2013 (r256168) +++ soc2013/ambarisha/head/usr.bin/dmget/dmget.c Mon Aug 19 20:48:15 2013 (r256169) @@ -96,11 +96,24 @@ static int mk_reqbuf(struct dmreq dmreq, char **reqbuf, char op) { - int bufsize = 0, i = 0; + int bufsize = 0, i = 0, csumlen = 0; + + switch(dmreq.chksum_type) { + case SHA1_CHKSUM: + csumlen = SHA_DIGEST_LENGTH; + break; + case MD5_CHKSUM: + csumlen = MD5_DIGEST_LENGTH; + break; + default: + break; + } bufsize += sizeof(bufsize); // Buffer size bufsize += 1; // Opcode - bufsize += sizeof(struct dmreq) - (3 * sizeof(char*)); // fix sizeof(dmreq) + bufsize += sizeof(struct dmreq); + bufsize -= (3 * sizeof(char*)) + sizeof(dmreq.chksum); // fix sizeof(dmreq) + bufsize += csumlen; bufsize += strlen(dmreq.i_filename) + 1; // bufsize += strlen(dmreq.URL) + 1; bufsize += strlen(dmreq.path) + 1; @@ -138,6 +151,22 @@ memcpy(*reqbuf + i, &(dmreq.T_secs), sizeof(dmreq.T_secs)); i += sizeof(dmreq.T_secs); + + memcpy(*reqbuf + i, &(dmreq.chksum_type), sizeof(dmreq.chksum_type)); + i += sizeof(dmreq.chksum_type); + + switch(dmreq.chksum_type) { + case SHA1_CHKSUM: + memcpy(*reqbuf + i, &(dmreq.chksum.sha1sum), SHA_DIGEST_LENGTH); + i += SHA_DIGEST_LENGTH; + break; + case MD5_CHKSUM: + memcpy(*reqbuf + i, &(dmreq.chksum.md5sum), MD5_DIGEST_LENGTH); + i += MD5_DIGEST_LENGTH; + break; + default: + break; + } memcpy(*reqbuf + i, &(dmreq.flags), sizeof(dmreq.flags)); i += sizeof(dmreq.flags); Modified: soc2013/ambarisha/head/usr.bin/dmget/fetch.c ============================================================================== --- soc2013/ambarisha/head/usr.bin/dmget/fetch.c Mon Aug 19 20:46:32 2013 (r256168) +++ soc2013/ambarisha/head/usr.bin/dmget/fetch.c Mon Aug 19 20:48:15 2013 (r256169) @@ -88,6 +88,9 @@ static long w_secs; /* -w: retry delay */ static int family = PF_UNSPEC; /* -[46]: address family to use */ +static int chksum_type = NO_CHKSUM; /* (SHA1/MD5/NO)_CHKSUM */ +static char chksum[MAX_CHKSUM_LEN]; + static int sigint; /* SIGINT received */ static long ftp_timeout = TIMEOUT; /* default timeout for FTP transfers */ @@ -264,6 +267,19 @@ if (i_flag) dmreq.i_filename = i_filename; else dmreq.i_filename = ""; + dmreq.chksum_type = chksum_type; + switch(chksum_type) { + case SHA1_CHKSUM: + memcpy(dmreq.chksum.sha1sum, chksum, SHA_DIGEST_LENGTH); + break; + case MD5_CHKSUM: + memcpy(dmreq.chksum.md5sum, chksum, MD5_DIGEST_LENGTH); + break; + default: + dmreq.chksum_type = NO_CHKSUM; + break; + } + dmreq.flags = 0; if (A_flag) dmreq.flags |= A_FLAG; if (F_flag) dmreq.flags |= F_FLAG; @@ -285,6 +301,37 @@ } static void +checksum(char *fn) +{ + FILE *f; + int ret, clen, i; + char csumstr[MAX_CHKSUM_LEN * 2]; + + f = fopen(fn, "r"); + ret = fread(csumstr, MAX_CHKSUM_LEN, 1, f); + if (ret == SHA_DIGEST_LENGTH * 2) { + chksum_type = SHA1_CHKSUM; + clen = SHA_DIGEST_LENGTH; + } else if (ret == MD5_DIGEST_LENGTH * 2) { + chksum_type = MD5_CHKSUM; + clen = MD5_DIGEST_LENGTH; + } else { + goto error; + } + + for (i = 0; i < clen; i++) { + if (sscanf(csumstr + (2 * i), "%2X", chksum + i) != 1) + goto error; + } + + return; + +error: + chksum_type = NO_CHKSUM; + return; +} + +static void usage(void) { fprintf(stderr, "%s\n%s\n%s\n%s\n", @@ -294,7 +341,6 @@ " [-T seconds] [-w seconds] [-i file] -h host -f file [-c dir]"); } - /* * Entry point */ @@ -308,7 +354,7 @@ int c, e, r; while ((c = getopt(argc, argv, - "146AaB:bc:dFf:Hh:i:lMmN:nPpo:qRrS:sT:tUvw:")) != -1) + "146AaB:bc:C:dFf:Hh:i:lMmN:nPpo:qRrS:sT:tUvw:")) != -1) switch (c) { case '1': once_flag = 1; @@ -337,6 +383,9 @@ case 'c': c_dirname = optarg; break; + case 'C': + checksum(optarg); + break; case 'd': d_flag = 1; break; Modified: soc2013/ambarisha/head/usr.bin/dms/dm.h ============================================================================== --- soc2013/ambarisha/head/usr.bin/dms/dm.h Mon Aug 19 20:46:32 2013 (r256168) +++ soc2013/ambarisha/head/usr.bin/dms/dm.h Mon Aug 19 20:48:15 2013 (r256169) @@ -6,10 +6,18 @@ #include #include +#include +#include /* TODO : Fix the path, make sure the perms on it are good */ #define DMS_UDS_PATH "/tmp/dms.uds" +#define NO_CHKSUM 0 +#define SHA1_CHKSUM 1 +#define MD5_CHKSUM 2 + +#define MAX_CHKSUM_LEN SHA_DIGEST_LENGTH /* TODO: Any better alternative? */ + struct dmres { int status; int errcode; @@ -24,8 +32,16 @@ off_t B_size; off_t S_size; long T_secs; - long flags; + int sha1; + char sha1sum[SHA_DIGEST_LENGTH]; + int chksum_type; + union { + char sha1sum[SHA_DIGEST_LENGTH]; + char md5sum[MD5_DIGEST_LENGTH]; + } chksum; + + long flags; #define A_FLAG (1 << 0) #define F_FLAG (1 << 1) #define O_STDOUT (1 << 2) Modified: soc2013/ambarisha/head/usr.bin/dms/dms.c ============================================================================== --- soc2013/ambarisha/head/usr.bin/dms/dms.c Mon Aug 19 20:46:32 2013 (r256168) +++ soc2013/ambarisha/head/usr.bin/dms/dms.c Mon Aug 19 20:48:15 2013 (r256169) @@ -170,7 +170,23 @@ memcpy(&(dmreq->T_secs), rcvbuf + i, sizeof(dmreq->T_secs)); i += sizeof(dmreq->T_secs); - + + memcpy(&(dmreq->chksum_type), rcvbuf + i, sizeof(dmreq->chksum_type)); + i += sizeof(dmreq->chksum_type); + + switch(dmreq->chksum_type) { + case SHA1_CHKSUM: + memcpy(dmreq->chksum.sha1sum, rcvbuf + i, SHA_DIGEST_LENGTH); + i += SHA_DIGEST_LENGTH; + break; + case MD5_CHKSUM: + memcpy(dmreq->chksum.md5sum, rcvbuf + i, MD5_DIGEST_LENGTH); + i += MD5_DIGEST_LENGTH; + break; + case NO_CHKSUM: + break; + } + memcpy(&(dmreq->flags), rcvbuf + i, sizeof(dmreq->flags)); i += sizeof(dmreq->flags); Modified: soc2013/ambarisha/head/usr.bin/dms/worker.c ============================================================================== --- soc2013/ambarisha/head/usr.bin/dms/worker.c Mon Aug 19 20:46:32 2013 (r256168) +++ soc2013/ambarisha/head/usr.bin/dms/worker.c Mon Aug 19 20:48:15 2013 (r256169) @@ -666,7 +666,7 @@ return (r); } -FILE * +static FILE * dmXGet(struct dmjob *dmjob, struct url_stat *us) { char flags[8]; @@ -676,12 +676,7 @@ struct dmreq *dmreq = dmjob->request; /* populate tmpjob */ - - /* TODO : Modify stat_* to udpate jobs of progress, - * right now we just put the msgs on stderr - * */ tmpjob.client = STDERR_FILENO; - tmpjob.request = &tmpreq; tmpreq.v_level = dmreq->v_level; tmpreq.ftp_timeout = dmjob->request->ftp_timeout; @@ -765,6 +760,52 @@ return f; } +static int +validate_and_copy(struct dmjob *dmjob, FILE *f, struct url_stat us) +{ + int ret, ret2; + SHA_CTX sha_ctx; + MD5_CTX md5_ctx; + char buf[1024], chksum[SHA_DIGEST_LENGTH]; /* TODO: MAX DIGEST LENGTH */ + + switch(dmjob->request->chksum_type) { + case SHA1_CHKSUM: + SHA1_Init(&sha_ctx); + + fseek(f, 0, SEEK_SET); + while ((ret = fread(buf, 1, 1024, f)) != 0) + SHA1_Update(&sha_ctx, buf, ret); + SHA1_Final(chksum, &sha_ctx); + + if (memcmp(chksum, dmjob->request->chksum.sha1sum, + SHA_DIGEST_LENGTH) != 0) { + fprintf(stderr, "dms: checksum missmatch\n"); + /* Notify the client of the same */ + return -1; + } + break; + case MD5_CHKSUM: + MD5_Init(&md5_ctx); + + fseek(f, 0, SEEK_SET); + while ((ret = fread(buf, 1, 1024, f)) != 0) + MD5_Update(&md5_ctx, buf, ret); + MD5_Final(chksum, &md5_ctx); + + if (memcmp(chksum, dmjob->request->chksum.md5sum, + MD5_DIGEST_LENGTH) != 0) { + fprintf(stderr, "dms: checksum mismatch\n"); + return -1; + } + break; + default: + break; + } + + fseek(f, 0, SEEK_SET); + return fetch(dmjob, f, us); +} + /* TODO: This handler isn't registered as SIGUSR1 interrupts the download * Figure out a proper way to handle this * */ @@ -892,8 +933,7 @@ if (f == NULL) { ret = -1; } else { - fseek(f, 0, SEEK_SET); - ret = fetch(dmjob, f, us); + ret = validate_and_copy(dmjob, f, us); } report.status = ret; From owner-svn-soc-all@FreeBSD.ORG Mon Aug 19 20:51:29 2013 Return-Path: Delivered-To: svn-soc-all@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id AF5C7F41 for ; Mon, 19 Aug 2013 20:51:29 +0000 (UTC) (envelope-from ambarisha@FreeBSD.org) Received: from socsvn.freebsd.org (socsvn.freebsd.org [IPv6:2001:1900:2254:206a::50:2]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 902472822 for ; Mon, 19 Aug 2013 20:51:29 +0000 (UTC) Received: from socsvn.freebsd.org ([127.0.1.124]) by socsvn.freebsd.org (8.14.7/8.14.7) with ESMTP id r7JKpTEg013842 for ; Mon, 19 Aug 2013 20:51:29 GMT (envelope-from ambarisha@FreeBSD.org) Received: (from www@localhost) by socsvn.freebsd.org (8.14.7/8.14.6/Submit) id r7JKpTk4013825 for svn-soc-all@FreeBSD.org; Mon, 19 Aug 2013 20:51:29 GMT (envelope-from ambarisha@FreeBSD.org) Date: Mon, 19 Aug 2013 20:51:29 GMT Message-Id: <201308192051.r7JKpTk4013825@socsvn.freebsd.org> X-Authentication-Warning: socsvn.freebsd.org: www set sender to ambarisha@FreeBSD.org using -f From: ambarisha@FreeBSD.org To: svn-soc-all@FreeBSD.org Subject: socsvn commit: r256170 - soc2013/ambarisha/head/usr.bin/dms MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-soc-all@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: SVN commit messages for the entire Summer of Code repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 Aug 2013 20:51:29 -0000 Author: ambarisha Date: Mon Aug 19 20:51:29 2013 New Revision: 256170 URL: http://svnweb.FreeBSD.org/socsvn/?view=rev&rev=256170 Log: Added mirror profiling DMS samples the download speeds from mirrors and picks the mirror which has the highest average speed over the past week. It picks a mirror which hasn't been tried before with preference. Added: soc2013/ambarisha/head/usr.bin/dms/mirror.c Modified: soc2013/ambarisha/head/usr.bin/dms/Makefile soc2013/ambarisha/head/usr.bin/dms/dms.c soc2013/ambarisha/head/usr.bin/dms/dms.h soc2013/ambarisha/head/usr.bin/dms/worker.c Modified: soc2013/ambarisha/head/usr.bin/dms/Makefile ============================================================================== --- soc2013/ambarisha/head/usr.bin/dms/Makefile Mon Aug 19 20:48:15 2013 (r256169) +++ soc2013/ambarisha/head/usr.bin/dms/Makefile Mon Aug 19 20:51:29 2013 (r256170) @@ -1,7 +1,7 @@ # $FreeBSD$ .include -SRCS= utils.c dms.c worker.c +SRCS= mirror.c utils.c dms.c worker.c PROG= dms CSTD?= c99 .if ${MK_OPENSSL} != "no" Modified: soc2013/ambarisha/head/usr.bin/dms/dms.c ============================================================================== --- soc2013/ambarisha/head/usr.bin/dms/dms.c Mon Aug 19 20:48:15 2013 (r256169) +++ soc2013/ambarisha/head/usr.bin/dms/dms.c Mon Aug 19 20:51:29 2013 (r256170) @@ -18,8 +18,12 @@ static char dm_errstr[512]; int stop; + struct dmjob *jobs; -pthread_mutex_t job_queue_mutex; +pthread_mutex_t job_queue_mutex; + +extern struct dmmirr *mirrors; +extern pthread_mutex_t mirror_list_mutex; void *run_worker(struct dmjob *job); @@ -114,6 +118,8 @@ static struct dmjob * mk_dmjob(struct dmreq *dmreq, int client) { + int ret; + struct dmmirr *cur; struct dmjob *dmjob = (struct dmjob *) malloc(sizeof(struct dmjob)); if (dmjob == NULL) { fprintf(stderr, "mk_dmjob: malloc: insufficient memory\n"); @@ -128,6 +134,7 @@ return NULL; } + dmjob->mirror = get_mirror(); dmjob->client = client; dmjob->sigint = 0; dmjob->sigalrm = 0; @@ -187,6 +194,7 @@ break; } + memcpy(&(dmreq->flags), rcvbuf + i, sizeof(dmreq->flags)); i += sizeof(dmreq->flags); @@ -292,7 +300,6 @@ if (ret == -1) { fprintf(stderr, "handle_request: Couldn't release " "job queue lock\n"); - goto error; } /* Job queue lock released */ @@ -322,11 +329,12 @@ void sigint_handler(int sig) { + save_mirrors(); stop = 1; exit(1); // Temporary } -static state_t +static int service_job(struct dmjob *job, fd_set *fdset) { int ret = 0; @@ -334,19 +342,23 @@ /* TODO: Worker can't handle this signal yet */ //pthread_kill(job->worker, SIGUSR1); } - return (job->state); + + return ret; } static void run_event_loop(int socket) { int i, ret, maxfd = socket; - state_t state; struct dmjob *cur; void *retptr; + fd_set fdset; + jobs = NULL; job_queue_mutex = PTHREAD_MUTEX_INITIALIZER; - fd_set fdset; + mirrors = NULL; + mirror_list_mutex = PTHREAD_MUTEX_INITIALIZER; + load_mirrors(); signal(SIGINT, sigint_handler); while (!stop) { @@ -398,8 +410,8 @@ cur = jobs; while (cur != NULL) { - state = service_job(cur, &fdset); - if (state == DONE) { + ret = service_job(cur, &fdset); + if (ret > 0) { close(cur->client); jobs = rm_job(jobs, cur); rm_dmjob(&cur); Modified: soc2013/ambarisha/head/usr.bin/dms/dms.h ============================================================================== --- soc2013/ambarisha/head/usr.bin/dms/dms.h Mon Aug 19 20:48:15 2013 (r256169) +++ soc2013/ambarisha/head/usr.bin/dms/dms.h Mon Aug 19 20:51:29 2013 (r256170) @@ -3,20 +3,29 @@ #include -typedef enum {RUNNING=0, DONE, DUPLICATE} state_t; +#define MAX_LISTEN_QUEUE 5 +#define MINBUFSIZE 4096 +#define MAX_SAMPLES 256 struct dmjob { int ofd; int client; - state_t state; int sigint; int sigalrm; int siginfo; int siginfo_en; unsigned timeout; + + enum { + RUNNING = 0, + DONE, + DUPLICATE + } state; + pthread_t worker; struct dmreq *request; struct url *url; + struct dmmirr *mirror; struct dmjob *next; struct dmjob *prev; @@ -28,9 +37,24 @@ char *errstr; }; -#define DEBUG 1 +struct dmmirr { + char name[512]; + int index; + + enum { + NOT_TRIED = 0, + ACTIVE, + FAILED + } remark; + + struct timeval timestamps[MAX_SAMPLES]; + double samples[MAX_SAMPLES]; + int nconns; -#define MAX_LISTEN_QUEUE 5 -#define MINBUFSIZE 4096 + struct dmmirr *next; + struct dmmirr *prev; +}; + +#define DEBUG 1 #endif Added: soc2013/ambarisha/head/usr.bin/dms/mirror.c ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ soc2013/ambarisha/head/usr.bin/dms/mirror.c Mon Aug 19 20:51:29 2013 (r256170) @@ -0,0 +1,310 @@ +#include +#include +#include "dm.h" +#include "dms.h" + +#define MAX_SAMPLES 256 +#define MAX_CONNS 5 +#define MIRRORS_FILE "mirrors.list" + +struct dmmirr *mirrors; +pthread_mutex_t mirror_list_mutex; + +static const char *MIRROR_LIST[] = { + "ftp.freebsd.org" +}; + +static struct dmmirr * +add_mirror(struct dmmirr *head, struct dmmirr *new) +{ + new->prev = NULL; + new->next = NULL; + + if (head == NULL) + return new; + + head->prev = new; + new->next = head; +} + +static struct dmmirr * +rm_mirror(struct dmmirr *head, struct dmmirr *mirror) +{ + if (mirror->next != NULL) + mirror->next->prev = mirror->prev; + + if (mirror->prev != NULL) + mirror->prev->next = mirror->next; + + if (mirror == head) + return mirror->next; + + return head; +} + +static double +get_speed(struct xferstat *xs) +{ + double delta = (xs->last.tv_sec + (xs->last.tv_usec / 1.e6)) + - (xs->last2.tv_sec + (xs->last2.tv_usec / 1.e6)); + if (delta == 0.0) + return -1.0; + return (xs->rcvd - xs->lastrcvd) / delta; +} + +static struct dmmirr * +read_mirror(FILE *f) +{ + int i; + struct dmmirr *mirror; + char buf[512], rem[64]; + + mirror = (struct dmmirr *) malloc(sizeof(struct dmmirr)); + if (mirror == NULL) { + fprintf(stderr, "read_mirror: Insufficient memory\n"); + return NULL; + } + + if (fgets(buf, 512, f) == NULL) { + free(mirror); + return NULL; + } + sscanf(buf, "%s\n", mirror->name); + + if (fgets(buf, 64, f) == NULL) { + fprintf(stderr, "WARNING: read_mirror: mirrors.list file corrupted\n"); + free(mirror); + return NULL; + } + sscanf(buf, "%s\n", rem); + + if (strcmp(rem, "NOT_TRIED") == 0) { + mirror->remark = NOT_TRIED; + } else if (strcmp(rem, "FAILED") == 0) { + mirror->remark = FAILED; + } else { + fprintf(stderr, "WARNING: Unknown mirror state in mirrors.list\n"); + } + + if (fgets(buf, 64, f) == NULL) { + fprintf(stderr, "WARNING: read_mirror: mirrors.list file corrupted\n"); + free(mirror); + return NULL; + } + sscanf(buf, "%d\n", &mirror->index); + + for(i = 0; i < MAX_SAMPLES; i++) { + fscanf(f, "%ld\t%f\n", &(mirror->timestamps[i].tv_sec), + &(mirror->samples[i])); + /* TODO: What if fscanf fails? */ + } + + return mirror; +} + +static void +write_mirror(struct dmmirr *mirror, FILE *f) +{ + int i; + + fputs(mirror->name, f); + fputc('\n', f); + + switch(mirror->remark) { + case NOT_TRIED: + fputs("NOT_TRIED\n", f); + break; + case FAILED: + fputs("FAILED\n", f); + break; + } + + for(i = 0; i < MAX_SAMPLES; i++) { + fprintf(f, "%ld\t%f\n", mirror->timestamps[i].tv_sec, + mirror->samples[i]); + } + + return; +} + +static int +init_mirrors_file(void) +{ + int i, j; + FILE *f = fopen(MIRRORS_FILE, "w"); + if (f == NULL) + return -1; + + for(i = 0; i < sizeof(MIRROR_LIST) / sizeof(MIRROR_LIST[0]); i++) { + fwrite(MIRROR_LIST[i], strlen(MIRROR_LIST[i]), 1, f); + fprintf(f, "\nNOT_TRIED\n"); + for (j = 0; j < MAX_SAMPLES; j++) + fprintf(f, "0\t0\n"); + } + + fclose(f); +} + +int +load_mirrors(void) +{ + int ret; + struct dmmirr *mirror; + + FILE *f = fopen(MIRRORS_FILE, "r"); + if (f == NULL && errno == ENOENT) { + init_mirrors_file(); + f = fopen(MIRRORS_FILE, "r"); + } else if (f == NULL) { + fprintf(stderr, "load_mirrors: fopen(%s) failed\n", + MIRRORS_FILE); + return -1; + } + + /* Profile list lock */ + ret = pthread_mutex_lock(&mirror_list_mutex); + if (ret == -1) { + fprintf(stderr, "get_mirror: Attempt to acquire" + " profile list mutex failed\n"); + return -1; + } + + mirror = read_mirror(f); + while(mirror != NULL) { + mirrors = add_mirror(mirrors, mirror); + mirror = read_mirror(f); + } + + ret = pthread_mutex_unlock(&mirror_list_mutex); + if (ret == -1) { + fprintf(stderr, "get_mirror: Couldn't release " + "profile list lock\n"); + return -1; + } + /* Profile list lock released */ + + fclose(f); + return 0; +} + +int +save_mirrors(void) +{ + int ret; + struct dmmirr *mirror = mirrors; + + FILE *f = fopen(MIRRORS_FILE, "w"); + + /* Profile list lock */ + ret = pthread_mutex_lock(&mirror_list_mutex); + if (ret == -1) { + fprintf(stderr, "get_mirror: Attempt to acquire" + " profile list mutex failed\n"); + return -1; + } + + while(mirror != NULL) { + write_mirror(mirror, f); + mirrors = rm_mirror(mirrors, mirror); + } + + ret = pthread_mutex_unlock(&mirror_list_mutex); + if (ret == -1) { + fprintf(stderr, "get_mirror: Couldn't release " + "profile list lock\n"); + return -1; + } + /* Profile list lock released */ + + fclose(f); + return 0; +} + +void +update_mirror(struct dmmirr *dmmirr, struct xferstat *xs) +{ + struct timeval tv; + double speed; + + gettimeofday(&tv, NULL); + if (tv.tv_sec - dmmirr->timestamps[dmmirr->index].tv_sec < 60) + return; + + speed = get_speed(xs); + + /* TODO: This assumes that workers and sites have 1-1 correspondence */ + dmmirr->index = (dmmirr->index + 1) % MAX_SAMPLES; + dmmirr->timestamps[dmmirr->index] = tv; + dmmirr->samples[dmmirr->index] = speed; + dmmirr->remark = ACTIVE; +} + +struct dmmirr * +get_mirror(void) +{ + struct dmmirr *cur, *tmp; + double tmpmax = -1.0; + int cnt, ret, i; + struct timeval now; + long week_sec; + double average; + + week_sec = 7 * 24 * 60 * 60; + + /* Profile list lock */ + ret = pthread_mutex_lock(&mirror_list_mutex); + if (ret == -1) { + fprintf(stderr, "get_mirror: Attempt to acquire" + " profile list mutex failed\n"); + return NULL; + } + + cur = mirrors; + tmp = NULL; + tmpmax = -1.0; + while (cur != NULL) { + if (cur->remark == NOT_TRIED) { + tmp = cur; + goto success; + } + + if (cur->remark == FAILED) + goto next; + if (cur->nconns > MAX_CONNS) + goto next; + + + i = cur->index; + cnt = 0; + average = 0.0; + do { + gettimeofday(&now, NULL); + if (cur->timestamps[i].tv_sec < now.tv_sec - week_sec) + break; + average = (average * cnt + cur->samples[i]) / (cnt + 1); + cnt++; + + i = (i - 1) % MAX_SAMPLES; + } while (i != cur->index); + + if (average > tmpmax) { + tmpmax = average; + tmp = cur; + } +next: + cur = cur->next; + } + + /* TODO: If we couldn't pick up a mirror? */ + +success: + ret = pthread_mutex_unlock(&mirror_list_mutex); + if (ret == -1) { + fprintf(stderr, "get_mirror: Couldn't release " + "profile list lock\n"); + return NULL; + } + /* Profile list lock released */ + + return tmp; +} Modified: soc2013/ambarisha/head/usr.bin/dms/worker.c ============================================================================== --- soc2013/ambarisha/head/usr.bin/dms/worker.c Mon Aug 19 20:48:15 2013 (r256169) +++ soc2013/ambarisha/head/usr.bin/dms/worker.c Mon Aug 19 20:51:29 2013 (r256170) @@ -180,6 +180,9 @@ xs->offset = offset; xs->rcvd = offset; xs->lastrcvd = offset; + + update_mirror(dmjob->mirror , xs); + if ((dmjob->request->flags & V_TTY) && dmjob->request->v_level > 0) stat_send(xs, 1); else if (dmjob->request->v_level > 0) @@ -190,6 +193,7 @@ stat_end(struct xferstat *xs, struct dmjob *dmjob) { gettimeofday(&xs->last, NULL); + update_mirror(dmjob->mirror , xs); if ((dmjob->request->flags & V_TTY) && dmjob->request->v_level > 0) { stat_send(xs, 2); putc('\n', stderr); @@ -203,10 +207,20 @@ stat_update(struct xferstat *xs, off_t rcvd, struct dmjob *dmjob) { xs->rcvd = rcvd; + update_mirror(dmjob->mirror , xs); if ((dmjob->request->flags & V_TTY) && dmjob->request->v_level > 0) stat_send(xs, 0); } +static void +select_mirror(struct dmjob *dmjob) +{ + dmjob->mirror = get_mirror(); + strcpy(dmjob->url->host, dmjob->mirror->name); + strcpy(dmjob->request->URL, dmjob->mirror->name); + strcat(dmjob->request->URL, dmjob->url->doc); +} + static int mk_url(struct dmjob *dmjob, char *flags) { @@ -228,12 +242,15 @@ fprintf(stderr, "warning: mk_url: URL empty\n"); goto failure; } + if ((dmjob->url = fetchParseURL(dmreq->URL)) == NULL) { warnx("%s: parse error", dmreq->URL); goto failure; } - /* if no scheme was specified, take a guess */ + /* Replace host name with the mirror name */ + select_mirror(dmjob); + if (!*(dmjob->url->scheme)) { if (!*(dmjob->url->host)) strcpy(dmjob->url->scheme, SCHEME_FILE); @@ -732,6 +749,7 @@ goto success; } */ + tmpreq.path = (char *) malloc(strlen(dmreq->path) + strlen(TMP_EXT)); if (tmpreq.path == NULL) { fprintf(stderr, "dmXGet: Insufficient memory\n"); From owner-svn-soc-all@FreeBSD.ORG Mon Aug 19 20:52:57 2013 Return-Path: Delivered-To: svn-soc-all@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 2EF2AF5F for ; Mon, 19 Aug 2013 20:52:57 +0000 (UTC) (envelope-from ambarisha@FreeBSD.org) Received: from socsvn.freebsd.org (socsvn.freebsd.org [IPv6:2001:1900:2254:206a::50:2]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 0231F282B for ; Mon, 19 Aug 2013 20:52:57 +0000 (UTC) Received: from socsvn.freebsd.org ([127.0.1.124]) by socsvn.freebsd.org (8.14.7/8.14.7) with ESMTP id r7JKquIw034678 for ; Mon, 19 Aug 2013 20:52:56 GMT (envelope-from ambarisha@FreeBSD.org) Received: (from www@localhost) by socsvn.freebsd.org (8.14.7/8.14.6/Submit) id r7JKqugJ034667 for svn-soc-all@FreeBSD.org; Mon, 19 Aug 2013 20:52:56 GMT (envelope-from ambarisha@FreeBSD.org) Date: Mon, 19 Aug 2013 20:52:56 GMT Message-Id: <201308192052.r7JKqugJ034667@socsvn.freebsd.org> X-Authentication-Warning: socsvn.freebsd.org: www set sender to ambarisha@FreeBSD.org using -f From: ambarisha@FreeBSD.org To: svn-soc-all@FreeBSD.org Subject: socsvn commit: r256171 - in soc2013/ambarisha/head/usr.bin: dmget dms MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-soc-all@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: SVN commit messages for the entire Summer of Code repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 Aug 2013 20:52:57 -0000 Author: ambarisha Date: Mon Aug 19 20:52:56 2013 New Revision: 256171 URL: http://svnweb.FreeBSD.org/socsvn/?view=rev&rev=256171 Log: Handling download failures (file not found, access denied etc.) with grace Modified: soc2013/ambarisha/head/usr.bin/dmget/dmget.c soc2013/ambarisha/head/usr.bin/dms/mirror.c soc2013/ambarisha/head/usr.bin/dms/worker.c Modified: soc2013/ambarisha/head/usr.bin/dmget/dmget.c ============================================================================== --- soc2013/ambarisha/head/usr.bin/dmget/dmget.c Mon Aug 19 20:51:29 2013 (r256170) +++ soc2013/ambarisha/head/usr.bin/dmget/dmget.c Mon Aug 19 20:52:56 2013 (r256171) @@ -423,6 +423,7 @@ rm_dmres(&dmres); goto success; } else { + fprintf(stderr, "dmget: download failed: %s\n", dmres->errstr); rm_dmres(&dmres); goto failure; } @@ -456,6 +457,7 @@ ret = -1; goto done; failure: + remove(dmreq.path); ret = 1; goto done; success: Modified: soc2013/ambarisha/head/usr.bin/dms/mirror.c ============================================================================== --- soc2013/ambarisha/head/usr.bin/dms/mirror.c Mon Aug 19 20:51:29 2013 (r256170) +++ soc2013/ambarisha/head/usr.bin/dms/mirror.c Mon Aug 19 20:52:56 2013 (r256171) @@ -203,7 +203,7 @@ return -1; } - while(mirror != NULL) { + while(mirrors != NULL) { write_mirror(mirror, f); mirrors = rm_mirror(mirrors, mirror); } Modified: soc2013/ambarisha/head/usr.bin/dms/worker.c ============================================================================== --- soc2013/ambarisha/head/usr.bin/dms/worker.c Mon Aug 19 20:51:29 2013 (r256170) +++ soc2013/ambarisha/head/usr.bin/dms/worker.c Mon Aug 19 20:52:56 2013 (r256171) @@ -753,10 +753,7 @@ tmpreq.path = (char *) malloc(strlen(dmreq->path) + strlen(TMP_EXT)); if (tmpreq.path == NULL) { fprintf(stderr, "dmXGet: Insufficient memory\n"); - free(tmpreq.i_filename); - free(tmpreq.path); - free(tmpreq.URL); - return NULL; + goto done; } strcpy(tmpreq.path, dmreq->path); strcat(tmpreq.path, TMP_EXT); @@ -764,11 +761,19 @@ tmpjob.ofd = open(tmpreq.path, O_RDWR | O_CREAT | O_TRUNC, S_IRUSR | S_IWUSR); FILE *f = fetchXGet(tmpjob.url, us, flags); + if (f == NULL) { + close(tmpjob.ofd); + remove(tmpreq.path); + fprintf(stderr, "dmXGet: Couldn't fetch remote file\n");//, fetchLastErrStr); + goto done; + } + fetch(&tmpjob, f, *us); fclose(f); f = fopen(tmpreq.path, "r"); +done: free(tmpjob.url->doc); free(tmpjob.url); free(tmpreq.i_filename); From owner-svn-soc-all@FreeBSD.ORG Tue Aug 20 08:21:42 2013 Return-Path: Delivered-To: svn-soc-all@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id B1FE2D70 for ; Tue, 20 Aug 2013 08:21:42 +0000 (UTC) (envelope-from dpl@FreeBSD.org) Received: from socsvn.freebsd.org (socsvn.freebsd.org [IPv6:2001:1900:2254:206a::50:2]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 9F70F2BEA for ; Tue, 20 Aug 2013 08:21:42 +0000 (UTC) Received: from socsvn.freebsd.org ([127.0.1.124]) by socsvn.freebsd.org (8.14.7/8.14.7) with ESMTP id r7K8LgEx097660 for ; Tue, 20 Aug 2013 08:21:42 GMT (envelope-from dpl@FreeBSD.org) Received: (from www@localhost) by socsvn.freebsd.org (8.14.7/8.14.6/Submit) id r7K8Lg59097655 for svn-soc-all@FreeBSD.org; Tue, 20 Aug 2013 08:21:42 GMT (envelope-from dpl@FreeBSD.org) Date: Tue, 20 Aug 2013 08:21:42 GMT Message-Id: <201308200821.r7K8Lg59097655@socsvn.freebsd.org> X-Authentication-Warning: socsvn.freebsd.org: www set sender to dpl@FreeBSD.org using -f From: dpl@FreeBSD.org To: svn-soc-all@FreeBSD.org Subject: socsvn commit: r256189 - soc2013/dpl/head/lib/libzcap MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-soc-all@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: SVN commit messages for the entire Summer of Code repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Aug 2013 08:21:42 -0000 Author: dpl Date: Tue Aug 20 08:21:42 2013 New Revision: 256189 URL: http://svnweb.FreeBSD.org/socsvn/?view=rev&rev=256189 Log: Modified to use global buffer. Debug fprintf deleted. Modified: soc2013/dpl/head/lib/libzcap/commands.c Modified: soc2013/dpl/head/lib/libzcap/commands.c ============================================================================== --- soc2013/dpl/head/lib/libzcap/commands.c Tue Aug 20 07:48:20 2013 (r256188) +++ soc2013/dpl/head/lib/libzcap/commands.c Tue Aug 20 08:21:42 2013 (r256189) @@ -82,7 +82,7 @@ extern pid_t pid; extern nvlist_t *sendCommand(nvlist_t *); -extern void * data; +extern void *data; nvlist_t *nvl, *args, *result; size_t gzfilesize = sizeof(struct gzFile_s); @@ -869,9 +869,9 @@ return(0); } nvlist_add_number(nvl, "command", ZCAPCMD_ZLIBCOMPILEFLAGS); + nvlist_add_nvlist(nvl, "args", args); result = sendCommand(nvl); - printf("after sendCommand()\n"); ret = nvlist_take_number(result, "result"); destroy(); From owner-svn-soc-all@FreeBSD.ORG Tue Aug 20 08:22:25 2013 Return-Path: Delivered-To: svn-soc-all@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 4CFB7DBA for ; Tue, 20 Aug 2013 08:22:25 +0000 (UTC) (envelope-from dpl@FreeBSD.org) Received: from socsvn.freebsd.org (socsvn.freebsd.org [IPv6:2001:1900:2254:206a::50:2]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 2C4172BF4 for ; Tue, 20 Aug 2013 08:22:25 +0000 (UTC) Received: from socsvn.freebsd.org ([127.0.1.124]) by socsvn.freebsd.org (8.14.7/8.14.7) with ESMTP id r7K8MPIG007270 for ; Tue, 20 Aug 2013 08:22:25 GMT (envelope-from dpl@FreeBSD.org) Received: (from www@localhost) by socsvn.freebsd.org (8.14.7/8.14.6/Submit) id r7K8MPOR007264 for svn-soc-all@FreeBSD.org; Tue, 20 Aug 2013 08:22:25 GMT (envelope-from dpl@FreeBSD.org) Date: Tue, 20 Aug 2013 08:22:25 GMT Message-Id: <201308200822.r7K8MPOR007264@socsvn.freebsd.org> X-Authentication-Warning: socsvn.freebsd.org: www set sender to dpl@FreeBSD.org using -f From: dpl@FreeBSD.org To: svn-soc-all@FreeBSD.org Subject: socsvn commit: r256190 - soc2013/dpl/head/lib/libzcap/test MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-soc-all@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: SVN commit messages for the entire Summer of Code repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Aug 2013 08:22:25 -0000 Author: dpl Date: Tue Aug 20 08:22:24 2013 New Revision: 256190 URL: http://svnweb.FreeBSD.org/socsvn/?view=rev&rev=256190 Log: Use err() instead of fprintf();exit(). Modified: soc2013/dpl/head/lib/libzcap/test/zcaplibtest.c Modified: soc2013/dpl/head/lib/libzcap/test/zcaplibtest.c ============================================================================== --- soc2013/dpl/head/lib/libzcap/test/zcaplibtest.c Tue Aug 20 08:21:42 2013 (r256189) +++ soc2013/dpl/head/lib/libzcap/test/zcaplibtest.c Tue Aug 20 08:22:24 2013 (r256190) @@ -2,6 +2,7 @@ #include #include +#include void testzlibVersion(void); void testdeflateInit(z_streamp strm); @@ -22,17 +23,13 @@ int main() { - if ( (strm = malloc(sizeof (z_stream))) == NULL) { - printf("zcaplibtest: malloc()"); - abort(); - } + if ( (strm = malloc(sizeof (z_stream))) == NULL) + err(1, "zcaplibtest: malloc()"); - if ( (data = malloc(10*1024)) == NULL) { - printf("zcaplibtest: malloc()"); - abort(); - } + if ( (data = malloc(10*1024)) == NULL) + err(1, "zcaplibtest: malloc()"); - for (i=0; i<(10*1024); i++) { + for (i=0; i < (10*1024); i++) { data[i] = rand(); } @@ -91,10 +88,9 @@ if ( (ret = zlibCompileFlags()) > 0) printf("OK\n"); - else - printf("Error\n"); - printf("zlibCompileFlags: %lu\n", ret); - + else + err(1, "Error\n"); + return; } @@ -108,41 +104,31 @@ void testchecksums(void) { - long data[2]; - - uLong first = 0; - uLong second = 0; - uLong all = 0; - uLong combined = 0; - int lsize = sizeof(long); + uLong first, second, combined; + first = second = combined = 0; + + first = adler32(0L, Z_NULL, 0); + printf("adler32: %ld\n", first ); + first = adler32(first, (void *)data, 5*1024); + printf("adler32: %ld\n", first ); + second = adler32(first, (void *)(data+5*1024), 5*1024); + printf("adler32: %ld\n", second ); - data[0] = random(); - data[1] = random(); + combined = adler32_combine(first, second, 5*1024); + printf("adler32_combine(): %ld\n", combined ); - first = adler32(0, (void *)data, sizeof(long)); - printf("adler32(0, %ld, %d): %ld\n", data[0], sizeof(long), first ); printf("adler32\t...\t"); - if (first != 0) + if (second == combined) printf("OK\n"); else - printf("Error\n"); - - second = adler32(0, (void *)(data+lsize), sizeof(long)); - printf("adler32(0, %ld, %d): %ld\n", data[1], sizeof(long), second ); - - combined = adler32_combine(first, second, sizeof(long)); - printf("adler32_combine(%ld, %ld, %d): %ld\n", first, second, sizeof(long), combined ); - - all = adler32(0L, (void *)data, 2*sizeof(long)); - printf("adler32(0, %ld, %d): %ld\n", data[0], 2*sizeof(long), all ); - + err(1, "Error\n"); printf("adler32_combine()\t...\t"); - if (all == combined) + if (second == combined) printf("OK\n"); else - printf("Error\n"); + err(1, "Error\n"); return; From owner-svn-soc-all@FreeBSD.ORG Tue Aug 20 08:23:30 2013 Return-Path: Delivered-To: svn-soc-all@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 9F429DEB for ; Tue, 20 Aug 2013 08:23:30 +0000 (UTC) (envelope-from dpl@FreeBSD.org) Received: from socsvn.freebsd.org (socsvn.freebsd.org [IPv6:2001:1900:2254:206a::50:2]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 7E64B2C09 for ; Tue, 20 Aug 2013 08:23:30 +0000 (UTC) Received: from socsvn.freebsd.org ([127.0.1.124]) by socsvn.freebsd.org (8.14.7/8.14.7) with ESMTP id r7K8NUw2021073 for ; Tue, 20 Aug 2013 08:23:30 GMT (envelope-from dpl@FreeBSD.org) Received: (from www@localhost) by socsvn.freebsd.org (8.14.7/8.14.6/Submit) id r7K8NUFU021064 for svn-soc-all@FreeBSD.org; Tue, 20 Aug 2013 08:23:30 GMT (envelope-from dpl@FreeBSD.org) Date: Tue, 20 Aug 2013 08:23:30 GMT Message-Id: <201308200823.r7K8NUFU021064@socsvn.freebsd.org> X-Authentication-Warning: socsvn.freebsd.org: www set sender to dpl@FreeBSD.org using -f From: dpl@FreeBSD.org To: svn-soc-all@FreeBSD.org Subject: socsvn commit: r256191 - soc2013/dpl/head/lib/libzcap/zlibworker MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-soc-all@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: SVN commit messages for the entire Summer of Code repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Aug 2013 08:23:30 -0000 Author: dpl Date: Tue Aug 20 08:23:30 2013 New Revision: 256191 URL: http://svnweb.FreeBSD.org/socsvn/?view=rev&rev=256191 Log: Improved commands, use global buffer. Improved error messages. Modified: soc2013/dpl/head/lib/libzcap/zlibworker/commands.c soc2013/dpl/head/lib/libzcap/zlibworker/zlibworker.c Modified: soc2013/dpl/head/lib/libzcap/zlibworker/commands.c ============================================================================== --- soc2013/dpl/head/lib/libzcap/zlibworker/commands.c Tue Aug 20 08:22:24 2013 (r256190) +++ soc2013/dpl/head/lib/libzcap/zlibworker/commands.c Tue Aug 20 08:23:30 2013 (r256191) @@ -6,6 +6,9 @@ #include nvlist_t * args; +extern int zero; +extern void *data; + size_t gzsize = sizeof(struct gzFile_s); size_t zstreamsize = sizeof(z_stream); @@ -80,15 +83,7 @@ void zcapcmd_deflateSetDictionary(nvlist_t *nvl, nvlist_t *result) { - uLong ret = -1; - - if ((args = nvlist_take_nvlist(nvl, "args")) == NULL) { - nvlist_take_binary(args, "strm", &zstreamsize); - ret = compressBound( - nvlist_take_number(args, "sourceLen") - ); - } - nvlist_add_number(result, "result", ret ); + ; } void @@ -255,11 +250,9 @@ { int ret = -1; int len = 0; - void * data = NULL; if ((args = nvlist_take_nvlist(nvl, "args")) != NULL) { len = nvlist_take_number(args, "len"); - data = malloc(len); ret = gzread( nvlist_take_binary(args, "file", &gzsize), data, @@ -269,6 +262,7 @@ nvlist_add_number(result, "result", ret); nvlist_add_binary(result, "data", data, len); + zero = 1; } void @@ -276,18 +270,18 @@ { int ret = -1; size_t len = 0; - void * data; if ((args = nvlist_take_nvlist(nvl, "args")) != NULL) { len = nvlist_take_number(args, "len"); data = malloc(len); memcpy(data, nvlist_take_binary(args, "data", &len), len); - ret = gzread( + ret = gzwrite( nvlist_take_binary(args, "file", &gzsize), data, len); } nvlist_add_number(result, "result", ret); + zero = 1; } void @@ -295,11 +289,9 @@ { int ret = -1; int len = 0; - void * data; if ((args = nvlist_take_nvlist(nvl, "args")) != NULL) { len = nvlist_take_number(args, "len"); - data = malloc(len); ret = gzprintf( nvlist_take_binary(args, "file", &gzsize), nvlist_take_string(args, "str") @@ -459,19 +451,16 @@ zcapcmd_adler32(nvlist_t *nvl, nvlist_t *result) { uLong ret = -1; - fprintf(stderr, "\nzcapcmd_adler32()\n"); if ((args = nvlist_take_nvlist(nvl, "args")) != NULL) { size_t len = nvlist_take_number(args, "len"); uLong adler = nvlist_take_number(args, "adler"); void *buf = nvlist_take_binary(args, "buf", &len); - fprintf(stderr, "zcapcmd_crc32: adler: %ld\n", adler); - fprintf(stderr, "zcapcmd_crc32: buf: %p\n", &buf); - fprintf(stderr, "zcapcmd_crc32: len: %d\n", len); - fflush(stderr); - ret = adler32(adler, buf, len); + memcpy(data, buf, len); + ret = adler32(adler, data, len); } nvlist_add_number(result, "result", ret ); + zero = 1; } Modified: soc2013/dpl/head/lib/libzcap/zlibworker/zlibworker.c ============================================================================== --- soc2013/dpl/head/lib/libzcap/zlibworker/zlibworker.c Tue Aug 20 08:22:24 2013 (r256190) +++ soc2013/dpl/head/lib/libzcap/zlibworker/zlibworker.c Tue Aug 20 08:23:30 2013 (r256191) @@ -4,6 +4,7 @@ #include #include +#include #include #include "../commands.h" @@ -65,7 +66,10 @@ extern void zcapcmd_crc32(nvlist_t *nvl, nvlist_t *result); extern void zcapcmd_crc32_combine(nvlist_t *nvl, nvlist_t *result); +/* Points to a 5kb buffer */ void * data; +/* Zero == 1 when we need to zero out data */ +int zero = 0; int main() @@ -75,22 +79,30 @@ /* Sandbox the process */ cap_enter(); - if ((data = malloc(5*1024)) == NULL) { + if ((data = calloc(5*1024, 1)) == NULL) { perror("malloc"); abort(); } - if ((result = nvlist_create(0)) == NULL) - return(-1); while(1) { + if ((result = nvlist_create(0)) == NULL) { + fprintf(stderr, "zlibworker: Can't create result.\n"); + exit(1); + } + if ((nvl = nvlist_recv(3)) != NULL) { - if (!nvlist_exists(nvl, "command")) - continue; - if (!nvlist_exists(nvl, "args")) - continue; + if (!nvlist_exists(nvl, "command")) { + fprintf(stderr, "zlibworker: No command.\n"); + exit(1); + } + if (!nvlist_exists(nvl, "args")) { + fprintf(stderr, "zlibworker: args doesn't exist\n"); + exit(1); + } } else { - continue; + fprintf(stderr, "zlibworker: nvl is NULL\n"); + exit(1); } // Switch for "command" @@ -247,7 +259,7 @@ zcapcmd_adler32(nvl, result); break; case (ZCAPCMD_ADLER32_COMBINE): - zcapcmd_compressBound(nvl, result); + zcapcmd_adler32_combine(nvl, result); break; case (ZCAPCMD_CRC32): zcapcmd_crc32(nvl, result); @@ -256,11 +268,19 @@ zcapcmd_crc32_combine(nvl, result); break; default: + perror("zlibworker: unknown command"); + exit(1); break; } - if( nvlist_send(3, result) != 0 ) + if( nvlist_send(3, result) != 0 ) { perror("zlibworker: Couldn't send response\n"); + exit(1); + } + + if (zero) + memset(data, 0, 5*1024); } + return (0); } \ No newline at end of file From owner-svn-soc-all@FreeBSD.ORG Tue Aug 20 12:23:34 2013 Return-Path: Delivered-To: svn-soc-all@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 7355C147 for ; Tue, 20 Aug 2013 12:23:34 +0000 (UTC) (envelope-from dpl@FreeBSD.org) Received: from socsvn.freebsd.org (socsvn.freebsd.org [IPv6:2001:1900:2254:206a::50:2]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 5F9C02B09 for ; Tue, 20 Aug 2013 12:23:34 +0000 (UTC) Received: from socsvn.freebsd.org ([127.0.1.124]) by socsvn.freebsd.org (8.14.7/8.14.7) with ESMTP id r7KCNYhO026611 for ; Tue, 20 Aug 2013 12:23:34 GMT (envelope-from dpl@FreeBSD.org) Received: (from www@localhost) by socsvn.freebsd.org (8.14.7/8.14.6/Submit) id r7KCNY0Y026598 for svn-soc-all@FreeBSD.org; Tue, 20 Aug 2013 12:23:34 GMT (envelope-from dpl@FreeBSD.org) Date: Tue, 20 Aug 2013 12:23:34 GMT Message-Id: <201308201223.r7KCNY0Y026598@socsvn.freebsd.org> X-Authentication-Warning: socsvn.freebsd.org: www set sender to dpl@FreeBSD.org using -f From: dpl@FreeBSD.org To: svn-soc-all@FreeBSD.org Subject: socsvn commit: r256193 - in soc2013/dpl/head/lib/libzcap: . zlibworker MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-soc-all@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: SVN commit messages for the entire Summer of Code repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Aug 2013 12:23:34 -0000 Author: dpl Date: Tue Aug 20 12:23:34 2013 New Revision: 256193 URL: http://svnweb.FreeBSD.org/socsvn/?view=rev&rev=256193 Log: Made slight advances. Modified: soc2013/dpl/head/lib/libzcap/capsicum.c soc2013/dpl/head/lib/libzcap/commands.h soc2013/dpl/head/lib/libzcap/zlibworker/zlibworker.c Modified: soc2013/dpl/head/lib/libzcap/capsicum.c ============================================================================== --- soc2013/dpl/head/lib/libzcap/capsicum.c Tue Aug 20 11:06:56 2013 (r256192) +++ soc2013/dpl/head/lib/libzcap/capsicum.c Tue Aug 20 12:23:34 2013 (r256193) @@ -9,6 +9,7 @@ #include #include #include +#include #define SUM 0 @@ -23,30 +24,27 @@ startChild(void) { if (socketpair(PF_LOCAL, SOCK_STREAM, 0, sv) < 0 ) - perror("zcaplib: socketpair"); + perror("zcaplib: socketpair()"); if( (pid = fork()) == 0 ){ cap_rights_limit(STDIN_FILENO, CAP_WRITE); cap_rights_limit(STDOUT_FILENO, CAP_READ); cap_rights_limit(STDERR_FILENO, CAP_WRITE); + //cap_rights_limit(3, CAP_WRITE|CAP_READ|CAP_EVENT); + closefrom(4); - close(sv[0]); - dup2(sv[1] , 3); /* 3 is now the socket */ - close(4); - /* open and execl() listener */ if (access("/usr/libexec/zlibworker", X_OK) < 0) { - perror("zcaplib: zlibworker:"); + perror("zcaplib: access():"); exit (-1); } if ( execl("/usr/libexec/zlibworker", "zlibworker", NULL) <0 ) { - perror("zcaplib: zlibworker:"); + perror("zcaplib: execl:"); exit (-1); } exit(0); } else { - close(sv[1]); atexit(killChild); } @@ -56,16 +54,11 @@ nvlist_t * sendCommand( nvlist_t *nvl ) { - if( nvlist_send(sv[0], nvl) != 0 ) { - perror("zcaplib: nvlist_send()"); - return NULL; - } + if( nvlist_send(sv[1], nvl) != 0 ) + err(1, "zcaplib: nvlist_send()"); - /* recv results */ - if ((nvl = nvlist_recv(sv[0])) == NULL) { - perror("zcaplib: nvlist_recv()"); - return NULL; - } + if ((nvl = nvlist_recv(sv[1])) == NULL) + err(1, "nvlist_recv(): nvl is NULL"); return (nvl); } Modified: soc2013/dpl/head/lib/libzcap/commands.h ============================================================================== --- soc2013/dpl/head/lib/libzcap/commands.h Tue Aug 20 11:06:56 2013 (r256192) +++ soc2013/dpl/head/lib/libzcap/commands.h Tue Aug 20 12:23:34 2013 (r256193) @@ -47,35 +47,35 @@ #define ZCAPCMD_ZLIBCOMPILEFLAGS 28 /* utility functions */ -#define ZCAPCMD_COMPRESSBOUND 30 +#define ZCAPCMD_COMPRESSBOUND 29 /* gzip file access functions */ -#define ZCAPCMD_GZOPEN 32 -#define ZCAPCMD_GZBUFFER 34 -#define ZCAPCMD_GZSETPARAMS 35 -#define ZCAPCMD_GZREAD 36 -#define ZCAPCMD_GZWRITE 37 -#define ZCAPCMD_GZPRINTF 38 -#define ZCAPCMD_GZPUTS 39 -#define ZCAPCMD_GZGETS 49 -#define ZCAPCMD_GZPUTC 50 -#define ZCAPCMD_GZGETC 51 -#define ZCAPCMD_GZUNGETC 52 -#define ZCAPCMD_GZFLUSH 53 -#define ZCAPCMD_GZSEEK 54 -#define ZCAPCMD_GZREWIND 55 -#define ZCAPCMD_GZTELL 56 -#define ZCAPCMD_GZOFFSET 57 -#define ZCAPCMD_GZEOF 58 -#define ZCAPCMD_GZDIRECT 59 -#define ZCAPCMD_GZCLOSE 60 -#define ZCAPCMD_GZCLOSE_R 61 -#define ZCAPCMD_GZCLOSE_W 62 -#define ZCAPCMD_GZERROR 63 -#define ZCAPCMD_GZCLEARERR 64 +#define ZCAPCMD_GZOPEN 30 +#define ZCAPCMD_GZBUFFER 31 +#define ZCAPCMD_GZSETPARAMS 32 +#define ZCAPCMD_GZREAD 33 +#define ZCAPCMD_GZWRITE 34 +#define ZCAPCMD_GZPRINTF 35 +#define ZCAPCMD_GZPUTS 36 +#define ZCAPCMD_GZGETS 46 +#define ZCAPCMD_GZPUTC 47 +#define ZCAPCMD_GZGETC 48 +#define ZCAPCMD_GZUNGETC 49 +#define ZCAPCMD_GZFLUSH 50 +#define ZCAPCMD_GZSEEK 51 +#define ZCAPCMD_GZREWIND 52 +#define ZCAPCMD_GZTELL 53 +#define ZCAPCMD_GZOFFSET 54 +#define ZCAPCMD_GZEOF 55 +#define ZCAPCMD_GZDIRECT 56 +#define ZCAPCMD_GZCLOSE 57 +#define ZCAPCMD_GZCLOSE_R 58 +#define ZCAPCMD_GZCLOSE_W 59 +#define ZCAPCMD_GZERROR 60 +#define ZCAPCMD_GZCLEARERR 61 /* checksum functions */ -#define ZCAPCMD_ADLER32 65 -#define ZCAPCMD_ADLER32_COMBINE 66 -#define ZCAPCMD_CRC32 67 -#define ZCAPCMD_CRC32_COMBINE 68 +#define ZCAPCMD_ADLER32 62 +#define ZCAPCMD_ADLER32_COMBINE 63 +#define ZCAPCMD_CRC32 64 +#define ZCAPCMD_CRC32_COMBINE 65 Modified: soc2013/dpl/head/lib/libzcap/zlibworker/zlibworker.c ============================================================================== --- soc2013/dpl/head/lib/libzcap/zlibworker/zlibworker.c Tue Aug 20 11:06:56 2013 (r256192) +++ soc2013/dpl/head/lib/libzcap/zlibworker/zlibworker.c Tue Aug 20 12:23:34 2013 (r256193) @@ -5,7 +5,7 @@ #include #include #include -#include +#include #include "../commands.h" /* Basic functions */ @@ -79,32 +79,23 @@ /* Sandbox the process */ cap_enter(); - if ((data = calloc(5*1024, 1)) == NULL) { - perror("malloc"); - abort(); - } + if ((data = calloc(5*1024, 1)) == NULL) + err(1, "malloc\n"); - while(1) { - if ((result = nvlist_create(0)) == NULL) { - fprintf(stderr, "zlibworker: Can't create result.\n"); - exit(1); - } + if ((result = nvlist_create(0)) == NULL) + err(1, "Can't create result.\n"); + //fprintf(stderr, "DEBUG: zlibworker waiting command\n"); if ((nvl = nvlist_recv(3)) != NULL) { - if (!nvlist_exists(nvl, "command")) { - fprintf(stderr, "zlibworker: No command.\n"); - exit(1); - } - if (!nvlist_exists(nvl, "args")) { - fprintf(stderr, "zlibworker: args doesn't exist\n"); - exit(1); - } - } else { - fprintf(stderr, "zlibworker: nvl is NULL\n"); - exit(1); - } + if (!nvlist_exists(nvl, "command")) + err(1, "No command.\n"); + if (!nvlist_exists(nvl, "args")) + err(1, "args doesn't exist\n"); + } else err(1, "nvl is NULL\n"); + + //fprintf(stderr, "DEBUG: zlibworker got command\n"); // Switch for "command" // Get args, and call the real lib. switch( nvlist_take_number(nvl, "command") ){ @@ -235,10 +226,10 @@ case (ZCAPCMD_GZSEEK): zcapcmd_gzseek(nvl, result); break; - /* - * All this commands have gzFile as argument, - * and return an int (or void, in the last case). - */ + /* + * All this commands have gzFile as argument, + * and return an int (or void, in the last case). + */ case (ZCAPCMD_GZGETC): case (ZCAPCMD_GZREWIND): case (ZCAPCMD_GZTELL): @@ -268,18 +259,16 @@ zcapcmd_crc32_combine(nvl, result); break; default: - perror("zlibworker: unknown command"); - exit(1); + err(1, "Wrong command"); break; } - if( nvlist_send(3, result) != 0 ) { - perror("zlibworker: Couldn't send response\n"); - exit(1); - } + //fprintf(stderr, "DEBUG: zlibworker responding\n"); + if( nvlist_send(3, result) != 0 ) + err(1, "Couldn't send response\n"); - if (zero) - memset(data, 0, 5*1024); + nvlist_destroy(nvl); + if (zero) memset(data, 0, 5*1024); } return (0); From owner-svn-soc-all@FreeBSD.ORG Tue Aug 20 14:26:24 2013 Return-Path: Delivered-To: svn-soc-all@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 5CE084C4 for ; Tue, 20 Aug 2013 14:26:24 +0000 (UTC) (envelope-from mattbw@FreeBSD.org) Received: from socsvn.freebsd.org (socsvn.freebsd.org [IPv6:2001:1900:2254:206a::50:2]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 3B2642391 for ; Tue, 20 Aug 2013 14:26:24 +0000 (UTC) Received: from socsvn.freebsd.org ([127.0.1.124]) by socsvn.freebsd.org (8.14.7/8.14.7) with ESMTP id r7KEQODd028863 for ; Tue, 20 Aug 2013 14:26:24 GMT (envelope-from mattbw@FreeBSD.org) Received: (from www@localhost) by socsvn.freebsd.org (8.14.7/8.14.6/Submit) id r7KEQOPx028859 for svn-soc-all@FreeBSD.org; Tue, 20 Aug 2013 14:26:24 GMT (envelope-from mattbw@FreeBSD.org) Date: Tue, 20 Aug 2013 14:26:24 GMT Message-Id: <201308201426.r7KEQOPx028859@socsvn.freebsd.org> X-Authentication-Warning: socsvn.freebsd.org: www set sender to mattbw@FreeBSD.org using -f From: mattbw@FreeBSD.org To: svn-soc-all@FreeBSD.org Subject: socsvn commit: r256201 - soc2013/mattbw/backend/query MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-soc-all@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: SVN commit messages for the entire Summer of Code repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Aug 2013 14:26:24 -0000 Author: mattbw Date: Tue Aug 20 14:26:23 2013 New Revision: 256201 URL: http://svnweb.FreeBSD.org/socsvn/?view=rev&rev=256201 Log: Add more ATF tests for ID/name conversion. I realise I'm spending too much time on this, so the next commit should be adapting the query mechanism. Modified: soc2013/mattbw/backend/query/id.c soc2013/mattbw/backend/query/id.h soc2013/mattbw/backend/query/id_test.c Modified: soc2013/mattbw/backend/query/id.c ============================================================================== --- soc2013/mattbw/backend/query/id.c Tue Aug 20 13:55:49 2013 (r256200) +++ soc2013/mattbw/backend/query/id.c Tue Aug 20 14:26:23 2013 (r256201) @@ -31,13 +31,13 @@ static char *join_namever(const char *name, const char *version); static char *strdup_null_if_empty(const char *in); static struct query_id *query_id_array_alloc(unsigned int count); -static struct query_id *query_id_array_from_strings(char **strings, unsigned int package_count, id_from_string_ptr id_from_string); +static struct query_id *query_id_array_from_strings(const char **strings, unsigned int package_count, id_from_string_ptr id_from_string); static void query_id_array_free(struct query_id **query_ids_p, unsigned int count); static void query_id_free_contents(struct query_id *query_id); /* Converts an array of PackageIDs to query IDs. */ struct query_id * -query_id_array_from_package_ids(gchar **package_ids, +query_id_array_from_package_ids(const gchar **package_ids, unsigned int package_count) { @@ -47,7 +47,7 @@ /* Converts an array of names (or name-version strings) to query IDs. */ struct query_id * -query_id_array_from_names(gchar **package_names, unsigned int package_count) +query_id_array_from_names(const gchar **package_names, unsigned int package_count) { return query_id_array_from_strings(package_names, package_count, @@ -59,7 +59,7 @@ * function. */ static struct query_id * -query_id_array_from_strings(char **strings, unsigned int package_count, +query_id_array_from_strings(const char **strings, unsigned int package_count, id_from_string_ptr id_from_string) { struct query_id *query_ids; @@ -69,19 +69,19 @@ query_ids = query_id_array_alloc(package_count); if (query_ids != NULL) { - bool error; + bool success; unsigned int i; - error = false; + success = true; for (i = 0; i < package_count; i++) { - error = id_from_string(strings[i], query_ids + i); - if (error) { + success = id_from_string(strings[i], query_ids + i); + if (!success) { query_id_array_free(&query_ids, package_count); break; } } - if (error) { + if (!success) { query_id_array_free(&query_ids, package_count); } } @@ -207,14 +207,15 @@ unsigned int i; assert(query_ids_p != NULL); - assert(*query_ids_p != NULL); - for (i = 0; i < count; i++) { - query_id_free_contents((*query_ids_p) + i); - } + if (*query_ids_p != NULL) { + for (i = 0; i < count; i++) { + query_id_free_contents((*query_ids_p) + i); + } - free(*query_ids_p); - *query_ids_p = NULL; + free(*query_ids_p); + *query_ids_p = NULL; + } } static void Modified: soc2013/mattbw/backend/query/id.h ============================================================================== --- soc2013/mattbw/backend/query/id.h Tue Aug 20 13:55:49 2013 (r256200) +++ soc2013/mattbw/backend/query/id.h Tue Aug 20 14:26:23 2013 (r256201) @@ -33,7 +33,7 @@ bool query_id_from_name(const char *name, struct query_id *query_id); bool query_id_from_package_id(const gchar *package_id, struct query_id *query_id); -struct query_id *query_id_array_from_names(gchar **package_names, unsigned int package_count); -struct query_id *query_id_array_from_package_ids(gchar **package_ids, unsigned int package_count); +struct query_id *query_id_array_from_names(const gchar **package_names, unsigned int package_count); +struct query_id *query_id_array_from_package_ids(const gchar **package_ids, unsigned int package_count); #endif /* !_PKGNG_BACKEND_QUERY_ID_H_ */ Modified: soc2013/mattbw/backend/query/id_test.c ============================================================================== --- soc2013/mattbw/backend/query/id_test.c Tue Aug 20 13:55:49 2013 (r256200) +++ soc2013/mattbw/backend/query/id_test.c Tue Aug 20 14:26:23 2013 (r256201) @@ -159,6 +159,10 @@ ATF_CHECK_STREQ(result.repo, "packagesite"); } +/* + * SINGLE NAMES + */ + ATF_TC(query_id_from_name_single_valid); ATF_TC_HEAD(query_id_from_name_single_valid, tc) { @@ -184,6 +188,10 @@ ATF_CHECK_EQ(result.repo, NULL); } +/* + * SINGLE NAMES + */ + ATF_TC(query_id_from_name_single_empty); ATF_TC_HEAD(query_id_from_name_single_empty, tc) { @@ -210,7 +218,103 @@ } /* - * SINGLE NAMES + * MULTIPLE ID FUNCTIONS + */ + +ATF_TC(query_id_from_name_multiple_valid); +ATF_TC_HEAD(query_id_from_name_multiple_valid, tc) +{ + + atf_tc_set_md_var(tc, "descr", + "Ensure valid arrays of package names convert to query IDs."); +} +ATF_TC_BODY(query_id_from_name_multiple_valid, tc) +{ + unsigned int i; + struct query_id *results; + const char *valid_names[3]; + const char *valid_1 = "pkg-1.1.4"; + const char *valid_2 = "brasero-1.2"; + const char *valid_3 = "PackageKit"; + + valid_names[0] = valid_1; + valid_names[1] = valid_2; + valid_names[2] = valid_3; + + results = query_id_array_from_names(valid_names, 3); + ATF_CHECK(results != NULL); + + if (results != NULL) { + for (i = 0; i < 3; i++) { + ATF_CHECK_STREQ(results[i].namever, valid_names[i]); + ATF_CHECK_EQ(results[i].arch, NULL); + ATF_CHECK_EQ(results[i].repo, NULL); + } + } +} + +ATF_TC(query_id_from_name_multiple_empty); +ATF_TC_HEAD(query_id_from_name_multiple_empty, tc) +{ + + atf_tc_set_md_var(tc, "descr", + "Ensure package name arrays with empty members fail to convert."); +} +ATF_TC_BODY(query_id_from_name_multiple_empty, tc) +{ + struct query_id *results; + const char *invalid_names[3]; + const char *invalid_1 = "pkg-1.1.4"; + const char *invalid_2 = ""; + const char *invalid_3 = "PackageKit"; + + invalid_names[0] = invalid_1; + invalid_names[1] = invalid_2; + invalid_names[2] = invalid_3; + + results = query_id_array_from_names(invalid_names, 3); + ATF_CHECK_EQ(results, NULL); +} + +ATF_TC(query_id_from_package_id_multiple_valid); +ATF_TC_HEAD(query_id_from_package_id_multiple_valid, tc) +{ + + atf_tc_set_md_var(tc, "descr", + "Ensure package ID arrays with all valid members convert."); +} +ATF_TC_BODY(query_id_from_package_id_multiple_valid, tc) +{ + struct query_id *results; + const char *valid_package_ids[3]; + const char *valid_1 = "pkg;1.1.4;freebsd:9:x86:32;packagesite"; + const char *valid_2 = "brasero;1.2;freebsd:9:x86:64;elsewhere"; + const char *valid_3 = "PackageKit;0.6;freebsd:10:x86:32;installed"; + + valid_package_ids[0] = valid_1; + valid_package_ids[1] = valid_2; + valid_package_ids[2] = valid_3; + + results = query_id_array_from_package_ids(valid_package_ids, 3); + ATF_CHECK(results != NULL); + + if (results != NULL) { + ATF_CHECK_STREQ(results[0].namever, "pkg-1.1.4"); + ATF_CHECK_STREQ(results[0].arch, "freebsd:9:x86:32"); + ATF_CHECK_STREQ(results[0].repo, "packagesite"); + + ATF_CHECK_STREQ(results[1].namever, "brasero-1.2"); + ATF_CHECK_STREQ(results[1].arch, "freebsd:9:x86:64"); + ATF_CHECK_STREQ(results[1].repo, "elsewhere"); + + ATF_CHECK_STREQ(results[2].namever, "PackageKit-0.6"); + ATF_CHECK_STREQ(results[2].arch, "freebsd:10:x86:32"); + ATF_CHECK_STREQ(results[2].repo, "installed"); + } +} + +/* + * TEST PACK */ ATF_TP_ADD_TCS(tp) @@ -225,5 +329,10 @@ ATF_TP_ADD_TC(tp, query_id_from_name_single_valid); ATF_TP_ADD_TC(tp, query_id_from_name_single_empty); + ATF_TP_ADD_TC(tp, query_id_from_name_multiple_valid); + ATF_TP_ADD_TC(tp, query_id_from_name_multiple_empty); + + ATF_TP_ADD_TC(tp, query_id_from_package_id_multiple_valid); + return atf_no_error(); } From owner-svn-soc-all@FreeBSD.ORG Tue Aug 20 14:48:40 2013 Return-Path: Delivered-To: svn-soc-all@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id DAA39C16 for ; Tue, 20 Aug 2013 14:48:40 +0000 (UTC) (envelope-from dpl@FreeBSD.org) Received: from socsvn.freebsd.org (socsvn.freebsd.org [IPv6:2001:1900:2254:206a::50:2]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id C7E6A2517 for ; Tue, 20 Aug 2013 14:48:40 +0000 (UTC) Received: from socsvn.freebsd.org ([127.0.1.124]) by socsvn.freebsd.org (8.14.7/8.14.7) with ESMTP id r7KEmeDs010075 for ; Tue, 20 Aug 2013 14:48:40 GMT (envelope-from dpl@FreeBSD.org) Received: (from www@localhost) by socsvn.freebsd.org (8.14.7/8.14.6/Submit) id r7KEmef8010065 for svn-soc-all@FreeBSD.org; Tue, 20 Aug 2013 14:48:40 GMT (envelope-from dpl@FreeBSD.org) Date: Tue, 20 Aug 2013 14:48:40 GMT Message-Id: <201308201448.r7KEmef8010065@socsvn.freebsd.org> X-Authentication-Warning: socsvn.freebsd.org: www set sender to dpl@FreeBSD.org using -f From: dpl@FreeBSD.org To: svn-soc-all@FreeBSD.org Subject: socsvn commit: r256202 - soc2013/dpl/head/contrib/xz/src/xz MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-soc-all@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: SVN commit messages for the entire Summer of Code repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Aug 2013 14:48:40 -0000 Author: dpl Date: Tue Aug 20 14:48:40 2013 New Revision: 256202 URL: http://svnweb.FreeBSD.org/socsvn/?view=rev&rev=256202 Log: Changed cap_init() for capsicum_enter(), since cap_init is a Casper public interface. Modified: soc2013/dpl/head/contrib/xz/src/xz/file_io.c soc2013/dpl/head/contrib/xz/src/xz/file_io.h soc2013/dpl/head/contrib/xz/src/xz/main.c Modified: soc2013/dpl/head/contrib/xz/src/xz/file_io.c ============================================================================== --- soc2013/dpl/head/contrib/xz/src/xz/file_io.c Tue Aug 20 14:26:23 2013 (r256201) +++ soc2013/dpl/head/contrib/xz/src/xz/file_io.c Tue Aug 20 14:48:40 2013 (r256202) @@ -1053,7 +1053,7 @@ } extern void -cap_init(void) +capsicum_enter(void) { cap_rights_t rights; Modified: soc2013/dpl/head/contrib/xz/src/xz/file_io.h ============================================================================== --- soc2013/dpl/head/contrib/xz/src/xz/file_io.h Tue Aug 20 14:26:23 2013 (r256201) +++ soc2013/dpl/head/contrib/xz/src/xz/file_io.h Tue Aug 20 14:48:40 2013 (r256202) @@ -109,5 +109,5 @@ extern void limitfd(file_pair *pair); /// \brief Enters Capability mode, and limit basic fds. -extern void cap_init(void); +extern void capsicum_enter(void); #endif Modified: soc2013/dpl/head/contrib/xz/src/xz/main.c ============================================================================== --- soc2013/dpl/head/contrib/xz/src/xz/main.c Tue Aug 20 14:26:23 2013 (r256201) +++ soc2013/dpl/head/contrib/xz/src/xz/main.c Tue Aug 20 14:48:40 2013 (r256202) @@ -293,7 +293,7 @@ /* Let the children compress */ wait(NULL); } else if (forkpid == 0){ - cap_init(); + capsicum_enter(); #endif run(pairs[i]); free(files[i]); From owner-svn-soc-all@FreeBSD.ORG Tue Aug 20 15:38:00 2013 Return-Path: Delivered-To: svn-soc-all@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id B354EA4F for ; Tue, 20 Aug 2013 15:38:00 +0000 (UTC) (envelope-from jmuniz@FreeBSD.org) Received: from socsvn.freebsd.org (socsvn.freebsd.org [IPv6:2001:1900:2254:206a::50:2]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 9E4CF282E for ; Tue, 20 Aug 2013 15:38:00 +0000 (UTC) Received: from socsvn.freebsd.org ([127.0.1.124]) by socsvn.freebsd.org (8.14.7/8.14.7) with ESMTP id r7KFc0WL027753 for ; Tue, 20 Aug 2013 15:38:00 GMT (envelope-from jmuniz@FreeBSD.org) Received: (from www@localhost) by socsvn.freebsd.org (8.14.7/8.14.6/Submit) id r7KFc0Xk027720 for svn-soc-all@FreeBSD.org; Tue, 20 Aug 2013 15:38:00 GMT (envelope-from jmuniz@FreeBSD.org) Date: Tue, 20 Aug 2013 15:38:00 GMT Message-Id: <201308201538.r7KFc0Xk027720@socsvn.freebsd.org> X-Authentication-Warning: socsvn.freebsd.org: www set sender to jmuniz@FreeBSD.org using -f From: jmuniz@FreeBSD.org To: svn-soc-all@FreeBSD.org Subject: socsvn commit: r256209 - in soc2013/jmuniz/personal: documentation ports/PackageKit-0.6.11/packagekit ports/PackageKit-0.6.11/packagekit/files ports/PackageKit-0.6.11/packagekit/files/netpackagekit... MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-soc-all@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: SVN commit messages for the entire Summer of Code repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Aug 2013 15:38:00 -0000 Author: jmuniz Date: Tue Aug 20 15:38:00 2013 New Revision: 256209 URL: http://svnweb.FreeBSD.org/socsvn/?view=rev&rev=256209 Log: Added: soc2013/jmuniz/personal/ports/PackageKit-0.6.11/packagekit/files/netpackagekitd/ soc2013/jmuniz/personal/ports/PackageKit-0.6.11/packagekit/files/netpackagekitd.in soc2013/jmuniz/personal/ports/PackageKit-0.6.11/packagekit/files/netpackagekitd/netpackagekit.c soc2013/jmuniz/personal/tools/testing.sh Modified: soc2013/jmuniz/personal/documentation/TODO soc2013/jmuniz/personal/ports/PackageKit-0.6.11/packagekit/Makefile soc2013/jmuniz/personal/ports/PackageKit-0.6.11/packagekit/pkg-plist soc2013/jmuniz/personal/ports/gnome-packagekit-2.32.0/gnome-packagekit/Makefile soc2013/jmuniz/personal/source/PackageKit-0.6.11/backends/Makefile.am soc2013/jmuniz/personal/source/PackageKit-0.6.11/configure soc2013/jmuniz/personal/source/PackageKit-0.6.11/configure.ac Modified: soc2013/jmuniz/personal/documentation/TODO ============================================================================== --- soc2013/jmuniz/personal/documentation/TODO Tue Aug 20 14:59:31 2013 (r256208) +++ soc2013/jmuniz/personal/documentation/TODO Tue Aug 20 15:38:00 2013 (r256209) @@ -5,57 +5,42 @@ Tasks are ordered hierarchically: the hyphens indicate the level of each node and any leafs they contain, and plus signs identify nodes with leafs. -This document -------------- -- delve more into specifics -+ areas that need work --- README --+ TESTING ---- instructions on configuring, building, and installing development sources --+ NEWS ---- track user oriented changes in this log to show current progress --+ INSTALL ---- instructions on configuring, building, and installing official releases --+ THANKS ---- add credits, urls, and licenses of works used --- FAQ --- Wiki --- Makefile --- PK-0.6.11 patches --- KPackageKit port --- KPackageKit patches --- gnome-packagekit port --- gnome-packagekit patches --- Patches for both front ends - README ------ -- repository description +- project / repository description - contact information - link to wiki for more information -- bug tracking - -BUGS ----- -+ document all bugs and limitations --- any bugs that are discovered during testing +- name of applicable software license NEWS ---- +- track user oriented changes in this log to show current progress INSTALL ------- +- once a plan is in place to publish an official release, the instructions to configure and install from the ports collection will be in this file THANKS ------ +- update to no longer reflect old hierarchy +- list assets used +- licenses +- names of contributors, or a group name if possible +- date that the source was accessed and the source version +- url to source +- project title FAQ --- - troubleshooting - usage -- requirements ++ requirements +-- lsof requires the kernel sources to build - bug tracking -- repository hierarchy +- repository directory hierarchy ++ status of each port +-- which ports are being actively developed +-- which ports are functioning TESTING ------- @@ -75,18 +60,25 @@ - make sure that milestones are appropriate - present reflective journals - request feedback in the form of bug reports, suggestions, feature requests, patches, regression alerts, and other potentially useful information -- add screenshots to present progress +- add screenshots and video to present progress - instructions for using pk-setter - use cases for overall project - test results (with seed data, test output, and performance results) - link to Google-Melange project page - add more links to external documentation (for related work) - point out that svnweb is not synchronized as well as socsvn.freebsd.org - -Makefile --------- -- contain all operations to test code -- running 'make' in the root of the repository will provide instructions on using the Makefile +- add "completed" column to milestone table +- refer to new documentation files and locations +- add links to new progress reports +- finish deliverables section, adding instructions and other information + +testing.sh +---------- +- keep track of steps in hidden file to allow for resuming +- configure / build / install / deinstall / reinstall +- patch +- restore to unpatched state +- internal dependency tracking for the ports in this repository, to make sure that all required ports are patched pk-setter --------- @@ -99,9 +91,10 @@ - configure pkg to use default repository - create a manual page - a way to reset PackageKit.conf to default using PackageKit.conf.sample +- make pk-setter a packagekit runtime dependency so it is checked for after packagekit is installed -PackageKit-0.6.11 patches -------------------------- +PackageKit-0.6.11 source +------------------------ - show unavailable file lists by downloading the plist from freebsd.org - find out if packagekit-cron could work @@ -112,9 +105,10 @@ - create a file on installation to contain a list of installed back ends for pk-setter + Makefile -- find a better way to add site_ruby/1.8/pkgtools to ruby 1.9 include path --- create an option to install pk-setter AFTER PK; this should prevent circular dependency -- replace manpage generation with a manpage patch -- save a copy of PackageKit.conf to PackageKit.conf.sample for default configuration recovery +-- make sure that pkgng back end option includes all required dependencies +-- change pkgng dependency check to library instead of executable + when Matt finishes the C back end -- update patches to permanent pkgng back end -- update pkg-plist to include the new files @@ -129,10 +123,10 @@ gnome-packagekit-2.32.0 port ---------------------------- + optional --- upgrade port to latest version +-- upgrade port to gnome-packagekit-3.8.2 -KPackageKit-0.6.3.3 patches ---------------------------- +KPackageKit-0.6.3.3 source +-------------------------- - enable help button, patch in help files - when the user clicks "install updates", any get-details transactions generated by update-viewer should be cancelled + user interface files @@ -140,42 +134,32 @@ -- preference option available to keep selections when navigating tabs (default: true) -- clicking on an item in a dependency list should open that items information -gnome-packagekit-2.32.0 patches -------------------------------- +gnome-packagekit-2.32.0 source +------------------------------ + user interface files -+ gpk-application.ui --- convenient access to view updates, install from local file, and preferences --+ gpk-backend-status.ui --+ gpk-client.ui --+ gpk-error.ui --+ gpk-eula.ui --+ gpk-log.ui --+ gpk-prefs.ui --+ gpk-repo.ui --+ gpk-service-pack.ui --+ gpk-signature.ui -+ gpk-update-viewer.ui +--- clicking install should cancel the transaction to get package descriptions + source code files + optional features to look into -- service packs (perhaps replace tarball with PBI) -- mime type installation -- catalogs (would allow for sharing of toolkits for example) -Patches for both front ends +Sources for both front ends --------------------------- - have front ends remember with which back end a port was installed with, and use that back end for updates - customizable permissions (so some users may only update packages, or view data) -- create a confirmation report, listing all changes +- create a pre-confirmation report, listing all changes - easy repository management - easily switch between ports and binary packages -- create mock up concepts - if front ends are new ports, combine the appearance of the front ends - update search results after each keystroke (using a timer delay to prevent extra overhead) - option to automatically start tray icons at boot - configure a port to change from development version to stable version once a specific stable version is released - add jail support, and easy selection of one or more jails to modify - manage remote systems and their jails in a centralized way -- improve transaction history to include more information on each transaction - import and export packagekit and front end configuration files - install specific available versions of software (over the internet) - fail over to another back end if version is not available (help to get latest versions) @@ -196,4 +180,6 @@ - easy means of deleting distfiles - easy means of removing orphaned dependencies - let user reinstall a package +- make sure Matt's back end provides the required output to communicate with front neds +- test front ends for complaints to make sure all ports back end output can communicate with front ends Modified: soc2013/jmuniz/personal/ports/PackageKit-0.6.11/packagekit/Makefile ============================================================================== --- soc2013/jmuniz/personal/ports/PackageKit-0.6.11/packagekit/Makefile Tue Aug 20 14:59:31 2013 (r256208) +++ soc2013/jmuniz/personal/ports/PackageKit-0.6.11/packagekit/Makefile Tue Aug 20 15:38:00 2013 (r256209) @@ -43,10 +43,11 @@ MAKE_ARGS= pkgconfigdir="${PREFIX}/libdata/pkgconfig" .if !defined(SLAVEPORT) -OPTIONS_DEFINE= DOCS +OPTIONS_DEFINE= DOCS NETWORKING OPTIONS_SINGLE= BACKEND OPTIONS_SINGLE_BACKEND= PORTS PKGNG DUMMY OPTIONS_DEFAULT=PORTS +NETWORKING_DESC= Ability to manage PackageKit from a remote host PORTS_DESC= Ruby back end for traditional or new binary packages PKGNG_DESC= C back end for PGN Next Generation, specifically DUMMY_DESC= The dummy back end for developers @@ -64,6 +65,13 @@ CONFIGURE_ARGS+=--disable-man-pages .endif +.if %{PORT_OPTIONS:MNETWORKING} +USE_RC_SUBR= netpackagekitd +PLIST_SUB+= NETWORKING="" +.else +PLIST_SUB+= NETWORKING="@comment " +.endif + .if ${PORT_OPTIONS:MDUMMY} CONFIGURE_ARGS+=--with-default-backend=dummy \ --enable-dummy @@ -73,7 +81,7 @@ .endif .if ${PORT_OPTIONS:MPORTS} -RUN_DEPENDS+= ${LOCALBASE}/sbin/portupgrade:${PORTSDIR}/ports-mgmt/portupgrade +BUILD_DEPENDS+= ${LOCALBASE}/sbin/portupgrade:${PORTSDIR}/ports-mgmt/portupgrade USE_RUBY= yes CONFIGURE_ARGS+=--enable-ports \ --with-default-backend=ports @@ -82,7 +90,7 @@ .endif .if ${PORT_OPTIONS:MPKGNG} -RUN_DEPENDS+= ${LOCALBASE}/sbin/pkg:${PORTSDIR}/ports-mgmt/pkg +BUILD_DEPENDS+= ${LOCALBASE}/sbin/pkg:${PORTSDIR}/ports-mgmt/pkg CONFIGURE_ARGS+=--enable-pkgng \ --with-default-backend=pkgng PLIST_SUB+= PKGNG="" \ @@ -128,6 +136,10 @@ @${INSTALL_DATA} ${WRKSRC}/data/transactions.db \ ${PREFIX}/share/PackageKit/transactions.db @-update-mime-database ${PREFIX}/share/mime > /dev/null + .if ${PORT_OPTIONS:MNETWORKING} + @${INSTALL_DATA} ${PATCHDIR}/netpackagekitd \ + ${LOCALBASE}/sbin/netpackagekitd + .endif .endif .include Added: soc2013/jmuniz/personal/ports/PackageKit-0.6.11/packagekit/files/netpackagekitd.in ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ soc2013/jmuniz/personal/ports/PackageKit-0.6.11/packagekit/files/netpackagekitd.in Tue Aug 20 15:38:00 2013 (r256209) @@ -0,0 +1,13 @@ +#!/bin/sh + +# PROVIDE: netpackagekitd + +. /etc/rc.subr + +name=netpackagekitd +rcvar=netpackagekitd_enable + +command="/usr/sbin/${name} + +load_rc_config $name +run_rc_command "$1" Added: soc2013/jmuniz/personal/ports/PackageKit-0.6.11/packagekit/files/netpackagekitd/netpackagekit.c ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ soc2013/jmuniz/personal/ports/PackageKit-0.6.11/packagekit/files/netpackagekitd/netpackagekit.c Tue Aug 20 15:38:00 2013 (r256209) @@ -0,0 +1,139 @@ +/* +* Copyright (C) 2013 Justin Edward Muniz +* +* Licensed under the GNU General Public License Version 2 +* This program is free software; you can redistribute it and/or modify +* it under the terms of the GNU General Public License as published by +* the Free Software Foundation; either version 2 of the License, or +* (at your option) any later version. +* +* Daemon for remote management of PackageKit +* This server listens on port 1200 to for instructions +*/ + +#include +/* +*These might be important for read(): +#include +#include +*/ + +#include +#include +#include + +/* +#include +#include +#include +*/ + +/* Only one client needs to connect at once */ +#define BACKLOG 1 + +/* +static void cleanupSSL() +{ + +} +*/ + +static void listenForConnections() +{ + register int listeningSocket, connectedSocket; + struct sockaddr_in socketInformation; + FILE *masterHost; + int socketAddressStructureSize; + int bytesRead; + char buffer[BUFSIZ+1]; + char userName[32]; +/* + BIO *bio; + + SSL_load_error_strings(); + SSL_load_BIO_strings(); + OpenSSL_add_all_algorithms(); +*/ + + if ((listeningSocket = socket(PF_INET, SOCK_STREAM, 0)) < 0) + { + perror("The socket couldn't be created"); + return 1; + } + + bzero(&socketInformation, sizeof socketInformation); + + socketInformation.sin_family = AF_INET; + socketInformation.sin_port = htons(1200); + + if (INADDR_ANY) + socketInformation.sin_addr.s_addr = htonl(INADDR_ANY); + + if (bind(listeningSocket, (struct sockaddr *)&socketInformation, sizeof socketInformation) < 0) + { + perror("Could not bind the socket"); + close(listeningSocket); + return 2; + } + + switch (fork()) + { + case -1: + perror("Could not create the child process"); + close(listeningSocket); + return 3; + break; + case 0: + break; + default: + close(listeningSocket); + return 0; + break; + } + + listen(listeningSocket, BACKLOG); + + while (true) + { + socketAddressStructureSize = sizeof socketInformation; + + if ((connectedSocket = accept(listeningSocket, (struct sockaddr *)&socketInformation, &socketAddressStructureSize)) < 0) + { + perror("Could not initialize incoming connection"); + close(listeningSocket); + return 4; + } + + if ((masterHost = fdopen(connectedSocket, "w+")) == NULL) + { + perror("Could not open socket file descriptor"); + close(connectedSocket); + close(listeningSocket); + return 5; + } + + /* Authenticate host */ + while (*userName='\0') + { + if (read(connectedSocket, buffer, BUFSIZ) > 0) + userName = buffer; + } + + /* Make sure that random numbers are exchanged and password hashes are generated with them, so that encrypted passwords do not traverse the network */ + + /* Receive and parse instructions from host */ + + /* Do work */ + + /* Send response back to server */ + + fclose(masterHost); + close(connectedSocket); + } +} + +int main(int argc, char *argv[]) +{ + listenForConnections(); +} + Modified: soc2013/jmuniz/personal/ports/PackageKit-0.6.11/packagekit/pkg-plist ============================================================================== --- soc2013/jmuniz/personal/ports/PackageKit-0.6.11/packagekit/pkg-plist Tue Aug 20 14:59:31 2013 (r256208) +++ soc2013/jmuniz/personal/ports/PackageKit-0.6.11/packagekit/pkg-plist Tue Aug 20 15:38:00 2013 (r256209) @@ -452,3 +452,4 @@ @dirrmtry etc/PackageKit @exec %%LOCALBASE%%/bin/update-mime-database %D/share/mime @unexec %%LOCALBASE%%/bin/update-mime-database %D/share/mime +%%NETWORKING%%sbin/netpackagekitd Modified: soc2013/jmuniz/personal/ports/gnome-packagekit-2.32.0/gnome-packagekit/Makefile ============================================================================== --- soc2013/jmuniz/personal/ports/gnome-packagekit-2.32.0/gnome-packagekit/Makefile Tue Aug 20 14:59:31 2013 (r256208) +++ soc2013/jmuniz/personal/ports/gnome-packagekit-2.32.0/gnome-packagekit/Makefile Tue Aug 20 15:38:00 2013 (r256209) @@ -23,7 +23,7 @@ unique-1.0:${PORTSDIR}/x11-toolkits/unique \ upower-glib.1:${PORTSDIR}/sysutils/upower \ canberra-gtk.0:${PORTSDIR}/audio/libcanberra \ - libgcrypt.so.18:${PORTSDIR}/security/libgcrypt + gcrypt:${PORTSDIR}/security/libgcrypt RUN_DEPENDS= yelp:${PORTSDIR}/x11/yelp USE_GMAKE= yes Modified: soc2013/jmuniz/personal/source/PackageKit-0.6.11/backends/Makefile.am ============================================================================== --- soc2013/jmuniz/personal/source/PackageKit-0.6.11/backends/Makefile.am Tue Aug 20 14:59:31 2013 (r256208) +++ soc2013/jmuniz/personal/source/PackageKit-0.6.11/backends/Makefile.am Tue Aug 20 15:38:00 2013 (r256209) @@ -81,7 +81,7 @@ endif if BACKEND_TYPE_PKGNG -SUBDIRS+= pkgng +SUBDIRS += pkgng endif clean-local : Modified: soc2013/jmuniz/personal/source/PackageKit-0.6.11/configure ============================================================================== --- soc2013/jmuniz/personal/source/PackageKit-0.6.11/configure Tue Aug 20 14:59:31 2013 (r256208) +++ soc2013/jmuniz/personal/source/PackageKit-0.6.11/configure Tue Aug 20 15:38:00 2013 (r256209) @@ -16944,7 +16944,7 @@ if test x$enable_strict != xno; then if test "$GCC" = "yes"; then - WARNINGFLAGS_CPP="$WARNINGFLAGS_CPP -Werror" +#$ WARNINGFLAGS_CPP="$WARNINGFLAGS_CPP -Werror" fi fi @@ -16961,7 +16961,7 @@ WARNINGFLAGS_CPP="$WARNINGFLAGS_CPP -Wreturn-type" WARNINGFLAGS_CPP="$WARNINGFLAGS_CPP -Wformat-nonliteral" WARNINGFLAGS_CPP="$WARNINGFLAGS_CPP -Wformat-security" - WARNINGFLAGS_CPP="$WARNINGFLAGS_CPP -Wmissing-include-dirs" + WARNINGFLAGS_CPP="$WARNINGFLAGS_CPP " WARNINGFLAGS_CPP="$WARNINGFLAGS_CPP -Wmissing-format-attribute" # WARNINGFLAGS_CPP="$WARNINGFLAGS_CPP -Wclobbered" # WARNINGFLAGS_CPP="$WARNINGFLAGS_CPP -Wempty-body" @@ -20749,7 +20749,7 @@ elif test -f /usr/local/sbin/portupgrade ; then with_default_backend=ports elif test -f /usr/sbin/pkgng ; then - with_default_backend=pkgng ; then + with_default_backend=pkgng else with_default_backend=dummy fi Modified: soc2013/jmuniz/personal/source/PackageKit-0.6.11/configure.ac ============================================================================== --- soc2013/jmuniz/personal/source/PackageKit-0.6.11/configure.ac Tue Aug 20 14:59:31 2013 (r256208) +++ soc2013/jmuniz/personal/source/PackageKit-0.6.11/configure.ac Tue Aug 20 15:38:00 2013 (r256209) @@ -878,7 +878,7 @@ Razor backend: ${enable_razor} Pacman backend: ${enable_pacman} PiSi backend: ${enable_pisi} - Pkgng backend: ${enable_ports} + Pkgng backend: ${enable_pkgng} poldek backend: ${enable_poldek} Portage backend: ${enable_portage} Ports backend: ${enable_ports} Added: soc2013/jmuniz/personal/tools/testing.sh ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ soc2013/jmuniz/personal/tools/testing.sh Tue Aug 20 15:38:00 2013 (r256209) @@ -0,0 +1,15 @@ +#!/bin/sh +# +# Copyright (C) 2013 Justin Edward Muniz +# +# Licensed under the GNU General Public License Version 2 +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# Function: Aid in quick testing of developing code, by way of simple sub-commands +# +# Arguments that can be passed... + + From owner-svn-soc-all@FreeBSD.ORG Tue Aug 20 19:23:56 2013 Return-Path: Delivered-To: svn-soc-all@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id B50E1C7E for ; Tue, 20 Aug 2013 19:23:56 +0000 (UTC) (envelope-from mattbw@FreeBSD.org) Received: from socsvn.freebsd.org (socsvn.freebsd.org [IPv6:2001:1900:2254:206a::50:2]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id A19F72632 for ; Tue, 20 Aug 2013 19:23:56 +0000 (UTC) Received: from socsvn.freebsd.org ([127.0.1.124]) by socsvn.freebsd.org (8.14.7/8.14.7) with ESMTP id r7KJNujD065930 for ; Tue, 20 Aug 2013 19:23:56 GMT (envelope-from mattbw@FreeBSD.org) Received: (from www@localhost) by socsvn.freebsd.org (8.14.7/8.14.6/Submit) id r7KJNuNl065925 for svn-soc-all@FreeBSD.org; Tue, 20 Aug 2013 19:23:56 GMT (envelope-from mattbw@FreeBSD.org) Date: Tue, 20 Aug 2013 19:23:56 GMT Message-Id: <201308201923.r7KJNuNl065925@socsvn.freebsd.org> X-Authentication-Warning: socsvn.freebsd.org: www set sender to mattbw@FreeBSD.org using -f From: mattbw@FreeBSD.org To: svn-soc-all@FreeBSD.org Subject: socsvn commit: r256216 - soc2013/mattbw/backend/query MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-soc-all@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: SVN commit messages for the entire Summer of Code repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Aug 2013 19:23:56 -0000 Author: mattbw Date: Tue Aug 20 19:23:56 2013 New Revision: 256216 URL: http://svnweb.FreeBSD.org/socsvn/?view=rev&rev=256216 Log: (Still broken) Start on the replacement for the emitter-based query flow. This doesn't do anything yet, but I've sketched out where the new ID transformation would occur. Further work to be done this week. Modified: soc2013/mattbw/backend/query/match.c Modified: soc2013/mattbw/backend/query/match.c ============================================================================== --- soc2013/mattbw/backend/query/match.c Tue Aug 20 18:22:04 2013 (r256215) +++ soc2013/mattbw/backend/query/match.c Tue Aug 20 19:23:56 2013 (r256216) @@ -26,8 +26,6 @@ #include "match.h" /* query_match_... */ - - /* * Runs a query over the PackageIDs selected in the backend that sends the * first match to an emitting function. @@ -36,17 +34,24 @@ query_match_id_to_emitter(PkBackend *backend, unsigned int load_flags, emit_ptr emitter) { - struct query_source s; - struct query_target t; - - s.unused = true; - s.position = 0; - s.total = 1; - s.type = QUERY_BACKEND_IDS; - - t.f = emitter; - t.load_flags = load_flags; - t.error_if_not_found = true; + bool success; + guint count; + gchar **package_ids; + struct query_id *query_ids; + + assert(backend != NULL); + assert(emitter != NULL); + + success = false; + + package_ids = pk_backend_get_strv("package_ids"); + assert(package_ids != NULL); + count = g_strv_length(package_ids); + + query_ids = query_id_array_from_package_ids(package_ids, count); + if (query_ids != NULL) { + /* More stuff here */ + } - return query_do(backend, &s, &t); + return success; } From owner-svn-soc-all@FreeBSD.ORG Tue Aug 20 19:47:53 2013 Return-Path: Delivered-To: svn-soc-all@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 64304454 for ; Tue, 20 Aug 2013 19:47:53 +0000 (UTC) (envelope-from mattbw@FreeBSD.org) Received: from socsvn.freebsd.org (socsvn.freebsd.org [IPv6:2001:1900:2254:206a::50:2]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 3728927BE for ; Tue, 20 Aug 2013 19:47:53 +0000 (UTC) Received: from socsvn.freebsd.org ([127.0.1.124]) by socsvn.freebsd.org (8.14.7/8.14.7) with ESMTP id r7KJlrlx054787 for ; Tue, 20 Aug 2013 19:47:53 GMT (envelope-from mattbw@FreeBSD.org) Received: (from www@localhost) by socsvn.freebsd.org (8.14.7/8.14.6/Submit) id r7KJlrT8054785 for svn-soc-all@FreeBSD.org; Tue, 20 Aug 2013 19:47:53 GMT (envelope-from mattbw@FreeBSD.org) Date: Tue, 20 Aug 2013 19:47:53 GMT Message-Id: <201308201947.r7KJlrT8054785@socsvn.freebsd.org> X-Authentication-Warning: socsvn.freebsd.org: www set sender to mattbw@FreeBSD.org using -f From: mattbw@FreeBSD.org To: svn-soc-all@FreeBSD.org Subject: socsvn commit: r256217 - soc2013/mattbw/backend/query MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-soc-all@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: SVN commit messages for the entire Summer of Code repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Aug 2013 19:47:53 -0000 Author: mattbw Date: Tue Aug 20 19:47:52 2013 New Revision: 256217 URL: http://svnweb.FreeBSD.org/socsvn/?view=rev&rev=256217 Log: (still broken) Flesh out the template from last commit. Focus will be on creating a new function, "query_match_ids", which takes a list of QueryIDs and a database, and returns a list of packages if and only if all QueryIDs could be matched to packages. Modified: soc2013/mattbw/backend/query/match.c Modified: soc2013/mattbw/backend/query/match.c ============================================================================== --- soc2013/mattbw/backend/query/match.c Tue Aug 20 19:23:56 2013 (r256216) +++ soc2013/mattbw/backend/query/match.c Tue Aug 20 19:47:52 2013 (r256217) @@ -26,6 +26,10 @@ #include "match.h" /* query_match_... */ +static bool emit_packages(PkBackend *backend, struct pkgdb *db, emit_ptr emitter, struct pkg *packages, unsigned int count); +static bool query_with_db(struct pkgdb *db, PkBackend *backend, unsigned int load_flags, emit_ptr emitter); + + /* * Runs a query over the PackageIDs selected in the backend that sends the * first match to an emitting function. @@ -35,14 +39,38 @@ emit_ptr emitter) { bool success; + + assert(backend != NULL); + assert(emitter != NULL); + + success = false; + + db = db_open_remote(backend); + if (db != NULL) { + success = query_with_db(db, backend, load_flags, emitter); + + db_close(&db); + } + return success; +} + +static bool +query_with_db(struct pkgdb *db, PkBackend *backend, unsigned int load_flags, + emit_ptr emitter) +{ + bool success; guint count; - gchar **package_ids; + struct pkg *packages; + struct pkgdb *db; struct query_id *query_ids; + gchar **package_ids; assert(backend != NULL); + assert(db != NULL); assert(emitter != NULL); success = false; + packages = NULL; package_ids = pk_backend_get_strv("package_ids"); assert(package_ids != NULL); @@ -50,7 +78,35 @@ query_ids = query_id_array_from_package_ids(package_ids, count); if (query_ids != NULL) { - /* More stuff here */ + packages = query_match_ids(query_ids, count, load_flags); + } + + if (packages != NULL) { + success = emit_packages(backend, db, emitter, packages, count); + query_free_packages(packages, count); + } + + return success; +} + +static bool +emit_packages(PkBackend *backend, struct pkgdb *db, emit_ptr emitter, + struct pkg *packages, unsigned int count) +{ + bool success; + unsigned int i; + + assert(backend != NULL); + assert(db != NULL); + assert(emitter != NULL); + assert(packages != NULL); + + success = true; + for (i = 0; i < count; i++) { + success = emitter(backend, packages + i); + if (success = false) { + break; + } } return success; From owner-svn-soc-all@FreeBSD.ORG Wed Aug 21 13:57:49 2013 Return-Path: Delivered-To: svn-soc-all@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 15AAE7DF for ; Wed, 21 Aug 2013 13:57:49 +0000 (UTC) (envelope-from dpl@FreeBSD.org) Received: from socsvn.freebsd.org (socsvn.freebsd.org [IPv6:2001:1900:2254:206a::50:2]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id E87A62D85 for ; Wed, 21 Aug 2013 13:57:48 +0000 (UTC) Received: from socsvn.freebsd.org ([127.0.1.124]) by socsvn.freebsd.org (8.14.7/8.14.7) with ESMTP id r7LDvmiB016161 for ; Wed, 21 Aug 2013 13:57:48 GMT (envelope-from dpl@FreeBSD.org) Received: (from www@localhost) by socsvn.freebsd.org (8.14.7/8.14.6/Submit) id r7LDvm5S016151 for svn-soc-all@FreeBSD.org; Wed, 21 Aug 2013 13:57:48 GMT (envelope-from dpl@FreeBSD.org) Date: Wed, 21 Aug 2013 13:57:48 GMT Message-Id: <201308211357.r7LDvm5S016151@socsvn.freebsd.org> X-Authentication-Warning: socsvn.freebsd.org: www set sender to dpl@FreeBSD.org using -f From: dpl@FreeBSD.org To: svn-soc-all@FreeBSD.org Subject: socsvn commit: r256237 - in soc2013/dpl/head/lib/libzcap: . test zlibworker MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-soc-all@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: SVN commit messages for the entire Summer of Code repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 Aug 2013 13:57:49 -0000 Author: dpl Date: Wed Aug 21 13:57:48 2013 New Revision: 256237 URL: http://svnweb.FreeBSD.org/socsvn/?view=rev&rev=256237 Log: Simplified code. Deleted: soc2013/dpl/head/lib/libzcap/inffast.c soc2013/dpl/head/lib/libzcap/inffast.h soc2013/dpl/head/lib/libzcap/inftrees.c soc2013/dpl/head/lib/libzcap/inftrees.h soc2013/dpl/head/lib/libzcap/trees.c soc2013/dpl/head/lib/libzcap/trees.h Modified: soc2013/dpl/head/lib/libzcap/Makefile soc2013/dpl/head/lib/libzcap/capsicum.c soc2013/dpl/head/lib/libzcap/commands.c soc2013/dpl/head/lib/libzcap/infback.c soc2013/dpl/head/lib/libzcap/inflate.c soc2013/dpl/head/lib/libzcap/inflate.h soc2013/dpl/head/lib/libzcap/test/zcaplibtest.c soc2013/dpl/head/lib/libzcap/zlibworker/commands.c soc2013/dpl/head/lib/libzcap/zlibworker/zlibworker.c Modified: soc2013/dpl/head/lib/libzcap/Makefile ============================================================================== --- soc2013/dpl/head/lib/libzcap/Makefile Wed Aug 21 08:01:52 2013 (r256236) +++ soc2013/dpl/head/lib/libzcap/Makefile Wed Aug 21 13:57:48 2013 (r256237) @@ -26,10 +26,7 @@ SRCS+= gzread.c SRCS+= gzwrite.c SRCS+= infback.c -SRCS+= inffast.c SRCS+= inflate.c -SRCS+= inftrees.c -SRCS+= trees.c SRCS+= uncompr.c SRCS+= zopen.c SRCS+= zutil.c Modified: soc2013/dpl/head/lib/libzcap/capsicum.c ============================================================================== --- soc2013/dpl/head/lib/libzcap/capsicum.c Wed Aug 21 08:01:52 2013 (r256236) +++ soc2013/dpl/head/lib/libzcap/capsicum.c Wed Aug 21 13:57:48 2013 (r256237) @@ -54,13 +54,12 @@ nvlist_t * sendCommand( nvlist_t *nvl ) { + nvlist_t *new; if( nvlist_send(sv[1], nvl) != 0 ) err(1, "zcaplib: nvlist_send()"); - - if ((nvl = nvlist_recv(sv[1])) == NULL) + if ((new = nvlist_recv(sv[1])) == NULL) err(1, "nvlist_recv(): nvl is NULL"); - - return (nvl); + return (new); } void killChild(void) { Modified: soc2013/dpl/head/lib/libzcap/commands.c ============================================================================== --- soc2013/dpl/head/lib/libzcap/commands.c Wed Aug 21 08:01:52 2013 (r256236) +++ soc2013/dpl/head/lib/libzcap/commands.c Wed Aug 21 13:57:48 2013 (r256237) @@ -98,8 +98,8 @@ void destroy(void) { - nvlist_destroy(nvl); nvlist_destroy(args); + nvlist_destroy(nvl); nvlist_destroy(result); } @@ -1259,8 +1259,9 @@ nvlist_add_binary(args, "buf", buf, len); nvlist_add_number(args, "len", len); nvlist_add_nvlist(nvl, "args", args); - + fprintf(stderr, "Before sending: %p\n", nvl); result = sendCommand(nvl); + fprintf(stderr, "After sending: %p\n", result); ret = nvlist_take_number(result, "result"); destroy(); return(ret); Modified: soc2013/dpl/head/lib/libzcap/infback.c ============================================================================== --- soc2013/dpl/head/lib/libzcap/infback.c Wed Aug 21 08:01:52 2013 (r256236) +++ soc2013/dpl/head/lib/libzcap/infback.c Wed Aug 21 13:57:48 2013 (r256237) @@ -4,9 +4,6 @@ */ #include "zutil.h" -#include "inftrees.h" -#include "inflate.h" -#include "inffast.h" extern int zcapcmd_inflateBackInit(); int ZEXPORT inflateBackInit_(strm, windowBits, window, version, stream_size) Modified: soc2013/dpl/head/lib/libzcap/inflate.c ============================================================================== --- soc2013/dpl/head/lib/libzcap/inflate.c Wed Aug 21 08:01:52 2013 (r256236) +++ soc2013/dpl/head/lib/libzcap/inflate.c Wed Aug 21 13:57:48 2013 (r256237) @@ -4,9 +4,7 @@ */ #include "zutil.h" -#include "inftrees.h" #include "inflate.h" -#include "inffast.h" #ifdef MAKEFIXED # ifndef BUILDFIXED Modified: soc2013/dpl/head/lib/libzcap/inflate.h ============================================================================== --- soc2013/dpl/head/lib/libzcap/inflate.h Wed Aug 21 08:01:52 2013 (r256236) +++ soc2013/dpl/head/lib/libzcap/inflate.h Wed Aug 21 13:57:48 2013 (r256237) @@ -3,10 +3,34 @@ * For conditions of distribution and use, see copyright notice in zlib.h */ -/* WARNING: this file should *not* be used by applications. It is - part of the implementation of the compression library and is - subject to change. Applications should only use zlib.h. - */ +typedef struct { + unsigned char op; /* operation, extra bits, table bits */ + unsigned char bits; /* bits in this part of the code */ + unsigned short val; /* offset in table or code value */ +} code; + + +/* Maximum size of the dynamic table. The maximum number of code structures is + 1444, which is the sum of 852 for literal/length codes and 592 for distance + codes. These values were found by exhaustive searches using the program + examples/enough.c found in the zlib distribtution. The arguments to that + program are the number of symbols, the initial root table size, and the + maximum bit length of a code. "enough 286 9 15" for literal/length codes + returns returns 852, and "enough 30 6 15" for distance codes returns 592. + The initial root table size (9 or 6) is found in the fifth argument of the + inflate_table() calls in inflate.c and infback.c. If the root table size is + changed, then these maximum sizes would be need to be recalculated and + updated. */ +#define ENOUGH_LENS 852 +#define ENOUGH_DISTS 592 +#define ENOUGH (ENOUGH_LENS+ENOUGH_DISTS) + +/* Type of code to build for inflate_table() */ +typedef enum { + CODES, + LENS, + DISTS +} codetype; /* define NO_GZIP when compiling if you want to disable gzip header and trailer decoding by inflate(). NO_GZIP would be used to avoid linking in Modified: soc2013/dpl/head/lib/libzcap/test/zcaplibtest.c ============================================================================== --- soc2013/dpl/head/lib/libzcap/test/zcaplibtest.c Wed Aug 21 08:01:52 2013 (r256236) +++ soc2013/dpl/head/lib/libzcap/test/zcaplibtest.c Wed Aug 21 13:57:48 2013 (r256237) @@ -29,7 +29,7 @@ if ( (data = malloc(10*1024)) == NULL) err(1, "zcaplibtest: malloc()"); - for (i=0; i < (10*1024); i++) { + for (i=0; i < (10*1024); i+=sizeof(long)) { data[i] = rand(); } @@ -42,7 +42,7 @@ /* testinflateEnd(strm);*/ testzlibCompileFlags(); -/* testCompressBound();*/ + testCompressBound(); testchecksums(); return 0; @@ -97,7 +97,12 @@ void testCompressBound(void) { + uLong ret = compressBound(10L); + printf("Compressbound: %lu\n", ret); printf("compressBound()\t...\t"); + + if (ret) printf("OK\n"); + else printf("Error\n"); return; } @@ -106,7 +111,7 @@ { uLong first, second, combined; first = second = combined = 0; - + first = adler32(0L, Z_NULL, 0); printf("adler32: %ld\n", first ); first = adler32(first, (void *)data, 5*1024); @@ -130,6 +135,5 @@ else err(1, "Error\n"); - return; } \ No newline at end of file Modified: soc2013/dpl/head/lib/libzcap/zlibworker/commands.c ============================================================================== --- soc2013/dpl/head/lib/libzcap/zlibworker/commands.c Wed Aug 21 08:01:52 2013 (r256236) +++ soc2013/dpl/head/lib/libzcap/zlibworker/commands.c Wed Aug 21 13:57:48 2013 (r256237) @@ -45,14 +45,14 @@ void zcapcmd_deflateEnd(nvlist_t *nvl, nvlist_t *result) -{ +{ /* uLong ret = -1; if ((args = nvlist_take_nvlist(nvl, "args")) == NULL) - ret = compressBound( + ret = deflateEnd( nvlist_take_number(args, "sourceLen") ); - nvlist_add_number(result, "result", ret ); + nvlist_add_number(result, "result", ret ); */ } void @@ -70,13 +70,13 @@ void zcapcmd_inflateEnd(nvlist_t *nvl, nvlist_t *result) { - uLong ret = -1; + /*uLong ret = -1; if ((args = nvlist_take_nvlist(nvl, "args")) == NULL) - ret = compressBound( + ret = inflateEnd( nvlist_take_number(args, "sourceLen") ); - nvlist_add_number(result, "result", ret ); + nvlist_add_number(result, "result", ret ); */ } /* Advanced functions */ Modified: soc2013/dpl/head/lib/libzcap/zlibworker/zlibworker.c ============================================================================== --- soc2013/dpl/head/lib/libzcap/zlibworker/zlibworker.c Wed Aug 21 08:01:52 2013 (r256236) +++ soc2013/dpl/head/lib/libzcap/zlibworker/zlibworker.c Wed Aug 21 13:57:48 2013 (r256237) @@ -267,7 +267,6 @@ if( nvlist_send(3, result) != 0 ) err(1, "Couldn't send response\n"); - nvlist_destroy(nvl); if (zero) memset(data, 0, 5*1024); } From owner-svn-soc-all@FreeBSD.ORG Thu Aug 22 06:51:01 2013 Return-Path: Delivered-To: svn-soc-all@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id E74DECAD for ; Thu, 22 Aug 2013 06:51:01 +0000 (UTC) (envelope-from jmuniz@FreeBSD.org) Received: from socsvn.freebsd.org (socsvn.freebsd.org [IPv6:2001:1900:2254:206a::50:2]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id C60ED22CF for ; Thu, 22 Aug 2013 06:51:01 +0000 (UTC) Received: from socsvn.freebsd.org ([127.0.1.124]) by socsvn.freebsd.org (8.14.7/8.14.7) with ESMTP id r7M6p1DB008649 for ; Thu, 22 Aug 2013 06:51:01 GMT (envelope-from jmuniz@FreeBSD.org) Received: (from www@localhost) by socsvn.freebsd.org (8.14.7/8.14.6/Submit) id r7M6p1Of008644 for svn-soc-all@FreeBSD.org; Thu, 22 Aug 2013 06:51:01 GMT (envelope-from jmuniz@FreeBSD.org) Date: Thu, 22 Aug 2013 06:51:01 GMT Message-Id: <201308220651.r7M6p1Of008644@socsvn.freebsd.org> X-Authentication-Warning: socsvn.freebsd.org: www set sender to jmuniz@FreeBSD.org using -f From: jmuniz@FreeBSD.org To: svn-soc-all@FreeBSD.org Subject: socsvn commit: r256282 - soc2013/jmuniz/personal/documentation MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-soc-all@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: SVN commit messages for the entire Summer of Code repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Aug 2013 06:51:02 -0000 Author: jmuniz Date: Thu Aug 22 06:51:01 2013 New Revision: 256282 URL: http://svnweb.FreeBSD.org/socsvn/?view=rev&rev=256282 Log: Updated credits to be readable and include three more packages recently added to the repository. Modified: soc2013/jmuniz/personal/documentation/THANKS Modified: soc2013/jmuniz/personal/documentation/THANKS ============================================================================== --- soc2013/jmuniz/personal/documentation/THANKS Thu Aug 22 05:53:47 2013 (r256281) +++ soc2013/jmuniz/personal/documentation/THANKS Thu Aug 22 06:51:01 2013 (r256282) @@ -1,25 +1,101 @@ -Thanks and Credit for Works in this Repository ----------------------------------------------- +Thanks and Credit for Works Used in This Repository +--------------------------------------------------- +--------------------------------------------------- + +Multiple software packages have been used in this repository, as foundations to base +code enhancements from. All code that has been modified is included in jmuniz/personal/ +in the Summer of Code repository. The corresponding original material is in jmuniz/vendor/. +The two subdirectories contain similar hierarchies, as documented in +jmuniz/personal/documenation/FAQ. + +The credits are divided into two groups for convenience and completeness: ports and source. +The original ports are all in jmuniz/vendor/ports/. The source is in jmuniz/vendor/source/. + +Please assist me in correcting mistakes, if any are found. + + +Modifications to the Ports Collection +------------------------------------- +------------------------------------- + + +gnome-packagekit-2.32.0 +----------------------- + +Maintainer: gnome@FreeBSD.org +URL: http://svn.freebsd.org/ports/head/ports-mgmt/gnome-packagekit/ + +KPackageKit-0.6.3.3 +------------------- + +Maintainer: kde@FreeBSD.org +URL: http://svn.freebsd.org/ports/head/ports-mgmt/kpackagekit/ + +PackageKit-0.6.11 +----------------- + +Maintainer: gnome@FreeBSD.org +URL: http://svn.freebsd.org/ports/head/ports-mgmt/packagekit/ + +PackageKit-0.8.9 +---------------- + +Maintainer: gnome@FreeBSD.org +URL: http://people.freebsd.org/~kwm/packagekit-0.8.9.diff + +PackageKit-Qt-0.6.11 +-------------------- + +Maintainer: gnome@FreeBSD.org +URL: http://svn.freebsd.org/ports/head/ports-mgmt/packagekit-qt4/ + +Packagekit-Qt-0.8.9 +------------------- + +Maintainer: gnome@FreeBSD.org +URL: http://people.freebsd.org/~kwm/packagekit-0.8.9.diff + + +Modifications to Source Code Distributions +------------------------------------------ +------------------------------------------ + + +Apper-0.8.0 +----------- + +Maintainer: dantti12@gmail.com +URL: http://mirrors.mit.edu/kde/stable/apper/0.8.0/src/apper-0.8.0.tar.bz2 + +gnome-packagekit-2.32.0 +----------------------- + +Maintainer: richard@hughsie.com +URL: http://ftp.gnome.org/pub/GNOME/sources/gnome-packagekit/2.32/gnome-packagekit-2.32.0.tar.bz2 + +gnome-packagekit-3.8.2 +----------------------- + +Maintainer: richard@hughsie.com +URL: http://ftp.gnome.org/pub/GNOME/sources/gnome-packagekit/3.8/gnome-packagekit-3.8.2.tar.xz + +KPackageKit-0.6.3.3 +------------------- + +Maintainer: dantti12@gmail.com +URL: http://opendesktop.org/CONTENT/content-files/84745-kpackagekit-0.6.3.3.tar.bz2 + +PackageKit-0.6.11 +----------------- + +Maintainer: richard@hughsie.com +URL: http://www.packagekit.org/releases/PackageKit-0.6.11.tar.gz + +PackageKit-0.8.9 +---------------- + +Maintainer: richard@hughsie.com +URLS: http://www.packagekit.org/releases/PackageKit-0.8.9.tar.xz + http://gitorious.org/packagekit/packagekit/trees/master/backends/ports -This document contains links to the original software that this repository is based off of, organized -by directory, in an effort to avoid plagiarism and share credit where it is due. -+ jmuniz ---+ KPackageKit -----+ Apper-0.7.2 -----+ KPackageKit-0.6.3.3 =>http://svn.freebsd.org/ports/head/ports-mgmt/kpackagekit/ ---+ PackageKit -----+ PackageKit-0.6.11 => http://svn.freebsd.org/ports/head/ports-mgmt/packagekit/ -----+ PackageKit-0.8.9 => http://people.freebsd.org/~kwm/packagekit-0.8.9.diff, http://gitorious.org/packagekit/packagekit/trees/master/backends/ports ---+ PackageKit-Setter -----+ packagekit-setter ---+ Packagekit-qt4 -----+ PackageKit-qt4-0.6.11 => http://svn.freebsd.org/ports/head/ports-mgmt/packagekit-qt4/ -----+ PackageKit-qt4-0.8.9 ---+ Patch_Overlays -----+ KPackageKit-0.6.3.3 => http://opendesktop.org/CONTENT/content-files/84745-kpackagekit-0.6.3.3.tar.bz2 -----+ PackageKit0.6.11 => http://www.packagekit.org/releases/PackageKit-0.6.11.tar.gz -----+ PackageKit-0.8.9 => http://www.packagekit.org/releases/PackageKit-0.8.9.tar.xz ---+ gnome-packagekit -----+ gnome-packagekit-2.32.0 => http://svn.freebsd.org/ports/head/ports-mgmt/gnome-packagekit/ -----+ gnome-packagekit-3.8.2 From owner-svn-soc-all@FreeBSD.ORG Thu Aug 22 07:22:18 2013 Return-Path: Delivered-To: svn-soc-all@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 9261D9F6 for ; Thu, 22 Aug 2013 07:22:18 +0000 (UTC) (envelope-from jmuniz@FreeBSD.org) Received: from socsvn.freebsd.org (socsvn.freebsd.org [IPv6:2001:1900:2254:206a::50:2]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 7EFF924A1 for ; Thu, 22 Aug 2013 07:22:18 +0000 (UTC) Received: from socsvn.freebsd.org ([127.0.1.124]) by socsvn.freebsd.org (8.14.7/8.14.7) with ESMTP id r7M7MIO0024244 for ; Thu, 22 Aug 2013 07:22:18 GMT (envelope-from jmuniz@FreeBSD.org) Received: (from www@localhost) by socsvn.freebsd.org (8.14.7/8.14.6/Submit) id r7M7MIEj024242 for svn-soc-all@FreeBSD.org; Thu, 22 Aug 2013 07:22:18 GMT (envelope-from jmuniz@FreeBSD.org) Date: Thu, 22 Aug 2013 07:22:18 GMT Message-Id: <201308220722.r7M7MIEj024242@socsvn.freebsd.org> X-Authentication-Warning: socsvn.freebsd.org: www set sender to jmuniz@FreeBSD.org using -f From: jmuniz@FreeBSD.org To: svn-soc-all@FreeBSD.org Subject: socsvn commit: r256290 - soc2013/jmuniz/personal/documentation MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-soc-all@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: SVN commit messages for the entire Summer of Code repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Aug 2013 07:22:18 -0000 Author: jmuniz Date: Thu Aug 22 07:22:18 2013 New Revision: 256290 URL: http://svnweb.FreeBSD.org/socsvn/?view=rev&rev=256290 Log: Made README more useful, by adding a brief description and source code links. Modified: soc2013/jmuniz/personal/documentation/README Modified: soc2013/jmuniz/personal/documentation/README ============================================================================== --- soc2013/jmuniz/personal/documentation/README Thu Aug 22 06:51:59 2013 (r256289) +++ soc2013/jmuniz/personal/documentation/README Thu Aug 22 07:22:18 2013 (r256290) @@ -1,8 +1,13 @@ Qt and QTK+ Front Ends for PKGNG -Google Summer of Code Project -Justin Edward Muniz -FreeBSD -2013 +Google Summer of Code 2013 + +A project to provide users of FreeBSD and derived systems with better GUI +package management tools. + +Student: Justin Muniz +Mentor: Eitan Adler Project homepage: https://wiki.freebsd.org/SummerOfCode2013/pkgQtGtk +Repository Web Browser: https://svnweb.freebsd.org/socsvn/soc2013/jmuniz/ +Repository SVN URL: https://socsvn.freebsd.org/socsvn/soc2013/jmuniz/ From owner-svn-soc-all@FreeBSD.ORG Thu Aug 22 08:10:16 2013 Return-Path: Delivered-To: svn-soc-all@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 4822C79D for ; Thu, 22 Aug 2013 08:10:16 +0000 (UTC) (envelope-from jmuniz@FreeBSD.org) Received: from socsvn.freebsd.org (socsvn.freebsd.org [IPv6:2001:1900:2254:206a::50:2]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 1BA0E2773 for ; Thu, 22 Aug 2013 08:10:16 +0000 (UTC) Received: from socsvn.freebsd.org ([127.0.1.124]) by socsvn.freebsd.org (8.14.7/8.14.7) with ESMTP id r7M8AFkO004777 for ; Thu, 22 Aug 2013 08:10:15 GMT (envelope-from jmuniz@FreeBSD.org) Received: (from www@localhost) by socsvn.freebsd.org (8.14.7/8.14.6/Submit) id r7M8AF4b004763 for svn-soc-all@FreeBSD.org; Thu, 22 Aug 2013 08:10:15 GMT (envelope-from jmuniz@FreeBSD.org) Date: Thu, 22 Aug 2013 08:10:15 GMT Message-Id: <201308220810.r7M8AF4b004763@socsvn.freebsd.org> X-Authentication-Warning: socsvn.freebsd.org: www set sender to jmuniz@FreeBSD.org using -f From: jmuniz@FreeBSD.org To: svn-soc-all@FreeBSD.org Subject: socsvn commit: r256294 - soc2013/jmuniz/personal/documentation MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-soc-all@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: SVN commit messages for the entire Summer of Code repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Aug 2013 08:10:16 -0000 Author: jmuniz Date: Thu Aug 22 08:10:15 2013 New Revision: 256294 URL: http://svnweb.FreeBSD.org/socsvn/?view=rev&rev=256294 Log: Updated task list in TODO to reflect documentation changes. Modified: soc2013/jmuniz/personal/documentation/TODO Modified: soc2013/jmuniz/personal/documentation/TODO ============================================================================== --- soc2013/jmuniz/personal/documentation/TODO Thu Aug 22 07:43:36 2013 (r256293) +++ soc2013/jmuniz/personal/documentation/TODO Thu Aug 22 08:10:15 2013 (r256294) @@ -5,12 +5,9 @@ Tasks are ordered hierarchically: the hyphens indicate the level of each node and any leafs they contain, and plus signs identify nodes with leafs. -README ------- -- project / repository description -- contact information -- link to wiki for more information -- name of applicable software license +All Ports +--------- +- validate with portlint and poudriere NEWS ---- @@ -20,16 +17,6 @@ ------- - once a plan is in place to publish an official release, the instructions to configure and install from the ports collection will be in this file -THANKS ------- -- update to no longer reflect old hierarchy -- list assets used -- licenses -- names of contributors, or a group name if possible -- date that the source was accessed and the source version -- url to source -- project title - FAQ --- - troubleshooting @@ -76,13 +63,12 @@ ---------- - keep track of steps in hidden file to allow for resuming - configure / build / install / deinstall / reinstall -- patch -- restore to unpatched state -- internal dependency tracking for the ports in this repository, to make sure that all required ports are patched +- patch all ports to /usr/ports +- restore to unpatched state (useful for updating the code to the latest revision) +- update ports to latest revision pk-setter --------- -- validate port with portlint and poudriere - create back end list from file saved during PK installation - verify that packagekitd starts with each new configuration, or roll back to last known good configuration - use GetDaemonState() to determine if packagekitd needs to be closed? @@ -100,7 +86,6 @@ PackageKit-0.6.11 port ---------------------- -- test with portlint and poudriere - make sure the back ends play nice with PC-BSD, since it relies on overlays - create a file on installation to contain a list of installed back ends for pk-setter + Makefile