From nobody Fri Mar 25 13:50:38 2022 X-Original-To: dev-commits-ports-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 6803D1A34582; Fri, 25 Mar 2022 13:50:41 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4KQ3Q80hg2z3h2H; Fri, 25 Mar 2022 13:50:38 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1648216240; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=EFiWILDopPJtsT1gtk9ocCL5UHZYTrOhk1ggkLafy5A=; b=XpJ6TDivmKyZEfrSmJKdUUvu+w8ZT4rHQAWeXWiIKi82JPA3b9qH9wxMkyCoyJZF9G+vt/ xxL35feqp9F5WnkGO2E5dQEHcL2aKEmfokhMFCIgLnkyMk7p3PcUW9XVp1qoZZ5/dUOD02 JkqNqMKa6hxZL3unMp55agYwaUKPuDnYWlYGFTchfcrla0gj8CWqnvLa0jt07fUhWbIxbn /XLXgnaH87P1qwhu7zPwil6wOERITetzJDZDo2c5qOehYz0g6XgcuPmh+duABu1haJQFQQ 4uLxA33ZoMA41/hdai+Dxhz5jkoE6dIYgxRlL0esliKNGF0THFUrHC3wUZRsQA== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id A25AC5548; Fri, 25 Mar 2022 13:50:38 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 22PDocQO037213; Fri, 25 Mar 2022 13:50:38 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 22PDocXC037212; Fri, 25 Mar 2022 13:50:38 GMT (envelope-from git) Date: Fri, 25 Mar 2022 13:50:38 GMT Message-Id: <202203251350.22PDocXC037212@gitrepo.freebsd.org> To: ports-committers@FreeBSD.org, dev-commits-ports-all@FreeBSD.org, dev-commits-ports-main@FreeBSD.org From: Po-Chuan Hsieh Subject: git: 331d3b8ebe69 - main - misc/py-xgboost: Fix build with setuptools 58.0.0+ List-Id: Commit messages for all branches of the ports repository List-Archive: https://lists.freebsd.org/archives/dev-commits-ports-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-ports-all@freebsd.org X-BeenThere: dev-commits-ports-all@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: sunpoet X-Git-Repository: ports X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: 331d3b8ebe694fa472655db79db12ad1c73ee1b3 Auto-Submitted: auto-generated ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1648216240; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=EFiWILDopPJtsT1gtk9ocCL5UHZYTrOhk1ggkLafy5A=; b=Fy72GLGllY/YhZTvta7w3hnslAq53UGsBXgKeLnfqOfeXlDe8clgbA0Tr/3hAIssq/Iyoc 8WUAKYTj3HaDcI95X3P4kx4XbqBbtOYJLHGGdkBKPdXUz4cl5CoZUNbf/yynkR0tGwwGKZ +80jrmJG88ICRhU26Ct4XOi8cZYaVYi1VNcMRFwycYZMgMC+tEgvFqW/vYpmpr7URsfK20 fms2T5gpcx0hDhpNCEeZT6kVXmpfA4eY0GYLbauNgZygitYAe760WVIJHb58FhsmPv4/Xq 1TNU1UgOetDZJXtdOcn64lovswA7I/Cq51Zm/mH02meJ4Djfv/9SeLsPF67F4g== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1648216240; a=rsa-sha256; cv=none; b=tT53KM+3QK6oZxwXpMz71KcnKz2QsuC+ZKhQpQEO4LBtuE0EC0VHj/V90DFUIUfeMFU9Z5 t4bkONokd8FeJ31qlJoNTmPT9EENONR43lKTY7zzkxT+AuvT6h+sA9J9FGk6xbc4GaQwmv /cKYeTOZmkh8tph4C9DmiopdRmssDvF2YHDNQxYHhFnfTqI9ahoZx+Qe/tDVciYPP6mEiX 9O6MgcduIAi9qajtLZ38q4Hu/K1jPPXq4TBNdjKuowaJ5kSPG+YYTR5s8T0xgT/pKIZYO6 kOzxN24sWS1TeyNGyypnEIryRc5Vr1SZeQmyahhD4iujSxZgdAYQtaAYa99xOw== ARC-Authentication-Results: i=1; mx1.freebsd.org; none X-ThisMailContainsUnwantedMimeParts: N The branch main has been updated by sunpoet: URL: https://cgit.FreeBSD.org/ports/commit/?id=331d3b8ebe694fa472655db79db12ad1c73ee1b3 commit 331d3b8ebe694fa472655db79db12ad1c73ee1b3 Author: Po-Chuan Hsieh AuthorDate: 2022-03-25 13:33:00 +0000 Commit: Po-Chuan Hsieh CommitDate: 2022-03-25 13:38:17 +0000 misc/py-xgboost: Fix build with setuptools 58.0.0+ With hat: python --- misc/py-xgboost/Makefile | 3 + misc/py-xgboost/files/patch-2to3 | 375 +++++++++++++++++++++++++++++++++++++++ 2 files changed, 378 insertions(+) diff --git a/misc/py-xgboost/Makefile b/misc/py-xgboost/Makefile index 33321c2507ae..ecce53c63555 100644 --- a/misc/py-xgboost/Makefile +++ b/misc/py-xgboost/Makefile @@ -35,6 +35,9 @@ POST_PLIST= fix-plist fix-plist: # https://github.com/dmlc/xgboost/issues/5705 @${REINPLACE_CMD} 's|.*libxgboost${PYTHON_EXT_SUFFIX}.so$$||' ${TMPPLIST} +post-install: + ${PYTHON_CMD} -m compileall -d ${PYTHON_SITELIBDIR} ${STAGEDIR}${PYTHON_SITELIBDIR} + do-test: # tests fail w/out CUDA: https://github.com/dmlc/xgboost/issues/6881 @cd ${WRKSRC}/.. && ${PYTHON_CMD} -m pytest diff --git a/misc/py-xgboost/files/patch-2to3 b/misc/py-xgboost/files/patch-2to3 new file mode 100644 index 000000000000..54eac41ed210 --- /dev/null +++ b/misc/py-xgboost/files/patch-2to3 @@ -0,0 +1,375 @@ +--- xgboost/callback.py.orig 2022-01-17 08:52:31 UTC ++++ xgboost/callback.py +@@ -319,7 +319,7 @@ def _aggcv(rlist): + cvmap[(metric_idx, k)].append(float(v)) + msg = idx + results = [] +- for (metric_idx, k), v in sorted(cvmap.items(), key=lambda x: x[0][0]): ++ for (metric_idx, k), v in sorted(list(cvmap.items()), key=lambda x: x[0][0]): + v = numpy.array(v) + if not isinstance(msg, STRING_TYPES): + msg = msg.decode() +@@ -595,10 +595,10 @@ class EarlyStopping(TrainingCallback): + evals_log: TrainingCallback.EvalsLog) -> bool: + epoch += self.starting_round # training continuation + msg = 'Must have at least 1 validation dataset for early stopping.' +- assert len(evals_log.keys()) >= 1, msg ++ assert len(list(evals_log.keys())) >= 1, msg + data_name = '' + if self.data: +- for d, _ in evals_log.items(): ++ for d, _ in list(evals_log.items()): + if d == self.data: + data_name = d + if not data_name: +@@ -672,8 +672,8 @@ class EvaluationMonitor(TrainingCallback): + + msg: str = f'[{epoch}]' + if rabit.get_rank() == self.printer_rank: +- for data, metric in evals_log.items(): +- for metric_name, log in metric.items(): ++ for data, metric in list(evals_log.items()): ++ for metric_name, log in list(metric.items()): + stdv: Optional[float] = None + if isinstance(log[-1], tuple): + score = log[-1][0] +--- xgboost/compat.py.orig 2022-01-17 08:52:31 UTC ++++ xgboost/compat.py +@@ -48,14 +48,14 @@ except ImportError: + + # sklearn + try: +- from sklearn.base import BaseEstimator +- from sklearn.base import RegressorMixin, ClassifierMixin +- from sklearn.preprocessing import LabelEncoder ++ from .sklearn.base import BaseEstimator ++ from .sklearn.base import RegressorMixin, ClassifierMixin ++ from .sklearn.preprocessing import LabelEncoder + + try: +- from sklearn.model_selection import KFold, StratifiedKFold ++ from .sklearn.model_selection import KFold, StratifiedKFold + except ImportError: +- from sklearn.cross_validation import KFold, StratifiedKFold ++ from .sklearn.cross_validation import KFold, StratifiedKFold + + SKLEARN_INSTALLED = True + +@@ -71,7 +71,7 @@ try: + def to_json(self): + '''Returns a JSON compatible dictionary''' + meta = {} +- for k, v in self.__dict__.items(): ++ for k, v in list(self.__dict__.items()): + if isinstance(v, np.ndarray): + meta[k] = v.tolist() + else: +@@ -82,7 +82,7 @@ try: + # pylint: disable=attribute-defined-outside-init + '''Load the encoder back from a JSON compatible dict.''' + meta = {} +- for k, v in doc.items(): ++ for k, v in list(doc.items()): + if k == 'classes_': + self.classes_ = np.array(v) + continue +--- xgboost/core.py.orig 2022-01-17 08:52:31 UTC ++++ xgboost/core.py +@@ -142,7 +142,7 @@ def _expect(expectations, got): + + def _log_callback(msg: bytes) -> None: + """Redirect logs from native library into Python console""" +- print(py_str(msg)) ++ print((py_str(msg))) + + + def _get_log_callback_func(): +@@ -479,7 +479,7 @@ def _deprecate_positional_args(f): + kwonly_args = [] + all_args = [] + +- for name, param in sig.parameters.items(): ++ for name, param in list(sig.parameters.items()): + if param.kind == Parameter.POSITIONAL_OR_KEYWORD: + all_args.append(name) + elif param.kind == Parameter.KEYWORD_ONLY: +@@ -1346,7 +1346,7 @@ class Booster(object): + def _configure_metrics(self, params: Union[Dict, List]) -> Union[Dict, List]: + if isinstance(params, dict) and 'eval_metric' in params \ + and isinstance(params['eval_metric'], list): +- params = dict((k, v) for k, v in params.items()) ++ params = dict((k, v) for k, v in list(params.items())) + eval_metrics = params['eval_metric'] + params.pop("eval_metric", None) + params = list(params.items()) +@@ -1577,7 +1577,7 @@ class Booster(object): + **kwargs + The attributes to set. Setting a value to None deletes an attribute. + """ +- for key, value in kwargs.items(): ++ for key, value in list(kwargs.items()): + if value is not None: + if not isinstance(value, STRING_TYPES): + raise ValueError("Set Attr only accepts string values") +@@ -1650,7 +1650,7 @@ class Booster(object): + value of the specified parameter, when params is str key + """ + if isinstance(params, Mapping): +- params = params.items() ++ params = list(params.items()) + elif isinstance(params, STRING_TYPES) and value is not None: + params = [(params, value)] + for key, val in params: +--- xgboost/dask.py.orig 2022-01-17 08:52:31 UTC ++++ xgboost/dask.py +@@ -49,9 +49,9 @@ from .sklearn import _cls_predict_proba + from .sklearn import XGBRanker + + if TYPE_CHECKING: +- from dask import dataframe as dd +- from dask import array as da +- import dask ++ from .dask import dataframe as dd ++ from .dask import array as da ++ from . import dask + import distributed + else: + dd = LazyLoader('dd', globals(), 'dask.dataframe') +@@ -152,7 +152,7 @@ def _start_tracker(n_workers: int) -> Dict[str, Any]: + + def _assert_dask_support() -> None: + try: +- import dask # pylint: disable=W0621,W0611 ++ from . import dask # pylint: disable=W0621,W0611 + except ImportError as e: + raise ImportError( + "Dask needs to be installed in order to use this module" +@@ -394,7 +394,7 @@ class DaskDMatrix: + # [(x0, x1, ..), (y0, y1, ..), ..] in delayed form + + # delay the zipped result +- parts = list(map(dask.delayed, zip(*parts))) # pylint: disable=no-member ++ parts = list(map(dask.delayed, list(zip(*parts)))) # pylint: disable=no-member + # At this point, the mental model should look like: + # [(x0, y0, ..), (x1, y1, ..), ..] in delayed form + +@@ -414,7 +414,7 @@ class DaskDMatrix: + + worker_map: Dict[str, "distributed.Future"] = defaultdict(list) + +- for key, workers in who_has.items(): ++ for key, workers in list(who_has.items()): + worker_map[next(iter(workers))].append(key_to_partition[key]) + + self.worker_map = worker_map +@@ -803,7 +803,7 @@ def _dmatrix_from_list_of_parts( + async def _get_rabit_args(n_workers: int, client: "distributed.Client") -> List[bytes]: + '''Get rabit context arguments from data distribution in DaskDMatrix.''' + env = await client.run_on_scheduler(_start_tracker, n_workers) +- rabit_args = [f"{k}={v}".encode() for k, v in env.items()] ++ rabit_args = [f"{k}={v}".encode() for k, v in list(env.items())] + return rabit_args + + # train and predict methods are supposed to be "functional", which meets the +@@ -930,7 +930,7 @@ async def _train_async( + + results = await client.gather(futures, asynchronous=True) + +- return list(filter(lambda ret: ret is not None, results))[0] ++ return list([ret for ret in results if ret is not None])[0] + + + def train( # pylint: disable=unused-argument +@@ -1579,7 +1579,7 @@ class DaskScikitLearnBase(XGBModel): + + def __getstate__(self) -> Dict: + this = self.__dict__.copy() +- if "_client" in this.keys(): ++ if "_client" in list(this.keys()): + del this["_client"] + return this + +@@ -1711,7 +1711,7 @@ class DaskXGBRegressor(DaskScikitLearnBase, XGBRegress + callbacks: Optional[List[TrainingCallback]] = None, + ) -> "DaskXGBRegressor": + _assert_dask_support() +- args = {k: v for k, v in locals().items() if k not in ("self", "__class__")} ++ args = {k: v for k, v in list(locals().items()) if k not in ("self", "__class__")} + return self._client_sync(self._fit_async, **args) + + +@@ -1814,7 +1814,7 @@ class DaskXGBClassifier(DaskScikitLearnBase, XGBClassi + callbacks: Optional[List[TrainingCallback]] = None + ) -> "DaskXGBClassifier": + _assert_dask_support() +- args = {k: v for k, v in locals().items() if k not in ("self", "__class__")} ++ args = {k: v for k, v in list(locals().items()) if k not in ("self", "__class__")} + return self._client_sync(self._fit_async, **args) + + async def _predict_proba_async( +@@ -2002,7 +2002,7 @@ class DaskXGBRanker(DaskScikitLearnBase, XGBRankerMixI + callbacks: Optional[List[TrainingCallback]] = None + ) -> "DaskXGBRanker": + _assert_dask_support() +- args = {k: v for k, v in locals().items() if k not in ("self", "__class__")} ++ args = {k: v for k, v in list(locals().items()) if k not in ("self", "__class__")} + return self._client_sync(self._fit_async, **args) + + # FIXME(trivialfis): arguments differ due to additional parameters like group and qid. +@@ -2067,7 +2067,7 @@ class DaskXGBRFRegressor(DaskXGBRegressor): + callbacks: Optional[List[TrainingCallback]] = None + ) -> "DaskXGBRFRegressor": + _assert_dask_support() +- args = {k: v for k, v in locals().items() if k not in ("self", "__class__")} ++ args = {k: v for k, v in list(locals().items()) if k not in ("self", "__class__")} + _check_rf_callback(early_stopping_rounds, callbacks) + super().fit(**args) + return self +@@ -2131,7 +2131,7 @@ class DaskXGBRFClassifier(DaskXGBClassifier): + callbacks: Optional[List[TrainingCallback]] = None + ) -> "DaskXGBRFClassifier": + _assert_dask_support() +- args = {k: v for k, v in locals().items() if k not in ("self", "__class__")} ++ args = {k: v for k, v in list(locals().items()) if k not in ("self", "__class__")} + _check_rf_callback(early_stopping_rounds, callbacks) + super().fit(**args) + return self +--- xgboost/plotting.py.orig 2022-01-17 08:52:31 UTC ++++ xgboost/plotting.py +@@ -81,7 +81,7 @@ def plot_importance(booster, ax=None, height=0.2, + tuples = sorted(tuples, key=lambda x: x[1])[-max_num_features:] + else: + tuples = sorted(tuples, key=lambda x: x[1]) +- labels, values = zip(*tuples) ++ labels, values = list(zip(*tuples)) + + if ax is None: + _, ax = plt.subplots(1, 1) +@@ -177,13 +177,13 @@ def to_graphviz(booster, fmap='', num_trees=0, rankdir + # squash everything back into kwargs again for compatibility + parameters = 'dot' + extra = {} +- for key, value in kwargs.items(): ++ for key, value in list(kwargs.items()): + extra[key] = value + + if rankdir is not None: + kwargs['graph_attrs'] = {} + kwargs['graph_attrs']['rankdir'] = rankdir +- for key, value in extra.items(): ++ for key, value in list(extra.items()): + if kwargs.get("graph_attrs", None) is not None: + kwargs['graph_attrs'][key] = value + else: +--- xgboost/sklearn.py.orig 2022-01-17 08:52:31 UTC ++++ xgboost/sklearn.py +@@ -455,7 +455,7 @@ class XGBModel(XGBModelBase): + booster : a xgboost booster of underlying model + """ + if not self.__sklearn_is_fitted__(): +- from sklearn.exceptions import NotFittedError ++ from .sklearn.exceptions import NotFittedError + raise NotFittedError('need to call fit or load_model beforehand') + return self._Booster + +@@ -476,7 +476,7 @@ class XGBModel(XGBModelBase): + + # this concatenates kwargs into parameters, enabling `get_params` for + # obtaining parameters from keyword parameters. +- for key, value in params.items(): ++ for key, value in list(params.items()): + if hasattr(self, key): + setattr(self, key, value) + else: +@@ -526,14 +526,14 @@ class XGBModel(XGBModelBase): + internal = {} + while stack: + obj = stack.pop() +- for k, v in obj.items(): ++ for k, v in list(obj.items()): + if k.endswith('_param'): +- for p_k, p_v in v.items(): ++ for p_k, p_v in list(v.items()): + internal[p_k] = p_v + elif isinstance(v, dict): + stack.append(v) + +- for k, v in internal.items(): ++ for k, v in list(internal.items()): + if k in params and params[k] is None: + params[k] = parse_parameter(v) + except ValueError: +@@ -549,7 +549,7 @@ class XGBModel(XGBModelBase): + "enable_categorical" + } + filtered = {} +- for k, v in params.items(): ++ for k, v in list(params.items()): + if k not in wrapper_specific and not callable(v): + filtered[k] = v + return filtered +@@ -568,7 +568,7 @@ class XGBModel(XGBModelBase): + + def save_model(self, fname: Union[str, os.PathLike]) -> None: + meta = {} +- for k, v in self.__dict__.items(): ++ for k, v in list(self.__dict__.items()): + if k == '_le': + meta['_le'] = self._le.to_json() + continue +@@ -607,7 +607,7 @@ class XGBModel(XGBModelBase): + return + meta = json.loads(meta_str) + states = {} +- for k, v in meta.items(): ++ for k, v in list(meta.items()): + if k == '_le': + self._le = XGBoostLabelEncoder() + self._le.from_json(v) +@@ -660,7 +660,7 @@ class XGBModel(XGBModelBase): + + def _set_evaluation_result(self, evals_result: TrainingCallback.EvalsLog) -> None: + if evals_result: +- for val in evals_result.items(): ++ for val in list(evals_result.items()): + evals_result_key = list(val[1].keys())[0] + evals_result[val[0]][evals_result_key] = val[1][evals_result_key] + self.evals_result_ = evals_result +@@ -1455,7 +1455,7 @@ class XGBRFClassifier(XGBClassifier): + feature_weights: Optional[array_like] = None, + callbacks: Optional[List[TrainingCallback]] = None + ) -> "XGBRFClassifier": +- args = {k: v for k, v in locals().items() if k not in ("self", "__class__")} ++ args = {k: v for k, v in list(locals().items()) if k not in ("self", "__class__")} + _check_rf_callback(early_stopping_rounds, callbacks) + super().fit(**args) + return self +@@ -1526,7 +1526,7 @@ class XGBRFRegressor(XGBRegressor): + feature_weights: Optional[array_like] = None, + callbacks: Optional[List[TrainingCallback]] = None + ) -> "XGBRFRegressor": +- args = {k: v for k, v in locals().items() if k not in ("self", "__class__")} ++ args = {k: v for k, v in list(locals().items()) if k not in ("self", "__class__")} + _check_rf_callback(early_stopping_rounds, callbacks) + super().fit(**args) + return self +--- xgboost/training.py.orig 2022-01-17 08:52:31 UTC ++++ xgboost/training.py +@@ -452,7 +452,7 @@ def cv(params, dtrain, num_boost_round=10, nfold=3, st + if 'eval_metric' in params: + params['eval_metric'] = _metrics + else: +- params = dict((k, v) for k, v in params.items()) ++ params = dict((k, v) for k, v in list(params.items())) + + if (not metrics) and 'eval_metric' in params: + if isinstance(params['eval_metric'], list): +@@ -506,7 +506,7 @@ def cv(params, dtrain, num_boost_round=10, nfold=3, st + results[key + '-std'].append(std) + + if should_break: +- for k in results.keys(): # pylint: disable=consider-iterating-dictionary ++ for k in list(results.keys()): # pylint: disable=consider-iterating-dictionary + results[k] = results[k][:(booster.best_iteration + 1)] + break + if as_pandas: