Cannot install synapser due to problems with MarkupSafe

Description

## Operating system Ubuntu 18.04.5 LTS ## Description of the problem There is an error when installing `MarkupSafe`: ``` > install.packages("synapser", repos=c("http://ran.synapse.org", "http://cran.fhcrc.org")) Installing package into ‘/home/genetica/R/x86_64-pc-linux-gnu-library/3.6’ (as ‘lib’ is unspecified) probando la URL 'http://ran.synapse.org/src/contrib/synapser_0.8.71.tar.gz' Content type 'application/x-tar' length 132104 bytes (129 KB) ================================================== downloaded 129 KB * installing *source* package ‘synapser’ ... ** using staged installation Loading required package: R6 Loading required package: rjson Loading required package: R6 Loading required package: rjson You are using pip version 18.1, however version 20.2.3 is available. You should consider upgrading via the 'pip install --upgrade pip' command. You are using pip version 18.1, however version 20.2.3 is available. You should consider upgrading via the 'pip install --upgrade pip' command. You are using pip version 18.1, however version 20.2.3 is available. You should consider upgrading via the 'pip install --upgrade pip' command. Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-install-c4ouwebu/MarkupSafe/ You are using pip version 18.1, however version 20.2.3 is available. You should consider upgrading via the 'pip install --upgrade pip' command. Traceback (most recent call last): File "<string>", line 1, in <module> File "/tmp/RtmpSd9Qnv/R.INSTALL8fb572759ef/synapser/inst/python/installPythonClient.py", line 137, in main ], localSitePackages) File "/tmp/RtmpSd9Qnv/R.INSTALL8fb572759ef/synapser/inst/python/installPythonClient.py", line 172, in _install_pip raise Exception("pip.main returned {} when installing {}".format(rc, package)) Exception : pip.main returned 1 when installing MarkupSafe==1.0 Error in PythonEmbedInR::pyExec(command) : An error has occurred while executing Python code. See traceback above. Ejecución interrumpida ERROR: configuration failed for package ‘synapser’ * removing ‘/home/genetica/R/x86_64-pc-linux-gnu-library/3.6/synapser’ The downloaded source packages are in ‘/tmp/RtmpWEd6TS/downloaded_packages’ Warning message: In install.packages("synapser", repos = c("http://ran.synapse.org", : installation of package ‘synapser’ had non-zero exit status ``` The problem is due to that synapser is trying to install an old version of MarkupSafe. When trying to install it in command line, directly with pip, I can install `MarkupSafe 1.1`: ``` > pip3 install MarkupSafe Collecting MarkupSafe Cache entry deserialization failed, entry ignored Using cached https://files.pythonhosted.org/packages/b2/5f/23e0023be6bb885d00ffbefad2942bc51a620328ee910f64abe5a8d18dd1/MarkupSafe-1.1.1-cp36-cp36m-manylinux1_x86_64.whl Installing collected packages: MarkupSafe Successfully installed MarkupSafe-1.1.1 ``` But not `MarkupSafe 1.0`: ``` > pip3 install MarkupSafe-1.0 Collecting MarkupSafe-1.0 Exception: Traceback (most recent call last): File "/usr/lib/python3/dist-packages/pip/basecommand.py", line 215, in main status = self.run(options, args) File "/usr/lib/python3/dist-packages/pip/commands/install.py", line 353, in run wb.build(autobuilding=True) File "/usr/lib/python3/dist-packages/pip/wheel.py", line 749, in build self.requirement_set.prepare_files(self.finder) File "/usr/lib/python3/dist-packages/pip/req/req_set.py", line 380, in prepare_files ignore_dependencies=self.ignore_dependencies)) File "/usr/lib/python3/dist-packages/pip/req/req_set.py", line 554, in _prepare_file require_hashes File "/usr/lib/python3/dist-packages/pip/req/req_install.py", line 278, in populate_link self.link = finder.find_requirement(self, upgrade) File "/usr/lib/python3/dist-packages/pip/index.py", line 465, in find_requirement all_candidates = self.find_all_candidates(req.name) File "/usr/lib/python3/dist-packages/pip/index.py", line 423, in find_all_candidates for page in self._get_pages(url_locations, project_name): File "/usr/lib/python3/dist-packages/pip/index.py", line 568, in _get_pages page = self._get_page(location) File "/usr/lib/python3/dist-packages/pip/index.py", line 683, in _get_page return HTMLPage.get_page(link, session=self.session) File "/usr/lib/python3/dist-packages/pip/index.py", line 795, in get_page resp.raise_for_status() File "/usr/share/python-wheels/requests-2.18.4-py2.py3-none-any.whl/requests/models.py", line 935, in raise_for_status raise HTTPError(http_error_msg, response=self) requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://pypi.org/simple/markupsafe-1-0/ ``` ### Expected behavior Install synapser. ### Actual behavior Synapser fails to install. ## Output of `sessionInfo()` ``` > sessionInfo() R version 3.6.3 (2020-02-29) Platform: x86_64-pc-linux-gnu (64-bit) Running under: Ubuntu 18.04.5 LTS Matrix products: default BLAS: /usr/lib/x86_64-linux-gnu/blas/libblas.so.3.7.1 LAPACK: /usr/lib/x86_64-linux-gnu/lapack/liblapack.so.3.7.1 locale: [1] LC_CTYPE=es_ES.UTF-8 LC_NUMERIC=C [3] LC_TIME=es_ES.UTF-8 LC_COLLATE=es_ES.UTF-8 [5] LC_MONETARY=es_ES.UTF-8 LC_MESSAGES=es_ES.UTF-8 [7] LC_PAPER=es_ES.UTF-8 LC_NAME=C [9] LC_ADDRESS=C LC_TELEPHONE=C [11] LC_MEASUREMENT=es_ES.UTF-8 LC_IDENTIFICATION=C attached base packages: [1] stats graphics grDevices utils datasets methods base loaded via a namespace (and not attached): [1] compiler_3.6.3 tools_3.6.3 ```

Environment

None

Activity

Show:
Jordan Kiang
October 14, 2020, 8:09 AM

This issue arises from a mix of two factors:

  1. This user has a specific version of setuptools installed in their user site-packages directory (e.g. .local/lib/python3.6/site-packages/) This is added to the sys.path of the embedded Python interpreter (also python 3.6). The version of setuptools bundled with synapser does not have this issue, but it is being overridden by the local version.

  2. A bug between particular versions of Markupsafe (1.0 which the current version of Synapser uses) and versions of setuptools.

An test version of synapser with upgraded Markupsafe resolves this for the user.

An alternate fix would be to ensure that the PYTHONPATH of the embedded Python’s site-packages preceded in the user’s site packages in the sys.path, which we could enforce in either synapser or PythonEmbedInR. However there may be cases where we want to allow the user the ability to upgrade a package so this seems overly restrictive.

Jordan Kiang
November 6, 2020, 8:59 PM

This user earlier reported that a test build from branch resolved their issue.

Jordan Kiang
November 13, 2020, 9:13 PM

The user reported this fixed in the RC via the GitHub issue:

https://github.com/Sage-Bionetworks/synapser/issues/286#issuecomment-708498352

Assignee

Jordan Kiang

Reporter

Boardgenius.io service account

Labels

Validator

Bruce Hoff

Development Area

None

Release Version History

None

Fix versions

Priority

Major
Configure