Skip to content

Commit 1be338b

Browse files
committed
Merge branch 'release/0.16.3'
2 parents 83d8860 + f8405bb commit 1be338b

File tree

13 files changed

+167
-52
lines changed

13 files changed

+167
-52
lines changed

CHANGELOG.md

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -6,12 +6,29 @@ The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/).
66

77
## [Unreleased]
88

9+
## [0.16.3] - 2025-09-30
10+
11+
### Changed
12+
- if none of the requested providers are online, raise an error
13+
- added retry for gencode ping (flaky connection)
14+
- increased minimum Python version to 3.9 due to:
15+
- importlib.resources.files()
16+
- tar.extractall(..., filter="data")
17+
18+
### Removed
19+
- removed norns dependency (which was using the deprecated pkg_resources)
20+
21+
### Fixed
22+
- pandas FutureWarning
23+
- unsafe tar.extractall
24+
925
## [0.16.2] - 2025-05-12
1026

1127
### Fixed
1228
- Ensembl release versions no longer includes unreleased versions
1329
- unit tests
1430
- upgraded formatters (and fixed the marked grammar & spelling errors)
31+
- UCSC no longer has genomes on its european mirror
1532

1633
## [0.16.1] - 2023-06-14
1734

@@ -485,6 +502,7 @@ The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/).
485502
- Added `-r` and `--match/--no-match` option to select sequences by regex.
486503

487504
[Unreleased]: https://github.com/vanheeringen-lab/genomepy/compare/master...develop
505+
[0.16.3]: https://github.com/vanheeringen-lab/genomepy/compare/0.16.2...0.16.3
488506
[0.16.2]: https://github.com/vanheeringen-lab/genomepy/compare/0.16.1...0.16.2
489507
[0.16.1]: https://github.com/vanheeringen-lab/genomepy/compare/0.16.0...0.16.1
490508
[0.16.0]: https://github.com/vanheeringen-lab/genomepy/compare/0.15.0...0.16.0

README.md

Lines changed: 6 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -5,9 +5,11 @@
55
[![PyPI version](https://badge.fury.io/py/genomepy.svg)](https://badge.fury.io/py/genomepy)
66
[![GitHub stars](https://badgen.net/github/stars/vanheeringen-lab/genomepy)](https://GitHub.com/vanheeringen-lab/genomepy/stargazers/)
77

8-
[![Build Status](https://app.travis-ci.com/vanheeringen-lab/genomepy.svg?branch=master)](https://app.travis-ci.com/github/vanheeringen-lab/genomepy/branches)
98
[![Maintainability](https://api.codeclimate.com/v1/badges/c4476820f1d21a3e0569/maintainability)](https://codeclimate.com/github/vanheeringen-lab/genomepy/maintainability)
10-
[![Test Coverage](https://api.codeclimate.com/v1/badges/c4476820f1d21a3e0569/test_coverage)](https://codeclimate.com/github/vanheeringen-lab/genomepy/test_coverage)
9+
[![CI/CD](https://github.com/vanheeringen-lab/genomepy/actions/workflows/ci-cd.yml/badge.svg?branch=master)](https://github.com/vanheeringen-lab/genomepy/actions/workflows/ci-cd.yml)
10+
11+
[//]: # ([![Build Status](https://app.travis-ci.com/vanheeringen-lab/genomepy.svg?branch=master)](https://app.travis-ci.com/github/vanheeringen-lab/genomepy/branches))
12+
[//]: # ([![Test Coverage](https://api.codeclimate.com/v1/badges/c4476820f1d21a3e0569/test_coverage)](https://codeclimate.com/github/vanheeringen-lab/genomepy/test_coverage))
1113

1214
[![bioinformatics](https://img.shields.io/badge/DOI-10.1093%2Fbioinformatics%2Fbtad119-%23167da4)](https://doi.org/10.1093/bioinformatics/btad119)
1315
[![zenodo](https://zenodo.org/badge/DOI/10.5281/zenodo.1010458.svg)](https://doi.org/10.5281/zenodo.1010458)
@@ -44,14 +46,14 @@ Don't be shy and [let us know](https://github.com/vanheeringen-lab/genomepy/issu
4446

4547
## Installation
4648

47-
genomepy requires Python 3.7+
49+
genomepy requires Python 3.9+
4850

4951
You can install genomepy via [bioconda](https://bioconda.github.io/), pip or git.
5052

5153
#### Bioconda
5254

5355
```bash
54-
$ conda install -c conda-forge -c bioconda 'genomepy>=0.15'
56+
$ conda install -c conda-forge -c bioconda 'genomepy>=0.16'
5557
```
5658

5759
#### Pip

docs/release_checklist.md

Lines changed: 10 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -29,15 +29,10 @@
2929
5. Check if release works on pypi:
3030

3131
```shell
32-
python -m build
33-
34-
# twine must be up to date (3.3.0 works). System installed twine can interfere.
35-
twine upload --repository-url https://test.pypi.org/legacy/ dist/genomepy-${new_version}*
36-
37-
pip uninstall genomepy
38-
39-
# the \ is to escape the ==, so the variable ${new_version} can be called
40-
pip install --extra-index-url https://test.pypi.org/simple/ genomepy\==${new_version}
32+
pip install uv
33+
uv pip install hatch
34+
hatch build -t wheel
35+
uv pip install --system dist/*.whl --force-reinstall
4136
4237
# tests
4338
genomepy --version
@@ -68,27 +63,24 @@
6863
git push --follow-tags origin develop master
6964
```
7065

71-
8. Upload to pypi:
66+
8. Release on pypi:
7267

73-
```shell
74-
python -m build
75-
twine upload dist/genomepy-${new_version}*
76-
```
68+
Done automatically by github actions.
7769

78-
9. Create release on github (if it not already exists)
70+
9. Release on github:
7971

8072
* Update release with CHANGELOG information from the latest version
8173
* Download the tarball from the github release (`.tar.gz`).
8274
* Attach downloaded tarball to release as binary (this way the download count get tracked).
8375

84-
10a. Update bioconda package
76+
10a. Release on bioconda
8577

8678
* wait for the bioconda bot to create a PR
87-
* update dependencies in the bioconda recipe.yaml if needed
79+
* in the PR, update dependencies in the bioconda recipe.yaml
8880
* approve the PR
8981
* comment: @bioconda-bot please merge
9082

91-
10b. Update bioconda package
83+
10b. Release on bioconda
9284

9385
* fork bioconda/bioconda-recipes
9486
* follow the steps in the [docs](https://bioconda.github.io/contributor/workflow.html)

environment.yml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -15,11 +15,11 @@ dependencies:
1515
- mygene
1616
- mysql <=8.4 # 9.3: 'mysql_native_password' cannot be loaded
1717
- mysql-connector-python <=8.4 # 9.3: 'mysql_native_password' cannot be loaded
18-
- norns >=0.1.6
1918
- numpy
2019
- pandas
2120
- pyfaidx >=0.7.2.1
22-
- python >=3.7
21+
- python >=3.9
22+
- pyyaml
2323
- requests
2424
- tqdm >=4.51
2525

genomepy/__about__.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
"""Metadata"""
22

3-
__version__ = "0.16.2"
3+
__version__ = "0.16.3"
44
__author__ = (
55
"Siebren Frölich, Maarten van der Sande, Tilman Schäfers and Simon van Heeringen"
66
)

genomepy/annotation/__init__.py

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,6 @@
55
from pathlib import Path
66
from typing import Iterable, Optional, Union
77

8-
import numpy as np
98
import pandas as pd
109
from loguru import logger
1110

@@ -242,7 +241,7 @@ def gene_coords(self, genes: Iterable[str], annot: str = "bed") -> pd.DataFrame:
242241
# 1 row per gene
243242
df = (
244243
df.groupby(["gene_name", "seqname", "strand"])
245-
.agg({"start": np.min, "end": np.max})
244+
.agg({"start": "min", "end": "max"})
246245
.reset_index(level=["seqname", "strand"])
247246
)
248247
gene_info = df[["seqname", "start", "end", "strand"]]

genomepy/config/__init__.py

Lines changed: 92 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -1,28 +1,108 @@
11
import os
2+
from collections.abc import MutableMapping as DictMixin
3+
from importlib.resources import files
24
from shutil import copyfile
35

46
from appdirs import user_config_dir
57
from loguru import logger
6-
from norns import config as cfg
8+
from yaml import dump, full_load
79

810
__all__ = ["config", "manage_config"]
911

10-
config = cfg("genomepy", default="config/default.yaml")
1112

13+
class Config(DictMixin):
14+
"""Configuration class.
1215
13-
def generate_config():
16+
State will be shared across config objects.
17+
"""
18+
19+
# Store shared state, Borg pattern
20+
__shared_state = {}
1421
config_dir = user_config_dir("genomepy")
15-
new_config = os.path.join(config_dir, "genomepy.yaml")
22+
config_file = os.path.join(config_dir, "genomepy.yaml")
23+
default_config_file = str(files("genomepy.config").joinpath("default.yaml"))
24+
25+
def __init__(self):
26+
"""
27+
Create a Config object and read a config file.
28+
"""
29+
self.__dict__ = self.__shared_state
30+
31+
# Lookup the config file according to XDG hierarchy
32+
if not os.path.exists(self.config_dir):
33+
os.makedirs(self.config_dir)
34+
if not os.path.exists(self.config_file):
35+
copyfile(self.default_config_file, self.config_file)
36+
self.config = {}
37+
self.load(self.config_file)
38+
39+
def load(self, path=None):
40+
"""
41+
Load yaml-formatted config file.
42+
43+
Parameters
44+
----------
45+
path : str | None
46+
path to config file
47+
"""
48+
if path is None:
49+
path = self.config_file
50+
with open(path) as f:
51+
self.config = full_load(f)
52+
if self.config is None:
53+
logger.warning("config file is empty!")
54+
self.config = {}
55+
56+
def save(self, path=None):
57+
"""
58+
Save current state of config dictionary.
59+
60+
Parameters
61+
----------
62+
path : str | None
63+
path to config file
64+
"""
65+
if path is None:
66+
path = self.config_file
67+
with open(path, "w") as f:
68+
f.write(dump(self.config, default_flow_style=False))
69+
70+
def __getitem__(self, key):
71+
return self.config.__getitem__(key)
1672

17-
# existing config must be removed before norns picks up the default again
18-
os.makedirs(config_dir, exist_ok=True)
19-
if os.path.exists(new_config):
20-
os.remove(new_config)
73+
def __delitem__(self, key):
74+
self.config.__delitem__(key)
2175

22-
default_config = cfg("genomepy", default="config/default.yaml").config_file
23-
copyfile(default_config, new_config)
24-
config.config_file = new_config
25-
logger.info(f"Created config file {new_config}")
76+
def __setitem__(self, key, value):
77+
return self.config.__setitem__(key, value)
78+
79+
def __len__(self):
80+
return self.config.__len__()
81+
82+
def __iter__(self):
83+
return self.config.__iter__()
84+
85+
def __str__(self):
86+
return (
87+
"{"
88+
+ ", ".join(
89+
": ".join([str(key), str(value)]) for key, value in self.items()
90+
)
91+
+ "}"
92+
)
93+
94+
def keys(self):
95+
return self.config.keys()
96+
97+
98+
config = Config()
99+
100+
101+
def generate_config():
102+
# overwrite existing config
103+
copyfile(config.default_config_file, config.config_file)
104+
config.load()
105+
logger.info(f"Created config file {config.config_file}")
26106

27107

28108
def manage_config(command):

genomepy/files.py

Lines changed: 20 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -188,7 +188,26 @@ def extract_tarball(fname, outfile=None, concat=True) -> Union[str, None]:
188188
# Extract files to temporary directory
189189
tmp_dir = mkdtemp(dir=os.path.dirname(outfile))
190190
with tarfile.open(fname) as tar:
191-
tar.extractall(path=tmp_dir)
191+
192+
def is_within_directory(directory, target):
193+
194+
abs_directory = os.path.abspath(directory)
195+
abs_target = os.path.abspath(target)
196+
197+
prefix = os.path.commonprefix([abs_directory, abs_target])
198+
199+
return prefix == abs_directory
200+
201+
def safe_extract(tar, path=".", members=None, *, numeric_owner=False):
202+
203+
for member in tar.getmembers():
204+
member_path = os.path.join(path, member.name)
205+
if not is_within_directory(path, member_path):
206+
raise Exception("Attempted Path Traversal in Tar File")
207+
208+
tar.extractall(path, members, numeric_owner=numeric_owner, filter="data")
209+
210+
safe_extract(tar, path=tmp_dir)
192211
for root, _, files in os.walk(tmp_dir):
193212
fnames += [os.path.join(root, fname) for fname in files]
194213

genomepy/providers/__init__.py

Lines changed: 11 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -85,27 +85,34 @@ def list_online_providers():
8585
return [p.name for p in PROVIDERS.values() if p.ping()]
8686

8787

88-
def online_providers(provider: str = None):
88+
def online_providers(name: str = None):
8989
"""
9090
Check if the provider can be reached, or any provider if none is specified.
9191
9292
Parameters
9393
----------
94-
provider : str, optional
94+
name : str, optional
9595
Only try to yield the specified provider.
9696
9797
Yields
9898
------
99-
provider
99+
generator
100100
Provider instances
101101
"""
102-
providers = [provider] if provider else list_providers()
102+
providers = [name] if name else list_providers()
103+
success = False
103104
for provider in providers:
104105
try:
105106
yield create(provider)
107+
success = True
106108
except ConnectionError as e:
107109
logger.warning(str(e))
108110

111+
# If none of the requested provider(s) is online, raise the error
112+
if not success:
113+
who = ", ".join(providers)
114+
raise ConnectionError(f"{who} appears to be offline")
115+
109116

110117
def search(term: str or int, provider: str = None, exact=False, size=False):
111118
"""

genomepy/providers/gencode.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -46,7 +46,7 @@ def __init__(self):
4646
@staticmethod
4747
def ping():
4848
"""Can the provider be reached?"""
49-
ftp_online = bool(check_url("ftp.ebi.ac.uk/pub/databases/gencode"))
49+
ftp_online = bool(check_url("ftp://ftp.ebi.ac.uk/pub/databases/gencode", 2))
5050
return ftp_online
5151

5252
def _genome_info_tuple(self, name, size=False):

0 commit comments

Comments
 (0)