Skip to content

Refactored scanpy.tools._sparse_nanmean to eliminate unnecessary data… #3570

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
Reovirus wants to merge 24 commits into
base: main
Choose a base branch
from
+73 −18

smaller relnote

8de0847
Select commit
Loading
Failed to load commit list.
Open

Refactored scanpy.tools._sparse_nanmean to eliminate unnecessary data… #3570

smaller relnote
8de0847
Select commit
Loading
Failed to load commit list.
scverse-benchmark / benchmark succeeded Jan 12, 2026 in 48m 26s

Benchmark

Benchmark run successful

Details

All benchmarks:

Change Before [98ff271] After [8de0847] Ratio Benchmark (Parameter)
308M 309M 1.00 preprocessing_counts.FastSuite.peakmem_calculate_qc_metrics('bmmc', 'counts')
309M 309M 1.00 preprocessing_counts.FastSuite.peakmem_calculate_qc_metrics('bmmc', 'counts-off-axis')
4.1G 4.1G 1.00 preprocessing_counts.FastSuite.peakmem_calculate_qc_metrics('lung93k', 'counts')
4.1G 4.1G 1.00 preprocessing_counts.FastSuite.peakmem_calculate_qc_metrics('lung93k', 'counts-off-axis')
368M 368M 1.00 preprocessing_counts.FastSuite.peakmem_calculate_qc_metrics('pbmc3k', 'counts')
368M 368M 1.00 preprocessing_counts.FastSuite.peakmem_calculate_qc_metrics('pbmc3k', 'counts-off-axis')
287M 287M 1.00 preprocessing_counts.FastSuite.peakmem_calculate_qc_metrics('pbmc68k_reduced', 'counts')
287M 287M 1.00 preprocessing_counts.FastSuite.peakmem_calculate_qc_metrics('pbmc68k_reduced', 'counts-off-axis')
311M 311M 1.00 preprocessing_counts.FastSuite.peakmem_log1p('bmmc', 'counts')
311M 311M 1.00 preprocessing_counts.FastSuite.peakmem_log1p('bmmc', 'counts-off-axis')
4.44G 4.44G 1.00 preprocessing_counts.FastSuite.peakmem_log1p('lung93k', 'counts')
4.44G 4.44G 1.00 preprocessing_counts.FastSuite.peakmem_log1p('lung93k', 'counts-off-axis')
377M 377M 1.00 preprocessing_counts.FastSuite.peakmem_log1p('pbmc3k', 'counts')
377M 377M 1.00 preprocessing_counts.FastSuite.peakmem_log1p('pbmc3k', 'counts-off-axis')
286M 286M 1.00 preprocessing_counts.FastSuite.peakmem_log1p('pbmc68k_reduced', 'counts')
286M 286M 1.00 preprocessing_counts.FastSuite.peakmem_log1p('pbmc68k_reduced', 'counts-off-axis')
399M 400M 1.00 preprocessing_counts.FastSuite.peakmem_normalize_total('bmmc', 'counts')
398M 399M 1.00 preprocessing_counts.FastSuite.peakmem_normalize_total('bmmc', 'counts-off-axis')
4.96G 4.96G 1.00 preprocessing_counts.FastSuite.peakmem_normalize_total('lung93k', 'counts')
4.96G 4.96G 1.00 preprocessing_counts.FastSuite.peakmem_normalize_total('lung93k', 'counts-off-axis')
470M 470M 1.00 preprocessing_counts.FastSuite.peakmem_normalize_total('pbmc3k', 'counts')
469M 470M 1.00 preprocessing_counts.FastSuite.peakmem_normalize_total('pbmc3k', 'counts-off-axis')
286M 286M 1.00 preprocessing_counts.FastSuite.peakmem_normalize_total('pbmc68k_reduced', 'counts')
286M 286M 1.00 preprocessing_counts.FastSuite.peakmem_normalize_total('pbmc68k_reduced', 'counts-off-axis')
13.7±1ms 13.2±0.4ms 0.97 preprocessing_counts.FastSuite.time_calculate_qc_metrics('bmmc', 'counts')
12.7±0.4ms 12.7±0.3ms 1.00 preprocessing_counts.FastSuite.time_calculate_qc_metrics('bmmc', 'counts-off-axis')
2.06±0s 2.08±0.01s 1.01 preprocessing_counts.FastSuite.time_calculate_qc_metrics('lung93k', 'counts')
1.64±0s 1.63±0.01s 1.00 preprocessing_counts.FastSuite.time_calculate_qc_metrics('lung93k', 'counts-off-axis')
37.5±0.4ms 37.8±0.5ms 1.01 preprocessing_counts.FastSuite.time_calculate_qc_metrics('pbmc3k', 'counts')
27.8±0.3ms 28.0±0.4ms 1.01 preprocessing_counts.FastSuite.time_calculate_qc_metrics('pbmc3k', 'counts-off-axis')
4.98±0.04ms 4.93±0.03ms 0.99 preprocessing_counts.FastSuite.time_calculate_qc_metrics('pbmc68k_reduced', 'counts')
4.95±0.03ms 4.90±0.03ms 0.99 preprocessing_counts.FastSuite.time_calculate_qc_metrics('pbmc68k_reduced', 'counts-off-axis')
1.53±0.08ms 1.52±0.1ms 1.00 preprocessing_counts.FastSuite.time_log1p('bmmc', 'counts')
1.59±0.03ms 1.61±0.05ms 1.01 preprocessing_counts.FastSuite.time_log1p('bmmc', 'counts-off-axis')
637±2ms 635±3ms 1.00 preprocessing_counts.FastSuite.time_log1p('lung93k', 'counts')
641±0.5ms 649±10ms 1.01 preprocessing_counts.FastSuite.time_log1p('lung93k', 'counts-off-axis')
7.07±0.05ms 7.13±0.02ms 1.01 preprocessing_counts.FastSuite.time_log1p('pbmc3k', 'counts')
7.23±0.1ms 7.21±0.1ms 1.00 preprocessing_counts.FastSuite.time_log1p('pbmc3k', 'counts-off-axis')
394±20μs 390±4μs 0.99 preprocessing_counts.FastSuite.time_log1p('pbmc68k_reduced', 'counts')
415±20μs 407±20μs 0.98 preprocessing_counts.FastSuite.time_log1p('pbmc68k_reduced', 'counts-off-axis')
2.57±0.2ms 2.57±0.04ms 1.00 preprocessing_counts.FastSuite.time_normalize_total('bmmc', 'counts')
6.57±0.2ms 6.85±0.5ms 1.04 preprocessing_counts.FastSuite.time_normalize_total('bmmc', 'counts-off-axis')
546±5ms 550±10ms 1.01 preprocessing_counts.FastSuite.time_normalize_total('lung93k', 'counts')
2.71±0.02s 2.63±0.02s 0.97 preprocessing_counts.FastSuite.time_normalize_total('lung93k', 'counts-off-axis')
8.67±0.2ms 8.64±0.6ms 1.00 preprocessing_counts.FastSuite.time_normalize_total('pbmc3k', 'counts')
32.3±2ms 34.2±0.4ms 1.06 preprocessing_counts.FastSuite.time_normalize_total('pbmc3k', 'counts-off-axis')
547±2μs 548±0.7μs 1.00 preprocessing_counts.FastSuite.time_normalize_total('pbmc68k_reduced', 'counts')
546±0.8μs 551±1μs 1.01 preprocessing_counts.FastSuite.time_normalize_total('pbmc68k_reduced', 'counts-off-axis')
422M 423M 1.00 preprocessing_counts.PreprocessingCountsSuite.peakmem_filter_cells('pbmc3k', 'counts')
422M 423M 1.00 preprocessing_counts.PreprocessingCountsSuite.peakmem_filter_cells('pbmc3k', 'counts-off-axis')
301M 301M 1.00 preprocessing_counts.PreprocessingCountsSuite.peakmem_filter_cells('pbmc68k_reduced', 'counts')
300M 301M 1.00 preprocessing_counts.PreprocessingCountsSuite.peakmem_filter_cells('pbmc68k_reduced', 'counts-off-axis')
422M 423M 1.00 preprocessing_counts.PreprocessingCountsSuite.peakmem_filter_genes('pbmc3k', 'counts')
422M 422M 1.00 preprocessing_counts.PreprocessingCountsSuite.peakmem_filter_genes('pbmc3k', 'counts-off-axis')
296M 297M 1.00 preprocessing_counts.PreprocessingCountsSuite.peakmem_filter_genes('pbmc68k_reduced', 'counts')
296M 296M 1.00 preprocessing_counts.PreprocessingCountsSuite.peakmem_filter_genes('pbmc68k_reduced', 'counts-off-axis')
1.11G 1.1G 1.00 preprocessing_counts.PreprocessingCountsSuite.peakmem_scrublet('pbmc3k', 'counts')
1.1G 1.1G 1.00 preprocessing_counts.PreprocessingCountsSuite.peakmem_scrublet('pbmc3k', 'counts-off-axis')
522M 518M 0.99 preprocessing_counts.PreprocessingCountsSuite.peakmem_scrublet('pbmc68k_reduced', 'counts')
519M 520M 1.00 preprocessing_counts.PreprocessingCountsSuite.peakmem_scrublet('pbmc68k_reduced', 'counts-off-axis')
55.5±4ms 55.5±4ms 1.00 preprocessing_counts.PreprocessingCountsSuite.time_filter_cells('pbmc3k', 'counts')
62.7±4ms 57.9±4ms 0.92 preprocessing_counts.PreprocessingCountsSuite.time_filter_cells('pbmc3k', 'counts-off-axis')
24.5±4ms 27.6±3ms ~1.12 preprocessing_counts.PreprocessingCountsSuite.time_filter_cells('pbmc68k_reduced', 'counts')
24.3±4ms 25.2±3ms 1.04 preprocessing_counts.PreprocessingCountsSuite.time_filter_cells('pbmc68k_reduced', 'counts-off-axis')
51.7±4ms 51.3±4ms 0.99 preprocessing_counts.PreprocessingCountsSuite.time_filter_genes('pbmc3k', 'counts')
50.5±0.4ms 50.4±4ms 1.00 preprocessing_counts.PreprocessingCountsSuite.time_filter_genes('pbmc3k', 'counts-off-axis')
11.1±0.8ms 10.3±0.8ms 0.93 preprocessing_counts.PreprocessingCountsSuite.time_filter_genes('pbmc68k_reduced', 'counts')
10.1±0.8ms 10.5±0.7ms 1.04 preprocessing_counts.PreprocessingCountsSuite.time_filter_genes('pbmc68k_reduced', 'counts-off-axis')
2.63±0.2s 2.56±0.3s 0.97 preprocessing_counts.PreprocessingCountsSuite.time_scrublet('pbmc3k', 'counts')
2.77±0.06s 2.36±2s ~0.85 preprocessing_counts.PreprocessingCountsSuite.time_scrublet('pbmc3k', 'counts-off-axis')
574±10ms 576±9ms 1.00 preprocessing_counts.PreprocessingCountsSuite.time_scrublet('pbmc68k_reduced', 'counts')
561±7ms 581±2ms 1.03 preprocessing_counts.PreprocessingCountsSuite.time_scrublet('pbmc68k_reduced', 'counts-off-axis')
429M 429M 1.00 preprocessing_log.PreprocessingSuite.peakmem_highly_variable_genes('pbmc3k', 'off-axis')
477M 476M 1.00 preprocessing_log.PreprocessingSuite.peakmem_highly_variable_genes('pbmc3k', None)
293M 293M 1.00 preprocessing_log.PreprocessingSuite.peakmem_highly_variable_genes('pbmc68k_reduced', 'off-axis')
295M 295M 1.00 preprocessing_log.PreprocessingSuite.peakmem_highly_variable_genes('pbmc68k_reduced', None)
561M 566M 1.01 preprocessing_log.PreprocessingSuite.peakmem_pca('pbmc3k', 'off-axis')
583M 582M 1.00 preprocessing_log.PreprocessingSuite.peakmem_pca('pbmc3k', None)
489M 494M 1.01 preprocessing_log.PreprocessingSuite.peakmem_pca('pbmc68k_reduced', 'off-axis')
492M 492M 1.00 preprocessing_log.PreprocessingSuite.peakmem_pca('pbmc68k_reduced', None)
n/a n/a n/a preprocessing_log.PreprocessingSuite.peakmem_regress_out('pbmc3k', 'off-axis')
n/a n/a n/a preprocessing_log.PreprocessingSuite.peakmem_regress_out('pbmc3k', None)
348M 346M 1.00 preprocessing_log.PreprocessingSuite.peakmem_regress_out('pbmc68k_reduced', 'off-axis')
348M 350M 1.01 preprocessing_log.PreprocessingSuite.peakmem_regress_out('pbmc68k_reduced', None)
1.29G 1.29G 1.00 preprocessing_log.PreprocessingSuite.peakmem_scale('pbmc3k', 'off-axis')
1.49G 1.49G 1.00 preprocessing_log.PreprocessingSuite.peakmem_scale('pbmc3k', None)
334M 339M 1.01 preprocessing_log.PreprocessingSuite.peakmem_scale('pbmc68k_reduced', 'off-axis')
337M 341M 1.01 preprocessing_log.PreprocessingSuite.peakmem_scale('pbmc68k_reduced', None)
39.0±2ms 39.2±2ms 1.01 preprocessing_log.PreprocessingSuite.time_highly_variable_genes('pbmc3k', 'off-axis')
55.9±0.5ms 55.8±0.5ms 1.00 preprocessing_log.PreprocessingSuite.time_highly_variable_genes('pbmc3k', None)
16.5±0.5ms 16.8±0.3ms 1.01 preprocessing_log.PreprocessingSuite.time_highly_variable_genes('pbmc68k_reduced', 'off-axis')
16.7±0.3ms 16.7±0.2ms 1.00 preprocessing_log.PreprocessingSuite.time_highly_variable_genes('pbmc68k_reduced', None)
1.94±0.01s 1.95±0.01s 1.00 preprocessing_log.PreprocessingSuite.time_pca('pbmc3k', 'off-axis')
2.15±0.01s 2.17±0.01s 1.01 preprocessing_log.PreprocessingSuite.time_pca('pbmc3k', None)
163±30ms 176±30ms 1.08 preprocessing_log.PreprocessingSuite.time_pca('pbmc68k_reduced', 'off-axis')
171±10ms 177±10ms 1.03 preprocessing_log.PreprocessingSuite.time_pca('pbmc68k_reduced', None)
n/a n/a n/a preprocessing_log.PreprocessingSuite.time_regress_out('pbmc3k', 'off-axis')
n/a n/a n/a preprocessing_log.PreprocessingSuite.time_regress_out('pbmc3k', None)
17.3±0.7ms 16.9±0.5ms 0.98 preprocessing_log.PreprocessingSuite.time_regress_out('pbmc68k_reduced', 'off-axis')
17.1±0.7ms 17.0±0.4ms 0.99 preprocessing_log.PreprocessingSuite.time_regress_out('pbmc68k_reduced', None)
507±4ms 503±3ms 0.99 preprocessing_log.PreprocessingSuite.time_scale('pbmc3k', 'off-axis')
560±0.7ms 555±0.7ms 0.99 preprocessing_log.PreprocessingSuite.time_scale('pbmc3k', None)
4.77±0.09ms 4.83±0.1ms 1.01 preprocessing_log.PreprocessingSuite.time_scale('pbmc68k_reduced', 'off-axis')
4.75±0.1ms 4.78±0.1ms 1.01 preprocessing_log.PreprocessingSuite.time_scale('pbmc68k_reduced', None)
285M 285M 1.00 tools.ToolsSuite.peakmem_diffmap
291M 291M 1.00 tools.ToolsSuite.peakmem_leiden
364M 359M 0.99 tools.ToolsSuite.peakmem_rank_genes_groups
+ 287M 340M 1.18 tools.ToolsSuite.peakmem_score_genes
447M 447M 1.00 tools.ToolsSuite.peakmem_umap
16.2±0.4ms 16.7±0.4ms 1.03 tools.ToolsSuite.time_diffmap
17.6±0.06ms 17.6±0.09ms 1.00 tools.ToolsSuite.time_leiden
60.8±4ms 61.4±5ms 1.01 tools.ToolsSuite.time_rank_genes_groups
19.6±1ms 17.6±0.4ms ~0.90 tools.ToolsSuite.time_score_genes
1.02±0s 1.02±0s 1.00 tools.ToolsSuite.time_umap
View more details on scverse-benchmark