Spatial Simulators
in developmentAssessing the quality of spatial transcriptomics simulators
Computational methods for spatially resolved transcriptomics (SRT) are frequently developed and assessed through data simulation. The effectiveness of these evaluations relies on the simulation methods' ability to accurately reflect experimental data. However, a systematic evaluation framework for spatial simulators is lacking. Here, we present SpatialSimBench, a comprehensive evaluation framework that assesses 13 simulation methods using 10 distinct STR datasets.
The research goal of this benchmark is to systematically evaluate and compare the performance of various simulation methods for spatial transcriptomics (ST) data. It aims to address the lack of a comprehensive evaluation framework for spatial simulators and explore the feasibility of leveraging existing single-cell simulators for ST data. The experimental setup involves collecting public spatial transcriptomics datasets and corresponding scRNA-seq datasets. The spatial and scRNA-seq datasets can originate from different study but should consist of similar cell types from similar tissues.
Contributors
Leaderboard
Methods ranked by scaled overall mean. Each cell encodes a score from 0 to 1 by size and intensity.
QC: Normalisation Visualisation 18 plots
Per metric: points placed by control-anchored scaled score (x); dashed lines mark scaled 0 and 1 (worst/best control); the lower axis shows the raw score. Points beyond [-0.2, 1.2] are clamped to the edge as triangles. Hover a dot or line to highlight it and read details.
- Celltype interactionlower better
- Centralized scorelower better
- clustering_arihigher better
- clustering_nmihigher better
- crosscor_cosinehigher better
- crosscor_mantelhigher better
- ctdeconcolute_jsdlower better
- ctdeconvolute_rmselower better
- Effective library sizelower better
- Frobenius norm of centralized scorelower better
- Frobenius norm of neighborhood enrichmentlower better
- Frobenius norm of transition matrixlower better
- L statisticslower better
- Library sizelower better
- Neighborhood enrichmentlower better
- svg_precisionhigher better
- svg_recallhigher better
- Transition matrixlower better
QC: Indicator table 146 errors11 warnings
Automated checks on the benchmark run and its results: missing values, score scaling, metric ranges and similar. Errors are high-severity issues that usually need a maintainer's attention; warnings are lower-severity signals. Findings that are expected for this task are listed separately as silenced.
146 high-severity issues need review. 462 of 619 checks passed.
- error Raw results Task number of results
Number of results should be equal to #datasets × #methods × #metrics Task: spatial_simulators Number of results: 1609 Number of datasets: 10 Number of methods: 11 Number of metrics: 46 Expected number of results: 5060
- error Raw results Dataset 'hindlimbmuscle' % missing
Percentage of missing results should be less than 10% Task: spatial_simulators Dataset: hindlimbmuscle Number of results: 151 Expected number of results: 506 Percentage missing: 70%
- error Raw results Dataset 'brain' % missing
Percentage of missing results should be less than 10% Task: spatial_simulators Dataset: brain Number of results: 133 Expected number of results: 506 Percentage missing: 74%
- error Raw results Dataset 'cortex' % missing
Percentage of missing results should be less than 10% Task: spatial_simulators Dataset: cortex Number of results: 179 Expected number of results: 506 Percentage missing: 65%
- error Raw results Dataset 'prostate' % missing
Percentage of missing results should be less than 10% Task: spatial_simulators Dataset: prostate Number of results: 143 Expected number of results: 506 Percentage missing: 72%
- error Raw results Dataset 'breast' % missing
Percentage of missing results should be less than 10% Task: spatial_simulators Dataset: breast Number of results: 167 Expected number of results: 506 Percentage missing: 67%
- error Raw results Dataset 'pdac' % missing
Percentage of missing results should be less than 10% Task: spatial_simulators Dataset: pdac Number of results: 140 Expected number of results: 506 Percentage missing: 72%
- error Raw results Dataset 'olfactorybulb' % missing
Percentage of missing results should be less than 10% Task: spatial_simulators Dataset: olfactorybulb Number of results: 188 Expected number of results: 506 Percentage missing: 63%
- error Raw results Dataset 'fibrosarcoma' % missing
Percentage of missing results should be less than 10% Task: spatial_simulators Dataset: fibrosarcoma Number of results: 185 Expected number of results: 506 Percentage missing: 63%
- error Raw results Dataset 'gastrulation' % missing
Percentage of missing results should be less than 10% Task: spatial_simulators Dataset: gastrulation Number of results: 130 Expected number of results: 506 Percentage missing: 74%
- error Raw results Dataset 'osteosarcoma' % missing
Percentage of missing results should be less than 10% Task: spatial_simulators Dataset: osteosarcoma Number of results: 193 Expected number of results: 506 Percentage missing: 62%
- error Raw results Method 'scdesign2' % missing
Percentage of missing results should be less than 10% Task: spatial_simulators Method: scdesign2 Number of results: 142 Expected number of results: 460 Percentage missing: 69%
- error Raw results Method 'scdesign3_nb' % missing
Percentage of missing results should be less than 10% Task: spatial_simulators Method: scdesign3_nb Number of results: 114 Expected number of results: 460 Percentage missing: 75%
- error Raw results Method 'scdesign3_poisson' % missing
Percentage of missing results should be less than 10% Task: spatial_simulators Method: scdesign3_poisson Number of results: 139 Expected number of results: 460 Percentage missing: 70%
- error Raw results Method 'sparsim' % missing
Percentage of missing results should be less than 10% Task: spatial_simulators Method: sparsim Number of results: 151 Expected number of results: 460 Percentage missing: 67%
- error Raw results Method 'splatter' % missing
Percentage of missing results should be less than 10% Task: spatial_simulators Method: splatter Number of results: 150 Expected number of results: 460 Percentage missing: 67%
- error Raw results Method 'srtsim' % missing
Percentage of missing results should be less than 10% Task: spatial_simulators Method: srtsim Number of results: 152 Expected number of results: 460 Percentage missing: 67%
- error Raw results Method 'symsim' % missing
Percentage of missing results should be less than 10% Task: spatial_simulators Method: symsim Number of results: 150 Expected number of results: 460 Percentage missing: 67%
- error Raw results Method 'zinbwave' % missing
Percentage of missing results should be less than 10% Task: spatial_simulators Method: zinbwave Number of results: 152 Expected number of results: 460 Percentage missing: 67%
- error Raw results Method 'positive' % missing
Percentage of missing results should be less than 10% Task: spatial_simulators Method: positive Number of results: 162 Expected number of results: 460 Percentage missing: 65%
- error Raw results Method 'negative_shuffle' % missing
Percentage of missing results should be less than 10% Task: spatial_simulators Method: negative_shuffle Number of results: 143 Expected number of results: 460 Percentage missing: 69%
- error Raw results Method 'negative_normal' % missing
Percentage of missing results should be less than 10% Task: spatial_simulators Method: negative_normal Number of results: 154 Expected number of results: 460 Percentage missing: 67%
- error Raw results Metric 'svg_precision' % missing
Percentage of missing results should be less than 10% Task: spatial_simulators Metric: svg_precision Number of results: 68 Expected number of results: 110 Percentage missing: 38%
- error Raw results Metric 'crosscor_cosine' % missing
Percentage of missing results should be less than 10% Task: spatial_simulators Metric: crosscor_cosine Number of results: 66 Expected number of results: 110 Percentage missing: 40%
- error Raw results Metric 'ks_statistic_frac_zero_genes_zstat' % missing
Percentage of missing results should be less than 10% Task: spatial_simulators Metric: ks_statistic_frac_zero_genes_zstat Number of results: 0 Expected number of results: 110 Percentage missing: 100%
- error Raw results Metric 'ks_statistic_frac_zero_cells_zstat' % missing
Percentage of missing results should be less than 10% Task: spatial_simulators Metric: ks_statistic_frac_zero_cells_zstat Number of results: 0 Expected number of results: 110 Percentage missing: 100%
- error Raw results Metric 'ks_statistic_lib_size_cells_zstat' % missing
Percentage of missing results should be less than 10% Task: spatial_simulators Metric: ks_statistic_lib_size_cells_zstat Number of results: 0 Expected number of results: 110 Percentage missing: 100%
- error Raw results Metric 'ks_statistic_efflib_size_cells_zstat' % missing
Percentage of missing results should be less than 10% Task: spatial_simulators Metric: ks_statistic_efflib_size_cells_zstat Number of results: 0 Expected number of results: 110 Percentage missing: 100%
- error Raw results Metric 'ks_statistic_tmm_cells_zstat' % missing
Percentage of missing results should be less than 10% Task: spatial_simulators Metric: ks_statistic_tmm_cells_zstat Number of results: 0 Expected number of results: 110 Percentage missing: 100%
- error Raw results Metric 'ks_statistic_scaled_var_cells_zstat' % missing
Percentage of missing results should be less than 10% Task: spatial_simulators Metric: ks_statistic_scaled_var_cells_zstat Number of results: 0 Expected number of results: 110 Percentage missing: 100%
- error Raw results Metric 'ks_statistic_scaled_mean_cells_zstat' % missing
Percentage of missing results should be less than 10% Task: spatial_simulators Metric: ks_statistic_scaled_mean_cells_zstat Number of results: 0 Expected number of results: 110 Percentage missing: 100%
- error Raw results Metric 'ks_statistic_lib_fraczero_cells_zstat' % missing
Percentage of missing results should be less than 10% Task: spatial_simulators Metric: ks_statistic_lib_fraczero_cells_zstat Number of results: 0 Expected number of results: 110 Percentage missing: 100%
- error Raw results Metric 'ks_statistic_pearson_cells_zstat' % missing
Percentage of missing results should be less than 10% Task: spatial_simulators Metric: ks_statistic_pearson_cells_zstat Number of results: 0 Expected number of results: 110 Percentage missing: 100%
- error Raw results Metric 'ks_statistic_scaled_var_genes_zstat' % missing
Percentage of missing results should be less than 10% Task: spatial_simulators Metric: ks_statistic_scaled_var_genes_zstat Number of results: 0 Expected number of results: 110 Percentage missing: 100%
- error Raw results Metric 'ks_statistic_scaled_mean_genes_zstat' % missing
Percentage of missing results should be less than 10% Task: spatial_simulators Metric: ks_statistic_scaled_mean_genes_zstat Number of results: 0 Expected number of results: 110 Percentage missing: 100%
- error Raw results Metric 'ks_statistic_pearson_genes_zstat' % missing
Percentage of missing results should be less than 10% Task: spatial_simulators Metric: ks_statistic_pearson_genes_zstat Number of results: 0 Expected number of results: 110 Percentage missing: 100%
- error Raw results Metric 'ks_statistic_mean_var_genes_zstat' % missing
Percentage of missing results should be less than 10% Task: spatial_simulators Metric: ks_statistic_mean_var_genes_zstat Number of results: 0 Expected number of results: 110 Percentage missing: 100%
- error Raw results Metric 'ks_statistic_mean_fraczero_genes_zstat' % missing
Percentage of missing results should be less than 10% Task: spatial_simulators Metric: ks_statistic_mean_fraczero_genes_zstat Number of results: 0 Expected number of results: 110 Percentage missing: 100%
- error Raw results Metric 'ks_statistic_frac_zero_genes_tstat' % missing
Percentage of missing results should be less than 10% Task: spatial_simulators Metric: ks_statistic_frac_zero_genes_tstat Number of results: 0 Expected number of results: 110 Percentage missing: 100%
- error Raw results Metric 'ks_statistic_frac_zero_cells_tstat' % missing
Percentage of missing results should be less than 10% Task: spatial_simulators Metric: ks_statistic_frac_zero_cells_tstat Number of results: 0 Expected number of results: 110 Percentage missing: 100%
- error Raw results Metric 'ks_statistic_lib_size_cells_tstat' % missing
Percentage of missing results should be less than 10% Task: spatial_simulators Metric: ks_statistic_lib_size_cells_tstat Number of results: 0 Expected number of results: 110 Percentage missing: 100%
- error Raw results Metric 'ks_statistic_efflib_size_cells_tstat' % missing
Percentage of missing results should be less than 10% Task: spatial_simulators Metric: ks_statistic_efflib_size_cells_tstat Number of results: 0 Expected number of results: 110 Percentage missing: 100%
- error Raw results Metric 'ks_statistic_tmm_cells_tstat' % missing
Percentage of missing results should be less than 10% Task: spatial_simulators Metric: ks_statistic_tmm_cells_tstat Number of results: 0 Expected number of results: 110 Percentage missing: 100%
- error Raw results Metric 'ks_statistic_scaled_var_cells_tstat' % missing
Percentage of missing results should be less than 10% Task: spatial_simulators Metric: ks_statistic_scaled_var_cells_tstat Number of results: 0 Expected number of results: 110 Percentage missing: 100%
- error Raw results Metric 'ks_statistic_scaled_mean_cells_tstat' % missing
Percentage of missing results should be less than 10% Task: spatial_simulators Metric: ks_statistic_scaled_mean_cells_tstat Number of results: 0 Expected number of results: 110 Percentage missing: 100%
- error Raw results Metric 'ks_statistic_lib_fraczero_cells_tstat' % missing
Percentage of missing results should be less than 10% Task: spatial_simulators Metric: ks_statistic_lib_fraczero_cells_tstat Number of results: 0 Expected number of results: 110 Percentage missing: 100%
- error Raw results Metric 'ks_statistic_pearson_cells_tstat' % missing
Percentage of missing results should be less than 10% Task: spatial_simulators Metric: ks_statistic_pearson_cells_tstat Number of results: 0 Expected number of results: 110 Percentage missing: 100%
- error Raw results Metric 'ks_statistic_scaled_var_genes_tstat' % missing
Percentage of missing results should be less than 10% Task: spatial_simulators Metric: ks_statistic_scaled_var_genes_tstat Number of results: 0 Expected number of results: 110 Percentage missing: 100%
- error Raw results Metric 'ks_statistic_scaled_mean_genes_tstat' % missing
Percentage of missing results should be less than 10% Task: spatial_simulators Metric: ks_statistic_scaled_mean_genes_tstat Number of results: 0 Expected number of results: 110 Percentage missing: 100%
- error Raw results Metric 'ks_statistic_pearson_genes_tstat' % missing
Percentage of missing results should be less than 10% Task: spatial_simulators Metric: ks_statistic_pearson_genes_tstat Number of results: 0 Expected number of results: 110 Percentage missing: 100%
- error Raw results Metric 'ks_statistic_mean_var_genes_tstat' % missing
Percentage of missing results should be less than 10% Task: spatial_simulators Metric: ks_statistic_mean_var_genes_tstat Number of results: 0 Expected number of results: 110 Percentage missing: 100%
- error Raw results Metric 'ks_statistic_mean_fraczero_genes_tstat' % missing
Percentage of missing results should be less than 10% Task: spatial_simulators Metric: ks_statistic_mean_fraczero_genes_tstat Number of results: 0 Expected number of results: 110 Percentage missing: 100%
- error Raw results Metric 'ks_statistic_L_stats' % missing
Percentage of missing results should be less than 10% Task: spatial_simulators Metric: ks_statistic_L_stats Number of results: 76 Expected number of results: 110 Percentage missing: 31%
- error Raw results Metric 'ks_statistic_celltype_interaction' % missing
Percentage of missing results should be less than 10% Task: spatial_simulators Metric: ks_statistic_celltype_interaction Number of results: 76 Expected number of results: 110 Percentage missing: 31%
- error Raw results Metric 'ks_statistic_nn_correlation' % missing
Percentage of missing results should be less than 10% Task: spatial_simulators Metric: ks_statistic_nn_correlation Number of results: 76 Expected number of results: 110 Percentage missing: 31%
- error Raw results Metric 'ks_statistic_morans_I' % missing
Percentage of missing results should be less than 10% Task: spatial_simulators Metric: ks_statistic_morans_I Number of results: 76 Expected number of results: 110 Percentage missing: 31%
- error Raw results Metric 'clustering_ari' number of control methods
Number of metric scores for control methods should be equal to #datasets × #control_methods Task: spatial_simulators Metric: clustering_ari Control method scores: 29 Expected control method scores: 30 Percentage succeeded: 97%
- error Raw results Metric 'clustering_nmi' number of control methods
Number of metric scores for control methods should be equal to #datasets × #control_methods Task: spatial_simulators Metric: clustering_nmi Control method scores: 29 Expected control method scores: 30 Percentage succeeded: 97%
- error Raw results Metric 'svg_recall' number of control methods
Number of metric scores for control methods should be equal to #datasets × #control_methods Task: spatial_simulators Metric: svg_recall Control method scores: 24 Expected control method scores: 30 Percentage succeeded: 80%
- error Raw results Metric 'svg_precision' number of control methods
Number of metric scores for control methods should be equal to #datasets × #control_methods Task: spatial_simulators Metric: svg_precision Control method scores: 9 Expected control method scores: 30 Percentage succeeded: 30%
- error Raw results Metric 'ctdeconvolute_rmse' number of control methods
Number of metric scores for control methods should be equal to #datasets × #control_methods Task: spatial_simulators Metric: ctdeconvolute_rmse Control method scores: 29 Expected control method scores: 30 Percentage succeeded: 97%
- error Raw results Metric 'ctdeconcolute_jsd' number of control methods
Number of metric scores for control methods should be equal to #datasets × #control_methods Task: spatial_simulators Metric: ctdeconcolute_jsd Control method scores: 29 Expected control method scores: 30 Percentage succeeded: 97%
- error Raw results Metric 'crosscor_mantel' number of control methods
Number of metric scores for control methods should be equal to #datasets × #control_methods Task: spatial_simulators Metric: crosscor_mantel Control method scores: 29 Expected control method scores: 30 Percentage succeeded: 97%
- error Raw results Metric 'crosscor_cosine' number of control methods
Number of metric scores for control methods should be equal to #datasets × #control_methods Task: spatial_simulators Metric: crosscor_cosine Control method scores: 29 Expected control method scores: 30 Percentage succeeded: 97%
- error Raw results Metric 'ks_statistic_frac_zero_genes_zstat' number of control methods
Number of metric scores for control methods should be equal to #datasets × #control_methods Task: spatial_simulators Metric: ks_statistic_frac_zero_genes_zstat Control method scores: 0 Expected control method scores: 30 Percentage succeeded: 0%
- error Raw results Metric 'ks_statistic_frac_zero_cells_zstat' number of control methods
Number of metric scores for control methods should be equal to #datasets × #control_methods Task: spatial_simulators Metric: ks_statistic_frac_zero_cells_zstat Control method scores: 0 Expected control method scores: 30 Percentage succeeded: 0%
- error Raw results Metric 'ks_statistic_lib_size_cells_zstat' number of control methods
Number of metric scores for control methods should be equal to #datasets × #control_methods Task: spatial_simulators Metric: ks_statistic_lib_size_cells_zstat Control method scores: 0 Expected control method scores: 30 Percentage succeeded: 0%
- error Raw results Metric 'ks_statistic_efflib_size_cells_zstat' number of control methods
Number of metric scores for control methods should be equal to #datasets × #control_methods Task: spatial_simulators Metric: ks_statistic_efflib_size_cells_zstat Control method scores: 0 Expected control method scores: 30 Percentage succeeded: 0%
- error Raw results Metric 'ks_statistic_tmm_cells_zstat' number of control methods
Number of metric scores for control methods should be equal to #datasets × #control_methods Task: spatial_simulators Metric: ks_statistic_tmm_cells_zstat Control method scores: 0 Expected control method scores: 30 Percentage succeeded: 0%
- error Raw results Metric 'ks_statistic_scaled_var_cells_zstat' number of control methods
Number of metric scores for control methods should be equal to #datasets × #control_methods Task: spatial_simulators Metric: ks_statistic_scaled_var_cells_zstat Control method scores: 0 Expected control method scores: 30 Percentage succeeded: 0%
- error Raw results Metric 'ks_statistic_scaled_mean_cells_zstat' number of control methods
Number of metric scores for control methods should be equal to #datasets × #control_methods Task: spatial_simulators Metric: ks_statistic_scaled_mean_cells_zstat Control method scores: 0 Expected control method scores: 30 Percentage succeeded: 0%
- error Raw results Metric 'ks_statistic_lib_fraczero_cells_zstat' number of control methods
Number of metric scores for control methods should be equal to #datasets × #control_methods Task: spatial_simulators Metric: ks_statistic_lib_fraczero_cells_zstat Control method scores: 0 Expected control method scores: 30 Percentage succeeded: 0%
- error Raw results Metric 'ks_statistic_pearson_cells_zstat' number of control methods
Number of metric scores for control methods should be equal to #datasets × #control_methods Task: spatial_simulators Metric: ks_statistic_pearson_cells_zstat Control method scores: 0 Expected control method scores: 30 Percentage succeeded: 0%
- error Raw results Metric 'ks_statistic_scaled_var_genes_zstat' number of control methods
Number of metric scores for control methods should be equal to #datasets × #control_methods Task: spatial_simulators Metric: ks_statistic_scaled_var_genes_zstat Control method scores: 0 Expected control method scores: 30 Percentage succeeded: 0%
- error Raw results Metric 'ks_statistic_scaled_mean_genes_zstat' number of control methods
Number of metric scores for control methods should be equal to #datasets × #control_methods Task: spatial_simulators Metric: ks_statistic_scaled_mean_genes_zstat Control method scores: 0 Expected control method scores: 30 Percentage succeeded: 0%
- error Raw results Metric 'ks_statistic_pearson_genes_zstat' number of control methods
Number of metric scores for control methods should be equal to #datasets × #control_methods Task: spatial_simulators Metric: ks_statistic_pearson_genes_zstat Control method scores: 0 Expected control method scores: 30 Percentage succeeded: 0%
- error Raw results Metric 'ks_statistic_mean_var_genes_zstat' number of control methods
Number of metric scores for control methods should be equal to #datasets × #control_methods Task: spatial_simulators Metric: ks_statistic_mean_var_genes_zstat Control method scores: 0 Expected control method scores: 30 Percentage succeeded: 0%
- error Raw results Metric 'ks_statistic_mean_fraczero_genes_zstat' number of control methods
Number of metric scores for control methods should be equal to #datasets × #control_methods Task: spatial_simulators Metric: ks_statistic_mean_fraczero_genes_zstat Control method scores: 0 Expected control method scores: 30 Percentage succeeded: 0%
- error Raw results Metric 'ks_statistic_frac_zero_genes_tstat' number of control methods
Number of metric scores for control methods should be equal to #datasets × #control_methods Task: spatial_simulators Metric: ks_statistic_frac_zero_genes_tstat Control method scores: 0 Expected control method scores: 30 Percentage succeeded: 0%
- error Raw results Metric 'ks_statistic_frac_zero_cells_tstat' number of control methods
Number of metric scores for control methods should be equal to #datasets × #control_methods Task: spatial_simulators Metric: ks_statistic_frac_zero_cells_tstat Control method scores: 0 Expected control method scores: 30 Percentage succeeded: 0%
- error Raw results Metric 'ks_statistic_lib_size_cells_tstat' number of control methods
Number of metric scores for control methods should be equal to #datasets × #control_methods Task: spatial_simulators Metric: ks_statistic_lib_size_cells_tstat Control method scores: 0 Expected control method scores: 30 Percentage succeeded: 0%
- error Raw results Metric 'ks_statistic_efflib_size_cells_tstat' number of control methods
Number of metric scores for control methods should be equal to #datasets × #control_methods Task: spatial_simulators Metric: ks_statistic_efflib_size_cells_tstat Control method scores: 0 Expected control method scores: 30 Percentage succeeded: 0%
- error Raw results Metric 'ks_statistic_tmm_cells_tstat' number of control methods
Number of metric scores for control methods should be equal to #datasets × #control_methods Task: spatial_simulators Metric: ks_statistic_tmm_cells_tstat Control method scores: 0 Expected control method scores: 30 Percentage succeeded: 0%
- error Raw results Metric 'ks_statistic_scaled_var_cells_tstat' number of control methods
Number of metric scores for control methods should be equal to #datasets × #control_methods Task: spatial_simulators Metric: ks_statistic_scaled_var_cells_tstat Control method scores: 0 Expected control method scores: 30 Percentage succeeded: 0%
- error Raw results Metric 'ks_statistic_scaled_mean_cells_tstat' number of control methods
Number of metric scores for control methods should be equal to #datasets × #control_methods Task: spatial_simulators Metric: ks_statistic_scaled_mean_cells_tstat Control method scores: 0 Expected control method scores: 30 Percentage succeeded: 0%
- error Raw results Metric 'ks_statistic_lib_fraczero_cells_tstat' number of control methods
Number of metric scores for control methods should be equal to #datasets × #control_methods Task: spatial_simulators Metric: ks_statistic_lib_fraczero_cells_tstat Control method scores: 0 Expected control method scores: 30 Percentage succeeded: 0%
- error Raw results Metric 'ks_statistic_pearson_cells_tstat' number of control methods
Number of metric scores for control methods should be equal to #datasets × #control_methods Task: spatial_simulators Metric: ks_statistic_pearson_cells_tstat Control method scores: 0 Expected control method scores: 30 Percentage succeeded: 0%
- error Raw results Metric 'ks_statistic_scaled_var_genes_tstat' number of control methods
Number of metric scores for control methods should be equal to #datasets × #control_methods Task: spatial_simulators Metric: ks_statistic_scaled_var_genes_tstat Control method scores: 0 Expected control method scores: 30 Percentage succeeded: 0%
- error Raw results Metric 'ks_statistic_scaled_mean_genes_tstat' number of control methods
Number of metric scores for control methods should be equal to #datasets × #control_methods Task: spatial_simulators Metric: ks_statistic_scaled_mean_genes_tstat Control method scores: 0 Expected control method scores: 30 Percentage succeeded: 0%
- error Raw results Metric 'ks_statistic_pearson_genes_tstat' number of control methods
Number of metric scores for control methods should be equal to #datasets × #control_methods Task: spatial_simulators Metric: ks_statistic_pearson_genes_tstat Control method scores: 0 Expected control method scores: 30 Percentage succeeded: 0%
- error Raw results Metric 'ks_statistic_mean_var_genes_tstat' number of control methods
Number of metric scores for control methods should be equal to #datasets × #control_methods Task: spatial_simulators Metric: ks_statistic_mean_var_genes_tstat Control method scores: 0 Expected control method scores: 30 Percentage succeeded: 0%
- error Raw results Metric 'ks_statistic_mean_fraczero_genes_tstat' number of control methods
Number of metric scores for control methods should be equal to #datasets × #control_methods Task: spatial_simulators Metric: ks_statistic_mean_fraczero_genes_tstat Control method scores: 0 Expected control method scores: 30 Percentage succeeded: 0%
- error Raw results Metric 'ks_statistic_L_stats' number of control methods
Number of metric scores for control methods should be equal to #datasets × #control_methods Task: spatial_simulators Metric: ks_statistic_L_stats Control method scores: 24 Expected control method scores: 30 Percentage succeeded: 80%
- error Raw results Metric 'ks_statistic_celltype_interaction' number of control methods
Number of metric scores for control methods should be equal to #datasets × #control_methods Task: spatial_simulators Metric: ks_statistic_celltype_interaction Control method scores: 24 Expected control method scores: 30 Percentage succeeded: 80%
- error Raw results Metric 'ks_statistic_nn_correlation' number of control methods
Number of metric scores for control methods should be equal to #datasets × #control_methods Task: spatial_simulators Metric: ks_statistic_nn_correlation Control method scores: 24 Expected control method scores: 30 Percentage succeeded: 80%
- error Raw results Metric 'ks_statistic_morans_I' number of control methods
Number of metric scores for control methods should be equal to #datasets × #control_methods Task: spatial_simulators Metric: ks_statistic_morans_I Control method scores: 24 Expected control method scores: 30 Percentage succeeded: 80%
- error Raw results Metric 'ks_statistic_transition_matrix' number of control methods
Number of metric scores for control methods should be equal to #datasets × #control_methods Task: spatial_simulators Metric: ks_statistic_transition_matrix Control method scores: 26 Expected control method scores: 30 Percentage succeeded: 87%
- error Raw results Metric 'ks_statistic_central_score' number of control methods
Number of metric scores for control methods should be equal to #datasets × #control_methods Task: spatial_simulators Metric: ks_statistic_central_score Control method scores: 26 Expected control method scores: 30 Percentage succeeded: 87%
- error Raw results Metric 'ks_statistic_enrichment' number of control methods
Number of metric scores for control methods should be equal to #datasets × #control_methods Task: spatial_simulators Metric: ks_statistic_enrichment Control method scores: 26 Expected control method scores: 30 Percentage succeeded: 87%
- error Raw results Metric 'ks_statistic_transition_scalef' number of control methods
Number of metric scores for control methods should be equal to #datasets × #control_methods Task: spatial_simulators Metric: ks_statistic_transition_scalef Control method scores: 26 Expected control method scores: 30 Percentage succeeded: 87%
- error Raw results Metric 'ks_statistic_central_score_scalef' number of control methods
Number of metric scores for control methods should be equal to #datasets × #control_methods Task: spatial_simulators Metric: ks_statistic_central_score_scalef Control method scores: 26 Expected control method scores: 30 Percentage succeeded: 87%
- error Raw results Metric 'ks_statistic_enrichment_scalef' number of control methods
Number of metric scores for control methods should be equal to #datasets × #control_methods Task: spatial_simulators Metric: ks_statistic_enrichment_scalef Control method scores: 26 Expected control method scores: 30 Percentage succeeded: 87%
- error Scaling Metric 'svg_precision' % outside range
Percentage of scaled scores outside control range should be less than 10% Task: spatial_simulators Metric: svg_precision Inside range: NA Scaled scores: 68 Percentage outside: NA%
- error Scaling Worst 'svg_precision' score for 'negative_shuffle'
Method 'negative_shuffle' performs much worse than controls for metric' svg_precision' Task: spatial_simulators Method: negative_shuffle Metric: svg_precision Worst score: NA Percentage outside range: 0%
- error Scaling Best 'svg_precision' score for 'negative_shuffle'
Method 'negative_shuffle' performs much better than controls for metric 'svg_precision' Task: spatial_simulators Method: negative_shuffle Metric: svg_precision Best score: NA Percentage outside range: 0%
- error Scaling Worst 'svg_precision' score for 'positive'
Method 'positive' performs much worse than controls for metric' svg_precision' Task: spatial_simulators Method: positive Metric: svg_precision Worst score: NA Percentage outside range: 0%
- error Scaling Best 'svg_precision' score for 'positive'
Method 'positive' performs much better than controls for metric 'svg_precision' Task: spatial_simulators Method: positive Metric: svg_precision Best score: NA Percentage outside range: 0%
- error Scaling Worst 'svg_precision' score for 'scdesign2'
Method 'scdesign2' performs much worse than controls for metric' svg_precision' Task: spatial_simulators Method: scdesign2 Metric: svg_precision Worst score: NA Percentage outside range: 0%
- error Scaling Best 'svg_precision' score for 'scdesign2'
Method 'scdesign2' performs much better than controls for metric 'svg_precision' Task: spatial_simulators Method: scdesign2 Metric: svg_precision Best score: NA Percentage outside range: 0%
- error Scaling Worst 'svg_precision' score for 'scdesign3_nb'
Method 'scdesign3_nb' performs much worse than controls for metric' svg_precision' Task: spatial_simulators Method: scdesign3_nb Metric: svg_precision Worst score: NA Percentage outside range: 0%
- error Scaling Best 'svg_precision' score for 'scdesign3_nb'
Method 'scdesign3_nb' performs much better than controls for metric 'svg_precision' Task: spatial_simulators Method: scdesign3_nb Metric: svg_precision Best score: NA Percentage outside range: 0%
- error Scaling Worst 'svg_precision' score for 'scdesign3_poisson'
Method 'scdesign3_poisson' performs much worse than controls for metric' svg_precision' Task: spatial_simulators Method: scdesign3_poisson Metric: svg_precision Worst score: NA Percentage outside range: 0%
- error Scaling Best 'svg_precision' score for 'scdesign3_poisson'
Method 'scdesign3_poisson' performs much better than controls for metric 'svg_precision' Task: spatial_simulators Method: scdesign3_poisson Metric: svg_precision Best score: NA Percentage outside range: 0%
- error Scaling Worst 'svg_precision' score for 'sparsim'
Method 'sparsim' performs much worse than controls for metric' svg_precision' Task: spatial_simulators Method: sparsim Metric: svg_precision Worst score: NA Percentage outside range: 0%
- error Scaling Best 'svg_precision' score for 'sparsim'
Method 'sparsim' performs much better than controls for metric 'svg_precision' Task: spatial_simulators Method: sparsim Metric: svg_precision Best score: NA Percentage outside range: 0%
- error Scaling Worst 'svg_precision' score for 'splatter'
Method 'splatter' performs much worse than controls for metric' svg_precision' Task: spatial_simulators Method: splatter Metric: svg_precision Worst score: NA Percentage outside range: 0%
- error Scaling Best 'svg_precision' score for 'splatter'
Method 'splatter' performs much better than controls for metric 'svg_precision' Task: spatial_simulators Method: splatter Metric: svg_precision Best score: NA Percentage outside range: 0%
- error Scaling Worst 'svg_precision' score for 'srtsim'
Method 'srtsim' performs much worse than controls for metric' svg_precision' Task: spatial_simulators Method: srtsim Metric: svg_precision Worst score: NA Percentage outside range: 0%
- error Scaling Best 'svg_precision' score for 'srtsim'
Method 'srtsim' performs much better than controls for metric 'svg_precision' Task: spatial_simulators Method: srtsim Metric: svg_precision Best score: NA Percentage outside range: 0%
- error Scaling Worst 'svg_precision' score for 'symsim'
Method 'symsim' performs much worse than controls for metric' svg_precision' Task: spatial_simulators Method: symsim Metric: svg_precision Worst score: NA Percentage outside range: 0%
- error Scaling Best 'svg_precision' score for 'symsim'
Method 'symsim' performs much better than controls for metric 'svg_precision' Task: spatial_simulators Method: symsim Metric: svg_precision Best score: NA Percentage outside range: 0%
- error Scaling Worst 'svg_precision' score for 'zinbwave'
Method 'zinbwave' performs much worse than controls for metric' svg_precision' Task: spatial_simulators Method: zinbwave Metric: svg_precision Worst score: NA Percentage outside range: 0%
- error Scaling Best 'svg_precision' score for 'zinbwave'
Method 'zinbwave' performs much better than controls for metric 'svg_precision' Task: spatial_simulators Method: zinbwave Metric: svg_precision Best score: NA Percentage outside range: 0%
- error Scaling Metric 'ks_statistic_celltype_interaction' % outside range
Percentage of scaled scores outside control range should be less than 10% Task: spatial_simulators Metric: ks_statistic_celltype_interaction Inside range: NA Scaled scores: 76 Percentage outside: NA%
- error Scaling Worst 'ks_statistic_celltype_interaction' score for 'negative_normal'
Method 'negative_normal' performs much worse than controls for metric' ks_statistic_celltype_interaction' Task: spatial_simulators Method: negative_normal Metric: ks_statistic_celltype_interaction Worst score: NaN Percentage outside range: 0%
- error Scaling Best 'ks_statistic_celltype_interaction' score for 'negative_normal'
Method 'negative_normal' performs much better than controls for metric 'ks_statistic_celltype_interaction' Task: spatial_simulators Method: negative_normal Metric: ks_statistic_celltype_interaction Best score: NaN Percentage outside range: 0%
- error Scaling Worst 'ks_statistic_celltype_interaction' score for 'negative_shuffle'
Method 'negative_shuffle' performs much worse than controls for metric' ks_statistic_celltype_interaction' Task: spatial_simulators Method: negative_shuffle Metric: ks_statistic_celltype_interaction Worst score: NaN Percentage outside range: 0%
- error Scaling Best 'ks_statistic_celltype_interaction' score for 'negative_shuffle'
Method 'negative_shuffle' performs much better than controls for metric 'ks_statistic_celltype_interaction' Task: spatial_simulators Method: negative_shuffle Metric: ks_statistic_celltype_interaction Best score: NaN Percentage outside range: 0%
- error Scaling Worst 'ks_statistic_celltype_interaction' score for 'positive'
Method 'positive' performs much worse than controls for metric' ks_statistic_celltype_interaction' Task: spatial_simulators Method: positive Metric: ks_statistic_celltype_interaction Worst score: NaN Percentage outside range: 0%
- error Scaling Best 'ks_statistic_celltype_interaction' score for 'positive'
Method 'positive' performs much better than controls for metric 'ks_statistic_celltype_interaction' Task: spatial_simulators Method: positive Metric: ks_statistic_celltype_interaction Best score: NaN Percentage outside range: 0%
- error Scaling Worst 'ks_statistic_celltype_interaction' score for 'scdesign2'
Method 'scdesign2' performs much worse than controls for metric' ks_statistic_celltype_interaction' Task: spatial_simulators Method: scdesign2 Metric: ks_statistic_celltype_interaction Worst score: NaN Percentage outside range: 0%
- error Scaling Best 'ks_statistic_celltype_interaction' score for 'scdesign2'
Method 'scdesign2' performs much better than controls for metric 'ks_statistic_celltype_interaction' Task: spatial_simulators Method: scdesign2 Metric: ks_statistic_celltype_interaction Best score: NaN Percentage outside range: 0%
- error Scaling Worst 'ks_statistic_celltype_interaction' score for 'scdesign3_nb'
Method 'scdesign3_nb' performs much worse than controls for metric' ks_statistic_celltype_interaction' Task: spatial_simulators Method: scdesign3_nb Metric: ks_statistic_celltype_interaction Worst score: NaN Percentage outside range: 0%
- error Scaling Best 'ks_statistic_celltype_interaction' score for 'scdesign3_nb'
Method 'scdesign3_nb' performs much better than controls for metric 'ks_statistic_celltype_interaction' Task: spatial_simulators Method: scdesign3_nb Metric: ks_statistic_celltype_interaction Best score: NaN Percentage outside range: 0%
- error Scaling Worst 'ks_statistic_celltype_interaction' score for 'scdesign3_poisson'
Method 'scdesign3_poisson' performs much worse than controls for metric' ks_statistic_celltype_interaction' Task: spatial_simulators Method: scdesign3_poisson Metric: ks_statistic_celltype_interaction Worst score: NaN Percentage outside range: 0%
- error Scaling Best 'ks_statistic_celltype_interaction' score for 'scdesign3_poisson'
Method 'scdesign3_poisson' performs much better than controls for metric 'ks_statistic_celltype_interaction' Task: spatial_simulators Method: scdesign3_poisson Metric: ks_statistic_celltype_interaction Best score: NaN Percentage outside range: 0%
- error Scaling Worst 'ks_statistic_celltype_interaction' score for 'sparsim'
Method 'sparsim' performs much worse than controls for metric' ks_statistic_celltype_interaction' Task: spatial_simulators Method: sparsim Metric: ks_statistic_celltype_interaction Worst score: NaN Percentage outside range: 0%
- error Scaling Best 'ks_statistic_celltype_interaction' score for 'sparsim'
Method 'sparsim' performs much better than controls for metric 'ks_statistic_celltype_interaction' Task: spatial_simulators Method: sparsim Metric: ks_statistic_celltype_interaction Best score: NaN Percentage outside range: 0%
- error Scaling Worst 'ks_statistic_celltype_interaction' score for 'splatter'
Method 'splatter' performs much worse than controls for metric' ks_statistic_celltype_interaction' Task: spatial_simulators Method: splatter Metric: ks_statistic_celltype_interaction Worst score: NaN Percentage outside range: 0%
- error Scaling Best 'ks_statistic_celltype_interaction' score for 'splatter'
Method 'splatter' performs much better than controls for metric 'ks_statistic_celltype_interaction' Task: spatial_simulators Method: splatter Metric: ks_statistic_celltype_interaction Best score: NaN Percentage outside range: 0%
- error Scaling Worst 'ks_statistic_celltype_interaction' score for 'srtsim'
Method 'srtsim' performs much worse than controls for metric' ks_statistic_celltype_interaction' Task: spatial_simulators Method: srtsim Metric: ks_statistic_celltype_interaction Worst score: NaN Percentage outside range: 0%
- error Scaling Best 'ks_statistic_celltype_interaction' score for 'srtsim'
Method 'srtsim' performs much better than controls for metric 'ks_statistic_celltype_interaction' Task: spatial_simulators Method: srtsim Metric: ks_statistic_celltype_interaction Best score: NaN Percentage outside range: 0%
- error Scaling Worst 'ks_statistic_celltype_interaction' score for 'symsim'
Method 'symsim' performs much worse than controls for metric' ks_statistic_celltype_interaction' Task: spatial_simulators Method: symsim Metric: ks_statistic_celltype_interaction Worst score: NaN Percentage outside range: 0%
- error Scaling Best 'ks_statistic_celltype_interaction' score for 'symsim'
Method 'symsim' performs much better than controls for metric 'ks_statistic_celltype_interaction' Task: spatial_simulators Method: symsim Metric: ks_statistic_celltype_interaction Best score: NaN Percentage outside range: 0%
- error Scaling Worst 'ks_statistic_celltype_interaction' score for 'zinbwave'
Method 'zinbwave' performs much worse than controls for metric' ks_statistic_celltype_interaction' Task: spatial_simulators Method: zinbwave Metric: ks_statistic_celltype_interaction Worst score: NaN Percentage outside range: 0%
- error Scaling Best 'ks_statistic_celltype_interaction' score for 'zinbwave'
Method 'zinbwave' performs much better than controls for metric 'ks_statistic_celltype_interaction' Task: spatial_simulators Method: zinbwave Metric: ks_statistic_celltype_interaction Best score: NaN Percentage outside range: 0%
Show 11 warnings
- warning Raw results Metric 'svg_recall' % missing
Percentage of missing results should be less than 10% Task: spatial_simulators Metric: svg_recall Number of results: 86 Expected number of results: 110 Percentage missing: 22%
- warning Raw results Metric component 'ks_statistic_sc_features' % failed
Percentage of failed processes should be less than 10% Task: spatial_simulators Metric component: ks_statistic_sc_features Succeeded processes: 76 Attempted processes: 107 Percentage failed: 29%
- warning Raw results Metric 'ks_statistic_transition_matrix' % missing
Percentage of missing results should be less than 10% Task: spatial_simulators Metric: ks_statistic_transition_matrix Number of results: 95 Expected number of results: 110 Percentage missing: 14%
- warning Raw results Metric 'ks_statistic_central_score' % missing
Percentage of missing results should be less than 10% Task: spatial_simulators Metric: ks_statistic_central_score Number of results: 95 Expected number of results: 110 Percentage missing: 14%
- warning Raw results Metric 'ks_statistic_enrichment' % missing
Percentage of missing results should be less than 10% Task: spatial_simulators Metric: ks_statistic_enrichment Number of results: 95 Expected number of results: 110 Percentage missing: 14%
- warning Raw results Metric 'ks_statistic_transition_scalef' % missing
Percentage of missing results should be less than 10% Task: spatial_simulators Metric: ks_statistic_transition_scalef Number of results: 95 Expected number of results: 110 Percentage missing: 14%
- warning Raw results Metric 'ks_statistic_central_score_scalef' % missing
Percentage of missing results should be less than 10% Task: spatial_simulators Metric: ks_statistic_central_score_scalef Number of results: 95 Expected number of results: 110 Percentage missing: 14%
- warning Raw results Metric 'ks_statistic_enrichment_scalef' % missing
Percentage of missing results should be less than 10% Task: spatial_simulators Metric: ks_statistic_enrichment_scalef Number of results: 95 Expected number of results: 110 Percentage missing: 14%
- warning Raw results Task number of successful processes
Number of successful processes should be equal to the number of attempted processes Task: spatial_simulators Succeeded processes: 483 Attempted processes: 538 Percentage failed: 10%
- warning Raw results Method 'scdesign3_nb' % failed
Percentage of failed processes should be less than 10% Task: spatial_simulators Method: scdesign3_nb Succeeded processes: 8 Attempted processes: 10 Percentage failed: 20%
- warning Raw results Metric component 'ks_statistic_spatial' % failed
Percentage of failed processes should be less than 10% Task: spatial_simulators Metric component: ks_statistic_spatial Succeeded processes: 95 Attempted processes: 107 Percentage failed: 11%
Method info 8
A transparent simulator that generates high-fidelity single-cell gene expression count data with gene correlations captured
scDesign2 is a transparent simulator that achieves all three goals (preserving genes, capturing gene correlations, and generating any number of cells with varying sequencing depths) and generates high-fidelity synthetic data for multiple single-cell gene expression count-based technologies.
A probabilistic model that unifies the generation and inference for single-cell and spatial omics data
scDesign3 offers a probabilistic model that unifies the generation and inference for single-cell and spatial omics data. The model's interpretable parameters and likelihood enable scDesign3 to generate customized in silico data and unsupervisedly assess the goodness-of-fit of inferred cell latent structures (for example, clusters, trajectories and spatial locations).
A probabilistic model that unifies the generation and inference for single-cell and spatial omics data
scDesign3 offers a probabilistic model that unifies the generation and inference for single-cell and spatial omics data. The model's interpretable parameters and likelihood enable scDesign3 to generate customized in silico data and unsupervisedly assess the goodness-of-fit of inferred cell latent structures (for example, clusters, trajectories and spatial locations).
SPARSim single cell is a count data simulator for scRNA-seq data.
SPARSim is a scRNA-seq count data simulator based on a Gamma-Multivariate Hypergeometric model. It allows to generate count data that resembles real data in terms of count intensity, variability and sparsity.
A single cell RNA-seq data simulator based on a gamma-Poisson distribution.
The Splat model is a gamma-Poisson distribution used to generate a gene by cell matrix of counts. Mean expression levels for each gene are simulated from a gamma distribution and the Biological Coefficient of Variation is used to enforce a mean-variance trend before counts are simulated from a Poisson distribution.
An SRT-specific simulator for scalable, reproducible, and realistic SRT simulations.
A key benefit of srtsim is its ability to maintain location-wise and gene-wise SRT count properties and preserve spatial expression patterns, enabling evaluation of SRT method performance using synthetic data.
Simulating multiple faceted variability in single cell RNA sequencing
SymSim is a simulator for modeling single-cell RNA-Seq data, accounting for three primary sources of variation: intrinsic transcription noise, extrinsic variation from different cell states, and technical variation from measurement noise and bias.
A general and flexible method for signal extraction from single-cell RNA-seq data
ZINB-WaVE is a general and flexible zero-inflated negative binomial model, which leads to low-dimensional representations of the data that account for zero inflation (dropouts), over-dispersion, and the count nature of the data.
Control method info 3
A negative control which generates normal distributed data.
This control method generates normal distributed data as a negative control. The mean and the sd are defined by the mean and sd of the input data.
A negative control method which shuffles the input data.
This control method shuffles the input data as a negative control.
A positive control method.
Metric info 46
KS statistic of the celltype interaction
The Kolmogorov-Smirnov statistic comparing the Celltype interaction in the real datasets versus the Celltype interaction in the simulated datasets.
Ks Statistic of the centralized score matrix.
The Kolmogorov-Smirnov statistic comparing the centralized score matrix of the real dataset versus the simulated dataset. The centralized score matrix is a vector of the group degree centrality (inter-cluster connectivity), average clustering coefficient (propensity for a spot within a spatial cluster to be connected to spots in another cluster), and the group closeness centrality (relative proximity or accessibility of one cluster to all spots in another).
Adjusted rand index (ARI) measures the similarity between two clusters in real and simulated datasets.
Adjusted Rand Index used in spatial clustering to measure the similarity between two data clusterings, adjusted for chance.
Normalized mutual information (NMI) measures of the mutual dependence between the real and simulated spatial clusters.
Normalized Mutual Information used in spatial clustering to measure the agreement between two different clusterings, scaled to [0, 1].
Cosine similarity measures similarity between bivariate Moran’s I of real dataset and that of in simulation dataset.
Cosine similarity used in spatial cross-correlation to measure the cosine of the angle between two non-zero vectors.
Mantel statistic is the test statistic for the Mantel test, which is a correlation coefficient calculated between bivariate Moran’s I of real dataset and that of in simulation dataset.
Mantel statistic used in spatial cross-correlation to test the correlation between two distance matrices.
Jensen-Shannon divergence (JSD) is calculated between the true and predicted proportion per cell type in all spots.
Jensen-Shannon Divergence used in cell type deconvolution to measure the similarity between two probability distributions.
Root Mean Square deviation is calculated between the true and predicted proportion of per cell type.
Root Mean Squared Error used in cell type deconvolution to measure the difference between observed and predicted values.
KS statistic of the effective library size.
The Kolmogorov-Smirnov statistic comparing the effective library size of the real datasets versus the effective library size of the simulated datasets.
KS statistic of the effective library size.
The Kolmogorov-Smirnov statistic comparing the effective library size of the real datasets versus the effective library size of the simulated datasets.
KS statistic of the effective library size.
The Kolmogorov-Smirnov statistic comparing the morans I of the real datasets versus the morans I of the simulated datasets.
KS statistic of the fraction of zeros per spot (cell).
The Kolmogorov-Smirnov statistic comparing the fraction of zeros per spot (cell) in the real datasets versus the fraction of zeros per spot (cell) in the simulated datasets.
KS statistic of the fraction of zeros per spot (cell).
The Kolmogorov-Smirnov statistic comparing the fraction of zeros per spot (cell) in the real datasets versus the fraction of zeros per spot (cell) in the simulated datasets.
KS statistic of the fraction of zeros per gene.
The Kolmogorov-Smirnov statistic comparing the fraction of zeros per gene in the real datasets versus the fraction of zeros per gene in the simulated datasets.
KS statistic of the fraction of zeros per gene.
The Kolmogorov-Smirnov statistic comparing the fraction of zeros per gene in the real datasets versus the fraction of zeros per gene in the simulated datasets.
Frobenius norm of the centralized score matrix.
The Frobenius norm of the difference between two matrices is calculated to assess the closeness of the spatial pattern of cell types between the simulated and real data.
Frobenius norm of the neighborhood enrichment.
The Frobenius norm of the difference between two matrices is calculated to assess the closeness of the spatial pattern of cell types between the simulated and real data.
Frobenius norm of the transition matrix.
The Frobenius norm of the difference between two matrices is calculated to assess the closeness of the spatial pattern of cell types between the simulated and real data.
KS statistic of the gene Pearson correlation.
The Kolmogorov-Smirnov statistic comparing the gene Pearson correlation of the real datasets versus the gene Pearson correlation of the simulated datasets.
KS statistic of the gene Pearson correlation.
The Kolmogorov-Smirnov statistic comparing the gene Pearson correlation of the real datasets versus the gene Pearson correlation of the simulated datasets.
KS statistic of the L statistics
The Kolmogorov-Smirnov statistic comparing the L statistics in the real datasets versus the L statistics in the simulated datasets.
KS statistic of the library size.
The Kolmogorov-Smirnov statistic comparing the total sum of UMI counts across all genes in the real datasets versus the total sum of UMI counts across all genes in the simmulated datasets.
KS statistic of the library size.
The Kolmogorov-Smirnov statistic comparing the total sum of UMI counts across all genes in the real datasets versus the total sum of UMI counts across all genes in the simmulated datasets.
KS statistic of the library size.
The Kolmogorov-Smirnov statistic comparing the nn correlation in the real datasets versus the nn correlation in the simmulated datasets.
KS statistic of the relationship between library size and the proportion of zeros per spot (cell).
The Kolmogorov-Smirnov statistic comparing the relationship between library size and the proportion of zeros per spot (cell) in the real datasets versus the simulated datasets.
KS statistic of the relationship between library size and the proportion of zeros per spot (cell).
The Kolmogorov-Smirnov statistic comparing the relationship between library size and the proportion of zeros per spot (cell) in the real datasets versus the simulated datasets.
KS statistic of the relationship between mean expression and the proportion of zero per gene.
The Kolmogorov-Smirnov statistic comparing the relationship between mean expression and the proportion of zero per gene in the real datasets versus the simulated datasets.
KS statistic of the relationship between mean expression and the proportion of zero per gene.
The Kolmogorov-Smirnov statistic comparing the relationship between mean expression and the proportion of zero per gene in the real datasets versus the simulated datasets.
KS statistic of the relationship between mean expression and variance expression.
The Kolmogorov-Smirnov statistic comparing the relationship between mean expression and variance expression in the real datasets versus the simulated datasets.
KS statistic of the relationship between mean expression and variance expression.
The Kolmogorov-Smirnov statistic comparing the relationship between mean expression and variance expression in the real datasets versus the simulated datasets.
Ks Statistic of the neighborhood enrichment.
The Kolmogorov-Smirnov statistic comparing the neighborhood enrichment matrices of the real dataset versus the simulated dataset. The neighborhood enrichment matrix quantifies the enrichment observed between each pair of spatial clusters. It serves to systematically assess the interaction between different clusters within a spatial context, providing insights into the relative connectivity between various spatial clusters.
KS statistic of the sample Pearson correlation.
The Kolmogorov-Smirnov statistic comparing the sample Pearson correlation of the real datasets versus the sample Pearson correlation of the simulated datasets.
KS statistic of the sample Pearson correlation.
The Kolmogorov-Smirnov statistic comparing the sample Pearson correlation of the real datasets versus the sample Pearson correlation of the simulated datasets.
KS statistic of the spot- (or cell-) level scaled mean of the expression matrix.
The Kolmogorov-Smirnov statistic comparing the z-score standardization of the mean of expression matrix in terms of log2(CPM) in the real datasets versus the simulated datasets.
KS statistic of the spot- (or cell-) level scaled mean of the expression matrix.
The Kolmogorov-Smirnov statistic comparing the z-score standardization of the mean of expression matrix in terms of log2(CPM) in the real datasets versus the simulated datasets.
KS statistic of the gene-level scaled mean of the expression matrix.
The Kolmogorov-Smirnov statistic comparing the gene-level z-score standardization of the mean of expression matrix in terms of log2(CPM) in the real datasets versus the simulated datasets.
KS statistic of the gene-level scaled mean of the expression matrix.
The Kolmogorov-Smirnov statistic comparing the gene-level z-score standardization of the mean of expression matrix in terms of log2(CPM) in the real datasets versus the simulated datasets.
KS statistic of the spot- (or cell-) level scaled variance of the expression matrix.
The Kolmogorov-Smirnov statistic comparing the spot-level z-score standardization of the variance of expression matrix in terms of log2(CPM) in the real datasets versus the simulated datasets.
KS statistic of the spot- (or cell-) level scaled variance of the expression matrix.
The Kolmogorov-Smirnov statistic comparing the spot-level z-score standardization of the variance of expression matrix in terms of log2(CPM) in the real datasets versus the simulated datasets.
KS statistic of the gene-level scaled variance of the expression matrix.
The Kolmogorov-Smirnov statistic comparing the gene-level z-score standardization of the variance of expression matrix in terms of log2(CPM) in the real datasets versus the simulated datasets.
KS statistic of the gene-level scaled variance of the expression matrix.
The Kolmogorov-Smirnov statistic comparing the gene-level z-score standardization of the variance of expression matrix in terms of log2(CPM) in the real datasets versus the simulated datasets.
Precision measures the proportion of correctly identified items in simulated datasets.
Precision used in identifying spatial variable genes, measuring the accuracy of positive predictions.
Recall measures the proportion of real SVG correctly identified in the simulated dataset.
Recall used in identifying spatial variable genes, measuring the true positive rate.
KS statistic of the weight trimmed mean of M-values normalization factor (TMM).
The Kolmogorov-Smirnov statistic comparing the weight trimmed mean of M-values normalization factor for the real datasets versus the weight trimmed mean of M-values normalization factor for the simulated datasets.
KS statistic of the weight trimmed mean of M-values normalization factor (TMM).
The Kolmogorov-Smirnov statistic comparing the weight trimmed mean of M-values normalization factor for the real datasets versus the weight trimmed mean of M-values normalization factor for the simulated datasets.
KS Statistic of the transition matrix.
The Kolmogorov-Smirnov statistic comparing the transition matrix of the real dataset versus the simulated dataset. The transition matrix elucidates the interrelationships among spatial clusters in each space. Each element in the matrix signifies the transition probability from one spatial cluster to another, thereby mapping the dynamic interplay of spatial clusters.
Dataset info 10
10X Visium spatial RNA-seq from adult mouse brain sections paired to single-nucleus RNA-seq
This datasets were generated matched single nucleus (sn, this submission) and Visium spatial RNA-seq (10X Genomics) profiles of adjacent mouse brain sections that contain multiple regions from the telencephalon and diencephalon.
A spatially resolved atlas of human breast cancers
This study presents a spatially resolved transcriptomics analysis of human breast cancers.
Scripts and source data for image processing, barcode calling, and cell type annotations in a seqFISH+ experiment.
The dataset includes processed image data, cell type annotations with Louvain clusters, gene IDs for transcript locations, and mRNA point locations, with additional data available on Zenodo.
Multi-resolution deconvolution of spatial transcriptomics data reveals continuous patterns of Tumor A1 of Tissue 1
Spatial transcriptomics of Tumor A1 of Tissue 1.
single-cell and spatial transcriptomic molecular map of mouse gastrulation
Single-Cell omics Data across Mouse Gastrulation and Highly multiplexed spatially resolved gene expression profiling of Early Organogenesis.
Spatial RNA sequencing of regenerating mouse hindlimb muscle
The spatial transcriptomics datasets regenerates mouse muscle tissue generated with the 10x Genomics Visium platform.
Single-cell and spatial transcriptomic of mouse olfactory bulb
Spatial profiling of human osteosarcoma cells.
Spatial transcriptome profiling by MERFISH reveals subcellular RNA compartmentalization and cell cycle-dependent gene expression.
Integrating microarray-based spatial transcriptomics and single-cell RNA-seq reveals tissue architecture in pancreatic ductal adenocarcinomas
We developed a multimodal intersection analysis method combining scRNA-seq with spatial transcriptomics to map and characterize the spatial organization and interactions of distinct cell subpopulations in complex tissues, such as primary pancreatic tumors..
Spatially resolved gene expression of human protate tissue slices treated with steroid hormones for 8 hours
Spatially resolved gene expression was prepard by dissociated hman prostate tissue to single cells, and collected & prepped for RNA-seq using the Visium Spatial Gene Expression kit.
References
- Journal of Machine Learning Technologies. (n.d.). 10.9735/2229-3981 ↗
- Baruzzo, G., Patuzzi, I., & Di Camillo, B. (2019). SPARSim single cell: a count data simulator for scRNA-seq data. 10.1093/bioinformatics/btz752 ↗
- Chacón, J. E., & Duong, T. (2018). Multivariate Kernel Smoothing and its Applications. 10.1201/9780429485572 ↗
- Drost, H.-G. (2018). Philentropy: Information Theory and Distance Quantification with R. 10.21105/joss.00765 ↗
- Eng, C.-H. L., Lawson, M., Zhu, Q., Dries, R., Koulena, N., Takei, Y., Yun, J., Cronin, C., Karp, C., Yuan, G.-C., & Cai, L. (2019). Transcriptome-scale super-resolved imaging in tissues by RNA seqFISH+. 10.1038/s41586-019-1049-y ↗
- Hodson, T. O. (2022). Root-mean-square error (RMSE) or mean absolute error (MAE): when to use them or not. 10.5194/gmd-15-5481-2022 ↗
- Kleshchevnikov, V., Shmatko, A., Dann, E., Aivazidis, A., King, H. W., Li, T., Elmentaite, R., Lomakin, A., Kedlian, V., Gayoso, A., Jain, M. S., Park, J. S., Ramona, L., Tuck, E., Arutyunyan, A., Vento-Tormo, R., Gerstung, M., James, L., Stegle, O., & Bayraktar, O. A. (2022). Cell2location maps fine-grained cell types in spatial transcriptomics. Nature Biotechnology, 40(5), 661–671. 10.1038/s41587-021-01139-4 ↗
- Legendre, P., Fortin, M., & Borcard, D. (2015). Should the Mantel test be used in spatial analysis? 10.1111/2041-210x.12425 ↗
- Leydesdorff, L. (2005). Similarity measures, author cocitation analysis, and information theory. 10.1002/asi.20130 ↗
- Liang, X., Cao, Y., & Hwa Yang, J. Y. (2024). Multi-task benchmarking of spatially resolved gene expression simulation models. 10.1101/2024.05.29.596418 ↗
- Lohoff, T., Ghazanfar, S., Missarova, A., Koulena, N., Pierson, N., Griffiths, J. A., Bardot, E. S., Eng, C.-H. L., Tyser, R. C. V., Argelaguet, R., Guibentif, C., Srinivas, S., Briscoe, J., Simons, B. D., Hadjantonakis, A.-K., Göttgens, B., Reik, W., Nichols, J., Cai, L., & Marioni, J. C. (2021). Integration of spatial and single-cell transcriptomic data elucidates mouse organogenesis. 10.1038/s41587-021-01006-2 ↗
- Lopez, R., Li, B., Keren-Shaul, H., Boyeau, P., Kedmi, M., Pilzer, D., Jelinski, A., Yofe, I., David, E., Wagner, A., Ergen, C., Addadi, Y., Golani, O., Ronchese, F., Jordan, M. I., Amit, I., & Yosef, N. (2022). DestVI identifies continuums of cell types in spatial transcriptomics data. Nature Biotechnology, 40(9), 1360–1369. 10.1038/s41587-022-01272-8 ↗
- McCray, T., Pacheco, J. V., Loitz, C. C., Garcia, J., Baumann, B., Schlicht, M. J., Valyi-Nagy, K., Abern, M. R., & Nonn, L. (2021). Vitamin D sufficiency enhances differentiation of patient-derived prostate epithelial organoids. 10.1016/j.isci.2021.102640 ↗
- McKellar, D. W., Walter, L. D., Song, L. T., Mantri, M., Wang, M. F. Z., De Vlaminck, I., & Cosgrove, B. D. (2020). Strength in numbers: Large-scale integration of single-cell transcriptomic data reveals rare, transient muscle progenitor cell states in muscle regeneration. 10.1101/2020.12.01.407460 ↗
- Risso, D., Perraudeau, F., Gribkova, S., Dudoit, S., & Vert, J.-P. (2018). A general and flexible method for signal extraction from single-cell RNA-seq data. 10.1038/s41467-017-02554-5 ↗
- Song, D., Wang, Q., Yan, G., Liu, T., Sun, T., & Li, J. J. (2023). scDesign3 generates realistic in silico data for multimodal single-cell and spatial omics. 10.1038/s41587-023-01772-1 ↗
- Ståhl, P. L., Salmén, F., Vickovic, S., Lundmark, A., Navarro, J. F., Magnusson, J., Giacomello, S., Asp, M., Westholm, J. O., Huss, M., Mollbrink, A., Linnarsson, S., Codeluppi, S., Borg, Å., Pontén, F., Costea, P. I., Sahlén, P., Mulder, J., Bergmann, O., … Frisén, J. (2016). Visualization and analysis of gene expression in tissue sections by spatial transcriptomics. 10.1126/science.aaf2403 ↗
- Sun, T., Song, D., Li, W. V., & Li, J. J. (2021). scDesign2: a transparent simulator that generates high-fidelity single-cell gene expression count data with gene correlations captured. 10.1186/s13059-021-02367-2 ↗
- Vinh, N. X., Epps, J., & Bailey, J. (2009). Information theoretic measures for clusterings comparison. 10.1145/1553374.1553511 ↗
- Wu, S. Z., Al-Eryani, G., Roden, D. L., Junankar, S., Harvey, K., Andersson, A., Thennavan, A., Wang, C., Torpy, J. R., Bartonicek, N., Wang, T., Larsson, L., Kaczorowski, D., Weisenfeld, N. I., Uytingco, C. R., Chew, J. G., Bent, Z. W., Chan, C.-L., Gnanasambandapillai, V., … Swarbrick, A. (2021). A single-cell and spatially resolved atlas of human breast cancers. Nature Genetics, 53(9), 1334–1347. 10.1038/s41588-021-00911-1 ↗
- Xia, C., Fan, J., Emanuel, G., Hao, J., & Zhuang, X. (2019). Spatial transcriptome profiling by MERFISH reveals subcellular RNA compartmentalization and cell cycle-dependent gene expression. 10.1073/pnas.1912459116 ↗
- Zappia, L., Phipson, B., & Oshlack, A. (2017). Splatter: simulation of single-cell RNA sequencing data. 10.1186/s13059-017-1305-0 ↗
- Zhang, X., Xu, C., & Yosef, N. (2019). Simulating multiple faceted variability in single cell RNA sequencing. 10.1038/s41467-019-10500-w ↗
- Zhu, J., Shang, L., & Zhou, X. (2023). SRTsim: spatial pattern preserving simulations for spatially resolved transcriptomics. 10.1186/s13059-023-02879-z ↗