Literature and study design · What you can do here
Choose and check a statistical test
Choose the correct statistical test directly from your experimental design, evaluate assumptions, and report effect sizes with confidence intervals rather than isolated p-values.
What this involves
Statistical tests must be chosen from the structure of your experiment before you analyse outcome columns. Selecting tests by trying several options to see which yields the lowest p-value produces spurious findings. Sound analysis requires establishing the genuine unit of independent observation, whether samples are paired or independent, the number of groups, and the distribution of residuals.
Doing this work properly means avoiding common testing fallacies. Two-sample continuous comparisons use Welch's t-test by default rather than gating on equal variance tests, ratio-like outcomes such as fold changes are log-transformed, and technical replicates are averaged or modelled hierarchically to prevent pseudo-replication. When multiple hypotheses are evaluated, family-wise error or false discovery rates are controlled systematically using Holm or Benjamini–Hochberg corrections.
The platform agent executes your statistical workflows using Python libraries including scipy, statsmodels, scikit-learn, and lifelines on its own environment. It extracts estimates, computes bootstrap confidence intervals, inspects diagnostic residuals, and returns complete reporting text without requiring any local package installation.
How it goes
- 1Define the experimental design and unit of analysisYou define the independent experimental unit, grouping structure, pairing, and outcome type, separating true biological replicates from technical replicates before selecting the test.
- 2Select and execute the statistical testThe agent runs the design-appropriate test in scipy or statsmodels, using Welch's t-test for two independent groups, exact permutation tests for small sample sizes below six, or generalised linear models for counts and proportions.
- 3Evaluate model diagnostics and assumptionsUsing statsmodels, the agent examines residual QQ plots, fitted-versus-residual curvature, heteroscedasticity, variance inflation factors, and Cook's distance to verify model specification.
- 4Apply multiple-comparison correctionsWhen testing families of hypotheses, the agent applies statsmodels multipletests using the Holm step-down procedure for confirmatory targets or the Benjamini–Hochberg method to control false discovery rates.
- 5Calculate effect sizes and confidence intervalsThe agent computes point estimates, standardised effect sizes, and 95% confidence intervals using scipy bootstrap routines to accompany all reported test statistics.
Start it here
Ask in plain language. The agent works on a Linux machine of its own — it installs and runs what the job needs, keeps every file, and reads the results back with the numbers explained. Nothing to set up, and a long run continues after you close the page.
What to watch for
- Treating technical replicates such as multiple wells or timepoints as independent units creates pseudo-replication and invalidates standard errors.
- Gating test selection on a Shapiro–Wilk test is unreliable because it lacks power at small sample sizes and rejects harmless deviations at large sample sizes.
- Testing for equal variance with Levene's test before running a t-test is unnecessary because Welch's t-test carries no penalty when variances are equal.
- Post-hoc or observed power is merely a monotone mathematical transformation of the p-value and conveys no independent information.
- Reporting a p-value without an effect size, its units, sample size, and a 95% confidence interval is uninformative.
- Evaluating multiple comparisons without Holm or Benjamini–Hochberg adjustments across the entire family of tests inflates false discovery.
Questions
Do I need to install anything to choose and check a statistical test?
No. The research agent does this work on a Linux machine of its own — it installs what the job needs, keeps the files, and reads the results back to you. Long runs continue if you close the page.
When should Welch's t-test be used instead of Student's standard t-test?
Welch's t-test should always be used for two independent groups by setting equal_var to False. It protects against type I error when variances differ while losing negligible power when variances are identical.
How should skewed positive outcomes like IC50 or fold change be handled?
Skewed, ratio-like measures should be log-transformed prior to running a t-test. This tests the geometric mean ratio directly, providing an interpretable estimate rather than relying on non-parametric rank tests.
Which multiple testing correction method does the agent run?
The agent uses statsmodels to apply the Holm correction for small, pre-planned confirmatory hypothesis sets, or the Benjamini–Hochberg false discovery rate procedure for broader screening panels.
How does the agent evaluate statistical power when a study yields a null result?
Prospective power is calculated prior to testing based on the smallest meaningful effect using statsmodels power solvers. For completed null studies, the agent reports the width of the confidence interval rather than post-hoc power.
Written from the method the research agent follows for this work and the documentation of the tools listed above. Statements citing a figure that does not appear there are removed before publication. Corollary is a non-commercial research demonstration; results are not for clinical use.