Clinical trial statistics answer different questions. An effect size describes the observed treatment benefit. A confidence interval shows how precisely that benefit has been estimated. Statistical power describes a trial’s ability to detect an assumed effect, while a p-value measures how incompatible the observed data are with a stated null hypothesis and its statistical assumptions. None of these numbers, taken alone, establishes that a drug deserves approval or has commercial value.
For biotech investors, the task is to connect the statistics to the treatment claim. A small p-value can accompany a benefit that patients would barely notice. A large observed benefit can remain uncertain because too few patients were studied. The starting point is the question established by the clinical trial’s design and patient selection, followed by the estimated effect, its uncertainty and the rules used to judge success. The examples below are hypothetical, not results from actual medicines.
What the Main Clinical Trial Statistics Tell You
A trial announcement may place its p-value in the headline, leaving the treatment difference and confidence interval several paragraphs below. Reverse that reading order. Identify what was measured, compare the treatment groups, then examine uncertainty and statistical testing. Keep the denominator visible throughout: 12 responses among 20 patients represent a different evidence base from 600 responses among 1,000 patients, even though both response rates equal 60%. The following distinctions prevent several common interpretation errors.
| Statistic | Question it addresses | What it does not establish |
|---|---|---|
| Effect size | How large was the estimated treatment difference? | Whether that estimate is precise or clinically worthwhile |
| Confidence interval | How much uncertainty surrounds the estimated effect? | The range of outcomes every patient will experience |
| P-value | How incompatible are the results with the null hypothesis under the analysis assumptions? | The probability that the drug works |
| Power | How often would the planned test detect an assumed true effect? | The probability that this development program will succeed |
| Significance threshold | What statistical decision rule was established for the test? | The minimum benefit patients or physicians would consider useful |
Effect Sizes: Measure the Benefit Before Judging the P-Value
Absolute and Relative Effects
An effect size expresses the magnitude of a treatment difference. Its form depends on the outcome: a difference in symptom scores, a ratio of response rates or a comparison of event hazards over time. Direction matters. A higher remission rate is favorable; a higher hospitalization rate, unfavorable. Before interpreting any number, establish what counts as a response, when it was assessed and whether the comparison uses all assigned patients. Those questions belong alongside the choice of clinical endpoint and its relevance to patients.
Suppose 90 of 200 patients receiving an experimental treatment respond at week 24, compared with 60 of 200 receiving the control. The response rates are 45% and 30%. The absolute improvement is 15 percentage points. The response risk ratio is 1.50, meaning the observed response rate is 50% higher relative to control. Both descriptions are correct, but they emphasize different aspects of the same result. Calling this a “50 percentage point improvement” would be wrong. Percentages and percentage points are not interchangeable.
| Measure | Calculation | Result |
|---|---|---|
| Treatment response rate | 90 ÷ 200 | 45% |
| Control response rate | 60 ÷ 200 | 30% |
| Absolute response difference | 45% − 30% | 15 percentage points |
| Response risk ratio | 45% ÷ 30% | 1.50 |
| Relative response increase | (45% − 30%) ÷ 30% | 50% |
| Response odds ratio | (90 ÷ 110) ÷ (60 ÷ 140) | Approximately 1.91 |
The odds ratio is larger than the risk ratio in this example because odds and probabilities use different denominators. A response probability divides responders by everyone in the group; response odds divide responders by nonresponders. Describing an odds ratio of 1.91 as “91% more patients responded” would therefore misstate these data. When an announcement reports an odds ratio, request the underlying group rates. The ratio alone leaves too much room for an exaggerated impression of benefit.
Baseline risk also changes the meaning of a relative reduction. Reducing an adverse outcome from 20% to 15% produces a 25% relative reduction and a five percentage point absolute reduction. Reducing it from 4% to 3% produces the same relative reduction, but only a one percentage point absolute reduction. Over the stated observation period, these imply numbers needed to treat of 20 and 100 respectively. That calculation takes the reciprocal of the absolute risk reduction expressed as a proportion. It needs a time horizon and a clearly defined outcome.
Continuous Outcomes and Clinical Importance
For continuous outcomes, retain the original units whenever possible. A four point improvement on a symptom scale is easier to evaluate when the scale, its direction and a justified threshold for meaningful change are supplied. A standardized effect divides a difference by a measure of variability, which can help compare measurements but removes their original units. Neither format settles whether the benefit matters. Ask whether the result changes patients’ function, treatment burden or disease experience enough to influence prescribing. The relevant commercial comparison is the benefit available from competing treatments, rather than an abstract preference for a larger number.
Hazard Ratios and Survival Outcomes
A hazard ratio compares event hazards between groups over time. The hazard concerns the instantaneous event rate among people who have not yet experienced the event. For an unfavorable event such as death or progression, a hazard ratio below one generally favors treatment. The National Cancer Institute’s hazard ratio definition identifies one as the no-difference value. A reported hazard ratio of 0.70 is commonly expressed as a 30% lower hazard. It does not mean that patients live 30% longer, or that 30% fewer patients will die by every possible follow-up date.
A single hazard ratio also needs a suitable interpretation of how the treatment effect behaves over time. If benefit appears late or survival curves cross, a constant relative effect may be a poor description. Examine the curves and survival estimates at relevant dates, rather than treating the headline ratio as a complete account. Restricted mean survival time can express the average event-free or survival time accumulated up to a chosen horizon; Royston and Parmar’s methodological research on restricted mean survival time develops this approach when proportional hazards are in doubt.
P-Values: What They Mean and What They Do Not
A p-value is calculated under a null hypothesis and the assumptions of the statistical test. For a conventional superiority comparison, the null often represents no treatment difference. The p-value is the probability, under that model, of obtaining a test statistic at least as incompatible with the null as the one observed. This is a conditional calculation. It does not reverse the question and tell you how likely the null hypothesis is after observing the data. The American Statistical Association’s principles on p-values reject that interpretation and distinguish statistical significance from effect size or practical importance.
Consider a trial reporting p = 0.03. That number does not mean there is a 3% probability the drug is ineffective, a 97% probability the treatment works or a 3% probability the result will fail to replicate. It also does not mean that “chance caused 3% of the benefit.” None of those statements describes the calculation. For investment analysis, avoid converting a p-value into a development probability. That requires a broader assessment of the evidence and the remaining clinical and regulatory success probabilities.
A significance threshold, commonly written as alpha, establishes a decision rule before results are examined. For an appropriately calibrated test with alpha of 0.05, the false positive probability is controlled at 5% under the null and the relevant assumptions. This does not mean that 5% of all positive drug trials are false findings. The latter question also depends on how often treatments genuinely work and how effectively studies detect those effects.
The distinction between p = 0.049 and p = 0.051 illustrates the difference between a decision rule and a scientific interpretation. A prespecified threshold may put those results on opposite sides of a formal success criterion. Their evidential difference, considered alone, is small. Read the effect estimates and intervals before treating one as convincing and the other as worthless. Equally, do not dismiss a failed primary test as an administrative technicality: the study’s stated objective still matters.
Check whether the reported p-value is one-sided or two-sided. A two-sided test considers departures in either direction; a one-sided test evaluates the prespecified direction. A one-sided p-value of 0.04 cannot simply be compared with a two-sided 0.05 threshold. The testing direction and error level must be established prospectively, rather than selected because one format gives the more attractive result.
Confidence Intervals: Separate Precision From Clinical Value
A confidence interval places the effect estimate in an uncertainty range. Under the frequentist interpretation, a valid 95% confidence interval procedure would cover the fixed true effect in about 95% of repeated applications under its assumptions. It does not assign a 95% probability to the true effect lying inside this particular calculated interval. Nor does it contain 95% of individual patient outcomes. The NIST explanation of confidence limits distinguishes the coverage of the procedure from the interpretation of one observed interval.
For practical reading, examine both boundaries. Suppose treatment improves a symptom score by six points, with a 95% confidence interval from two to ten points, where positive values favor treatment. The result estimates benefit, but its size remains uncertain. If eight points is a justified threshold for a worthwhile treatment difference, this interval includes effects below and above that threshold. Evidence against zero benefit does not establish that the benefit is large enough to justify treatment.
Compare that with an interval from minus one to thirteen points. It includes no difference, slight harm and a substantial benefit. Calling the result “negative” without further qualification hides this uncertainty. An interval from minus one to one point tells a different story: it also includes zero, but rules out much larger effects under the model. The same statistical label can therefore cover an inconclusive study or a precise estimate of a negligible difference. That distinction matters when deciding whether another trial is worth funding.
For difference measures, the usual no-effect value is zero. For ratios, it is one. With a conventional two-sided test and a matching 95% confidence interval, excluding the no-effect value corresponds to rejecting that null at the 0.05 level. Matching is the operative word. A nominal interval may not reflect adjustments for multiple endpoints or interim testing. Check the analysis method before assuming that an interval excluding zero establishes formal trial success.
Confidence intervals describe statistical uncertainty under their assumptions; they do not repair bias or make an unsuitable endpoint useful. A precise estimate can still answer the wrong clinical question. Interpret the interval beside treatment burden and adverse events, discontinuations and benefit–risk balance. Also use the interval for the treatment contrast itself. Looking only at whether separate treatment-group confidence intervals overlap is not a reliable substitute for the planned between-group comparison.
Statistical Power: A Planning Assumption, Not a Success Forecast
Power is the probability that a test rejects the null when a stated alternative effect is true. An 80% powered design would detect that assumed effect in about 80% of repeated trials conducted under the planning assumptions. The remaining 20% represents the corresponding Type II error probability. Power depends on the assumed effect, sample size, variability and testing threshold. The NIST sample size calculations show these relationships explicitly. Power is conditional on the assumptions, not a probability that the drug’s biological hypothesis is correct.
Suppose a trial is planned around a ten point treatment difference, but the true difference is five points. Its stated power for ten points does not carry across unchanged. Detecting the smaller effect is harder. Under a simple comparison of means, with variability, allocation and testing requirements held constant, halving the target difference requires roughly four times as many participants to retain the same power. That arithmetic explains why optimistic assumptions can produce a manageable trial budget and an unhelpful experiment.
Scrutinize where the expected effect came from. Was it estimated in a small exploratory cohort? Did the earlier study use a different population or a weaker control? A trial designed around an unusually favorable early estimate may struggle even when the treatment has some genuine activity. The purpose of the different clinical trial phases affects how much weight to give those assumptions. Early evidence can justify further investigation without establishing a dependable effect size for a larger confirmatory study.
Variability, Sample Size and Event Counts
Standard deviation describes variation among observations. Standard error describes uncertainty in an estimate. They answer different questions, so check which one appears beside an error bar. In a simple independent sample, increasing patient numbers reduces the standard error even if patient-to-patient variation remains unchanged. A larger trial can therefore detect a small average difference without making individual responses more predictable. More precise does not mean more useful to every patient.
For an event-driven survival trial, information depends heavily on the number of observed events, rather than enrollment alone. “Fully enrolled” does not necessarily mean that the primary analysis is close. If events occur more slowly than expected, the study may need longer follow-up. For an investor, that creates a second question beyond statistical power: can the company fund the additional time? Connect event assumptions with cash burn and runway through the next milestone, instead of treating enrollment completion as equivalent to an imminent readout.
Multiple Testing: Why Every P-Value Below 0.05 Is Not a Confirmed Finding
A study can generate many opportunities to find a favorable result: different endpoints, doses, assessment dates and patient groups. If success can be declared from any one of several tests, applying an unadjusted threshold to each can inflate the chance of a false positive. The FDA guidance on multiple endpoints in clinical trials addresses methods for controlling that risk. These include allocating error across tests and ordering hypotheses in a prespecified testing sequence.
A simple illustration shows the problem. If 20 independent tests each have a 5% false positive probability, and every null hypothesis is true, the probability of at least one false positive is 1 − 0.9520, approximately 64%. Actual trial outcomes are often correlated, so this is not a universal estimate for studies with 20 analyses. It demonstrates why “we found one p-value below 0.05” is a weak claim when the search involved many chances to find it.
In a simple fixed testing sequence, a secondary endpoint may be tested formally only after the preceding endpoint succeeds. If the primary test fails, a favorable secondary result can remain descriptive rather than confirmatory, even if its nominal p-value is below 0.05. Co-primary endpoints create a different arrangement: when success requires every co-primary endpoint to pass, the study has fewer ways to succeed, which affects power. The protocol must make clear whether all required outcomes or any one of several outcomes establish success.
When reading an announcement, identify which results survived the planned multiplicity procedure and which were reported without that protection. “Nominally significant” is not necessarily an invalid calculation; it describes evidence that may lack the required error control for a confirmatory claim. Be especially cautious when a failed primary result is followed by a long selection of favorable subgroups. The separate guide to selective reporting and subgroup claims covers that reporting problem in more detail.
Interim Analyses: Apply the Correct Stopping Boundary
Repeatedly testing accumulating efficacy data creates another route to false positive findings. A planned interim analysis therefore needs rules that account for the opportunities to stop early. Group sequential methods can set different boundaries at interim and final analyses while controlling the trial’s error probability. Early efficacy stopping may require much stronger evidence than the familiar p < 0.05 convention. The FDA guidance on adaptive clinical trial designs describes prospective stopping boundaries, error control and the distinction between efficacy and futility decisions.
Suppose an interim analysis produces p = 0.02, but its prespecified efficacy boundary is 0.005. The trial has not crossed that boundary. Continuing the study does not contradict the reported p-value. Equally, a statement that a trial passed a futility review does not mean efficacy has been established: it may mean only that the available data did not trigger a recommendation to stop. For investment catalysts around interim readouts, distinguish a continuation decision from a successful efficacy analysis. The difference can change what the next announcement still needs to establish.
Analysis Populations and Missing Data Can Change the Answer
Before comparing statistical outputs, identify exactly which treatment effect the analysis aims to estimate. An estimand defines that target, including the population, treatment comparison and outcome, with a strategy for events such as treatment discontinuation or rescue medication. These events do not automatically make later outcomes missing or irrelevant. The ICH E9(R1) framework for estimands and sensitivity analysis connects the clinical question to data collection, estimation and the assumptions tested in sensitivity analyses.
Consider a patient who stops the experimental medicine because of nausea and starts rescue treatment. One question asks what happens after assignment to the treatment strategy, including subsequent care. Another asks what might have happened without rescue treatment. Those questions need different handling of the same patient’s later measurements. Neither becomes the correct investment interpretation simply because it produces the smaller p-value. Ask which question matches the proposed use of the medicine and whether the necessary outcomes were collected.
A completer-only result also needs scrutiny. If patients with poor responses leave the treatment group, the remaining patients may give an attractive but incomplete impression. Write down how many patients were assigned, how many contributed to the primary estimate and why others did not. Then inspect whether the result changes under plausible alternative assumptions about unobserved outcomes. A sensitivity analysis should challenge assumptions while addressing the same target effect; switching to a different clinical question is a supplementary analysis.
Look for the protocol and statistical analysis plan, including their dates and amendments. Establish whether the highlighted analysis was planned before comparative outcomes were known. A change is not automatically improper, but it needs an explanation and a clear account of what changed. The practical guide to trial registries, scientific papers and financial filings provides a broader research workflow. For statistical interpretation, the immediate goal is narrower: match the announced result to the analysis the trial was designed to perform.
Subgroups and Non-Inferiority Require Different Questions
A statistically significant result in one subgroup and a nonsignificant result in another does not, by itself, demonstrate that treatment effects differ between them. One group may simply be larger or more precisely measured. An interaction analysis addresses differences in treatment effect across groups. For non-inferiority, the question changes again: the aim is to rule out a loss of efficacy exceeding a justified, prespecified margin. Failing to demonstrate superiority does not establish non-inferiority. These distinctions are set out in ICH E9 statistical principles for clinical trials.
Suppose a hypothetical non-inferiority study measures treatment minus control response rates, with a margin of minus ten percentage points. An estimated difference of minus two points with a matching interval from minus six to plus two would exclude losses beyond that margin. An interval from minus twelve to plus two would not. Both intervals include zero. That is why looking only for a p-value above 0.05 on a superiority test answers the wrong question. Check the margin, its clinical justification and the confidence level required by the planned analysis.
For subgroup claims, ask whether the patient definition was established in advance and whether it can be applied reliably in practice. A biomarker-positive population may have a stronger biological rationale, but a smaller commercial pool. Keep those consequences separate. Evidence for a restricted population should feed into an estimate of eligible patients and the addressable market, rather than being combined with the revenue assumptions for a much broader population.
A Worked Example: Reading a Positive Response Result
Return to the hypothetical trial with 200 patients in each group and week 24 response rates of 45% versus 30%. Assume this is the prespecified primary comparison, that everyone’s response status is known and that a conventional unadjusted analysis is appropriate. The absolute response difference is 15 percentage points. A simple large-sample 95% confidence interval for that difference is approximately 5.6 to 24.4 percentage points. A conventional unadjusted two-group comparison gives a two-sided p-value of about 0.002, with the precise calculation depending on the test used.
Those results support a treatment difference under the stated assumptions. They do not make the magnitude certain. The interval includes an improvement of roughly six additional responders per 100 treated patients and one of roughly 24 additional responders. That spread matters if the investment case depends on a very large advantage. Write a factual interpretation before making a valuation decision: the study estimated a 15 percentage point improvement, with uncertainty spanning more modest and larger benefits.
Now add a commercial assumption: the medicine would need at least a ten percentage point advantage to compensate for a burdensome administration schedule. The point estimate exceeds that benchmark, but the interval extends below it. The trial can therefore meet its statistical objective while leaving uncertainty about the commercial proposition. That is not a contradiction. It is the difference between evidence for some treatment benefit and evidence that the benefit clears the threshold used in the investment model.
The next review should examine response durability, treatment burden and the result’s relevance to available care. It should also verify that the confidence interval and p-value correspond to the announced primary analysis, rather than an attractive alternative. A broader process for reading trial results and assessing their investment importance then connects this statistical finding to the rest of the clinical evidence.
Translate Statistical Uncertainty Into Valuation Assumptions
Do not enter a p-value into a spreadsheet as though it were an approval probability. Start by stating what uncertainty the trial has reduced. Has it provided evidence of biological activity, estimated a clinically relevant benefit against a suitable comparator, or supported a narrower patient population than previously expected? Each answer changes a different part of the investment case. A risk-adjusted pipeline valuation should reflect those distinctions instead of giving every statistically positive readout the same treatment.
Use the estimated effect and its uncertainty to challenge assumptions about adoption. If the commercial forecast depends on a large efficacy advantage, ask how the forecast behaves when the advantage is modest. If convenience is the proposed selling point, ask whether the estimated efficacy leaves room for that positioning. Avoid treating the confidence interval’s endpoints as ready-made bear and bull cases with assigned probabilities. They describe uncertainty in an estimated treatment effect, not the probability distribution of future sales.
Keep clinical uncertainty separate from commercial uncertainty. Even a precise benefit estimate does not determine price, reimbursement, manufacturing capacity or physician adoption. Conversely, an uncertain benefit estimate does not justify an elaborate sales forecast with narrow error bars. Use valuation sensitivity analysis to show which assumptions drive the result. If a small change in treatment differentiation eliminates most of the estimated value, the investment thesis remains fragile despite a successful primary test.
Build a Short Statistical Review Before the Next Readout
Prepare the review before the announcement arrives. Record the primary endpoint and assessment date, the target treatment difference, the planned analysis population and any known testing sequence. Leave space for the observed effect and confidence interval. This makes it harder for an attractive secondary result to replace the original success criterion after the fact. It also gives you a record of which assumptions were yours and which were established by the trial protocol.
- Write the treatment contrast in plain English. Name the groups, outcome and assessment period.
- Record the effect in useful units. Include absolute group results when available, rather than copying only a relative ratio.
- Read both confidence limits. Check whether they include harm, negligible benefit or effects below the threshold your investment case requires.
- Verify the statistical decision. Match the p-value to the planned test, multiplicity procedure and any interim boundary.
- List unresolved questions. Separate missing data and analysis concerns from safety, durability and commercial assumptions.
Keep that review beside the valuation model and the planned position size around the clinical readout. When a release omits the treatment difference, its interval or the required testing details, mark the claim as unresolved rather than filling the gap with a favorable assumption. Request the missing analysis. A precise question is more useful than a confident interpretation of an incomplete result.