Background Calculation of numbers needed to treat (NNT) is more complex from meta-analysis than from single trials. Treating the data as if it all came from one trial may lead to misleading results when the trial arms are imbalanced. Discussion An example is shown from a published Cochrane review in which the benefit of nursing intervention for smoking cessation is shown by formal meta-analysis of the individual trial results. However if these patients were added together as if they all came from one trial the direction of the effect appears to be reversed (due to Simpson's paradox). Whilst NNT from meta-analysis can be calculated from pooled Risk Differences, this is unlikely to be a stable method unless the event rates in the control groups are very similar. Since in practice event rates vary considerably, the use a relative measure, such as Odds Ratio or Relative Risk is advocated. These can be applied to different levels of baseline risk to generate a risk specific NNT for the treatment. Summary The method used to calculate NNT from meta-analysis should be clearly stated, and adding the patients from separate trials as if they all came from one trial should be avoided.

Analysis of subgroup results in a clinical trial is surprisingly unreliable, even in a large trial. This is the result of a combination of reduced statistical power, increased variance and the play of chance. Reliance on such analyses is likely to be more erroneous, and hence harmful, than application of the overall proportional (or relative) result in the whole trial to the estimate of absolute risk in that subgroup. Plausible explanations can usually be found for effects that are, in reality, simply due to the play of chance. When clinicians believe such subgroup analyses, there is a real danger of harm to the individual patient.

Background Acceptability curves have been proposed for quantifying the probability that a treatment under investigation in a clinical trial is cost-effective. Various definitions and estimation methods have been proposed. Loosely speaking, all the definitions, Bayesian or otherwise, relate to the probability that the treatment under consideration is cost-effective as a function of the value placed on a unit of effectiveness. These definitions are, in fact, expressions of the certainty with which the current evidence would lead us to believe that the treatment under consideration is cost-effective, and are dependent on the amount of evidence (i.e. sample size). Methods An alternative for quantifying the probability that the treatment under consideration is cost-effective, which is independent of sample size, is proposed. Results Non-parametric methods are given for point and interval estimation. In addition, these methods provide a non-parametric estimator and confidence interval for the incremental cost-effectiveness ratio. An example is provided. Conclusions The proposed parameter for quantifying the probability that a new therapy is cost-effective is superior to the acceptability curve because it is not sample size dependent and because it can be interpreted as the proportion of patients who would benefit if given the new therapy. Non-parametric methods are used to estimate the parameter and its variance, providing the appropriate confidence intervals and test of hypothesis.

Randomized trials and observational studies, such as case-control studies, are often seen as opposing approaches. However, in many instances results obtained by different designs may complement each other. For instance, case-control studies on aetiology of disease may help to give the direction of future trials. In this commentary, the author discusses the purpose of randomization and observation, and under which conditions one design may be preferred to another. Randomization is useful to combat 'confounding by indication', and is therefore the design of choice for most therapeutic trials. When this confounding is not an issue, as in studies of genetic risk factors or side-effects, then case-control studies are preferred.

In one of the biggest dilemmas facing cardiovascular clinical research, clinical trials are increasingly being required to show benefits on clinical end-points rather than surrogate end-points, while at the same time the incremental benefits of newer treatments are getting smaller. These two factors have a huge impact on sample size, which has led some investigators to design trials to show that the new treatment has an effect similar to that of the standard, rather than outright superiority. Recent examples of fibrinolytic trials that have demonstrated similar effects of two drugs are ASSENT (Assessment of the Safety and Efficacy of a New Thrombolytic)-2, GUSTO (Global Use of Strategies to Open Occluded Coronary Arteries)-III, and COBALT (Continuous Infusion Versus Double-Bolus Administration of Alteplase) [1,2,3,4]. However, as discussed by several authors [5,6,7,8], there are issues with trials of this type that make them considerably less credible than superiority trials.

Background Many randomized trials involve measuring a continuous outcome - such as pain, body weight or blood pressure - at baseline and after treatment. In this paper, I compare four possibilities for how such trials can be analyzed: post-treatment; change between baseline and post-treatment; percentage change between baseline and post-treatment and analysis of covariance (ANCOVA) with baseline score as a covariate. The statistical power of each method was determined for a hypothetical randomized trial under a range of correlations between baseline and post-treatment scores. Results ANCOVA has the highest statistical power. Change from baseline has acceptable power when correlation between baseline and post-treatment scores is high;when correlation is low, analyzing only post-treatment scores has reasonable power. Percentage change from baseline has the lowest statistical power and was highly sensitive to changes in variance. Theoretical considerations suggest that percentage change from baseline will also fail to protect from bias in the case of baseline imbalance and will lead to an excess of trials with non-normally distributed outcome data. Conclusions Percentage change from baseline should not be used in statistical analysis. Trialists wishing to report this statistic should use another method, such as ANCOVA, and convert the results to a percentage change by using mean baseline scores.

Background Readers may question the interpretation of findings in clinical trials when multiple outcome measures are used without adjustment of the p-value. This question arises because of the increased risk of Type I errors (findings of false "significance") when multiple simultaneous hypotheses are tested at set p-values. The primary aim of this study was to estimate the need to make appropriate p-value adjustments in clinical trials to compensate for a possible increased risk in committing Type I errors when multiple outcome measures are used. Discussion The classicists believe that the chance of finding at least one test statistically significant due to chance and incorrectly declaring a difference increases as the number of comparisons increases. The rationalists have the following objections to that theory: 1) P-value adjustments are calculated based on how many tests are to be considered, and that number has been defined arbitrarily and variably; 2) P-value adjustments reduce the chance of making type I errors, but they increase the chance of making type II errors or needing to increase the sample size. Summary Readers should balance a study's statistical significance with the magnitude of effect, the quality of the study and with findings from other studies. Researchers facing multiple outcome measures might want to either select a primary outcome measure or use a global assessment measure, rather than adjusting the p-value.

Background The study of cost-effectiveness comparisons between competing medical interventions has led to a variety of proposals for quantifying cost-effectiveness. The differences between the various approaches can be subtle, and one purpose of this article is to clarify some important distinctions. Discussion We discuss alternative measures in the framework of individual, patient-level, incremental net benefits. In particular we examine the probability of cost-effectiveness for an individual, proposed by Willan. Summary We argue that this is a useful addition to the range of cost-effectiveness measures, but will be of secondary interest to most decision makers. We also demonstrate that Willan's proposed estimate of this probability is logically flawed.

Clinical trialists and statisticians are very wary of subgroup analysis, for good reasons. Clinicians have to deal with situations in which subgroups of patients differ widely from one another in their prognosis and response to treatment. Few trials are large enough to demonstrate convincingly these differences in outcome, but often provide suggestive evidence. Should we ignore this and treat all patients as the same, or should we allow dubious statistical evidence to buttress biological plausibility in making clinical decisions?

The focus of this report is to describe the statistical inference procedures used to produce design-based estimates as presented in the 2017 detailed tables and the 2017 FFR, which are based on restricted-use data.