What are the lessons learned?

The process by which we arrived at a workable model, bringing together participants with a wide variety of perspectives, scientific, epidemiological, statistical and clinical, was crucial. In addition, impartial facilitation ensured that everyone could contribute to the discussion and engage in vigorous discussions that were focussed on the specific topics derived from the PrOACT-URL framework.

Quantification and shaping of judgements through a process of deliberative discourse and model-testing helped to make explicit what is normally implicit. As one participant said, "It requires each participant to explain publicly his or her opinion to other stakeholders; it requires that participant to construct the argument with some rationale instead of a rough global intuition." Another commented that "In addition the public sharing of discussion leads to a cross fertilisation in individual experiences or knowledge, which in return contributes to the evidence-base or rationale of the expressed opinion." In this way, assessing the benefit-risk balance becomes a collaborative process.

The group experienced occasional frustration at the difficulty of interpreting published data. Authors of published papers often report only sample sizes, means, confidence intervals and significance levels. While this may be sufficient for making statistical inferences, it may not be adequate for the purposes of an MCDA, particularly for sensitivity analyses and probabilistic simulation. The group also were frustrated by the lack of individual patient data available to the general public, and they recognised that benefits and risks are correlated, so many of the simulation combinations are not realistic. If the intercorrelations were taken into account, it is not clear whether this would weaken the output of the probabilistic simulation or strengthen it even more.

The important insight of the findings is that working with point estimates of the data instead of their underlying probability distributions could restrict medical regulators and other decision makers from seeing the full picture. Despite the fact of being an adequate benefit-risk balancing technique, using a deterministic MCDA alone is not necessarily sufficient to tell the whole story. That may not matter for simpler cases, but for complex ones, like rosiglitazone, a fuller consideration of all the clinical data could lead to a different conclusion.

This case study shows it is feasible to create a model of a drug's benefit-risk balance that could assist decision makers in the pharmaceutical industry, regulators and prescribing physicians. The MCDA model handles all sorts of evidence and makes explicit the difference between that evidence and its therapeutic and clinical relevance. Adding probabilistic simulation makes it possible to make statements about how much more likely the benefit-risk balance of a drug is compared to any other alternative. Such a statement could be useful to any decision maker.