Probability of Success for Dual Endpoints (PFS & OS)

Gabriel Potvin, Valeria A. G. Mazzanti, J. Kyle Wathen

May 01, 2025

The following scripts are related to both the Integration Point: Response and the Integration Point: Analysis. Click on the links for more information about these integration points.

Introduction

This example demonstrates how to compute the probability of success of a clinical trial by extending East Horizon’s single-endpoint framework to handle dual endpoints using custom R scripts for the Response (Patient Simulation) and the Analysis integration points. This example uses Progression Free Survival and Overall Survival as endpoints, however, this could be extended to other types of endpoints by modifying the R code.

To try out this example, start by creating a new project on East Horizon using the Two Arm Confirmatory study objective and the Time to Event single endpoint, and then create an input set using the Explore task.

The figure below illustrates where this example fits within the R integration points of Cytel products, accompanied by flowcharts outlining the general steps performed by the R code.

Once CyneRgy is installed, you can load this example in RStudio with the following commands:

CyneRgy::RunExample( "ProbabilitySuccessDualEndpointsPFSOS" )

Running the command above will load the RStudio project in RStudio.

RStudio Project File: DualEndpoint.Rproj

In the R directory of this example you will find the following R files:

1. AnalyzePFSAndOS.R - This file provides the code used for the Analysis integration point below.

2. Simulate2EndpointTTEWithMultiState.R - This file provides the code used for the Response integration point below.

Response (Patient Simulation) Integration Point

This endpoint is related to this R file: Simulate2EndpointTTEWithMultiState.R

We are using East Horizon’s single-endpoint framework, which we customize to support dual endpoints through the Response Integration Point via our R script linked above. The two endpoints of interest are:

• Progression-Free Survival (PFS): The duration during which a patient lives with a disease without experiencing further progression.
• Overall Survival (OS): The total time elapsed from treatment until death.

We use a multi-state model to simulate event times for each patient in every simulated trial. This model captures the relationship between PFS and OS by generating both outcomes together, rather than independently. Information generated from this simulation will be used later for the Analysis Integration Point.

The diagram above illustrates the multistate model for survival analysis, with three states:

1. State 0: Initial, no progression.
2. State 1: Progression, event occurs that makes the disease progress.
3. State 2: Death.

There are also three transitions between states:

1. $\alpha_{01}$: Transition rate from Initial to Progression.
2. $\alpha_{02}$: Transition rate from Initial to Death.
3. $\alpha_{12}$: Transition rate from Progression to Death.

From Meller et al. [1], the probability of progression before death is:

$$q = \frac{\alpha_{01}}{\alpha_{01} + \alpha_{02}}$$

Where $q = 1 - p$, with $p$ being the probability of Death before Progression.

Using this relationship:

$$\alpha_{02} = \frac{\alpha_{01} \cdot p}{q}$$

Or equivalently:

$$\alpha_{01} = \frac{\alpha_{02} \cdot q}{p}$$

Progression-Free Survival (PFS) is treated as the minimum of two competing times:

• $X_1 \sim \text{Exp}(\alpha_{01})$: Time from Initial to Progression.
• $X_2 \sim \text{Exp}(\alpha_{02})$: Time from Initial to Death without Progression.

Using the property of the minimum of two exponential random variables [2]:

$$Z = \min(X_1, X_2) \sim \text{Exp}(\alpha_{01} + \alpha_{02})$$

The median of an exponential distribution is given by:

$$M = \frac{\log(2)}{\alpha_{01} + \alpha_{02}}$$

Substituting $alpha_{01}$ in terms of $alpha_{02}$:

$$M = \frac{1}{\alpha_{01} + \alpha_{02}} = \frac{1}{\frac{\alpha_{02} \cdot q}{p} + \alpha_{02}} = \frac{1}{\alpha_{02} \cdot \left(\frac{q}{p} + 1\right)}$$

Solving for $alpha_{02}$:

$$\alpha_{02} = \frac{\log(2)}{M \cdot \left(\frac{q}{p} + 1\right)}$$

Once $alpha_{02}$ is calculated, $alpha_{01}$ can be derived using:

$$\alpha_{01} = \frac{\alpha_{02} \cdot q}{p}$$

Overall Survival (OS) accounts for both pathways:

1. Directly Initial to Death without Progression:
   • Occurs with probability $p$.
   • Survival time is $OS = X_2$ (time to death without progression).
2. Initial to Progression to Death:
   • Occurs with probability $q$.
   • Survival time is $OS = X_1 + X_3$, where:
     • $X_1 \sim \text{Exp}(\alpha_{01})$ (time to progression),
     • $X_3 \sim \text{Exp}(\alpha_{12})$ (time from progression to death).

The median OS ($M_{OS}$) is approximated as a weighted combination of these two pathways:

$$M_{OS} = \text{Median}(X_2 \cdot p + (X_1 + X_3) \cdot q) = p \cdot \frac{\log(2)}{\alpha_{02}} + q \cdot (\frac{\log(2)}{\alpha_{01}} + \frac{\log(2)}{\alpha_{12}})$$

To compute $alpha_{12}$, the model numerically solves for the rate using uniroot, ensuring consistency with the user-provided median OS.

Sources

• [1] Joint Modeling of progression-free and overall survival and computation of correlation measures, Meller M, Beyersmann J, Rufibach K, 2018
• [2] Purdue University, minimum_two_exponentials.pdf

Example Using Median Times

We begin by directly inputting the median times and probabilities of death before progression into the script as user parameters. Refer to the table below for the definitions of these user-defined parameters.

User parameter	Definition	Value
dMedianPFS0	Median time to PFS event for control group	12
dMedianPFS1	Median time to PFS event for treatment group	18
dMedianOS0	Median time to OS event for control group	18
dMedianOS1	Median time to OS event for treatment group	27
dProbOfDeathBeforeProgression0	Probability of death before PFS for control group	0.2
dProbOfDeathBeforeProgression1	Probability of death before PFS for treatment group	0.2

Here, the median times are in months and the hazard ratio for both endpoints (PFS & OS) is equal to $\frac{12}{18} = \frac{18}{27} = 0.6667$. The probability of death before progression is 20% for both control and treatment arms.

Example Using Prior Distributions

Now we customize how patient data is simulated by building a more realistic model for both PFS and OS outcomes using prior distributions instead of directly using median times and probabilities. Using prior distributions allows us to account for uncertainty around the true treatment effect, which enables users to identify the Probability of Success of a trial rather than statistical Power. Here’s how the event data is generated:

• Time to PFS event is sampled from a Gamma distribution.
• Time to OS event is also sampled from a Gamma distribution.
• The probability that the OS event happens before the PFS event is sampled from a Beta distribution.

All three parameters are treated as random variables, sampled from prior distributions, allowing each simulated trial to reflect a range of possible real-world outcomes.

Refer to the table below for the definitions of the user-defined parameters used in this example.

User Parameter	Definition
dMedianPFS0PriorShape	Shape parameter for the median time to PFS event for control group
dMedianPFS0PriorRate	Rate parameter for the median time to PFS event for control group
dMedianPFS1PriorShape	Shape parameter for the median time to PFS event for treatment group
dMedianPFS1PriorRate	Rate parameter for the median time to PFS event for treatment group
dMedianOS0PriorShape	Shape parameter for the median time to OS event for control group
dMedianOS0PriorRate	Rate parameter for the median time to OS event for control group
dMedianOS1PriorShape	Shape parameter for the median time to OS event for treatment group
dMedianOS1PriorRate	Rate parameter for the median time to OS event for treatment group
dProbOfDeathBeforeProgression0Param1	Alpha parameter for probability of death before progression for control group
dProbOfDeathBeforeProgression0Param2	Beta parameter for probability of death before progression for control group
dProbOfDeathBeforeProgression1Param1	Alpha parameter for probability of death before progression for treatment group
dProbOfDeathBeforeProgression1Param2	Beta parameter for probability of death before progression for treatment group

The shape and rate parameters can be calculated from the assumed mean (i.e. median survival time) and its variance. You can use the following tool to compute these required parameters.

Enter your desired mean and variance to calculate the shape and rate parameters for the Gamma distribution:

Mean:
Variance:

Calculate

Shape: -

Rate: -

If you have the scale parameter for your assumption instead of the rate parameter, you can convert from one to the other using the formula $Rate = \frac{1}{Scale}$.

Scenario 1: Alternative Hypothesis & Equal Variance

In this first scenario, we want:

• A variance of 10 for both endpoints (PFS and OS) and both arms (control and treatment).
• For the control arm:
  • A mean time to PFS event of 12 months.
  • A mean time to OS event of 18 months.
• For the treatment arm:
  • A mean time to PFS event of 18 months.
  • A mean time to OS event of 27 months.
• A probability of death before progression of 20% for both arms.

Using the tool above, we get:

• For $mean = 12$, $variance = 10$$\rightarrow$$shape = 14.4$, $rate = 1.2$
• For $mean = 18$, $variance = 10$$\rightarrow$$shape = 32.4$, $rate = 1.8$
• For $mean = 27$, $variance = 10$$\rightarrow$$shape = 72.9$, $rate = 2.7$

Refer to the table below for the values of all user-defined parameters used in this example.

User parameter	Value
dMedianPFS0PriorShape	14.4
dMedianPFS0PriorRate	1.2
dMedianPFS1PriorShape	32.4
dMedianPFS1PriorRate	1.8
dMedianOS0PriorShape	32.4
dMedianOS0PriorRate	1.8
dMedianOS1PriorShape	72.9
dMedianOS1PriorRate	2.7
dProbOfDeathBeforeProgression0Param1	20
dProbOfDeathBeforeProgression0Param2	80
dProbOfDeathBeforeProgression1Param1	20
dProbOfDeathBeforeProgression1Param2	80

Scenario 2: Alternative Hypothesis & Higher Variance

In this second scenario, we want:

• For the control arm:
  • A variance of 10 for both endpoints (PFS and OS).
  • A mean time to PFS event of 12 months.
  • A mean time to OS event of 18 months.
• For the treatment arm:
  • A variance of 20 for both endpoints (PFS and OS).
  • A mean time to PFS event of 18 months.
  • A mean time to OS event of 27 months.
• A probability of death before progression of 20% for both arms.

Using the tool above, we get:

• For $mean = 12$, $variance = 10$$\rightarrow$$shape = 14.4$, $rate = 1.2$
• For $mean = 18$, $variance = 10$$\rightarrow$$shape = 32.4$, $rate = 1.8$
• For $mean = 18$, $variance = 20$$\rightarrow$$shape = 16.2$, $rate = 0.9$
• For $mean = 27$, $variance = 20$$\rightarrow$$shape = 36.45$, $rate = 1.35$

Refer to the table below for the values of all user-defined parameters used in this example.

User parameter	Value
dMedianPFS0PriorShape	14.4
dMedianPFS0PriorRate	1.2
dMedianPFS1PriorShape	16.2
dMedianPFS1PriorRate	0.9
dMedianOS0PriorShape	32.4
dMedianOS0PriorRate	1.8
dMedianOS1PriorShape	36.45
dMedianOS1PriorRate	1.35
dProbOfDeathBeforeProgression0Param1	20
dProbOfDeathBeforeProgression0Param2	80
dProbOfDeathBeforeProgression1Param1	20
dProbOfDeathBeforeProgression1Param2	80

Scenario 3: Null Hypothesis & Equal Variance

In this third scenario, we want:

• A variance of 10 for both endpoints (PFS and OS) and both arms (control and treatment).
• For the control arm:
  • A mean time to PFS event of 12 months.
  • A mean time to OS event of 18 months.
• For the treatment arm:
  • A mean time to PFS event of 12 months.
  • A mean time to OS event of 18 months.
• A probability of death before progression of 20% for both arms.

Using the tool above, we get:

• For $mean = 12$, $variance = 10$$\rightarrow$$shape = 14.4$, $rate = 1.2$
• For $mean = 18$, $variance = 10$$\rightarrow$$shape = 32.4$, $rate = 1.8$

Refer to the table below for the values of all user-defined parameters used in this example.

User parameter	Value
dMedianPFS0PriorShape	14.4
dMedianPFS0PriorRate	1.2
dMedianPFS1PriorShape	14.4
dMedianPFS1PriorRate	1.2
dMedianOS0PriorShape	32.4
dMedianOS0PriorRate	1.8
dMedianOS1PriorShape	32.4
dMedianOS1PriorRate	1.8
dProbOfDeathBeforeProgression0Param1	20
dProbOfDeathBeforeProgression0Param2	80
dProbOfDeathBeforeProgression1Param1	20
dProbOfDeathBeforeProgression1Param2	80

Scenario 4: Null Hypothesis & Higher Variance

In this final scenario, we want:

• For the control arm:
  • A variance of 10 for both endpoints (PFS and OS)
  • A mean time to PFS event of 12 months.
  • A mean time to OS event of 18 months.
• For the treatment arm:
  • A variance of 20 for both endpoints (PFS and OS)
  • A mean time to PFS event of 12 months.
  • A mean time to OS event of 18 months.
• A probability of death before progression of 20% for both arms.

Using the tool above, we get:

• For $mean = 12$, $variance = 10$$\rightarrow$$shape = 14.4$, $rate = 1.2$
• For $mean = 18$, $variance = 10$$\rightarrow$$shape = 32.4$, $rate = 1.8$
• For $mean = 12$, $variance = 20$$\rightarrow$$shape = 7.2$, $rate = 0.6$
• For $mean = 18$, $variance = 20$$\rightarrow$$shape = 16.2$, $rate = 0.9$

Refer to the table below for the values of all user-defined parameters used in this example.

User parameter	Value
dMedianPFS0PriorShape	14.4
dMedianPFS0PriorRate	1.2
dMedianPFS1PriorShape	7.2
dMedianPFS1PriorRate	0.6
dMedianOS0PriorShape	32.4
dMedianOS0PriorRate	1.8
dMedianOS1PriorShape	16.2
dMedianOS1PriorRate	0.9
dProbOfDeathBeforeProgression0Param1	20
dProbOfDeathBeforeProgression0Param2	80
dProbOfDeathBeforeProgression1Param1	20
dProbOfDeathBeforeProgression1Param2	80

Analysis Integration Point

This endpoint is related to this R file: AnalyzePFSAndOS.R

Using the file above, the Analysis element of East Horizon’s simulation is customized to compute the probability of success (PoS) for the trial, based on dual endpoints: Progression-Free Survival (PFS) and Overall Survival (OS). The file uses information from the simulation (SimData variable) that is generated by the Response element of East Horizon’s simulation. See the Response section above for more information about the PFS and OS endpoints generation.

The criteria for declaring trial success are as follows. Both criteria must be met to declare success:

• PFS Endpoint: Statistical significance must be achieved by crossing the predefined efficacy boundary (defined by East Horizon, using a frequentist analysis).
• OS Endpoint: A positive trend must be observed, defined as the OS hazard ratio being below a pre-specified threshold (defined with user parameters).

Note: At an interim analysis, we usually set stricter criteria for a positive trend on OS compared to the final analysis. This is because we have less data early on, so we want to be more confident in any decision we make at that stage. Therefore, the threshold defining a positive trend at the interim is typically lower than the threshold used at the final analysis.

Refer to the table below for the definitions of the user-defined parameters used in this example.

User parameter	Definition
HazardRatioCutoffIA	OS hazard ratio threshold used for the interim analysis
HazardRatioCutoffFA	OS hazard ratio threshold used for the final analysis

Option 1: Fixed Sample Design

The first option is a fixed sample design with a hazard ratio threshold of 0.9. This option shows that we can still use this script without interim analyses. Refer to the table below for the values of the user-defined parameters used in this option.

User parameter	Value
HazardRatioCutoffFA	0.9

Option 2: Group Sequential Design

The second option is a group sequential design with hazard ratio thresholds of 0.8 (for interim analysis) and 0.9 (for final analysis). Refer to the table below for the values of the user-defined parameters used in this option.

User parameter	Value
HazardRatioCutoffIA	0.8
HazardRatioCutoffFA	0.9

Option 3: Low PoS Group Sequential Design

The third option is a group sequential design with a lower hazard ratio threshold of 0.5 for both interim and final analyses. Refer to the table below for the values of the user-defined parameters used in this option.

User parameter	Value
HazardRatioCutoffIA	0.5
HazardRatioCutoffIA	0.5

Option 4: High PoS Group Sequential Design

The fourth option is a group sequential design with a lower hazard ratio threshold of 0.5 (for interim analysis) and a higher hazard ratio threshold of 1.2 (for final analysis). Refer to the table below for the values of the user-defined parameters used in this option.

User parameter	Value
HazardRatioCutoffIA	0.5
HazardRatioCutoffIA	1.2

Conclusion

In the Results section, Power now refers to the probability of success, where success is defined as a statistically significant difference in time to progression-free survival (PFS) between the control and treatment arm as well as a positive trend in overall survival (OS), i.e. the time to overall survival (OS) is longer in magnitude for the patients in the treatment arm compared to those in the control arm. Below is an example of the heatmap that could be generated by East Horizon following the simulation. Each square represents the simulated probability of success for a trial, based on a specific combination of Response scenario (columns) and Analysis option (rows).