Biomedical

Assessing the properties of patient-specific treatment effect estimates from causal forest algorithms under essential heterogeneity






  Peer Reviewed

Abstract

Key Questions

1. What are Causal Forest Algorithms (CFAs)?

CFAs are machine learning methods used to estimate patient-specific treatment effects from observational data, accounting for heterogeneity in treatment responses.

2. What is essential heterogeneity, and how does it affect CFAs?

Essential heterogeneity occurs when unmeasured patient factors influencing treatment effects also affect treatment choice. Under this condition, CFAs may produce biased estimates, particularly for untreated patients.

3. How well did CFA-GRF perform in the study?

CFA-GRF performed well when treatment effect heterogeneity did not influence treatment choice but overestimated effects for untreated patients under essential heterogeneity.

4. What recommendations were made for researchers using CFAs?

Researchers should develop conceptual frameworks of treatment choice before estimation to guide the interpretation of CFA results, especially in real-world settings.

Background

Background Treatment variation from observational data has been used to estimate patient-specific treatment effects. Causal Forest Algorithms (CFAs) developed for this task have unknown properties when treatment effect heterogeneity from unmeasured patient factors influences treatment choice – essential heterogeneity.

Methods

Methods We simulated eleven populations with identical treatment effect distributions based on patient factors. The populations varied in the extent that treatment effect heterogeneity influenced treatment choice. We used the generalized random forest application (CFA-GRF) to estimate patient-specific treatment effects for each population. Average differences between true and estimated effects for patient subsets were evaluated.

Results

Results CFA-GRF performed well across the population when treatment effect heterogeneity did not influence treatment choice. Under essential heterogeneity, however, CFA-GRF yielded treatment effect estimates that reflected true treatment effects only for treated patients and were on average greater than true treatment effects for untreated patients.

Conclusions

Conclusions Patient-specific estimates produced by CFAs are sensitive to why patients in real-world practice make different treatment choices. Researchers using CFAs should develop conceptual frameworks of treatment choice prior to estimation to guide estimate interpretation ex post.