Reflection on the PENTACON Trial: Lessons learned from an unpublished study
- 1University Medical Center, Utrecht, the Netherlands
Abstract
As a clinician-scientist, the journey through the PENTACON trial, a multicenter randomized clinical study comparing two surgical techniques for corneal transplantation in keratoconus patients, was both enlightening and challenging. The study, which served as the centerpiece of my PhD research in 2015, ticked all the regulatory boxes and was ethically approved but ultimately fell short due to being gravely underpowered. This reflection aims to dissect the experiences, challenges, and lessons learned from this endeavor. Most importantly, while courses on clinical trial design such as Good Clinical Practice (GCP) certifications can teach important principles, the core value of any study remains somewhat elusive. How can one find the thin golden line between scientific value, innovation, patient recruitment, and overall trial feasibility? The outcomes of a study can never be completely sure; if we knew them ahead of time, it wouldn't be called research. However, in this particular example, there were tell-tale signs of failure early on. Therefore, I believe there are lessons for fellow researchers to be found.
Keywords: trial and error, clinical trial design, scientific equipoise, adversity, underpowered study
Personal and professional growth
The failure to publish the outcomes of the biggest endeavor of my PhD was a significant setback, but it also catalyzed my growth in unexpected ways. We conceptualized this clinical trial in 2010, based on the innovative and experimental work of Professor Busin in Forlí, Italia. His team developed many novel surgical approaches in corneal surgery, of which one innovation was particularly impactful: the microkeratome assisted anterior lamellar keratoplasty (Busin et al., 2012). Prof. Busin was awarded the "best of show" video session covering this technique by the American Academy of Ophthalmology annual conference in 2005, suggesting widespread appreciation of his approach in the field. Even today, he achieves scientific acclaim for his innovations in corneal surgery, as evidenced by top-tier publications considering the same type of procedures in the same population (see Bovone et al., 2024).
Our team in Utrecht was nimble, and this clinical trial was my first prospective project. My supervisor was a corneal surgeon and immunologist, with an established track record in immunological research, uveitis, and onchocerciasis. She performed these surgeries and collaborated with the Busin group. With great enthusiasm we embarked on this project, secured funding, and designed the clinical trial, dubbed the PENTACON. The public video library aiding surgeons in how to perform the surgery is still online (Utrecht Trial Videos, 2012).
The PENTACON report is clear; the trial never reached the needed participants to produce reliable scientific conclusions. After much postponement, we decided to cancel the trial due to insufficient recruitment. The adage "what doesn’t kill you makes you stronger" resonated deeply with me during this period. The disappointment spurred me to explore other research avenues to meet the PhD requirements, leading to novel collaborations in corneal immunology and epidemiology. These new directions not only broadened my expertise but also reinforced my resilience and adaptability as a researcher. Yet sometimes I cannot resist the what-if question: What if we had been successful? Would my career be substantially different? Now, 10 years later, I’m not sure, but I think my other work compensated at least.
The experience of navigating the aftermath of the PENTACON trial taught me to value resilience and adaptability. This pivot was not just about salvaging a career path but about embracing the broader scope of scientific inquiry. I ventured into corneal immunology and epidemiology, fields that were not initially on my radar but have been an integral to my professional identity. This adaptability is a testament to the dynamic nature of scientific research, where setbacks can often lead to unanticipated breakthroughs and growth.
The Intricacies of study design
One of the crucial lessons learned from the PENTACON trial was the essential nature of study design in clinical research. The trial’s underpowered status highlighted the importance of seasoned advice from experienced mentors with a proven track record of successfully completing clinical trials. My supervisor at the time lacked this specific expertise, and as a junior researcher, I was not fully equipped to recognize this gap. When I got sound advice from a respected senior researcher, I didn’t listen. He politely refused to let his center participate in this trial for the exact reason the study eventually failed: a too complex procedure in a niche population. Keratoconus patients are not particularly rare, as we’ve identified ourselves in an epidemiological study (Godefrooij et al., 2017). Suitable candidates needed to be advanced cases, otherwise a corneal transplant is too costly and high-risk, yet end-stage disease with scars and post-corneal hydrops was an exclusion criterion. The trouble of patient recruitment was apparent at the start, for those doctors willing to admit and see patients. Rather than consider their concerns, I listened to our external, seasoned surgeon based in Italy whose technique we planned to test. He was not involved enough in the study to provide the detailed feedback we needed. This experience underscored the necessity of having a robust support system and guidance from seasoned researchers who can anticipate and navigate the potential pitfalls of clinical trials.
This experience taught me that effective study design is not just about theoretical knowledge but also about practical wisdom. The guidance from mentors is invaluable. Their insights can often spell the difference between a study’s success and failure. This trial was a stark reminder of the importance of listening to experienced voices and valuing their input, even when it challenges our preconceptions or plans.
Challenges in multicenter studies
Based on the relative rarity of suitable candidates for trial enrollment, a multicenter study was inevitable. From a scientific perspective, this was also the preferred route, since multicenter study outcomes are often more generalizable and less prone to reflect local circumstances not controlled for during the study.
The multicenter nature of the PENTACON trial presented unique challenges, particularly with patient recruitment and center participation. Our largest anticipated center in Rotterdam failed to include a single patient despite extensive efforts to clear all ethical checks and barriers and several site visits. This experience taught me that not all centers will deliver as expected in a multicenter study. My advice: hedge for that event. Recruit more centers than are needed, since trial inclusion can run slow for a myriad of reasons. Effective communication, commitment, and follow-through from all participating centers are paramount for the success of multicenter trials. We painfully learned that (future) multicenter studies should carefully evaluate and select participating centers based on their demonstrated capability and commitment to deliver. In retrospect, the multicenter approach, while intended to enhance the trial’s robustness, became one of its downfalls. The variability in center participation highlighted the critical need for stringent pre-trial assessments of each site’s readiness and commitment. Several sites just included one or two patients, which further reduced the scientific value of the study. In summary, true multicenter studies involve more than ethical clearances and polite site visits; they require a deep dive into each center’s patient population, logistical capabilities, and the willingness of local investigators to engage fully with the trial’s demands. Investigators’ personal networks are a huge asset in this aspect.
Importance of feasibility
The ability to quickly include patients is a critical factor in the success of clinical trials. The PENTACON trial suffered from a slow inclusion rate, partly due to the rarity of eligible patients and the complexity of the surgical technique. This highlights the necessity of ensuring a steady and sufficient patient pool within the clinic or network before embarking on a trial. If a disease is rare, be aware. For early-stage career researchers, leveraging their network to gain more volume in inclusions sounds sensible, but is your network ready to deliver? Unless you have clear recruitment pathways, it may be more sensible to investigate an interesting disease using a different approach, capturing more data and delivering better conclusions.
The slow patient inclusion rate was a significant hurdle. In planning future trials, it is imperative to conduct thorough feasibility studies that accurately estimate the patient pool and the rate of inclusion. This involves not just statistical projections but also practical considerations about patient availability and the operational capacity of each site. A clear understanding of these factors can prevent the kind of slow inclusion that plagued the PENTACON trial and ultimately contributed to its downfall.
Rigidity in protocols and its impact
Another significant lesson from the PENTACON trial was the impact of protocol rigidity over extended periods. The trial spanned several years, during which we wanted to update the surgical technique. Surgeons in this field are creative and always tinker; my supervisor was no exception. However, altering the protocol would have jeopardized the internal consistency of the study. This rigidity prevented iterative improvements and adaptations to the surgical procedure, ultimately contributing to the decision to terminate the trial. The relatively high rate of complications in both treatment arms in the PENTACON trial showed that the premise of a safe alternative was not met. As clinicians, we started to reject study participation for our patients, believing the technique under investigation to be inferior. Ergo, we lost our scientific equipoise: the genuine belief that both treatment options have equal value. This emphasized the need for a limited window of inclusions and adaptive trial designs that can accommodate advancements and refinements without compromising the study’s integrity. Pallmann and colleagues (2018) advocate for adaptive trial methodologies that allow real-time adjustments, such as modifying sample sizes, adjusting randomization ratios, or incorporating interim analyses, thus enhancing the study’s responsiveness without compromising validity. These factors can be anticipated in discussions with ethical boards prior and warrant input from experienced methodologists.
The tension between maintaining protocol fidelity and allowing for methodological advancements is a common challenge in clinical trials (Lawton et al., 2011). The PENTACON trial’s rigidity prevented us from incorporating improvements that could have potentially salvaged the study. Future trials should consider more flexible designs that allow for adaptations in response to new developments, provided these changes are methodologically sound and ethically justified.
Ethical obligation and reporting negative results
Despite the trial’s termination and underpowered status, it was our ethical obligation to report the results. The manuscript became a chapter in my PhD (Wisse, 2015), and while it is not peer-reviewed, we can consider it publicly available (grey literature). Naturally, PhD dissertations are not indexed by search engines and scientific catalogs, and their outcomes are difficult to find. The PENTACON trial demonstrated that the PET technique was not as safe as anticipated, and both techniques showed a high rate of intra-operative complications. Although the article lacks detailed examples, our findings did highlight several conditions, such as the relatively high complication rates in both treatment arms, and the steep learning curve associated with the PET technique. The latter was not reported before in literature. Reporting these findings was crucial to inform the medical community and contribute to collective knowledge, preventing future researchers from encountering similar pitfalls. Especially at that time, the caveats and complications of these surgical techniques were gravely underreported in the scientific literature. No room for failure, apparently. This bias against failure has luckily changed, with rigorous and large studies being published that include the complications of complex corneal surgery (e.g., Feizi et al., 2023).
The ethical imperative to report all findings, regardless of their nature, cannot be overstated. Negative or inconclusive results are as crucial as positive ones in building a comprehensive understanding of medical interventions. Reluctance to publish negative findings perpetuates a skewed view of clinical efficacy and safety, leading to potential repetition of avoidable mistakes. Reporting bias is a known problem, jeopardizing the quality of our overall scientific endeavors and the value of funding research (Gill, 2012). By attempting to publish the results of the PENTACON trial, we aimed to fill a critical gap in the literature and foster a more transparent and informative scientific discourse. This was unfortunately not recognized by the peer-review process.
Reflecting on broader impacts and future directions
The PENTACON trial, though unsuccessful in achieving its primary objectives, was a profound learning experience that shaped my career and research philosophy. It underscored the importance of robust study designs, realistic feasibility assessments, effective multicenter collaboration, protocol flexibility, and ethical reporting. These lessons have been invaluable in guiding my subsequent research endeavors and have contributed to my growth as a resilient and adaptive clinician-scientist.
Secondly, trial designs must incorporate flexibility to adapt to new developments and findings. Adaptive trial designs, which allow for modifications based on interim results, can enhance the relevance and applicability of clinical studies without compromising their integrity. This approach requires careful planning and ethical considerations but can significantly improve the efficiency and impact of clinical research.
Lastly, the scientific community must continue to emphasize the importance of publishing negative results. Journals and funding bodies should encourage the dissemination of all trial outcomes, fostering a more balanced and comprehensive understanding of medical interventions. This shift can help prevent the recurrence of past mistakes and guide future research more effectively.
Conclusion
Through this reflection, I hope to share these insights with the broader medical community, fostering a culture of continuous learning and improvement in clinical research. The PENTACON trial, despite its shortcomings, has been a cornerstone in my development as a researcher. One learns the most from one’s mistakes. The failed trial highlighted the intricacies of clinical trial design, the importance of experienced mentorship, the challenges of multicenter studies, and the ethical duty to report all findings. The lessons of this study have shaped my approach to research and will undoubtedly influence my future endeavors. Moreover, they serve as a reminder of the dynamic and often unpredictable nature of scientific inquiry, where every setback is an opportunity for growth and learning. I hope this reflection can prevent fellow researchers from making these same mistakes.
References
Bovone, C., De Rosa, L., Pellegrini, M., Ruzza, A., Ferrari, S., Camposampiero, D., Ponzin, D., Zauli, G., Yu, A. C., & Busin, M. (2024). Deep anterior lamellar keratoplasty using dehydrated versus standard organ culture-stored donor corneas: Prospective randomized trial. Ophthalmology, 131(6), 674-681. https://doi.org/10.1016/j.ophtha.2023.12.027
Busin, M., Scorcia, V., Zambianchi, L., & Ponzin, D. (2012). Outcomes from a modified microkeratome-assisted lamellar keratoplasty for keratoconus. Archives of Ophthalmology, 130(6), 776-782. https://doi.org/10.1001/archophthalmol.2011.1546
Feizi, S., Javadi, M. A., Karimian, F., Bayat, K., Bineshfar, N., & Esfandiari, H. (2023). Penetrating keratoplasty versus deep anterior lamellar keratoplasty for advanced stage of keratoconus. American Journal of Ophthalmology, 248, 107-115. https://doi.org/10.1016/j.ajo.2022.11.019
Gill, C. J. (2012). How often do US-based human subjects research studies register on time, and how often do they post their results? A statistical analysis of the Clinicaltrials.gov database. BMJ Open, 2(4), Article e001186. https://doi.org/10.1136/bmjopen-2012-001186
Godefrooij, D. A., de Wit, G. A., Uiterwaal, C. S., Imhof, S. M., & Wisse, R. P. (2017). Age-specific incidence and prevalence of keratoconus: A nationwide registration study. American Journal of Ophthalmology, 175, 169-172. https://doi.org/10.1016/j.ajo.2016.12.015
Lawton, J., Jenkins, N., Darbyshire, J. L., Holman, R. R., Farmer, A. J., & Hallowell, N. (2011). Challenges of maintaining research protocol fidelity in a clinical care setting: A qualitative study of the experiences and views of patients and staff participating in a randomized controlled trial. Trials, 12, Article 108. https://doi.org/10.1186/1745-6215-12-108
Pallmann, P., Bedding, A. W., Choodari-Oskooei, B., Dimairo, M., Flight, L., Hampson, L. V., Holmes, J., Odoni, L., Sydes, M. R., Villar, S. S., Wason, J. M. S., Weir, C. J., Wheeler, G. M., Yap, C., & Jaki, T. (2018). Adaptive designs in clinical trials: Why use them, and how to run and report them. BMC Medicine, 16, Article 16. https://doi.org/10.1186/s12916-018-1017-7
Utrecht Trial Videos. (2012, May 3). MALK for Keratoconus - The partial endothelial trepanation. Vimeo. https://vimeo.com/37978511
Wisse, R. P. L. (2015). Keratoconus : Inflammatory associations and treatment characteristics. [Doctoral Dissertation, Utrecht University]. Utrecht University Repository. https://dspace.library.uu.nl/handle/1874/325112
