An AI System to Help Predict ED Patient Deterioration
Our PhD students Yiqiu Shen, Nan Wu, and Aakash Kaku alongside other CDS and NYU colleagues and peers, have recently published a research paper on the development of an AI system that could potentially help to predict the deterioration of emergency room COVID-19 patients.
The paper’s abstract is as follows:
During the COVID-19 pandemic, rapid and accurate triage of patients at the emergency department is critical to inform decision-making. We propose a data-driven approach for automatic prediction of deterioration risk using a deep neural network that learns from chest X-ray images, and a gradient boosting model that learns from routine clinical variables. Our AI prognosis system, trained using data from 3,661 patients, achieves an AUC of 0.786 (95% CI: 0.742–0.827) when predicting deterioration within 96 hours. The deep neural network extracts informative areas of chest X-ray images to assist clinicians in interpreting the predictions, and performs comparably to two radiologists in a reader study. In order to verify performance in a real clinical setting, we silently deployed a preliminary version of the deep neural network at NYU Langone Health during the first wave of the pandemic, which produced accurate predictions in real-time. In summary, our findings demonstrate the potential of the proposed system for assisting front-line physicians in the triage of COVID-19 patients
It was around April of this year CDS affiliated professor Krzysztof J. Geras called for volunteers to work on COVID-19 data as NYU Langone had collected a chest X-ray dataset from COVID patients. At the time, the ER department was quite overwhelmed by the number of patients affected by COVID-19. After some discussion with CDS professor Carlos Fernandez-Granda, NYU Abu Dhabi professor Farah Shamout and NYU School of Medicine professor Yindalon Aphinyanaphongs, they realized they could potentially use this dataset to develop a model that could assist in determining which patients were more vulnerable to clinical deterioration. (The team defines deterioration — the target to be predicted by their models — as the occurrence of one of three adverse events: admission to the intensive care unit (ICU), intubation, and in-hospital mortality. If multiple events occurred, they only considered the time of the initial event.)
The dataset was collected from patients seen between March 3, 2020 and June 28, 2020. It consisted of chest X-ray images obtained from patients who tested positive for COVID-19 using the polymerase chain reaction (PCR) test — “a real-time reverse transcription polymerase chain reaction (rRT-PCR) test for the qualitative detection of nucleic acid from SARS-CoV-2 in upper and lower respiratory specimens” (1) — in addition to “the clinical variables recorded closest to the time of image acquisition (e.g. vital signs, laboratory test results, and patient characteristics.”) (2)
The training set consists of 5,617 chest X-ray images used for model development and hyperparameter tuning while the test set images used to report the final results consists of 832 results. Both sets were disjointed with no patient overlap. Additionally, both sets are similar in terms of age, BMI, and proportion of females. They’ve noted that there is a higher proportion of chest X-ray images in the test set that are associated with deterioration across all time windows. Since the test set only includes chest X-ray images collected from emergency department patients, this suggests that there is a higher incidence of adverse events amongst emergency department patients than inpatients.
Ultimately, the project’s immediate goal is to provide support for critical clinical decision-making involving patients arriving at the emergency department that are in immediate need of care. It’s a system designed to fulfill a clinical need of frontline physicians by predicting the risk of deterioration when the patient’s vital signs are measured. In the long-term, their goal is to deploy this system in existing clinical workflows to assist clinicians.
About the Project’s CDS PhD Student Participants:
Yiqiu Shen is a CDS PhD student. His studies focus on deep learning models for medical image analysis, under the supervision of CDS professors Krzysztof J. Geras and Kyunghyun Cho.
Nan Wu is a CDS Ph.D. student. She works on deep learning research for medical imaging with professors Krzysztof J. Geras and Kyunghyun Cho.
Aakash Kaku is a CDS PhD student. His works on solving problems in the healthcare domain using machine learning under the supervision of professor Carlos Fernandez-Granda…
The team also includes CDS alum Jungkyu Park, now a PhD student at NYU School of Medicine, Taro Makino, an incoming CDS PhD student, Narges Razavian, a CDS affiliated professor, and Stanisław Jastrzębski, a CDS affiliated postdoc.
To view the paper in its entirety, please visit the paper’s arXiv page.
References
By Ashley C. McDonald
