Development of machine learning models for predicting outcome in patients with distal medium vessel occlusions: a retrospective study

Introduction

Acute ischemic stroke (AIS) is caused by distal medium vessel occlusions (DMVOs) in 24–40% of cases and can result in significant morbidity. DMVOs are recently defined as occlusions in the M2–M4 middle cerebral artery, posterior inferior cerebellar artery, posterior cerebral artery, anterior inferior cerebellar artery, anterior cerebral artery and superior cerebellar artery (1). DMVOs can arise as a primary thromboemboli or an unintended endovascular thrombectomy result (2). They often result in severe deficits and disabilities, contrary to the conventional belief that they cause only minor symptoms (3). However, even though they are a significant cause of stroke, DMVOs are difficult to diagnose using standard-of-care methods. Furthermore, there are no clearly defined treatment recommendations and prognostic factors are not well known (4,5).

Accurate short-term and long-term outcome prediction in DMVO-AIS patients can facilitate evidence-based clinical decision-making, inform the selection of appropriate treatment strategies, and enable more tailored prognostic assessments for individual patients, ultimately improving patient outcomes and addressing existing treatment gaps. Numerous prognostic factors have been reported for large vessel occlusion (LVO)-AIS patients, but the same cannot be said for DMVOs (6). Since efficacy and safety of thrombectomy for this group remain unknown, the research is focused and limited to prognostic factors related to mechanical thrombectomy outcomes in DMVOs (2,7-9). In addition to managing prognostic expectations, accurate outcome prediction is required to improve personalization in managing DMVO-AIS patients prior to treatment, regardless of treatment choice.

The current gold standard for diagnosing DMVOs, which can be challenging, is computed tomography (CT) angiography (CTA) plus nonenhanced CT (4). In addition, cerebral perfusion imaging is now routinely used in assessing AIS (10). It has been shown that perfusion imaging provides prognostic information about functional outcome in AIS patients (11). Moreover, Amukotuwa et al. demonstrated that even novice readers could accurately and quickly identify DMVOs on time-to-maximum of the tissue residue function (T_max) maps, a quantitative parameter acquired by postprocessing CT perfusion (CTP) data (4). Furthermore, Muehlen et al. showed that the hypoperfusion intensity ratio (HIR), a parameter derived from CTP-based automated perfusion software platforms, can provide helpful information on functional outcome in patients with LVO stroke who have poor revascularization after mechanical thrombectomy (12). Supporting these findings, Wan et al. showed that HIR was associated with the functional outcome of LVO-AIS patients (13). Furthermore, Guenego et al. showed that a HIR <0.3 is linked to a reduction in infarct growth following successful recanalization of DMVO (14).

Machine learning (ML) is a collection of computational methods for discovering complex patterns and relationships in data (15). ML algorithms have the ability to utilize high-dimensional clinical data effectively, enabling the creation of precise patient risk assessment models. These models not only aid in the development of intelligent guidelines but also have the potential to impact healthcare decisions by customizing care. Advanced ML algorithms provide substantial benefits over traditional prognostic models, which typically utilize some form of logistic regression. Firstly, ML rarely requires prior knowledge of key predictors (16). Furthermore, advanced ML algorithms often allow for a greater number of predictors in a dataset compared to logistic regression, proving useful in large datasets with numerous predictors where relationships between predictors and outcomes may not be immediately evident. Finally, they can detect complex, nonlinear relationships within datasets that logistic regression cannot capture (17). As a result, advanced ML algorithms tend to be more dependable and precise than logistic regression techniques when analyzing the same dataset (18). In stroke, ML models have frequently been used and have successfully predicted prognosis (15,19-21).

Considering CTP’s expected impact on stroke diagnosis and management, we hypothesize that ML algorithms using CTP, clinical and laboratory variables as input parameters can successfully predict the outcome in DMVO patients. Thus, we used several ML algorithms to develop a model predicting the National Institutes of Health Stroke Scale (NIHSS) shift score of DMVO AIS patients based on demographic, clinical, and laboratory variables and CTP postprocessing quantitative parameters, regardless of the preferred treatment method. The NIHSS is a broadly investigated instrument useful for determining prognosis in the preliminary examination of AIS patients. We present this article in accordance with the TRIPOD reporting checklist (available at https://qims.amegroups.com/article/view/10.21037/qims-23-154/rc).

Methods

Patient selection

We conducted a retrospective cohort study at two comprehensive stroke centers, namely Johns Hopkins Hospital (JHH) and Johns Hopkins Bayview Medical Center (JHBMC). To identify eligible participants, we screened consecutive patients admitted between January 1, 2017, and September 1, 2022. An arterial occlusion involving the anterior cerebral artery, M2–M4 middle cerebral artery, posterior cerebral artery, posterior inferior cerebellar artery, anterior inferior cerebellar artery, and superior cerebellar artery was defined as a DMVO (1). The diagnosis of AIS was established based on clinical assessment and confirmed through the use of brain CT or magnetic resonance imaging (MRI). The study encompassed patients who satisfied the following inclusion criteria: (I) admitted within 24 hours of symptom onset; (II) ≥18 years of age; (III) initial non-contrast brain CT scan data to exclude the intracranial hemorrhage; (IV) primary DMVO diagnosis based on CTA and CTP; and (V) utilization of an automated perfusion software platform (RAPID 4.9, iSchemaView, Menlo Park, CA, USA) for hypoperfusion analysis based on CTP. Patients who had incomplete outcome data, individuals who experienced DMVO as a result of emboli caused by endovascular treatment for a different occlusion (referred to as secondary DMVO), and those who were discharged with a diagnosis of transient ischemic attack were not included in the study. The study was conducted in accordance with the Declaration of Helsinki (as revised in 2013). The study was approved by ethics board of Johns Hopkins Hospital and individual consent for this retrospective analysis was waived.

Data extraction

Retrospectively, demographic and clinical data were obtained from electronic medical records. The following variables were acquired: race, sex, age, comorbidities (hypertension, diabetes mellitus, coronary artery disease, atrial fibrillation), history of malignancy, admission vitals (heart rate, systolic blood pressure, diastolic blood pressure), admission and discharge NIHSS score, mechanical thrombectomy and intravenous (IV) tissue plasminogen activator (tPA) treatment. Based on the recent study from Meyer et al., each patient’s NIHSS shift score (admission NIHSS score − discharge NIHSS score) is computed and designated as the clinical outcome (22). The clinical outcome was separated into two groups based on the median NIHSS shift score. Patients with an NIHSS shift score higher than the median were allocated to the favorable outcome group, while those with a score lower than the median were designated to the unfavorable outcome group. Our models predicted these favorable and unfavorable groups.

In accordance with our local stroke care standard protocol, peripheral venous blood samples were collected from all patients upon arrival at the emergency department. The blood samples were uniformly collected, processed using the same method, and analyzed at the identical clinical laboratory. The following baseline parameters were extracted retrospectively: hemoglobin, hematocrit, mean corpuscular volume (MCV), mean platelet volume (MPV), platelet count (PC), neutrophil-to-platelet ratio (NPR), blood urea nitrogen/creatinine ratio (BUN/Cr), glucose, sodium, potassium, and calcium.

Imaging notes were used to collect radiologic variables. Additionally, a board-certified neuroradiologist with 6 years of experience (VSY) evaluated the CTAs of all patients in the study. This evaluation was performed in conjunction with the analysis of all available imaging and clinical data. The presence and exact location of any DMVO were documented, and the same neuroradiologist also gathered and confirmed information such as the Alberta Stroke Program Early CT Score (ASPECTS), the occluded vessel, the specific segment of occlusion, the level of occlusion, the laterality of the occlusion, and the occurrence of hemorrhagic transformation.

Thrombectomy procedures were conducted by either one of four skilled interventional neuroradiologists or endovascular neurosurgeons. The choice of the thrombectomy device used during the procedure was determined by the neurointerventionalist’s discretion and limited to those approved by the Food and Drug Administration. The collected data consisted of the reperfusion grade assessed by the treating interventionalist using the modified treatment in cerebral ischemia (mTICI) score, the number of passes made during the thrombectomy procedure, and the time taken from groin puncture to recanalization, measured in minutes.

Imaging protocols

Baseline comprehensive CT imaging was performed in JHH and JHBMC using helical scanners on the Siemens Flash and Drive (Siemens Healthineers, Erlangen, Germany). All imaging parameters used in the study were consistent with those reported in a previously published study (23).

The extracted data consisted of relative cerebral blood flow (rCBF) <20%, rCBF <30%, rCBF <34%, rCBF <38%, T_max >4 s, T_max >6 s, T_max >8 s, T_max >10 s, cerebral blood volume (CBV) <34%, CBV <38%, CBV <42%, mismatch volume, hypoperfusion index (HI), diffusion-weighted imaging (DWI) volume on follow up MRI within 7 days of symptom onset, infarct growth (assessed as the difference between the follow up DWI volume and the initial rCBF<30% volume on CTP), and follow up post treatment non-contrast CT infarct volume. The mismatch volume was calculated by subtracting the perfusion deficit volume from the ischemic core volume. HI is defined as the ratio obtained by dividing the volume of the lesion with T_max greater than 10 seconds by the volume of the lesion with T_max greater than 6 seconds.

Data preprocessing

Values for some of the variables that apply only to the patients who underwent mechanical thrombectomy, such as mTICI score or time from groin puncture to recanalization in minutes, were assigned as ‘not attempted’ for categorical variables and ‘0’ for continuous variables. Apart from that, we used imputation to prevent bias from being introduced by excluding patients with missing values. At least one value was missing from twelve continuous variables. After excluding variables with missing values for more than 25% of the patient population, a nearest neighbor (NN) imputation technique was employed to fill in the missing values for the remaining continuous variables. The only categorical variable contained missing values was the variable ‘occlusion location’ and its missing values were credited as ‘unknown’.

To scale continuous variables and take outliers into consideration, the robust scaler was used. Normalization is crucial in order to standardize and assign equal weight to all feature values, ensuring that they are on the same scale. Using a min-max normalization, each continuous variable [such as body mass index (BMI) or laboratory data] was placed in the [0, 1] range. Categorical non-binary variables, such as race and mTICI Score, were encoded using a one-hot encoding technique. On the other hand, variables with ordinal characteristics, such as mTICI score and number of passes, were encoded using an ordinal encoding method. Data were divided into training and test sets in a 70:30 ratio. For hyperparameter tuning, one-fourth of the training set was utilized as the validation set. Investigators conducting the ML analysis were blinded to the outcome categories.

Since admission and discharge NIHSS scores are directly related with our outcome, NIHSS shift, they were not included in the predictor variables. For selecting 10 features out of all other variables, we adopted the feature selection approach known as recursive feature elimination (RFE), which fits a model and eliminates the weakest feature (or features) until the required number of features is attained. Features are ordered according to importance, and RFE works to get rid of any dependencies and collinearity that could be present in the model by iteratively deleting a small number of features every loop.

ML analyses

XGBoost, LightGBM, CatBoost, multi-layer perceptron, random forest, and logistic regression were six supervised ML algorithms that we used to predict the outcome using the predictor variables (24-28). Supervised ML algorithms are methods where the model learns from data that is already labeled—that is, it already knows the outcome it should arrive at. The chosen algorithms each have unique strengths. XGBoost, LightGBM, and CatBoost, for instance, are powerful gradient boosting frameworks that build models in a stage-wise fashion, learning from the mistakes of previous stages, which makes them highly accurate. The multi-layer perceptron is a type of neural network that can model complex patterns, while random forest builds multiple decision trees to make robust predictions. Lastly, logistic regression, despite its simplicity, is highly effective when the relationships in the data are linear. We optimized these algorithms using the Optuna optimization package, aiming to maximize the area under the receiver operating characteristic (ROC) curve (AUROC), which is a commonly used metric to evaluate the performance of binary classification tasks (29). Optuna is a software framework that is used for hyperparameter tuning in ML. Hyperparameters are settings in the model that we can adjust to change its performance. Optuna makes the tuning process efficient by providing a variety of optimization approaches. The Bayesian optimization technique used in this process was the Tree-Structured Parzen Estimator Sampler (TPESampler). Bayesian optimization methods, such as the TPESampler, help in guiding the optimization process by building a probability model of the objective function and use it to select the most promising hyperparameters to evaluate in the real-world. After the hyperparameter tuning, we used the optimized settings along with the entire training dataset to construct the final models for predicting the outcome(s). The ML analyses were performed in Python version 3.7.15 (RRID:SCR_008394), a popular programming language in data science due to its simplicity and the availability of numerous scientific computation libraries. The model’s source code can be found in the project’s GitHub repository (https://github.com/mertkarabacak/DMVO).

Performance evaluation

The performance of the models was assessed through visual analysis using ROC and precision-recall curve (PRC). Additionally, numerical evaluation was conducted using metrics such as AUROC, area under the PRC curve (AUPRC), accuracy, precision, recall, and Matthews correlation coefficient (MCC). We used Shapley additive explanations (SHAP) in addition to performance plots and metrics to look at the relative weights of predictor variables. To understand how models make predictions, the visualization technique SHAP is often employed in ML.

Statistical analysis

To analyze and compare the low and high NIHSS shift groups, we utilized various statistical tests based on the nature of the variables. For normally distributed continuous variables with equal variances, an independent t-test was conducted. In cases where the variances were unequal, Welch’s t-test was employed. For non-normally distributed continuous variables, the Mann-Whitney U test was utilized. Categorical variables were evaluated using Pearson’s chi-squared test. Normality of variables was assessed using the Shapiro-Wilk test, while Levene’s test was used to examine the equality of variances. Statistical significance was considered at a threshold of P<0.05. All statistical analyses were performed using Python version 3.7.15 (RRID:SCR_008394).

Results

The study initially encompassed 148 DMVO-AIS patients admitted to JHH and JHBMC. However, we excluded 26 patients due to incomplete outcome data and another 53 due to the absence of quantitative CTP data. Thus, the final analysis included a cohort of 69 patients. The group with a favorable outcome consisted of 35 patients, while the group with an unfavorable outcome included 34 patients. The majority of both groups were female, with 60% in the favorable outcome group and 52.9% in the unfavorable outcome group. The average age was 67.51±13.54 years for the favorable outcome group and 62.5±12.75 years for the unfavorable outcome group. The admission NIHSS score had a median of 11.0 in the favorable outcome group and 4.5 in the unfavorable outcome group. Thrombectomy was performed in 62.9% of the favorable outcome group and only 11.8% of the unfavorable outcome group. IV-tPA treatment was administered to 45.7% of patients in the favorable outcome group and 29.4% in the unfavorable outcome group. Table 1 provides a comprehensive overview of the baseline clinical, laboratory, and radiologic characteristics of the patient population.

RFE yielded the following selected features: serum potassium, serum hemoglobin, MCV, T_max >6 s, T_max >8 s, T_max >10 s, mismatch volume, HI, DWI volume. Using those 10 selected features, XGBoost showed the best performance in predicting unfavorable outcome with an AUROC of 0.894 and an AUPRC of 0.756. The XGBoost algorithm attained a precision of 1.000, a recall of 0.571, an F1 score of 0.727, an accuracy of 0.714, and an MCC of 0.555. Table 2 displays the metrics that assess the performance of the algorithms. AUROC and AUPRC curves for all models are shown in Figures 1,2. The following features used by XGBoost were ranked in order of importance by the SHAP analysis: mismatch volume, T_max >6 s, DWI volume, NPR, MCV, T_max >10 s, hemoglobin, potassium, HI, and T_max >8 s. SHAP plot of the best performing model in terms of AUROC, XGBoost is shown in Figure 3. Figure 4A-4E exhibit the remaining SHAP plots.

Figure 1 Receiver operating characteristic curves. AUROC, area under the receiver operating characteristic curve.

Figure 2 Precision-recall curves. AUPRC, area under the precision-recall curve.

Figure 3 SHPA for XGBoost. T_max, time-to-maximum of the tissue residue function; DWI, diffusion-weighted imaging; MCV, mean corpuscular volume; SHAP, Shapley additive explanations.

Figure 4 SHAP for (A) CatBoost, (B) LightGBM, (C) multi-layer perceptron, (D) random forest, and (E) logistic regression. T_max, time-to-maximum of the tissue residue function; DWI, diffusion-weighted imaging; MCV, mean corpuscular volume; SHAP, Shapley additive explanations.

Discussion

With more research indicating that mechanical thrombectomy is safe and effective for DMVOs, accurate diagnosis and prognostic evaluation are needed (1,2,8,9,22). Our study showed that ML algorithms could accurately predict short-term outcomes (NIHSS shift score) patients with AIS due to DMVO, regardless of applied treatment. Using RFE-selected features, XGBoost demonstrated the best performance predicting unfavorable outcomes, with an AUROC of 0.894 and an AUPRC of 0.756. While we acknowledge the potential influence of discharge time and duration of hospital stay on patient outcomes, our ML model was still able to accurately predict the NIHSS shift scores using the available data. Furthermore, although presentation NIHSS is known to be a strong predictor of AIS outcomes, we decided not to include it as an input in our ML models to focus on evaluating the predictive performance of the CTP parameters and other variables in our dataset independently of presentation NIHSS.

Because DMVOs are challenging to diagnose with CTA, CTP has been incorporated into routine AIS evaluation, with CTP being used more frequently in clinical practice (4,30). Becks et al. demonstrated that the brain CTP maps enhance the performance for intracranial vessel occlusion detection on CTA by improving the detection of distal and posterior circulation vessel occlusions (31). Furthermore, Amukotuwa et al. demonstrated that even inexperienced readers could correctly and quickly identify DMVOs on T_max maps (4). Aside from these diagnostic advances, research has been conducted to demonstrate the prognostic value of quantitative CTP parameters in AIS patients. Fainardi et al. demonstrated that T_max target mismatch could predict clinical outcome (modified Rankin score) in patients with LVO-AIS who received endovascular treatment within 24 hours from onset (32). Furthermore, Keenan et al. showed that CTP-based lesion thresholds could predict poor outcomes (discharge to a skilled nursing facility or death at discharge) in AIS patients who received IV-tPA within 4.5 hours of symptom onset (33). In addition, Wan et al. discovered a significant positive correlation between HIR and functional outcome (modified Rankin score) in patients with LVO-AIS, consistent across subgroups (13). Furthermore, in the case of middle cerebral artery occlusions, Seker et al. showed that T_max >6 s, T_max >8 s, and T_max >10 s were higher in patients with poor outcome compared to patients with a good outcome (34). In our study, T_max >6 s, T_max >8 s, and T_max >10 s were selected by RFE to predict the short-term prognosis, supporting the earlier finding. Our ML model, which incorporates quantitative baseline CTP parameters, accurately predicts the short-term outcome in DMVO patients, thus corroborating these findings. As a side note, our study did not directly address potential concerns regarding the reliability and interpretation of CTP imaging within the first 6 hours of AIS onset, which could impact the reliability of CTP-derived perfusion parameters used in our ML models for outcome prediction; however, our focus was on assessing the predictive value of these parameters, rather than validating the accuracy of CTP imaging itself within this timeframe.

In the algorithm with the best performance, the XGBoost, mismatch volume was the most significant variable for predicting the outcome according to the SHAP. In patients with minor AIS, it has been shown that patients treated with IV thrombolysis followed by mechanical thrombectomy (bridging therapy) with a mismatch volume <40 mL were associated with a worse outcome than patients treated with IV thrombolysis alone (35). This effect was not observed in patients with a mismatch volume >40 mL. Patients with M1 or M2 occlusion were included in this multi-institutional study. This finding shows that mismatch volume may help identify patients who respond poorly to bridging therapy. Our study showed that mismatch volume could also help predict the short-term outcome in patients with a DMVO. Furthermore, Jiang et al. showed that DWI volume could provide prognostic information about AIS patients after thrombectomy (36). In our study, the DWI volume was one of the selected features with a high SHAP for predicting the outcome, which is supported by the aforementioned study. While these findings are important, prospective studies with larger sample sizes should be conducted to reveal the true potential of mismatch volume and DWI volume in AIS patients.

In a particular study, researchers evaluated the HI as a means to estimate the initial rate of core progression in patients with medium vessel occlusion (MeVO) in comparison to those with LVO stroke over an extended time period following the onset of stroke (37). The findings indicated that an HI value greater than 0.5 was linked to rapid progression, and it could effectively distinguish individuals with fast progression from those with slow progression in both MeVO and LVO strokes within the initial 24 hours of AIS. This suggests that considering core progression rate at the time of stroke evaluation may have implications in the selection of patients with MeVO and LVO stroke for reperfusion therapy. Furthermore, in another study, Nomani et al. conducted a study involving 106 patients with occlusion of any type to compare different thresholds for infarct progression. They evaluated the effectiveness of HI in distinguishing between fast and slow rates of infarct progression (38). The study found that HI categorized 100% of the cohort with an optimal cutoff of 0.5 for any-type occlusion, showing high sensitivity and specificity. This indicated eligibility for reperfusion and clinical outcomes, with better results for those with HI ≤0.5. In our study, HI played a significant role as one of the top features in the ML model. This further highlights the importance of considering HI in the evaluation and management of patients with AIS.

In our study, several laboratory values were selected after RFE and used in the model. In patients with AIS, He et al. discovered that a high NPR was associated with an increased risk of hemorrhagic transformation (39). Furthermore, it has been demonstrated that the platelet-to-neutrophil ratio (PNR) could be a protective factor in the prediction of the prognosis of AIS (40). Kim et al. reported similar results (41). NPR was one of our study’s selected features with a high SHAP, showing the importance of this value in the short-term prognosis of patients with DMVO-AIS. Furthermore, it has been shown that an abnormal hemoglobin level could be associated with a higher risk of poor functional outcome and stroke recurrence (42,43). In a study, it has been suggested that MCV could predict AIS short-term mortality (44). In addition, the relationship between serum potassium levels and stroke risk or stroke recurrence has been shown in the literature (45,46). Our ML algorithm included serum potassium levels, MCV, and hemoglobin in the model after RFE. Further research on the aforementioned relationships is needed.

Although our study provides valuable insights, it is important to acknowledge its limitations. Our study has its intrinsic limitations of retrospective studies. To validate our ML model, it would be necessary to conduct prospective studies with a larger sample size. A limitation of our study is that we focused on short-term outcomes, which may not always translate into subsequent standard 90-day outcomes in AIS; ideally, the inclusion of 90-day outcome data would have provided a more comprehensive assessment, as it is the standard outcome measure in AIS studies. It is important to recognize that algorithms utilizing gradient boosting can be sensitive to outliers and have a tendency to overfit. To handle outliers, the robust scaler was used, and RFE was used to prevent overfitting. Although the number of patients in this study appears to be small, it included more patients than most published studies on DMVO in the literature (2,5,7,14,47-49). Furthermore, excluding a significant proportion of the sample size may have an impact on the generalizability and robustness of our ML models. Another limitation of our study was that the generalization of our CTP-based model to posterior circulation stroke is questionable, and the accuracy of CT/CTP for posterior circulation stroke is debatable. Moreover, our study did not specifically assess the impact of atherosclerotic disease on patient outcomes, which could have resulted in a different behavior within this particular cohort, potentially affecting the generalizability of our findings. Also, our study did not account for the potential impact of medication use related to thrombectomies, such as antiplatelet/anticoagulation medications, which may have influenced patient outcomes and could be considered in future analyses to assess their contribution to prediction accuracy. In the future, another set of parameters related to the time aspect of thrombectomy may be added as well. Finally, even though it was not among the selected features, the operator reported mTICI scores might be overestimated (50).

Conclusions

In an effort to curb rising healthcare costs, emphasis has been placed on using registries and databases to monitor and determine risk-adjusted estimates for patient outcomes. Consequently, clinicians are tasked with managing extensive volumes of intricate data, necessitating more advanced analytical techniques. As risk identification and shared decision-making play crucial roles in patient care, incorporating ML classifiers into clinical prediction models can provide a significant advantage over traditional methods. Our ML models, which accurately predicted short-term outcomes in patients with DMVOs using baseline quantitative CT parameters, can assist clinicians in making informed treatment decisions, personalizing care plans, and optimizing resource allocation. Furthermore, our findings may contribute to the development of clinical guidelines and decision support tools, ultimately improving patient outcomes and overall stroke care. Our ML model trained on baseline quantitative CTP parameters and laboratory data was able to predict the short-term outcome, NIHSS shift score, in patients with DMVO-AIS. According to the SHAP in the best-performing model, the most important variable was the mismatch volume in predicting the clinical outcome. Notably, admission NIHSS score, and age were not used in our model to predict the prognosis of stroke patients, which are normally very important predictors of prognosis. Using CTP parameters, we were able to accurately predict the prognosis of DMVO-AIS patients, demonstrating the importance of imaging. We hope that our study serves as a starting point for future research in this area.

Acknowledgments

Funding: None.

Footnote

Reporting Checklist: The authors have completed the TRIPOD reporting checklist. Available at https://qims.amegroups.com/article/view/10.21037/qims-23-154/rc

Conflicts of Interest: All authors have completed the ICMJE uniform disclosure form (available at https://qims.amegroups.com/article/view/10.21037/qims-23-154/coif). VSY reports that he is a consultant for RAPID IschemaView (Menlo Park, CA, USA) and MRI Online (Cincinnati, OH, USA). The other authors have no conflicts of interest to declare.

Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. The study was conducted in accordance with the Declaration of Helsinki (as revised in 2013). The study was approved by ethics board of Johns Hopkins Hospital and individual consent for this retrospective analysis was waived.

Open Access Statement: This is an Open Access article distributed in accordance with the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License (CC BY-NC-ND 4.0), which permits the non-commercial replication and distribution of the article with the strict proviso that no changes or edits are made and the original work is properly cited (including links to both the formal publication through the relevant DOI and the license). See: https://creativecommons.org/licenses/by-nc-nd/4.0/.

References

1. Saver JL, Chapot R, Agid R, Hassan A, Jadhav AP, Liebeskind DS, Lobotesis K, Meila D, Meyer L, Raphaeli G, Gupta RDistal Thrombectomy Summit Group. Thrombectomy for Distal, Medium Vessel Occlusions: A Consensus Statement on Present Knowledge and Promising Directions. Stroke 2020;51:2872-84. [Crossref] [PubMed]
2. Guenego A, Mine B, Bonnet T, Elens S, Vazquez Suarez J, Jodaitis L, Ligot N, Naeije G, Lubicz B. Thrombectomy for distal medium vessel occlusion with a new generation of Stentretriever (Tigertriever 13). Interv Neuroradiol 2022;28:444-54. [Crossref] [PubMed]
3. Ospel JM, Bala F, McDonough RV, Volny O, Kashani N, Qiu W, Menon BK, Goyal M. Interrater Agreement and Detection Accuracy for Medium-Vessel Occlusions Using Single-Phase and Multiphase CT Angiography. AJNR Am J Neuroradiol 2022;43:93-7. [Crossref] [PubMed]
4. Amukotuwa SA, Wu A, Zhou K, Page I, Brotchie P, Bammer R. Distal Medium Vessel Occlusions Can Be Accurately and Rapidly Detected Using Tmax Maps. Stroke 2021;52:3308-17. [Crossref] [PubMed]
5. Fischer S, Will L, Phung T, Weber W, Maus V, Nordmeyer H. The Tigertriever 13 for mechanical thrombectomy in distal and medium intracranial vessel occlusions. Neuroradiology 2022;64:775-83. [Crossref] [PubMed]
6. Raza SA, Rangaraju S. A Review of Pre-Intervention Prognostic Scores for Early Prognostication and Patient Selection in Endovascular Management of Large Vessel Occlusion Stroke. Interv Neurol 2018;7:171-81. [Crossref] [PubMed]
7. Hulscher F, Farouki Y, Mine B, Bonnet T, Wang M, Elens S, Suarez JV, Jodaitis L, Ligot N, Naeije G, Lubicz B, Guenego A. Predictors of Good Clinical Outcome after Thrombectomy for Distal Medium Vessel Occlusions. World Neurosurg 2022;160:e566-72. [Crossref] [PubMed]
8. Marchal A, Bretzner M, Casolla B, Kyheng M, Labreuche J, Personnic T, Cordonnier C, Henon H, Bricout N. Endovascular Thrombectomy for Distal Medium Vessel Occlusions of the Middle Cerebral Artery: A Safe and Effective Procedure. World Neurosurg 2022;160:e234-41. [Crossref] [PubMed]
9. Bilgin C, Hardy N, Hutchison K, Pederson JM, Mebane A, Olaniran P, Kobeissi H, Kallmes KM, Fiorella D, Kallmes DF, Brinjikji W. First-line thrombectomy strategy for distal and medium vessel occlusions: a systematic review. J Neurointerv Surg 2023;15:539-46. [Crossref] [PubMed]
10. Tarpley J, Franc D, Tansy AP, Liebeskind DS. Use of perfusion imaging and other imaging techniques to assess risks/benefits of acute stroke interventions. Curr Atheroscler Rep 2013;15:336. [Crossref] [PubMed]
11. Demeestere J, Wouters A, Christensen S, Lemmens R, Lansberg MG. Review of Perfusion Imaging in Acute Ischemic Stroke: From Time to Tissue. Stroke 2020;51:1017-24. [Crossref] [PubMed]
12. Muehlen I, Borutta M, Siedler G, Engelhorn T, Hock S, Knott M, Hoelter P, Volbers B, Schwab S, Doerfler A. Prognostic Accuracy of CTP Summary Maps in Patients with Large Vessel Occlusive Stroke and Poor Revascularization after Mechanical Thrombectomy-Comparison of Three Automated Perfusion Software Applications. Tomography 2022;8:1350-62. [Crossref] [PubMed]
13. Wan Z, Meng Z, Xie S, Fang J, Li L, Chen Z, Liu J, Jiang G. Correlation between Hypoperfusion Intensity Ratio and Functional Outcome in Large-Vessel Occlusion Acute Ischemic Stroke: Comparison with Multi-Phase CT Angiography. J Clin Med 2022;11:5274. [Crossref] [PubMed]
14. Guenego A, Farouki Y, Mine B, Bonnet T, Hulscher F, Wang M, Elens S, Vazquez Suarez J, Jodaitis L, Ligot N, Naeije G, Lubicz B. Hypoperfusion Intensity Ratio Predicts Infarct Growth After Successful Thrombectomy for Distal Medium Vessel Occlusion. Clin Neuroradiol 2022;32:849-56. [Crossref] [PubMed]
15. Brugnara G, Neuberger U, Mahmutoglu MA, Foltyn M, Herweh C, Nagel S, Schönenberger S, Heiland S, Ulfert C, Ringleb PA, Bendszus M, Möhlenbruch MA, Pfaff JAR, Vollmuth P. Multimodal Predictive Modeling of Endovascular Treatment Outcome for Acute Ischemic Stroke Using Machine-Learning. Stroke 2020;51:3541-51. [Crossref] [PubMed]
16. Cruz JA, Wishart DS. Applications of machine learning in cancer prediction and prognosis. Cancer Inform 2007;2:59-77.
17. Kuhle S, Maguire B, Zhang H, Hamilton D, Allen AC, Joseph KS, Allen VM. Comparison of logistic regression with machine learning methods for the prediction of fetal growth abnormalities: a retrospective cohort study. BMC Pregnancy Childbirth 2018;18:333. [Crossref] [PubMed]
18. Oermann EK, Rubinsteyn A, Ding D, Mascitelli J, Starke RM, Bederson JB, Kano H, Lunsford LD, Sheehan JP, Hammerbacher J, Kondziolka D. Using a Machine Learning Approach to Predict Outcomes after Radiosurgery for Cerebral Arteriovenous Malformations. Sci Rep 2016;6:21161. [Crossref] [PubMed]
19. Heo J, Yoon JG, Park H, Kim YD, Nam HS, Heo JH. Machine Learning-Based Model for Prediction of Outcomes in Acute Stroke. Stroke 2019;50:1263-5. [Crossref] [PubMed]
20. Nishi H, Oishi N, Ishii A, Ono I, Ogura T, Sunohara T, Chihara H, Fukumitsu R, Okawa M, Yamana N, Imamura H, Sadamasa N, Hatano T, Nakahara I, Sakai N, Miyamoto S. Predicting Clinical Outcomes of Large Vessel Occlusion Before Mechanical Thrombectomy Using Machine Learning. Stroke 2019;50:2379-88. [Crossref] [PubMed]
21. Hamann J, Herzog L, Wehrli C, Dobrocky T, Bink A, Piccirelli M, Panos L, Kaesmacher J, Fischer U, Stippich C, Luft AR, Gralla J, Arnold M, Wiest R, Sick B, Wegener S. Machine-learning-based outcome prediction in stroke patients with middle cerebral artery-M1 occlusions and early thrombectomy. Eur J Neurol 2021;28:1234-43. [Crossref] [PubMed]
22. Meyer L, Stracke CP, Jungi N, Wallocha M, Broocks G, Sporns PB, et al. Thrombectomy for Primary Distal Posterior Cerebral Artery Occlusion Stroke: The TOPMOST Study. JAMA Neurol 2021;78:434-44. [Crossref] [PubMed]
23. Ozkara BB, Karabacak M, Hamam O, Wang R, Kotha A, Khalili N, Hoseinyazdi M, Chen MM, Wintermark M, Yedavalli VS. Prediction of Functional Outcome in Stroke Patients with Proximal Middle Cerebral Artery Occlusions Using Machine Learning Models. J Clin Med 2023;12:839. [Crossref] [PubMed]
24. Chen T, Guestrin C. XGBoost: A Scalable Tree Boosting System. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. San Francisco: ACM, 2016:785-94.
25. Ke G, Meng Q, Finley T, Wang T, Chen W, Ma W, et al. LightGBM: A Highly Efficient Gradient Boosting Decision Tree. In: Guyon I, Luxburg UV, Bengio S, Wallach H, Fergus R, Vishwanathan S, et al. editors. Advances in Neural Information Processing Systems. Red Hook: Curran Associates, Inc., 2017.
26. Dorogush AV, Ershov V, Gulin A. CatBoost: gradient boosting with categorical features support. arXiv preprint arXiv:1810.11363, 2018.
27. sklearn.neural_network.MLPClassifier. scikit-learn. Available online: https://scikit-learn/stable/modules/generated/sklearn.neural_network.MLPClassifier.html (Accessed 2 Nov 2022).
28. sklearn.ensemble.RandomForestClassifier. scikit-learn. Available online: https://scikit-learn/stable/modules/generated/sklearn.ensemble.RandomForestClassifier.html (Accessed 2 Nov 2022).
29. Akiba T, Sano S, Yanase T, Ohta T, Koyama M. Optuna: A Next-generation Hyperparameter Optimization Framework. In: Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. Anchorage: ACM, 2019:2623-31.
30. Vagal A, Wintermark M, Nael K, Bivard A, Parsons M, Grossman AW, Khatri P. Automated CT perfusion imaging for acute ischemic stroke: Pearls and pitfalls for real-world use. Neurology 2019;93:888-98. [Crossref] [PubMed]
31. Becks MJ, Manniesing R, Vister J, Pegge SAH, Steens SCA, van Dijk EJ, Prokop M, Meijer FJA. Brain CT perfusion improves intracranial vessel occlusion detection on CT angiography. J Neuroradiol 2019;46:124-9. [Crossref] [PubMed]
32. Fainardi E, Busto G, Rosi A, Scola E, Casetta I, Bernardoni A, Saletti A, Arba F, Nencini P, Limbucci N, Mangiafico S, Demchuk A, Almekhlafi MA, Goyal M, Lee TY, Menon BK, Morotti A T. (max) Volumes Predict Final Infarct Size and Functional Outcome in Ischemic Stroke Patients Receiving Endovascular Treatment. Ann Neurol 2022;91:878-88. [Crossref] [PubMed]
33. Keenan KJ, Christensen S, Inoue M, Mlynash M, Albers GW, Smith WS. Validation and iteration of CT perfusion defined malignant profile thresholds for acute ischemic stroke. Int J Stroke 2020;15:55-60. [Crossref] [PubMed]
34. Seker F, Pfaff J, Potreck A, Mundiyanapurath S, Ringleb PA, Bendszus M, Möhlenbruch MA. Correlation of T(max) volumes with clinical outcome in anterior circulation stroke. Brain Behav 2017;7:e00772. [Crossref] [PubMed]
35. Seners P, Arquizan C, Fontaine L, Ben Hassen W, Heldner MR, Strambo D, et al. Perfusion Imaging and Clinical Outcome in Acute Minor Stroke With Large Vessel Occlusion. Stroke 2022;53:3429-38. [Crossref] [PubMed]
36. Jiang L, Peng M, Chen H, Geng W, Zhao B, Yin X, Chen YC, Su H. Diffusion-weighted imaging (DWI) ischemic volume is related to FLAIR hyperintensity-DWI mismatch and functional outcome after endovascular therapy. Quant Imaging Med Surg 2020;10:356-67. [Crossref] [PubMed]
37. Nomani AZ, Kamtchum Tatuene J, Rempel JL, Jeerakathil T, Winship IR, Khan KA, Buck BH, Shuaib A, Jickling GC. Association of CT-Based Hypoperfusion Index With Ischemic Core Enlargement in Patients With Medium and Large Vessel Stroke. Neurology 2021;97:e2079-87. [Crossref] [PubMed]
38. Nomani AZ, Rempel JL, Khan KA, Shuaib A, Jickling GC. How to Define Fast and Slow Progressors in Any-Type Occlusion Acute Ischemic Stroke. Can J Neurol Sci 2023;50:268-73. [Crossref] [PubMed]
39. He W, Ruan Y, Yuan C, Cheng Q, Cheng H, Zeng Y, Chen Y, Huang G, Chen H, He J. High Neutrophil-to-Platelet Ratio Is Associated With Hemorrhagic Transformation in Patients With Acute Ischemic Stroke. Front Neurol 2019;10:1310. [Crossref] [PubMed]
40. Jin P, Li X, Chen J, Zhang Z, Hu W, Chen L, Feng X, Shao B. Platelet-to-neutrophil ratio is a prognostic marker for 90-days outcome in acute ischemic stroke. J Clin Neurosci 2019;63:110-5. [Crossref] [PubMed]
41. Kim SY, Yi HJ, Shin DS, Kim BT. Prognostic significance of platelet-to-lymphocyte and platelet-to-neutrophil ratios in patients with mechanical thrombectomy for acute ischemic stroke. J Cerebrovasc Endovasc Neurosurg 2022;24:221-31. [Crossref] [PubMed]
42. Chang JY, Lee JS, Kim BJ, Kim JT, Lee J, Cha JK, et al. Influence of Hemoglobin Concentration on Stroke Recurrence and Composite Vascular Events. Stroke 2020;51:1309-12. [Crossref] [PubMed]
43. Zhang R, Xu Q, Wang A, Jiang Y, Meng X, Zhou M, Wang Y, Liu G. Hemoglobin Concentration and Clinical Outcomes After Acute Ischemic Stroke or Transient Ischemic Attack. J Am Heart Assoc 2021;10:e022547. [Crossref] [PubMed]
44. Hatamian H, Saberi A, Pourghasem M. The relationship between stroke mortality and red blood cell parameters. Iran J Neurol 2014;13:237-40.
45. Wang A, Cao S, Tian X, Zuo Y, Meng X, Chen P, Li H, Zhao L, Wang Y. Lower Serum Potassium Levels at Admission are Associated with the Risk of Recurrent Stroke in Patients with Acute Ischemic Stroke or Transient Ischemic Attack. Cerebrovasc Dis 2022;51:304-12. [Crossref] [PubMed]
46. Green DM, Ropper AH, Kronmal RA, Psaty BM, Burke GLCardiovascular Health Study. Serum potassium level and dietary potassium intake as risk factors for stroke. Neurology 2002;59:314-20. [Crossref] [PubMed]
47. Wang M, Farouki Y, Hulscher F, Mine B, Bonnet T, Elens S, Vazquez Suarez J, Jodaitis L, Ligot N, Naeije G, Lubicz B, Guenego A. Early Neurological Improvement Predicts Clinical Outcome After Thrombectomy for Distal Medium Vessel Occlusions. Front Neurol 2022;13:809066. [Crossref] [PubMed]
48. Grieb D, Greling B, Schulz K, Boxberg F, Melber K, Abu-Fares O, Lanfermann H, Schlunz-Hendann M, Meila D. Endovascular treatment of distal medium vessel occlusions using microcatheter aspiration thrombectomy. Interv Neuroradiol 2022; Epub ahead of print. [Crossref]
49. Farouki Y, Bonnet T, Mine B, Hulscher F, Wang M, Elens S, Vazquez Suarez J, Jodaitis L, Ligot N, Naeije G, Walker G, Lubicz B, Guenego A. First-Pass Effect Predicts Clinical Outcome and Infarct Growth After Thrombectomy for Distal Medium Vessel Occlusions. Neurosurgery 2022;91:913-9. [Crossref] [PubMed]
50. Ibrahim MK, Shehata MA, Ghozy S, Bilgin C, Jabal MS, Heiferman DM, Kadirvel R, Kallmes DF. Operator assessment versus core laboratory adjudication of recanalization following endovascular treatment of acute ischemic stroke: a systematic review and meta-analysis. J Neurointerv Surg 2023;15:133-8. [Crossref] [PubMed]