Presentation Title
Identifying predictive brain structure features in alcohol dependent subjects
Abstract
Identifying predictive brain structure features in alcohol dependent subjects
Sage Hahn, Nicholas Allgaier, Scott Mackey, Hugh Garavan
Typical neuroimaging analysis tends to remain bounded in exploring linear effects between features and/or groupings of features. The objective of this study was to explore the merit of machine learning techniques, generally capable of modeling more complex nonlinear effects, in predicting alcohol dependence from measurements of brain structure, previously shown possible by (Mackey et al. 2018). Of particular interest was in isolating subsets of features responsible for accurate cross validated predictions. A dataset of 911 individuals, 640 diagnosed as alcohol dependent, was collected by the Enhancing Neuro-Imaging Genetics Through Meta-Analysis (ENIGMA) Addiction Working Group (Mackey et al. 2016). Freesurfer 5.3 was utilized to process each patients structural weighted T1 MRI scan extracting volume measurements for 7 bilateral subcortical regions and measurements corresponding to thickness and surface volume for 34 bilateral cortical regions. Measurements were then residualized according to age, sex, intra cranial volume and study site.
In order to reliably determine classifier performance repeated (n=50) random 3-fold stratified cross validation (CV) was employed, where specifically a support vector machine (SVM) with a radial basis function kernel was trained and evaluated on all available measurements, with SVM parameters chosen from a randomized parameter search (n=100) using further nested 3-fold CV (Suykens et al. 1999). Baseline SVM performance when trained on all 150 available measurements achieved an average area under the receiver operating characteristic curve (ROC AUC) of .779 +- .027, in comparison to when trained on only the 14 subcortical volume measurements with a ROC AUC .626 +- .032, 64 measures of surface area with a ROC AUC .605 +- .030, and 64 measures of average thickness with a ROC AUC .780 +- .027. These results suggest that only measurements corresponding to average thickness contribute to classifier performance, though notably there still remains 2^68 - 1 (1.8 * 10^19) possible combinations of features potentially responsible. A multi-objective evolutionary search algorithm was designed with the goal of finding both the smallest set of useful thickness measurements possible as well as the most predictive. Outputted feature sets from the search were then thresholded, retaining only sets of features with a ROC AUC > .77 under the previously introduced evaluation methodology. After 4 searches, 28 separate groupings of 11 to 18 features met this criteria. These sets were then analyzed for predictive importance with the assumption that a particular features importance is directly related to the fraction of feature sets in which it appears. Two regions in particular, the right posterior cingulate cortex and right middle temporal gyrus appeared in 90+% of sets, along with a total of 10 features occurring in over 50% out of 45 which appeared at least once.
The ability of a machine learning classifier to predict alcohol dependence from
measurements of cortical thickness alone represents an encouraging result towards
the development of dependence related neuroimaging biomarkers. Likewise, efforts towards isolating the specific sets of thickness measurements responsible move closer towards that goal. Future experiments will run additional evolutionary searches as well as seek to replicate classifier performance on unseen datasets.
Mackey, Scott, et al. "Mega-Analysis of Gray Matter Volume in Substance Dependence: General and Substance-Specific Regional Effects." American Journal of Psychiatry (2018): appi-ajp.
Mackey, Scott, et al. "Genetic imaging consortium for addiction medicine: From neuroimaging to genes." Progress in brain research. Vol. 224. Elsevier, 2016. 203-223.
Suykens, Johan AK, and Joos Vandewalle. "Least squares support vector machine classifiers." Neural processing letters9.3 (1999): 293-300.
Primary Faculty Mentor Name
Hugh Garavan
Secondary Mentor Name
Nick Allgaier
Status
Graduate
Student College
College of Engineering and Mathematical Sciences
Program/Major
Complex Systems
Primary Research Category
Health Sciences
Identifying predictive brain structure features in alcohol dependent subjects
Identifying predictive brain structure features in alcohol dependent subjects
Sage Hahn, Nicholas Allgaier, Scott Mackey, Hugh Garavan
Typical neuroimaging analysis tends to remain bounded in exploring linear effects between features and/or groupings of features. The objective of this study was to explore the merit of machine learning techniques, generally capable of modeling more complex nonlinear effects, in predicting alcohol dependence from measurements of brain structure, previously shown possible by (Mackey et al. 2018). Of particular interest was in isolating subsets of features responsible for accurate cross validated predictions. A dataset of 911 individuals, 640 diagnosed as alcohol dependent, was collected by the Enhancing Neuro-Imaging Genetics Through Meta-Analysis (ENIGMA) Addiction Working Group (Mackey et al. 2016). Freesurfer 5.3 was utilized to process each patients structural weighted T1 MRI scan extracting volume measurements for 7 bilateral subcortical regions and measurements corresponding to thickness and surface volume for 34 bilateral cortical regions. Measurements were then residualized according to age, sex, intra cranial volume and study site.
In order to reliably determine classifier performance repeated (n=50) random 3-fold stratified cross validation (CV) was employed, where specifically a support vector machine (SVM) with a radial basis function kernel was trained and evaluated on all available measurements, with SVM parameters chosen from a randomized parameter search (n=100) using further nested 3-fold CV (Suykens et al. 1999). Baseline SVM performance when trained on all 150 available measurements achieved an average area under the receiver operating characteristic curve (ROC AUC) of .779 +- .027, in comparison to when trained on only the 14 subcortical volume measurements with a ROC AUC .626 +- .032, 64 measures of surface area with a ROC AUC .605 +- .030, and 64 measures of average thickness with a ROC AUC .780 +- .027. These results suggest that only measurements corresponding to average thickness contribute to classifier performance, though notably there still remains 2^68 - 1 (1.8 * 10^19) possible combinations of features potentially responsible. A multi-objective evolutionary search algorithm was designed with the goal of finding both the smallest set of useful thickness measurements possible as well as the most predictive. Outputted feature sets from the search were then thresholded, retaining only sets of features with a ROC AUC > .77 under the previously introduced evaluation methodology. After 4 searches, 28 separate groupings of 11 to 18 features met this criteria. These sets were then analyzed for predictive importance with the assumption that a particular features importance is directly related to the fraction of feature sets in which it appears. Two regions in particular, the right posterior cingulate cortex and right middle temporal gyrus appeared in 90+% of sets, along with a total of 10 features occurring in over 50% out of 45 which appeared at least once.
The ability of a machine learning classifier to predict alcohol dependence from
measurements of cortical thickness alone represents an encouraging result towards
the development of dependence related neuroimaging biomarkers. Likewise, efforts towards isolating the specific sets of thickness measurements responsible move closer towards that goal. Future experiments will run additional evolutionary searches as well as seek to replicate classifier performance on unseen datasets.
Mackey, Scott, et al. "Mega-Analysis of Gray Matter Volume in Substance Dependence: General and Substance-Specific Regional Effects." American Journal of Psychiatry (2018): appi-ajp.
Mackey, Scott, et al. "Genetic imaging consortium for addiction medicine: From neuroimaging to genes." Progress in brain research. Vol. 224. Elsevier, 2016. 203-223.
Suykens, Johan AK, and Joos Vandewalle. "Least squares support vector machine classifiers." Neural processing letters9.3 (1999): 293-300.