Measuring Confidence in Classification Decisions for Clinical Decision Support Systems: A Gaussian Bayes Optimization Approach

Han, Dongxu (2022) Measuring Confidence in Classification Decisions for Clinical Decision Support Systems: A Gaussian Bayes Optimization Approach. Doctoral thesis, The University Of Buckingham.

Text DongxuHan_1303092 Final Thesis -.pdf - Submitted Version Available under License Creative Commons Attribution Non-commercial No Derivatives. Download (2MB)

Abstract

This thesis generally investigated various aspects of designing and developing Clinical Decision Support Systems (CDSSs), but in particular exploited machine learning techniques in supporting medical diagnosis decisions. Having reviewed the fundamental functional components of existing modern CDSSs, it shows that most such systems were lacking a trusted decision evaluation module that provides reliable information about decision strengths. Therefore a refined CDSS system framework was first proposed, which centralises the concept of confidence-based classification by coupling eventual decision outcomes with a level of decision reliability. Based on measure theory, a unified Decision Score measure of the decision reliability was introduced, which combines the decision outcomes in terms of positive or negative signs together with the decision strength in percentage values. Furthermore, the behaviour of the proposed decision score measure was investigated in more complex and diverse feature spaces of high dimensionality, where the challenges of the “curse of dimensionality” are encountered. Such challenge was handled by revisiting the problem under orthogonal projections of the feature space, and have developed a new measure in performing quantified evaluations on the decision score measure, known as the Decision Sensitivity measure. The key influencing factors for the sensitivity of decisions were found to include not only the dimensionality of the selected features, but also the standard deviation of each feature used in the transformed orthogonal space. After the basic concept of the decision score measure is established, this thesis further extended the uses of the decision score measure in a multiple classifiers setting. This thesis first reviewed the principles and rationales behind various well-established information fusion schemes and tested their strengths and limitations in adapting the proposed decision score measure. Moreover, a correlation-based decision fusion scheme was proposed in maximising the potentials of the decision score measure in complex scenarios. Based on the evaluation results across different datasets, it proves that fusion schemes improve the robustness of the decision models while maintaining a good level of diagnostic accuracy in general. As clinical decision making normally faces new unseen cases and unpredictable challenges, it is essential to maintain a degree of adaptivity in a CDSS for post-deployment robustness of the system. Therefore, the last piece of the research reported in this thesis focused on investigating possible ways to refine the CDSS decision scores model in a time-efficient manner, spontaneously. In particular, this thesis reviewed several commonly used metrics and methods for monitoring and refining prediction models, and further adapted these methods to the proposed decision score measure.

Actions (login required)

View Item