SCIENTIA SINICA Informationis, Volume 47 , Issue 9 : 1164-1182(2017) https://doi.org/10.1360/N112017-00075

Multimodal aided neurological disease diagnosis with synergy of cloud and client

More info
  • ReceivedApr 18, 2017
  • AcceptedJul 27, 2017
  • PublishedSep 5, 2017


Funded by





[1] Wirdefeldt K, Adami H O, Cole P, et al. Epidemiology and etiology of Parkinson's disease: a review of the evidence. Eur J Epidemiol, 2011, 26: 1--58. Google Scholar

[2] Morgado P M. Automated diagnosis of Alzheimer's disease using PET images a study of alternative procedures for feature extraction and selection electrical and computer engineering. Dissertation for M.S. Degree. Lisbon: Technical University of Lisbon, 2012. Google Scholar

[3] Smith E E, O'Donnell M, Dagenais G. Early cerebral small vessel disease and brain volume, cognition, and gait.. Ann Neurol, 2015, 77: 251-261 CrossRef PubMed Google Scholar

[4] Rijk M C D, Launer L J, Berger K, et al. Prevalence of Parkinson's disease in Europe: a collaborative study of population-based cohorts. Neurology, 2000, 54: 21--23. Google Scholar

[5] Prince M, Ali G C, Guerchet M. Recent global trends in the prevalence and incidence of dementia, and survival with dementia.. Alz Res Ther, 2016, 8: 23 CrossRef PubMed Google Scholar

[6] Kim J S. Small vessel disease: not a small problem.. J Stroke, 2015, 17: 1 CrossRef PubMed Google Scholar

[7] Skodda S, Gronheit W, Mancinelli N, et al. Progression of voice and speech impairment in the course of Parkinson's disease: a longitudinal study. Parkinson's Dis, 2013, 2013: 389195. Google Scholar

[8] Khedher L, Ramírez J, Górriz J M. Early diagnosis of Alzheimer?s disease based on partial least squares, principal component analysis and support vector machine using segmented MRI images. Neurocomputing, 2015, 151: 139-150 CrossRef Google Scholar

[9] Jaimes A, Sebe N. Multimodal human?Ccomputer interaction: A survey. Comp Vision Image Understanding, 2007, 108: 116-134 CrossRef Google Scholar

[10] Zhang W, Smith M L, Smith L N. Eye center localization and gaze gesture recognition for human-computer interaction. J Opt Soc Am A, 2016, 33: 314-325 CrossRef ADS Google Scholar

[11] Zhang H, Wang Y, Bai X. Cyclic tensile strain on vocal fold fibroblasts inhibits cigarette smoke-induced inflammation: implications for Reinke edema.. J Voice, 2015, 29: 13-21 CrossRef PubMed Google Scholar

[12] Roy N, Barkmeier-Kraemer J, Eadie T. Evidence-Based Clinical Voice Assessment: A Systematic Review. Am J Speech Lang Pathol, 2013, 22: 212-226 CrossRef Google Scholar

[13] MacCallum J K, Zhang Y, Jiang J J. Vowel selection and its effects on perturbation and nonlinear dynamic measures.. Folia Phoniatr Logop, 2011, 63: 88-97 CrossRef PubMed Google Scholar

[14] Instructions for Contributors. J Voice, 2016, 30: A7-A9 CrossRef Google Scholar

[15] Moers C, M?bius B, Rosanowski F. Vowel- and text-based cepstral analysis of chronic hoarseness.. J Voice, 2012, 26: 416-424 CrossRef PubMed Google Scholar

[16] Alnasheri A, Muhammad G, Alsulaiman M, et al. An investigation of multidimensional voice program parameters in three different databases for voice pathology detection and classification. J Voice, 2016, 31: 9--18. Google Scholar

[17] Bin L, Zheng D, Yu N. An Improved Weighted Centroid Localization Algorithm. IJFGCN, 2013, 6: 45-52 CrossRef Google Scholar

[18] Tsanas A, Gómez-Vilda P. Novel robust decision support tool assisting early diagnosis of pathological voices using acoustic analysis of sustained vowels. In: Proceedings of Multidisciplinary Conference of Users of Voice, Speech and Singing, Las Palmas, 2013. 3--12. Google Scholar

[19] Ünlü A, Brause R, Krakow K. Handwriting analysis for diagnosis and prognosis of parkinson's disease. In: Proceedings of the 7th International Conference on Biological and Medical Data Analysis, Thessaloniki, 2006. 441--450. Google Scholar

[20] Standardized Handwriting to Assess Bradykinesia, Micrographia and Tremor in Parkinson's Disease. PLoS ONE, 2014, 9: e97614 CrossRef PubMed ADS Google Scholar

[21] Kim H, Cho Y S, Do E Y. Context-bounded refinement filter algorithm: improving recognizer accuracy of handwriting in clock drawing test. In: Proceedings of National Conference on Artificial Intelligence, Atlanta, 2010. 53--60. Google Scholar

[22] Davis R, Libon D, Au R. THink: Inferring Cognitive Status from Subtle Behaviors. AIMag, 2015, 36: 49-60 CrossRef Google Scholar

[23] Hagler S, Jimison H B, Pavel M. Assessing executive function using a computer game: computational modeling of cognitive processes.. IEEE J Biomed Health Inform, 2014, 18: 1442-1452 CrossRef PubMed Google Scholar

[24] Drake J M, Griffen B D. Early warning signals of extinction in deteriorating environments. Nature, 2010, 467: 456-459 CrossRef PubMed ADS Google Scholar

[25] Chen D Y, Ding H, Zhou X Y, et al. A system of measuring and analyzing gait of human walking. Chin J Biomed Eng, 1997, 16: 133--141. Google Scholar

[26] Sinha A, Chakravarty K, Bhowmick B, et al. Person identification using skeleton information from kinect. In: Proceedings of International Conference on Advances in Computer-Human Interaction, Nice, 2013. 101--108. Google Scholar

[27] Paul M, Haque S M E, Chakraborty S. Human detection in surveillance videos and its applications - a review. EURASIP J Adv Signal Process, 2013, 2013: 176 CrossRef ADS Google Scholar

[28] Lan K C, Shih W Y. Early Diagnosis of Parkinson's Disease Using a Smartphone. Procedia Comp Sci, 2014, 34: 305-312 CrossRef Google Scholar

[29] Chen Y, Huang M, Hu C, et al. A coarse-to-fine feature selection method for accurate detection of cerebral small vessel disease. In: Proceedings of International Symposium on Neural Networks, Saint Petersburg, 2016. 2609--2616. Google Scholar

[30] Dirican A C, G?ktürk M. Psychophysiological measures of human cognitive states applied in human computer interaction. Procedia Comp Sci, 2011, 3: 1361-1367 CrossRef Google Scholar

[31] Altun K, Barshan B. Human activity recognition using inertial/magnetic sensor units. In: Proceedings of International Conference on Pattern Recognition, Istanbul, 2010. 38--51. Google Scholar

[32] Patel S, Lorincz K, Hughes R. Monitoring motor fluctuations in patients with Parkinson's disease using wearable sensors.. IEEE Trans Inform Technol Biomed, 2009, 13: 864-873 CrossRef PubMed Google Scholar

[33] Robichaud J A, Pfann K D, Leurgans S, et al. Variability of EMG patterns: a potential neurophysiological marker of Parkinson's disease? Clin Neurophysiol, 2009, 120: 390--397. Google Scholar

[34] Jordan K G. Emergency EEG and continuous EEG monitoring in acute ischemic stroke. Clin Neurophysiol, 2004, 21: 341--352. Google Scholar

[35] Armbrust M, Fox A, Griffith R, et al. A view of cloud computing. Commun ACM, 2010, 53: 50--58. Google Scholar

[36] Kamara S, Lauter K. Cryptographic cloud storage. In: Proceedings of International Conference on Financial Cryptography and Data Security, Tenerife, 2010. 136--149. Google Scholar

[37] Subashini S, Kavitha V. A survey on security issues in service delivery models of cloud computing. J Network Comp Appl, 2011, 34: 1-11 CrossRef Google Scholar

[38] Wang H, Yi X, Bertino E. Protecting outsourced data in cloud computing through access management. Concurrency Computat-Pract Exper, 2016, 28: 600-615 CrossRef Google Scholar

[39] Tian F, Dai G Z, Chen Y D, et al. Specification and structure design of 3D interaction tasks. J Softw, 2002, 13: 2099--2105. Google Scholar

[40] Huang J, Chen Y N, Liu J, et al. Multimodal human-computer interaction model for nerve function assessment in mobile environment. J Softw, 2016, 27: 156--171. Google Scholar

[41] Hanchuan Peng , Fuhui Long , Ding C. Feature selection based on mutual information: criteria of max-dependency, max-relevance, and min-redundancy.. IEEE Trans Pattern Anal Machine Intell, 2005, 27: 1226-1238 CrossRef PubMed Google Scholar

[42] Chandrashekar G, Sahin F. A survey on feature selection methods. Comp Electrical Eng, 2014, 40: 16-28 CrossRef Google Scholar

[43] Kumari B, Swarnkar T. Filter versus wrapper feature subset selection in large dimensionality micro array: a review. Int J Comput Sci Inf Technol, 2011, 2: 1048--1053. Google Scholar

[44] Polikar R. Ensemble based systems in decision making. IEEE Circuits Syst Mag, 2006, 6: 21-45 CrossRef Google Scholar

[45] Miao K H, Miao J H, Miao G J. Diagnosing coronary heart disease using ensemble machine learning. Int J Adv Comput Sci Appl, 2016, 7: 30--39. Google Scholar

[46] Yang J J, Li J, Shen R. Exploiting ensemble learning for automatic cataract detection and grading.. Comp Methods Programs Biomed, 2016, 124: 45-57 CrossRef PubMed Google Scholar

[47] Inzamam-Ul-Hossain M, MacKinnon L, Islam M R. Parkinson disease detection using ensemble method in PASW benchmark. In: Proceedings of IEEE International Advance Computing Conference, Banglore, 2015. 666--670. Google Scholar

[48] Lezak M D. Neuropsychological Assessment. New York: Oxford University Press, 1995. 650--680. Google Scholar

[49] Arnett J A, Labovitz S S. Effect of physical layout in performance of the Trail Making Test.. Psychological Assessment, 1995, 7: 220-221 CrossRef Google Scholar

[50] Sofranko J L, Prosek R A. The effect of levels and types of experience on judgment of synthesized voice quality.. J Voice, 2014, 28: 24-35 CrossRef PubMed Google Scholar

[51] Tsanas A, Little M A, McSharry P E. Accurate telemonitoring of Parkinson's disease progression by noninvasive speech tests.. IEEE Trans Biomed Eng, 2010, 57: 884-893 CrossRef PubMed Google Scholar

[52] Yumoto E, Gould W J, Baer T. Harmonics-to-noise ratio as an index of the degree of hoarseness. J Acoust Soc Am, 1982, 71: 1544-1550 CrossRef ADS Google Scholar

[53] Srinivasan B, Spinner T, Rengaswamy R. Control loop performance assessment using detrended fluctuation analysis (DFA). Automatica, 2012, 48: 1359-1363 CrossRef Google Scholar

[54] Godino-Llorente J I, Osma-Ruiz V, Sáenz-Lechón N. The effectiveness of the glottal to noise excitation ratio for the screening of voice disorders.. J Voice, 2010, 24: 47-56 CrossRef PubMed Google Scholar

[55] Arias-Londo?o J D, Godino-Llorente J I, Sáenz-Lechón N. An improved method for voice pathology detection by means of a HMM-based feature space transformation. Pattern Recognition, 2010, 43: 3100-3112 CrossRef Google Scholar

[56] Hall M A. Correlation-based feature subset selection for machine learning. Dissertation for Ph.D. Degree. Hamilton: University of Waikato, 1998. Google Scholar

[57] Sellam V, Jagadeesan J. Classification of normal and pathological voice using SVM and RBFNN. J Signal Inf Process, 2014, 5: 1--7. Google Scholar

[58] Seixas F L, Zadrozny B, Laks J. A Bayesian network decision model for supporting the diagnosis of dementia, Alzheimer?s disease and mild cognitive impairment.. Comp Biol Med, 2014, 51: 140-158 CrossRef PubMed Google Scholar

[59] Kalyani R R, Corriere M, Ferrucci L. Age-related and disease-related muscle loss: the effect of diabetes, obesity, and other diseases. Lancet Diabetes Endocrinology, 2014, 2: 819-829 CrossRef Google Scholar

[60] Ransmayr G, Künig G, Neubauer M. Effect of age and disease duration on parkinsonian motor scores under levodopa therapy. J Neural Transm Gen Sect, 1995, 9: 177-188 CrossRef Google Scholar

  • Figure 1

    Cloud-Client synergy based multi-modal aided neurological diseasediagnosis framework

  • Figure 2

    Neurological disease diagnosis based on multi-modal interaction

  • Figure 3

    Case study of neurological disease diagnosis based on pen and audio interaction

  • Figure 4

    (Color online) (a) Voice data collection; (b) pen based trail making test

  • Figure 5

    (Color online) A diagram of Stacking ensemble_learning classifier

  • Figure 6

    (Color online) Classification effects of five models on different datasets

  • Table 1   Extraction characteristics of voice data and their meanings
    DFA1一段时间内声波信号的包络变化趋势, 噪声的随机自相似性
    发音器官运动MFCC84声波的整体衡量, 频域表示, 短时能量谱, 波形的细微扰动
  • Table 2   Extraction characteristics of pen gesture and their meanings
    位移 Acceleration112笔尖的瞬时加速度, 反映手部运动的力量变化
    UMO40笔尖位置与当前笔迹意图中心的偏离程度, 反映无意识运动程度
    Curvature76当前笔迹在该笔尖位置上的弯曲程度, 反映笔迹的笔直程度
    压力Pressure speed36笔尖压力的变化速度
    Pressure accuracy36笔尖压力的变化加速度, 反映手部运动的纵向力量变化
    笔身姿态 Posture speed148笔身角度的变化速度
    Posture accuracy148笔身角度的变化加速度, 反映手部运动在三维空间中的力量变化
  • Table 3   Classification effects of five different models
    ModelAccuracy (%)PrecisionRecall$F$ value
    DMEM (Ex)89.220.9050.8790.885
  • Table 4   Classification effects of DMEM (Ex) on different ages and genders
    Sample setAccuracy (%)PrecisionRecall$F$ value
    Age 0 $\sim$ 6486.950.8980.8270.846
    Age 65 $\sim$ 7490.400.8460.8520.834
    Age above 75 94.620.8080.8410.819
  • Table 5   Classification effects of DMEM (Ex) on different ages
    Age rangeAccuracy (%)PrecisionRecall$F$ value
    0 $\sim$ 6489.140.9160.8580.873
    65 $\sim$ 7486.990.8450.8240.813
    Age above 75 94.470.8060.8200.807
  • Table 6   Classification effects of DMEM (Ex) on different genders
    ModelAccuracy (%)PrecisionRecall$F$ value