SCIENTIA SINICA Informationis, Volume 51 , Issue 3 : 383(2021) https://doi.org/10.1360/SSI-2020-0234

Esthetic preference mining of Chinese typefaces using multiview cluster analysis

More info
  • ReceivedAug 1, 2020
  • AcceptedSep 16, 2020
  • PublishedFeb 23, 2021


Funded by



[1] Zhang J. A survey of digital calligraphy. Sci Sin-Inf, 2019, 49: 143-158 CrossRef Google Scholar

[2] Bargh J A, Ferguson M J. Beyond behaviorism: On the automaticity of higher mental processes.. Psychological Bull, 2000, 126: 925-945 CrossRef Google Scholar

[3] H?fel L, Jacobsen T. Electrophysiological indices of processing aesthetics: Spontaneous or intentional processes?. Int J PsychoPhysiol, 2007, 65: 20-31 CrossRef Google Scholar

[4] Handy T C, Smilek D, Geiger L. ERP Evidence for Rapid Hedonic Evaluation of Logos. J Cognitive Neuroscience, 2010, 22: 124-138 CrossRef Google Scholar

[5] Sutton S, Tueting P, Zubin J. Information Delivery and the Sensory Evoked Potential. Science, 1967, 155: 1436-1439 CrossRef ADS Google Scholar

[6] Jacobsen T, H?fel L. Aesthetics Electrified: An Analysis of Descriptive Symmetry and Evaluative Aesthetic Judgment Processes Using Event-Related Brain Potentials. Empirical Studies Arts, 2001, 19: 177-190 CrossRef Google Scholar

[7] Barton J J S, Sekunova A, Sheldon C. Reading words, seeing style: The neuropsychology of word, font and handwriting perception. Neuropsychologia, 2010, 48: 3868-3877 CrossRef Google Scholar

[8] Nakatsuka M, Thabit M N, Koganemaru S. Writing's Shadow: Corticospinal Activation during Letter Observation. J Cognitive Neuroscience, 2012, 24: 1138-1148 CrossRef Google Scholar

[9] Li R, Qin R, Zhang J. The esthetic preference of Chinese typefaces - An event-related potential study. Brain Res, 2015, 1598: 57-65 CrossRef Google Scholar

[10] Liu L, Chen R, Wolf L. Optimizing Photo Composition. Comput Graphics Forum, 2010, 29: 469-478 CrossRef Google Scholar

[11] Redies C, Amirshahi S A, Koch M, et al. PHOG-derived aesthetic measures applied to color photographs of artworks, natural scenes and objects. In: Proceedings of European Conference on Computer Vision. Berlin: Springer, 2012. 522--531. Google Scholar

[12] Hagoort P. Integration of Word Meaning and World Knowledge in Language Comprehension. Science, 2004, 304: 438-441 CrossRef ADS Google Scholar

[13] Hellige J, Adamson M. Hemispheric differences in processing handwritten cursive. Brain Language, 2007, 102: 215-227 CrossRef Google Scholar

[14] Lu Y, Zheng W L, Li B, et al. Combining eye movements and EEG to enhance emotion recognition. In: Proceedings of the 24th International Joint Conference on Artificial Intelligence, 2015. 1170--1176. Google Scholar

[15] Zheng W L, Lu B L. Personalizing EEG-based affective models with transfer learning. In: Proceedings of the 25th International Joint Conference on Artificial Intelligence, 2016. 2732--2738. Google Scholar

[16] Stober S, Cameron D J, Grahn J A. Using convolutional neural networks to recognize rhythm stimuli from electroencephalography recordings. In: Proceedings of the Neural Information Processing Systems, 2014. 1449--1457. Google Scholar

[17] Schirrmeister R T, Springenberg J T, Fiederer L D J. Deep learning with convolutional neural networks for EEG decoding and visualization. Hum Brain Mapp, 2017, 38: 5391-5420 CrossRef Google Scholar

[18] Li Y, Zheng W, Cui Z, et al. A novel neural network model based on cerebral hemispheric asymmetry for EEG emotion recognition. In: Proceedings of the 27th International Joint Conference on Artificial Intelligence, 2018. 1561--1567. Google Scholar

[19] Bos D O. EEG-based emotion recognition. The Influence of Visual and Auditory Stimuli, 2006, 56(3): 1-17. Google Scholar

[20] Nie D, Wang X W, Shi L C, et al. EEG-based emotion recognition during watching movies. In: Proceedings of the 5th International IEEE/EMBS Conference on Neural Engineering, 2011. 667--670. Google Scholar

[21] Wang X W, Nie D, Lu B L. Emotional state classification from EEG data using machine learning approach. Neurocomputing, 2014, 129: 94-106 CrossRef Google Scholar

[22] Vidaurre C, Kr?mer N, Blankertz B. Time Domain Parameters as a feature for EEG-based Brain-Computer Interfaces. Neural Networks, 2009, 22: 1313-1319 CrossRef Google Scholar

[23] Jenke R, Peer A, Buss M. Feature Extraction and Selection for Emotion Recognition from EEG. IEEE Trans Affective Comput, 2014, 5: 327-339 CrossRef Google Scholar

[24] Xie Y, Tao D, Zhang W. On Unifying Multi-view Self-Representations for Clustering by Tensor Multi-rank Minimization. Int J Comput Vis, 2018, 126: 1157-1179 CrossRef Google Scholar

[25] Cristianini N, Shawe-Taylor J, Kandola J S. Spectral kernel methods for clustering. In: Proceedings of the Neural Information Processing Systems, 2002. 649--655. Google Scholar

[26] Hofmann T, Sch?lkopf B, Smola A J. Kernel methods in machine learning. Ann Statist, 2008, 36: 1171-1220 CrossRef Google Scholar

[27] Lin Z, Chen M, Ma Y. The augmented lagrange multiplier method for exact recovery of corrupted low-rank matrices. 2010,. arXiv Google Scholar

[28] Lin Z, Liu R, Su Z. Linearized alternating direction method with adaptive penalty for low-rank representation. In: Proceedings of the Neural Information Processing Systems, 2011. 612--620. Google Scholar

[29] Qu Y, Liu J, Xie Y, et al. Robust kernelized multi-view self-representations for clustering by tensor multi-rank minimization. 2017,. arXiv Google Scholar

[30] Li M, Lu B L. Emotion classification based on gamma-band EEG. In: Proceedings of the 2009 Annual International Conference of the IEEE Engineering in Medicine and Biology Society, 2009. 1223--1226. Google Scholar

[31] Ray W, Cole H. EEG Alpha Activity Reflects Attentional Demands, and Beta Activity Reflects Emotional and Cognitive Processes. Science, 1985, 228: 750-752 CrossRef ADS Google Scholar

  • Figure 1

    EEG cap layout for 62 electrodes

  • Figure 2

    (Color online) Overview of the proposed model. (a) EEG's feature extraction from five frequency bands ($\delta$: protectłinebreak 1$\sim$3 Hz, $\theta$: 4$\sim$7 Hz, $\alpha$: 8$\sim$13 Hz, $\beta$: 14$\sim$30 Hz, $\gamma$: 31$\sim$50 Hz) for multi-view representation; (b) multi-view self-representation in the kernel induced feature space and tensor construction by stacking the kernel multi-view self-representation; (c) visualization of brain correlation map through the input-perturbation correlation method to process the ERP-based feature matrix and cluster labels

  • Figure 3

    (Color online) Input-perturbation correlation map of ERP waves in terms of Dislike, Like and Neutral preference

  • Figure 4

    (Color online) The green color means this electrode correlates with like preference. The red color is more correlated with dislike preference and the blue one correlates with neutral preference

  • Figure 5

    (Color online) The grand-average ERP responses of selected electrodes from three esthetic preferences (like/dislike/neutral)

  • Table 1   Average recognition results for the combination of different views
    View$\rm{\delta~\theta}$ $\rm{\delta~\alpha}$$\rm{\delta~\beta}$ $\rm{\delta~\gamma}$ $\rm{\theta~\alpha}$ $\rm{\theta~\beta}$$\rm{\theta~\gamma}$ $\rm{\alpha~\beta}$$\rm{\alpha~\gamma}$$\rm{\beta~\gamma}$$\rm{\delta~\theta~\alpha}$$\rm{\delta~\theta\beta}$$\rm{\delta~\theta\gamma}$
    NMI 0.7590.8000.7100.7970.6810.6210.7340.5030.7490.6660.8910.7570.823
    ACC 0.8940.9380.8600.9340.8560.8350.9140.7700.9210.8690.9710.8840.945
    AR 0.7770.8330.7250.8280.7010.6460.7740.5230.7890.7000.9170.7710.856
    F-score 0.8510.8880.8160.8850.8000.7630.8490.6810.8590.7970.9440.8460.903
    Precision 0.8510.8870.8150.8840.7990.7610.8480.6800.8580.7960.9440.8460.904
    Recall 0.8500.8890.8170.8850.8010.7640.8500.6810.8600.7980.9430.8460.903
    View$\rm{\delta~\alpha\beta}$$\rm{\delta~\alpha\gamma}$$\rm{\delta~\beta~\gamma}$$\rm{\theta~\alpha~\beta}$$\rm{\theta~\alpha\gamma}$$\rm{\theta~\beta~\gamma}$$\rm{\alpha~\beta~\gamma}$ $\rm{\delta~\theta~\alpha~\beta}$$\rm{\delta~\theta~\alpha~\gamma}$$\rm{\delta~\theta~\beta~\gamma}$$\rm{\delta~\alpha~\beta\gamma}$$\rm{\theta~\alpha~\beta~\gamma}$$\rm{\delta~\theta~\alpha~\beta~\gamma}$
    NMI 0.797 0.8400.8080.7670.8040.7140.7390.8450.8580.8330.8420.7770.875
    ACC 0.9160.9500.9390.909 0.9380.8920.9060.9370.9570.9480.9520.9200.963
    AR 0.8140.868 0.8390.734 0.8350.7440.7740.8000.8840.8630.8700.8030.899
    F-score 0.8760.9110.8920.8620.8900.8290.8490.9060.9220.9090.9130.8680.930
    Precision 0.8760.9110.8920.861 0.8890.8280.8490.9050.9220.9080.9130.8680.932
    Recall 0.8760.9120.8930.862 0.8890.8300.8490.9070.9210.9090.9120.8690.931

    Algorithm 1 Input-perturbation correlation mapping algorithm

    Input: Multi-view feature matrices of ERP signal: ${\boldsymbol{X}}^{(1)},{\boldsymbol{X}}^{(2)},\ldots,{\boldsymbol{X}}^{(V)}$, $\lambda$, cluster number $c$, electrodes number $e$, sampling frequency $f$, the best-combined result of frequency bands and time of perturbation $K$; Output:correlation martix $C$

    Initialize ${\boldsymbol{Z}}^{(v)}={\boldsymbol{Y}}_v=0$, ${\boldsymbol{P}}^{(v)}=I_n$, $i=1,\ldots,V$; $\mathcal{G}=\mathcal{W}=0$;

    $\mu=\rho=10^{-5}$, $\mu_{\rm~max}=10^{13}$, $\varepsilon=10^{-7}$;

    while not converge do

    Update subproblem $\boldsymbol{Z}^{(v)}$, $\boldsymbol{P}$ and $\mathcal{G}$ by using (7), (8) and (11) according to [29];

    Obtain $\tilde{\mathcal{Z}}=\Phi({{\boldsymbol{Z}}^{(1)}},{{\boldsymbol{Z}}^{(2)}},\ldots,{{\boldsymbol{Z}}^{(V)}})$;

    Update $\boldsymbol{Y}^{(v)}$, $\mathcal{W}$ and parameters $\mu$, $\rho$ by using (12)$\sim$(15), respectively;

    end while

    Obtain the affinity matrix $\boldsymbol{A}$ by (16);

    Obtain cluster labels and feature matrix $U^{o}$ by applying the spectral clustering method with the affinity matrix $\boldsymbol{A}$, and obtain $U_o^*$ at the same time;

    for $k=1:K$

    Perturbing EEG signals by applying the randomly generated noise matrix $N_k$;

    Replacing the multi-view feature matrix of the ERP signal with the perturbed result, and using steps 1$\sim$8 to obtain the affinity matrix ${\boldsymbol~A}_k$;

    Obtain cluster labels and feature matrix $U_k$ by applying the spectral clustering method with the affinity matrix ${\boldsymbol~A}_k$, and obtain $U_k^*$ at the same time;

    Obtain $Q_k$ by using (18), and obtain $C_k$ by using (19);

    end for

    After taking the average of all $C_{k}$, and then superimpose it on all frequencies to generate the correlation matrix $C$;

    return correlation matrix $C$.

  • Table 2   Average recognition results for single band
    View$\rm{\delta}$ $\rm{\theta~}$ $\rm{\alpha}$$\rm{\beta}$ $\rm{\gamma}$
    NMI 0.2020.1990.1990.2120.220
    ACC 0.5570.5610.5700.5710.585
    AR 0.1240.1300.1330.1380.157
    F-score 0.4720.4720.4690.4710.487
    Precision 0.3920.3960.3990.4020.416
    Recall 0.5960.5870.5730.5730.579
  • Table 3   Average recognition results for the combination of different features
    View$\rm{TDP~}$ $\rm{BP~}$ $\rm{TDP~\&~BP}$
    NMI 0.2200.2270.920
    ACC 0.5970.5730.979
    AR 0.1910.1450.939
    F-score 0.4880.4760.959
    Precision 0.4440.4060.959
    Recall 0.5490.5790.958
  • Table 4   Classification accuracies of SVM for different electrode sets with 10-fold cross-validation
    Selected electrode numbersACC (BP) (%)ACC (TDP) (%)
    62 $74.71$$70.25$
    12 $79.13$$84.58$
    9 $84.13$$83.50$
    6 $80.29$$82.83$
    3 $\textbf{89.17}$$\textbf{89.16}$