SCIENTIA SINICA Informationis, Volume 51 , Issue 3 : 383(2021) https://doi.org/10.1360/SSI-2020-0234

## Esthetic preference mining of Chinese typefaces using multiview cluster analysis

• AcceptedSep 16, 2020
• PublishedFeb 23, 2021
Share
Rating

### References

[1] Zhang J. A survey of digital calligraphy. Sci Sin-Inf, 2019, 49: 143-158 CrossRef Google Scholar

[2] Bargh J A, Ferguson M J. Beyond behaviorism: On the automaticity of higher mental processes.. Psychological Bull, 2000, 126: 925-945 CrossRef Google Scholar

[3] H?fel L, Jacobsen T. Electrophysiological indices of processing aesthetics: Spontaneous or intentional processes?. Int J PsychoPhysiol, 2007, 65: 20-31 CrossRef Google Scholar

[4] Handy T C, Smilek D, Geiger L. ERP Evidence for Rapid Hedonic Evaluation of Logos. J Cognitive Neuroscience, 2010, 22: 124-138 CrossRef Google Scholar

[5] Sutton S, Tueting P, Zubin J. Information Delivery and the Sensory Evoked Potential. Science, 1967, 155: 1436-1439 CrossRef ADS Google Scholar

[6] Jacobsen T, H?fel L. Aesthetics Electrified: An Analysis of Descriptive Symmetry and Evaluative Aesthetic Judgment Processes Using Event-Related Brain Potentials. Empirical Studies Arts, 2001, 19: 177-190 CrossRef Google Scholar

[7] Barton J J S, Sekunova A, Sheldon C. Reading words, seeing style: The neuropsychology of word, font and handwriting perception. Neuropsychologia, 2010, 48: 3868-3877 CrossRef Google Scholar

[8] Nakatsuka M, Thabit M N, Koganemaru S. Writing's Shadow: Corticospinal Activation during Letter Observation. J Cognitive Neuroscience, 2012, 24: 1138-1148 CrossRef Google Scholar

[9] Li R, Qin R, Zhang J. The esthetic preference of Chinese typefaces - An event-related potential study. Brain Res, 2015, 1598: 57-65 CrossRef Google Scholar

[10] Liu L, Chen R, Wolf L. Optimizing Photo Composition. Comput Graphics Forum, 2010, 29: 469-478 CrossRef Google Scholar

[11] Redies C, Amirshahi S A, Koch M, et al. PHOG-derived aesthetic measures applied to color photographs of artworks, natural scenes and objects. In: Proceedings of European Conference on Computer Vision. Berlin: Springer, 2012. 522--531. Google Scholar

[12] Hagoort P. Integration of Word Meaning and World Knowledge in Language Comprehension. Science, 2004, 304: 438-441 CrossRef ADS Google Scholar

[13] Hellige J, Adamson M. Hemispheric differences in processing handwritten cursive. Brain Language, 2007, 102: 215-227 CrossRef Google Scholar

[14] Lu Y, Zheng W L, Li B, et al. Combining eye movements and EEG to enhance emotion recognition. In: Proceedings of the 24th International Joint Conference on Artificial Intelligence, 2015. 1170--1176. Google Scholar

[15] Zheng W L, Lu B L. Personalizing EEG-based affective models with transfer learning. In: Proceedings of the 25th International Joint Conference on Artificial Intelligence, 2016. 2732--2738. Google Scholar

[16] Stober S, Cameron D J, Grahn J A. Using convolutional neural networks to recognize rhythm stimuli from electroencephalography recordings. In: Proceedings of the Neural Information Processing Systems, 2014. 1449--1457. Google Scholar

[17] Schirrmeister R T, Springenberg J T, Fiederer L D J. Deep learning with convolutional neural networks for EEG decoding and visualization. Hum Brain Mapp, 2017, 38: 5391-5420 CrossRef Google Scholar

[18] Li Y, Zheng W, Cui Z, et al. A novel neural network model based on cerebral hemispheric asymmetry for EEG emotion recognition. In: Proceedings of the 27th International Joint Conference on Artificial Intelligence, 2018. 1561--1567. Google Scholar

[19] Bos D O. EEG-based emotion recognition. The Influence of Visual and Auditory Stimuli, 2006, 56(3): 1-17. Google Scholar

[20] Nie D, Wang X W, Shi L C, et al. EEG-based emotion recognition during watching movies. In: Proceedings of the 5th International IEEE/EMBS Conference on Neural Engineering, 2011. 667--670. Google Scholar

[21] Wang X W, Nie D, Lu B L. Emotional state classification from EEG data using machine learning approach. Neurocomputing, 2014, 129: 94-106 CrossRef Google Scholar

[22] Vidaurre C, Kr?mer N, Blankertz B. Time Domain Parameters as a feature for EEG-based Brain-Computer Interfaces. Neural Networks, 2009, 22: 1313-1319 CrossRef Google Scholar

[23] Jenke R, Peer A, Buss M. Feature Extraction and Selection for Emotion Recognition from EEG. IEEE Trans Affective Comput, 2014, 5: 327-339 CrossRef Google Scholar

[24] Xie Y, Tao D, Zhang W. On Unifying Multi-view Self-Representations for Clustering by Tensor Multi-rank Minimization. Int J Comput Vis, 2018, 126: 1157-1179 CrossRef Google Scholar

[25] Cristianini N, Shawe-Taylor J, Kandola J S. Spectral kernel methods for clustering. In: Proceedings of the Neural Information Processing Systems, 2002. 649--655. Google Scholar

[26] Hofmann T, Sch?lkopf B, Smola A J. Kernel methods in machine learning. Ann Statist, 2008, 36: 1171-1220 CrossRef Google Scholar

[27] Lin Z, Chen M, Ma Y. The augmented lagrange multiplier method for exact recovery of corrupted low-rank matrices. 2010,. arXiv Google Scholar

[28] Lin Z, Liu R, Su Z. Linearized alternating direction method with adaptive penalty for low-rank representation. In: Proceedings of the Neural Information Processing Systems, 2011. 612--620. Google Scholar

[29] Qu Y, Liu J, Xie Y, et al. Robust kernelized multi-view self-representations for clustering by tensor multi-rank minimization. 2017,. arXiv Google Scholar

[30] Li M, Lu B L. Emotion classification based on gamma-band EEG. In: Proceedings of the 2009 Annual International Conference of the IEEE Engineering in Medicine and Biology Society, 2009. 1223--1226. Google Scholar

[31] Ray W, Cole H. EEG Alpha Activity Reflects Attentional Demands, and Beta Activity Reflects Emotional and Cognitive Processes. Science, 1985, 228: 750-752 CrossRef ADS Google Scholar

• Figure 1

EEG cap layout for 62 electrodes

• Figure 2

(Color online) Overview of the proposed model. (a) EEG's feature extraction from five frequency bands ($\delta$: protectłinebreak 1$\sim$3 Hz, $\theta$: 4$\sim$7 Hz, $\alpha$: 8$\sim$13 Hz, $\beta$: 14$\sim$30 Hz, $\gamma$: 31$\sim$50 Hz) for multi-view representation; (b) multi-view self-representation in the kernel induced feature space and tensor construction by stacking the kernel multi-view self-representation; (c) visualization of brain correlation map through the input-perturbation correlation method to process the ERP-based feature matrix and cluster labels

• Figure 3

(Color online) Input-perturbation correlation map of ERP waves in terms of Dislike, Like and Neutral preference

• Figure 4

(Color online) The green color means this electrode correlates with like preference. The red color is more correlated with dislike preference and the blue one correlates with neutral preference

• Figure 5

(Color online) The grand-average ERP responses of selected electrodes from three esthetic preferences (like/dislike/neutral)

• Table 1   Average recognition results for the combination of different views
 View $\rm{\delta~\theta}$ $\rm{\delta~\alpha}$ $\rm{\delta~\beta}$ $\rm{\delta~\gamma}$ $\rm{\theta~\alpha}$ $\rm{\theta~\beta}$ $\rm{\theta~\gamma}$ $\rm{\alpha~\beta}$ $\rm{\alpha~\gamma}$ $\rm{\beta~\gamma}$ $\rm{\delta~\theta~\alpha}$ $\rm{\delta~\theta\beta}$ $\rm{\delta~\theta\gamma}$ NMI 0.759 0.800 0.710 0.797 0.681 0.621 0.734 0.503 0.749 0.666 0.891 0.757 0.823 ACC 0.894 0.938 0.860 0.934 0.856 0.835 0.914 0.770 0.921 0.869 0.971 0.884 0.945 AR 0.777 0.833 0.725 0.828 0.701 0.646 0.774 0.523 0.789 0.700 0.917 0.771 0.856 F-score 0.851 0.888 0.816 0.885 0.800 0.763 0.849 0.681 0.859 0.797 0.944 0.846 0.903 Precision 0.851 0.887 0.815 0.884 0.799 0.761 0.848 0.680 0.858 0.796 0.944 0.846 0.904 Recall 0.850 0.889 0.817 0.885 0.801 0.764 0.850 0.681 0.860 0.798 0.943 0.846 0.903 View $\rm{\delta~\alpha\beta}$ $\rm{\delta~\alpha\gamma}$ $\rm{\delta~\beta~\gamma}$ $\rm{\theta~\alpha~\beta}$ $\rm{\theta~\alpha\gamma}$ $\rm{\theta~\beta~\gamma}$ $\rm{\alpha~\beta~\gamma}$ $\rm{\delta~\theta~\alpha~\beta}$ $\rm{\delta~\theta~\alpha~\gamma}$ $\rm{\delta~\theta~\beta~\gamma}$ $\rm{\delta~\alpha~\beta\gamma}$ $\rm{\theta~\alpha~\beta~\gamma}$ $\rm{\delta~\theta~\alpha~\beta~\gamma}$ NMI 0.797 0.840 0.808 0.767 0.804 0.714 0.739 0.845 0.858 0.833 0.842 0.777 0.875 ACC 0.916 0.950 0.939 0.909 0.938 0.892 0.906 0.937 0.957 0.948 0.952 0.920 0.963 AR 0.814 0.868 0.839 0.734 0.835 0.744 0.774 0.800 0.884 0.863 0.870 0.803 0.899 F-score 0.876 0.911 0.892 0.862 0.890 0.829 0.849 0.906 0.922 0.909 0.913 0.868 0.930 Precision 0.876 0.911 0.892 0.861 0.889 0.828 0.849 0.905 0.922 0.908 0.913 0.868 0.932 Recall 0.876 0.912 0.893 0.862 0.889 0.830 0.849 0.907 0.921 0.909 0.912 0.869 0.931
•

Algorithm 1 Input-perturbation correlation mapping algorithm

Input: Multi-view feature matrices of ERP signal: ${\boldsymbol{X}}^{(1)},{\boldsymbol{X}}^{(2)},\ldots,{\boldsymbol{X}}^{(V)}$, $\lambda$, cluster number $c$, electrodes number $e$, sampling frequency $f$, the best-combined result of frequency bands and time of perturbation $K$; Output:correlation martix $C$

Initialize ${\boldsymbol{Z}}^{(v)}={\boldsymbol{Y}}_v=0$, ${\boldsymbol{P}}^{(v)}=I_n$, $i=1,\ldots,V$; $\mathcal{G}=\mathcal{W}=0$;

$\mu=\rho=10^{-5}$, $\mu_{\rm~max}=10^{13}$, $\varepsilon=10^{-7}$;

while not converge do

Update subproblem $\boldsymbol{Z}^{(v)}$, $\boldsymbol{P}$ and $\mathcal{G}$ by using (7), (8) and (11) according to [29];

Obtain $\tilde{\mathcal{Z}}=\Phi({{\boldsymbol{Z}}^{(1)}},{{\boldsymbol{Z}}^{(2)}},\ldots,{{\boldsymbol{Z}}^{(V)}})$;

Update $\boldsymbol{Y}^{(v)}$, $\mathcal{W}$ and parameters $\mu$, $\rho$ by using (12)$\sim$(15), respectively;

end while

Obtain the affinity matrix $\boldsymbol{A}$ by (16);

Obtain cluster labels and feature matrix $U^{o}$ by applying the spectral clustering method with the affinity matrix $\boldsymbol{A}$, and obtain $U_o^*$ at the same time;

for $k=1:K$

Perturbing EEG signals by applying the randomly generated noise matrix $N_k$;

Replacing the multi-view feature matrix of the ERP signal with the perturbed result, and using steps 1$\sim$8 to obtain the affinity matrix ${\boldsymbol~A}_k$;

Obtain cluster labels and feature matrix $U_k$ by applying the spectral clustering method with the affinity matrix ${\boldsymbol~A}_k$, and obtain $U_k^*$ at the same time;

Obtain $Q_k$ by using (18), and obtain $C_k$ by using (19);

end for

After taking the average of all $C_{k}$, and then superimpose it on all frequencies to generate the correlation matrix $C$;

return correlation matrix $C$.

• Table 2   Average recognition results for single band
 View $\rm{\delta}$ $\rm{\theta~}$ $\rm{\alpha}$ $\rm{\beta}$ $\rm{\gamma}$ NMI 0.202 0.199 0.199 0.212 0.220 ACC 0.557 0.561 0.570 0.571 0.585 AR 0.124 0.130 0.133 0.138 0.157 F-score 0.472 0.472 0.469 0.471 0.487 Precision 0.392 0.396 0.399 0.402 0.416 Recall 0.596 0.587 0.573 0.573 0.579
• Table 3   Average recognition results for the combination of different features
 View $\rm{TDP~}$ $\rm{BP~}$ $\rm{TDP~\&~BP}$ NMI 0.220 0.227 0.920 ACC 0.597 0.573 0.979 AR 0.191 0.145 0.939 F-score 0.488 0.476 0.959 Precision 0.444 0.406 0.959 Recall 0.549 0.579 0.958
• Table 4   Classification accuracies of SVM for different electrode sets with 10-fold cross-validation
 Selected electrode numbers ACC (BP) (%) ACC (TDP) (%) 62 $74.71$ $70.25$ 12 $79.13$ $84.58$ 9 $84.13$ $83.50$ 6 $80.29$ $82.83$ 3 $\textbf{89.17}$ $\textbf{89.16}$

Citations

Altmetric