logo

SCIENTIA SINICA Informationis, Volume 48 , Issue 4 : 390-405(2018) https://doi.org/10.1360/N112017-00227

Tangible user interface: origins, development, and future trends

More info
  • ReceivedNov 6, 2017
  • AcceptedFeb 7, 2018
  • PublishedApr 9, 2018

Abstract


Funded by

国家重点研发计划(2016YFB1001402)

国家自然科学基金重点项目(61232013)

国家自然科学基金青年项目(61402250)


Acknowledgment

戴国忠研究员为促成本文成稿做了许多工作, 特此致谢. 姚远, 麦龙辉完成了本文诸多图片绘制工作. 路奇为本文排版做了大量工作. 在此一并致谢.


References

[1] Frazer J, Frazer P. Intelligent physical three-dimensional modelling systems. Comput Graph, 1980, 80: 359--370. Google Scholar

[2] Perlman R. Using computer technology to provide a creative learning environment for preschool children. MIT Logo Memo, 1976. Google Scholar

[3] Abrams R. Adventures in tangible computing: the work of interaction designer “Durrell bishop" in context. South Asia Res, 1999, 28: 23--48. Google Scholar

[4] Fitzmaurice G W, Ishii H, Buxton W A S. Bricks: laying the foundations for graspable user interfaces. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, Denver, 1995. 442--449. Google Scholar

[5] Ishii H, Ullmer B. Tangible bits: towards seamless interfaces between people, bits and atoms. In: Proceedings of the ACM SIGCHI Conference on Human Factors in Computing Systems, Atlanta, 1997. 234--241. Google Scholar

[6] Ullmer B, Ishii H. The metaDESK: models and prototypes for tangible user interface. In: Proceedings of the 10th Annual ACM Symposium on User Interface Software and Technology, Banff, 1997. 223--232. Google Scholar

[7] Ishii H, Ratti C, Piper B. Bringing Clay and Sand into Digital Design - Continuous Tangible user Interfaces. BT Tech J, 2004, 22: 287-299 CrossRef Google Scholar

[8] Weiss M, Jennings R, Khoshabeh R, et al. SLAP widgets: bridging the gap between virtual and physical controls on tabletops. In: Proceedings of CHI'09 Extended Abstracts on Human Factors in Computing Systems, Boston, 2009. 481--490. Google Scholar

[9] Pangaro G, Maynes-Aminzade D, Ishii H. The actuated workbench: computer-controlled actuation in tabletop tangible interfaces. ACM Trans Graph, 2003, 22: 181--190. Google Scholar

[10] Pedersen E W, Hornbæ k K. Tangible bots: interaction with active tangibles in tabletop interfaces. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, Vancouver, 2011. 2975--2984. Google Scholar

[11] Le Goc M, Kim L H, Parsaei A, et al. Zooids: building blocks for swarm user interfaces. In: Proceedings of the 29th Annual Symposium on User Interface Software and Technology, Tokyo, 2016. 97--109. Google Scholar

[12] Raffle H S, Parkes A J, Ishii H. Topobo: a constructive assembly system with kinetic memory. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, Vienna, 2004. 647--654. Google Scholar

[13] Baudisch P, Becker T, Rudeck F. Lumino: tangible building blocks based on glass fiber bundles. In: Proceedings of ACM SIGGRAPH 2010 Emerging Technologies, Los Angeles, 2010. Google Scholar

[14] Wang M, Mi H, Xu Y Q. TwistBlocks: non-rigid interface for 3D-aware design and actuation. In: Proceedings of China Human Computer Interaction Conference, Shanghai, 2016. Google Scholar

[15] Holman D, Vertegaal R. Organic user interfaces: designing computers in any way, shape, or form. Commun ACM Org User Interface, 2008, 51: 48--55. Google Scholar

[16] Minuto A, Vyas D, Poelman W, et al. Smart material interfaces: a vision. In: Proceedings of International Conference on Intelligent Technologies for Interactive Entertainment, Genoa, 2011. 57--62. Google Scholar

[17] Yao L N, Niiyama R, Ou J F, et al. PneUI: pneumatically actuated soft composite materials for shape changing interfaces. In: Proceedings of the 26th Annual ACM Symposium on User Interface Software and Technology, St. Andrews, 2013. 13--22. Google Scholar

[18] Follmer S, Leithinger D, Olwal A, et al. inFORM: dynamic physical affordances and constraints through shape and object actuation. In: Proceedings of the 26th Annual ACM Symposium on User Interface Software and Technology, St. Andrews, 2013. 417--426. Google Scholar

[19] Lu Q Y, Mao C P, Wang L Y, et al. LIME: liquid metal interfaces for non-rigid interaction. In: Proceedings of the 29th Annual Symposium on User Interface Software and Technology, Tokyo, 2016. 449--452. Google Scholar

[20] Ullmer B, Ishii H. Emerging frameworks for tangible user interfaces. IBM Syst J, 2000, 39: 915-931 CrossRef Google Scholar

[21] Mi H. Actuated tangible user interface: an extensible method for tabletop interaction. Dissertation for Ph.D. Degree. Tokyo: University of Tokyo, 2011. Google Scholar

[22] Ishii H, Lakatos D, Bonanni L, et al. Radical atoms: beyond tangible bits, toward transformable materials. Interactions, 2012, 19: 38--51. Google Scholar

[23] Ullmer B, Ishii H, Jacob R J K. Token+constraint systems for tangible interaction with digital information. ACM Trans Comput-Hum Interact, 2005, 12: 81-118 CrossRef Google Scholar

[24] Fishkin K. A taxonomy for and analysis of tangible interfaces. Pers Ubiquit Comput, 2004, 8: 347--358. Google Scholar

[25] Jansen Y, Karrer T, Borchers J. MudPad: tactile feedback and haptic texture overlay for touch surfaces. In: Proceedings of ACM International Conference on Interactive Tabletops and Surfaces, Saarbrücken, 2010. 4351--4356. Google Scholar

[26] Yao L N, Ou J F, Cheng C Y, et al. BioLogic: natto cells as nanoactuators for shape changing interfaces. In: Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, Seoul, 2015. 1--10. Google Scholar

[27] Probst K, Seifried T, Haller M, et al. Move-it: interactive sticky notes actuated by shape memory alloys. In: Proceedings of CHI'11 Extended Abstracts on Human Factors in Computing Systems, Vancouver, 2011. 1393--1398. Google Scholar

[28] Ogata M, Fukumoto M. FluxPaper: reinventing paper with dynamic actuation powered by magnetic flux. In: Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, Seoul, 2015. 29--38. Google Scholar

[29] Ono M, Shizuki B, Tanaka J. Touch & activate: adding interactivity to existing objects using active acoustic sensing. In: Proceedings of the 26th Annual ACM Symposium on User Interface Software and Technology, St. Andrews, 2013. 31--40. Google Scholar

[30] Li H, Ye C, Sample A P. IDSense: a human object interaction detection system based on passive UHF RFID. In: Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, Seoul, 2015. 2555--2564. Google Scholar

[31] Ou J F, Yao L N, Tauber D, et al. jamSheets: thin interfaces with tunable stiffness enabled by layer jamming. In: Proceedings of the 8th International Conference on Tangible, Embedded and Embodied Interaction, Munich, 2014. 65--72. Google Scholar

[32] Jiao Y, Gong J, Xu Y Q. Graille: design research of graphical tactile display for the visually impaired (in Chinese). Art Des, 2016, 01: 94--96. Google Scholar

[33] O'Malley C, Fraser D S. Literature review in learning with tangible technologies. NESTA Futurelab Report 12, 2004. https://www.nfer.ac.uk/publications/FUTL69/FUTL69.pdf. Google Scholar

[34] Antle A N. The CTI framework: informing the design of tangible systems for children. In: Proceedings of the 1st International Conference on Tangible and Embedded Interaction, Baton Rouge, 2007. 195--202. Google Scholar

[35] Kaltenbrunner M, Bencina R. reacTIVision: a computer-vision framework for table-based tangible interaction. In: Proceedings of the 1st International Conference on Tangible and Embedded Interaction, Baton Rouge, 2007. 69--74. Google Scholar

[36] Wang M, Mi H, Liu Y, et al. PolyHinge: shape changing TUI on tabletop. In: Proceedings of the 8th Conference on Interfaces and Human Computer Interaction, Lisbon, 2017. Google Scholar

[37] Perelman G, Serrano M, Raynal M, et al. The roly-poly mouse: designing a rolling input device unifying 2D and 3D interaction. In: Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, Seoul, 2015. 327--336. Google Scholar

[38] Nakagaki K, Follmer S, Ishii H. LineFORM: actuated curve interfaces for display, interaction, and constraint. In: Proceedings of the 28th Annual ACM Symposium on User Interface Software & Technology, Charlotte, 2015. 333--339. Google Scholar

[39] Shaer O, Horn M S, Jacob R J K. Tangible user interface laboratory: Teaching tangible interaction design in practice. AIEDAM, 2009, 23: 251-261 CrossRef Google Scholar

  • Figure 1

    (a) A Roman abacus and (b) a Chinese abacus

  • Figure 2

    (a) Frazer's 3D manipulating system and (b) Bishop's marble answering machine

  • Figure 3

    MCRit model

  • Figure 4

    ATUI model

  • Figure 5

    From GUI to Radical atoms

  • Figure 6

    Classifications of passive TUIs

  • Figure 7

    Classifications of new TUIs

  • Table 1   Tangible user interface: origins and development
    特征 代表性工作 特点 发表年份
    早期先驱工作 Slot machine [1] 使用物理卡片进行编程 1976
    1970s–1997 早期萌芽 Marble answering machine [2] 使用物理实体指代信息 1995
    Graspable interface [3] 可抓握界面 1995
    metaDesk [4] 桌面上的实物控件操作 1997
    无源 SandScape [5] 柔性任意塑形的信息耦合呈现 2002
    SLAP widget 桌面上较完备的实物操作组件 2009
    Tangible bits Topobo [6] 拼插结构与动作记忆 2004
    组装 Lumino 光纤束结构实现无源多模块组装 2010
    1997– TwistBlock [7] 传感器网络结构实现复杂组装 2017
    Actuated workbench [8] 电磁阵列实现多物体动态反馈 2002
    自驱动 Tangbile bots [9] 小机器人实现复杂动态反馈 2011
    zooids [10] 实现动态反馈的微型机器人平台 2017
    PneUI [11] 气动控制形状改变 2013
    Radical atoms 物理属性 inForm [12] 垂直运动阵列实现复杂表面形态变化 2013
    bioLogic [13] 生物材料实现形状改变 2015
    2012– 形状改变 LIME [14] 液态材料实现物理特征及形状改变 2016
    Programmable droplets$^{1)}$ 水滴受控运动及变形 2018

    1) http://tangible.media.mit.edu/project/programmable-droplets/.