Hacking the holodeck: an introduction to Touch and Virtual reality


Prerequisites: Students should have either a working knowledge of programming, electronics, or 3D modelling. Additional experience with Unity and Arduino is a bonus. Students will work in small cross-disciplinary groups and will be provided with a basic software toolchain and hardware on which to develop their projects.

Load: 10AM to noon Jan. 9-13 and 17-20 plus significant time outside of class.

Time Outside Class: 2 to 4 hours daily outside class

Special Instructions: Students are required to bring their own laptop with Unity and the Arduino IDE installed.


In 1965, Ivan Sutherland proposed the vision of an “Ultimate Display”: a room that could render data so realistically that it would allow users to interact with information as if it were a real, physical object. Since then, the idea of such an environment has become increasingly popular and has captured people’s imagination through the Star Trek Holodeck, The Matrix, and most recently, Westworld.

Virtual and Augmented Reality interfaces have brought us closer this vision, but current technologies still often lack a key ingredient for building this ultimate display: the ability to physically touch information.

In this course, students will learn how to build immersive computer interfaces that allow us to feel and touch data in real life. Building on the rich history of VR at MIT (and beyond), we will explore the state of the art in shape changing displays, soft robotics, haptic interfaces, and wearables, while combining them with virtual reality interfaces such as the HTC Vive and Google Cardboard.

Through this hybrid lecture and studio workshop students will learn how to create physical user interfaces and haptics for virtual reality. Hands-on experience with common VR devices and prototyping tools such as Unity and Arduino will introduce students to current state of the art research and provide a platform for development and experimentation. Bridging design and creative disciplines, such as product design, fashion, architecture, graphic design, and filmmaking, students will ideate and prototype new computer interfaces as teams, learning to cohesively blend virtual and physical realities.

Students should have either a working knowledge of programming, electronics, or 3D modelling. Additional experience with Unity and Arduino is a bonus. Students will work in small cross-disciplinary groups and will be provided with a basic software toolchain and hardware on which to develop their projects.