With Flexpad, paper, plastic film or another flexible material is transformed into a computer input device and moveable display.
Recently at the 2013 IFA international trade show for consumer electronics and home appliances in Berlin, major electronics manufacturers displayed new types of displays that are thin, and even curved, but expensive. IT experts in Saarbrücken have gone a step further. Their more cost-effective approach, called Flexpad, allows a simple, standard sheet of paper to be transformed into a moveable, flexible display. Already today, this could help patients better review the results of a computer tomography, for example. In the long term, the IT experts want to discover what new applications are viable in future for ultra-thin, deformable, mobile end devices, and how they can best be operated.
Human organs shimmer in red on a sheet of paper displaying a longitudinal view of the human abdomen. The spinal column and pelvic bones form contrasting yellow islands. As the sheet of paper is bent downwards at the ends, the bones appear to come into the foreground while the soft tissue recedes (see video). What appears to be science fiction at first glance, is the result of the “Flexpad” research project developed under the leadership of Jürgen Steimle in the Media Lab at the Massachusetts Institute of Technology in the US and the Max Planck Institute for Informatics in Saarbrücken, in cooperation with Kiel University. In the meantime, Steimle heads the Embodied Interaction research group at the Multimodal Computing and Interaction Cluster of Excellence.
“We routinely deform objects intuitively in many different ways. We bend back pages in books, deflate balls, fold paper, and sculpt clay”, explains Jürgen Steimle. “And by projecting user interface elements onto tangible, deformable objects we can control computers and other technical devices better and more easily.”
Flexpad thereby works as follows: The motion sensor records the user and the paper, capturing the paper’s deformation and movement. So that the recording takes place precisely and in real time despite the rather coarse image data from the Kinect camera, the researchers have developed and implemented two algorithms. The first initially subtracts out the interference caused by the fingers and hands of the user. If the user moves the paper - whether left or right, or bends it into an arc or wavy form - the camera senses this. A specially developed computer model subsequently describes these movements in fractions of a second, so that the projector can reproduce it on the sheet in near real time.
Nevertheless, Flexpad has certain limits: The user must stand in a particular area under the camera and projector for the system to work properly. Therefore, the user cannot move freely around the room.
“The paper takes on two simultaneous functions in our system”, explains Steimle. “It is input device and display at the same time.” The user can interact with the device in a similar way to using a mouse for controlling a computer. Other materials besides paper are also suitable, for example sheets of plastic or other deformable materials. The only important thing is that they possess a certain malleability and flexibility.