Microsoft Touch Screen - Surgery VR Potential

There has been a lot in the lay press about microsoft’s new touch tablet display.  As CNET writes (video link here):

Microsoft has just announced its Surface Computing technology, a project that has been kept under wraps for five years. Using a giant table-like display, users are able to draw, interact with media, and use another new technology called domino tagging, in which a real-life object on the computer’s surface is identified and becomes an on-screen object that can be interacted with

What is unique about the screen is that it is a collaborative touch table interface.  That is, multiple users can interact and manipulate objects.  While they claim Microsoft has kept this under wraps for 5 years it is not a new concept at all.  Heck, even Minority Report ( my surgical version here) used this concept in its image manipulation scenes. 

Last year at Nextfest GE showed a similar protoype and I had an opportunity to speak with the engineer about potential medical applications (he would not let me take a photo of the table they had).  It’s best use to date in their mind was for collaborative radiology.  However a VERY limited market!  I can only see that being used for teaching sessions for trainees (very good in that setting) or for multi-speciality team surgery planning for complex cases (ie conjoined twins etc).    Much larger uses will be in military and building and geology.

Could this be a prototype of a gesture control interface for surgery?  Noir for real time but more so for planning.  I had an extensive discussion of these systems in a late night conference call With Andy Van Dam (you know Computer Science Professor at Brown, Founder of SIGGRAPH, namesake of Andy in Toy Story, invetor of VR and the military Cave technology).  Andy feels 100% that surgery should be immersive - that we will be in the image and use gestures to move the images around us.  Others disagreee 100%.  Many of the the old CAVE researchers (see imersive VR info here)are abandoning that approach for this table like interface.  Here you look into the 3-D space and move the object in 3D but NOT immersed.  See how much immersion is enough?  When I look at a 3D VR travel down the bowel do I want it to look like I am inside the bowel fantastic Voyage style (Andy’s approach) or do I want to stretch it out, slice it open and manipulate the 3D image in front of me twisting and turning it in space (new method)?

Much more on our discussion and the surgical options to follow.