New 3D Display Technology

philips-3d.bmpthis sure beats the old system!

3d_glasses.jpgBetter for Creature Features than the OR

Phillips just demo’d an intriguing display at the Berlin consumer-electronics show. It is an amalgam of 9 x 42-inch displays on a grid creating a 132 inch display that reportedly can display 3D images without the need for glasses. 

Why this is so important: 3D display technology is badly needed for endoscopic surgery. In order to see in 3D you need stereo vision which requires 2 separate images taken from slighly different angles and them superimposed.  You body does this with your 2 eyes slighly separate on your face.  In traditional laparoscopic surgery there is a single telescope and a single camera so all the images are in 2D.  Unfortunately, depth perception is lost.  How does the surgen operate then?  What heppens with training and practice is that your brain picks up and other clues primarily shadowing and touch perception from your hands and the surgeon becomes able to interpolate a 3D space even though all of the visual skills are mising.  This is one of the hardest if not the hardest step to learn when I teach surgeons to first perform laparoscopic surgery and some people just have a much harder time than others.  Interestingly, with HD displays there is a pseudo-enhancement of depth perception that engineers and visual scientists tell me is due to the enhanced color fidelity and resolution and shadowing which allows the brain to pick up more 3D clues of the space from the 2D image!  Still, the lack of true 3D data increases the difficulty of the procedures especially complex ones requiring suturing. 

What is available today:  Currently there are some attempts to address this limitation.  They have required the use of head mounted displays with separate displays for each eye and separate imaging chips or lenses on the scopes but these have been heavy and cumbersome to use.  Others such as some of the robotic solutions have immersed the surgeon’s head in a remote 2-panel display station but this also is a very complex solution.  For years I have seen many many attempts at no-glasses 3D displays from various companies but all suffered from narrow viewing angles or poor resolution or other design issues.

2dpd.jpg 2d

How this solution works.  This is a display technology that they call 2D + depth.  In order to generate a 3D image, the display requires a regular 2D representation of the image and a depth-map. This depth-map indicates the distance between each pixel and the viewer. The 2D image and the depth-map are used to create images on the screen, and these images are then merged by the viewer’s brain into a 3D sensation.

Lenticular Screen:  The system works with lenses on the screen that provide a slightly different view for each eye (without the red-green glasses of the 50′s).  A sheet of transparent lenses, is fixed on an LCD screen. This sheet sends different images to each eye, and so a person sees two images. These two images are combined by our brain, to create a 3D effect.

lenticular.jpg

I’ll have to get ahold of one of these displays to see if it holds promise for the OR…

Share

Docinthemachine Research Featured on 20/20! MedTechno Insights From the Day

2020logo.jpg

I previously wrote about the upcoming National Geographic Special Inside the Living Body and my work featured in the special. I was delighted when the producers of 20/20 called to request an interview with me on my research featured on the show and my vision of the future technological transformation of medicine.   Bob Brown was interested in coming to interview me.  They have already posted a description of the upcoming interview and a summary of the show. 

They call it an “Unprecedented Journey Inside the Living Body- ‘We’re Seeing Things That We Had Never Seen Before,‘ Says Scientist (that’s me).

On their website they write:

Recent technological advances have allowed for such dramatic and amazing views of the inside of our bodies that watching the footage can feel like you’re in a science fiction film or on an imaginary expedition…In such a science fiction journey, the 1966 film “Fantastic Voyage,” a group of scientists and their submarine were miniaturized so they could be injected into a body in order to eliminate an otherwise unreachable brain clot.

“I use clips from that movie when I lecture about these new technologies,” said Dr. Steven Palter, the medical and scientific director of Gold Coast IVF in Syosset, N.Y. “Now, physicians can actually see the workings of the body and understand it in a way that they never could before.”

Palter, who has a medical technology blog called docinthemachine.com, is a pioneer of methods capable of producing spectacular high-definition surgical images.  Palter obtained his footage by advancing well-established procedures that allow doctors to insert cameras through small incisions and view the target areas of their surgeries. He successfully hooked up high-definition cameras and, he said, was awestruck by the results.

“With high definition, we’re seeing things that we had never seen before … with depth perception, clarity and detail … because now it’s enormously clear and magnified. We have views that you don’t get with your naked eye.”

They also write about my autofluorescent laparoscopy research: “New Way of Seeing Ourselves”

The technology used for the National Geographic Channel is also clearly on its way to helping revolutionize medical care. Palter contributed to the development of what’s called an auto-fluorescent laparoscope, which exposes diseased tissue inside the body that a surgeon couldn’t otherwise see.

“Instead of using visible light, it makes the disease fluoresce,” Palter said. “If you look with your naked eye, you see nothing. When you switch on the light and the filters, all of a sudden the disease is glowing green, and you can see disease that’s beyond the resolution of your naked eye.”

setup.jpgThe setup

Details and Insights from the Interview: It really was an amazing morning.  I have done countless interviews and seminars with the media over the years and this really stood out for me.  Perhaps most enjoyable was the genuine interest and fascination with the topic of their correspondent Bob Brown (who was also a first rate nice guy).  They showed up at 8AM and took 1.5 hours to dismantle my office and set up the lighting.  We started extra early with the fertility patients that day so they could be finished and out the door before the TV crew came in to protect their confidentiality and to not make them feel uncormfortable (always a key issue in my fertility practice Gold Coast IVF).

joep.jpgDirecting the shoot

The cameras and the Crew:  Being the techno videophile guy that I am I jumped at the chance to talk with independent film crew brought to shoot me.  They had 3 cameramen/directors and there were 2 producers from 20/20, Bob Brown the correspondent, and a media relations rep from National Geographic (in case questions came up about their part).  They set-up a 2 camera shoot in my office with blazingly hot spot lights to ensure I would be nice and sweaty on camera.  They shot in standard BetacamSP.  Of course I could not resist to ask them why they did not shoot in HD.  They answered that the news shows inthe studios shoot in HD but that in the US all field work is done in SD.  This is because there are countless freelancers and crews out there all using different equipment and all waiting for some semblance of an HD standard to evolve before they invest hundreds of thousands of dollars in new HD cameras and editing and risk it being the “wrong format”.  Wow- how similar to the confusion in the medical and consumer video sectors! I continued my fact-finding quest and asked about who was using what systems and the relative advantages of each- panasonic sony JVC image sensors, color fidelity, native chip resolution tape vs disk vs solid state recording editing etc etc all trying to gleam insights I could take back to medicine and the OR. 

bobandi.jpgShowing Bob Brown (and cameraman) a Laparoscope  

The interview and turning the tables:  Bob interviewed me for 2.5 hours until they ran out of film. I was excited to share my excitement and passion for the subject of the future of medicine and surgery and how my work fits into this vision.  Bob was interested in the medical technology behind the show.  He asked a very wide range of questions from how I thought to merge HD video and surgerr back in 1999 to what I think is more beautiful – the earth from space or the vista of the internal human organs, to how will we pay for these new technology developments.  He was interested in everything I was working on and what I thought would have the most impact.  We discussed robotic assited surgery, natural orifice surgery (NOTES), augmented reality and head mounted displays, surgical simulators for training and the potential for real-dataset preoperative practice, virtual colonoscopy and 3D/4D ultrasound etc etc.

I had a chance to turn the tables a bit and ask him why they chose this topic and how they felt it would appeal to the lay public.  He told me that TV shows like 20/20 they basically track viewers interest levels minute by minute as they shows air.  He added that the medical pieces they ran have huge audience ratings and the more real the higher the appeal.  We discussed how the netorks know that on shows like CSI it is often the medical technology that draws the audience in.  He has a special talent in reporting human interest segments and has an amazing ability to distill down the high tech medicine we discussed and share with non-medical viewers how it will affect their lives. 

Sharing the footage:  After the interview he wanted to watch some of my HD surgical footage that I shot for National Geographic with the true HD 1080 16:9 system which I fortunately had available on HD XDCAM with a Sony ultrahigh resolution 24″ LCD HD monitor. Both the 20/20 people and the video crew were amazed by the resolution of the images and one of them remarked “If I need surgery I want them to use that   Being video people the film crew and director’s understanding of the power of HD in the OR was immediate when they saw just a few seconds of the images.  I continue to have the same degree of awe and fascination each time I operate with these systems.

Bob Brown was especially interested in my research on the development of autofluorescent laparoscopy and my concept of “FutureVision“- where surgical technology surpasses inate human senses and we watched those videos as well.

They finished off with few minutes of B-roll footage of Bob and I walking and talking in front of the hospital and requests for room cam OR footage and my AF surgery footage(all of which I was happy to share with them).

all4.jpgBob Brown, the Producers, and the docinthemachine

The 20/20 show airs this Friday September 7th at 10PM on ABC- check it out!

Share

DITM NG Special Website up- Interview with Wired Magazine

wired.gif

I previously wrote about the upcoming National Geographic Special Inside the Living Body and my work featured in the special.  I was also interviewed by Wired Magazine about the show and the technology behind it.  You can read their take on it here (note – I have to email the author Sonia and explain that it is not a good idea to use the descriptor “Organ Porn” in conjunction with the work of  gynecologist!). 

ng.jpgOne of the CG shots from the show

nglogo.gif 

National Geographic has set up a website dedicated to the show with photos videos and facts.  Lot’s of fascinating info and images to check out!  They write:

From our first cry to our last breath, our bodies undergo a continuous second-by-second transformation. Every move we make and every outside stimulus triggers a reaction through the skin, bones, organs, muscles and cells. We breathe, on average, 700 million breaths in a lifetime; an adult skeleton is replaced every seven to 10 years; we shed as many as 30,000 dead skin cells every minute; and the food we eat travels 30 feet (9 meters) on its journey through our bodies. Now, the National Geographic Channel (NGC) takes you beneath the skin to reveal how our bodies evolve from birth to old age, and the amazing biological systems we need to thrive.
From the producers of NGC’s critically acclaimed In the Womb series, Inside the Living Body traces one “everywoman’s story”, using milestones to examine the everyday workings of a living, functioning body in ways not seen on television until now. Cutting-edge miniature endoscopic HD cameras delve deep inside the mouth, throat, heart, lungs, digestive tract, brain and reproductive organs to shed new light on how and why our bodies do what they do. Stunning photography in this two-hour special reveals universal moments in human development at the most minute level, providing insight into both our own individual metamorphosis and our shared human experiences.

(the bold is my part!)

The Show airs September 16th on the National Geographic Channel (and the NG HD channel!) at 8PM.

Share