I am so excited to report that I have performed the world’s first laparoscopy in 4K - using cameras with a resolution 4 times that of HD. I presented the details and shared the images with a crowd of 3500 at the 65th Annual Meeting of the ASRM this week in Atlanta in my invited plenary lecture. We showed the audience how the digital technology being developed to transform movies could be directly applied to take surgical performance to tthe next level.
“Dr. Palter’s research and vision of surgery’s technological future opened the eyes and minds of the audience to fantastic treatments beyond what can be done today” said R. Dale McClure, MD President of ASRM.
Handheld and 4 foot Jib arm methods of 4K surgery test
What was New This Year:
Last year the same team worked togother (along with FotoKem) on our proofof concept project. Then we took the camera into the OR and shot with the Hollywood version on a tripod looking around the OR– but it was not used for the surgery itself. This year we directly attached the camera to an operating laparocope and used it for the surgery itself.
What we Did: Laparoscopic, or keyhole, surgery is when telescopes are used to perform minimally invasive procedures with faster recovery and no major incisions. The surgeon works by remote looking on a television monitor. Usually this is in regular standard definition and lately high definition systems are being used. As detail and resolution increases surgeons will see and perform better. For this reason I set out to see if images 4 times the resolution ofHD could be obtained through our surgical scopes and if the next generation of Hollywood 4k cameras could be used for surgery. In a pilot project we successfully connected the camera of the future to our surgical scope and obtained the highest resolution surgical images of body ever directly in the procedure. The audience nicknamed me Dr. Steven Spielberg.
Since these scopes had never been attached to such a digital creature as this camera we had to create a coupling system. We went back to models and systems used in the 1940′s when film was just invented to devise a system. Thanks to brainstorming and testing by USC’s Richard Weinberg in the School of Cinematic Arts and Karl Storz Endoscopy we figured out and developed a system.
Why we did it- the Hollywood connection: New cinematic technologies are transforming the film business today. The two major revolutionary developments are 1) beyond high def “4k” technology – which brings resolution to 4 times that of HD and 2) realistic immersive high definition 3D. I set out to introduce these technologies to the medical world and to see if we could for the first time directly perform surgery in 4k. Setting the goal to once again use technological innovation to improve our patient outcomes.
The Equipment We Used
Our Partners From Hollywood:
Red Digital Cinema Camera Company makes the Red One 4K camera.
Sony Electronics, Inc. makes the incredible SRXR-220 projector– 4096×1920 resolution capable of 3D HD and 18,000 lumens. Not to mention $200,000 and 700 pounds.
Here’s the control room including the 3D SRW deck. With the technical guru’s from Sony Mark Woudsma and Vinh Vo doing their magic. Evan Kratchman from the Medical Division helped brig all the diverse Sony divisions together and coordinate the event to make it happen.
The incredible Mark Woudsma of Sony – master of all future tech roadshows setting digital inputs on one of our 20 foot scaffolds (we had three of these folks!)
Offhollywood is the uber-cool NY Soho-based masters of the Red One camera and digital post production house. The head of the facility in NY Mark Pederson immediately signed onto the project to try to figure out with me and Red how to do 4K through the scope. He sent out the cameraman Eric Camp who ran the Red and the 4 foot jib arm we needed for stabilization. We then spent a late night in post production color correcting and editing the footage in Assimilate’s Scratch. Finally Pliny produced the DCP (4K digital file format) file that the digital theaters could play. It only took 8 hours to render… Not to mention the XML packaging that the players require and the Hard drive that had to be deliverd via courier. BTW– the photo here is me with mark and Ted from Red with the full Hollywood camera and a prototype of the 3K handheld Scarlet version…
For 3D in HD we used the Sony SRW Deck and 3D circular polarized theater glasses matched to theatrical lenses from RealD. Amazing collaborations from 3Ality allowed us to get Hollywood footage including U2 in 3D in concert! Followed that with 4K Tom Cruise, Will Smith, Julia Roberts, Steven Spielberg, Harrison Ford, and Ben Stiller among others in cameo blew the audience away. We shifted for each technology from Hollywood to medical use showing the overlap in technical development… Andrew Stucker from Sony Digital Cinemas helped make it all happen and secure rights… We projected the largest high definition (HD) 3d surgical images ever. We used the same system as the 3D Hollywood blockbuster Cloudy With a Chance of Meatball to allow the surgeons to feel as if they could “reach out and join the operation”. These images were enabled by converting Sony’s ultrahigh definition and 3d theatrical systems to show medical footage. For medical footgae we use Intuitive Surgical’s DaVinci robot which images in resouding HD 3D.
Background
Virtually all of my surgery is endoscopic- performed through tiny telescopes and viewed on a TV monitor. In 2000 I performed the world’s firs HDTV surgery and demonstrated how increased resolution improved the surgeon’s visualization and performance of procedures. For those interested in the history of HDTV surgery and the details of its development further details of my work are here from the New York Times and here from Science Daily and here from MIT Technology Review. Over the past two years I refined this work with even better performing camera systems and this work was featured on 20/20 and on the National Geographic Channel’s first ever HD medical show – Inside the Living Body (as reported in Wired).
Hollywood is embracing its digital future by adopting (with $1 billion in financing and a follow-up deal by Sony) planned conversion of 20,000 theaters to ultrahigh definition 4k (4096x 2048) video. The revolutionary Red One camera is one of the few that can natively record this type of file. Having heard about it I went to vegas for NAB and saw with my own eyes the amazing realistic movies being made with it. While there I met with Ted Schilowitz, Red’s “Leader of the Rebellion” and we began our collaboration to take surgery into imaging’s future.
I also partnered with Sony’s Electronics Medical and Broadcast Divisions who make the best highest resolution 4k and 3D projectors in the world– usually used for next genertion digtal movie theaters.
By increasing resolution to this level we allow the surgeon to be actually immersed in images that surpass the live surgical experience. The resolution approaches that of the human eye but it is combined with 10 fold magnification through the telescopes which operate just inches away from the disease. The progress from regular surgical film technology is like comparing sitting in an HD home theater to watching a video on a cell phone.
Ultra high resolution digital cameras are transforming the art of cinema. Leading Hollywood directors such as Peter Jackson and Stephen Soderbergh today have just started filming the next generation of cinema blockbusters using cameras with “4K” resolution, four times the resolution of High Definition (HD) with 4096 lines of resolution to give audiences unprecedented realism. The Red One digital cinema camera is the at the forefront of the revolution. Director Soderberg previously described this technology as “This is the camera I’ve been waiting for my whole career: Red is going to change everything….Shooting with Red is like hearing The Beatles for the first time. Red sees the way I see.”
Amazingly, the surgeons in the conference were able to visualize the surgery they were watching better than if they had been in the operating room live. If it can transform the immersive experience of the movies with unprecedented realism wouldn’t you want that degree of vision in your surgeon’s hands? By combining unprecedented resolution and magnification the surgical images were beyond what a surgeon would have standing live in the operating room. Those in the audience predicted this technology would further revolutionize minimally invasive surgery as it becomes incorporated into the OR of the future.
The 4K system, manufactured by RED Digital Cinema Camera Company, was used to film Jumper, Crossing the Line, and The Argentine. This recording represents its first direct use for a surgical procedure in the world.
I’ll post more details soon….
I was involved in the keynote general session at the 38th Annual International Congress of Gynecologic Endoscopy (The 
