DITM Podcast on Future of Video for Entertainment & Medicine With Sony Exec

Posted on November 12, 2009 by Steven F. Palter, MD

I have been getting great feedback on my DITM medical technology podcast which starts the podcast series I’ll be doing regularly. Several people wrote to ask if I could post a version with just the interview since they loved it so much and wanted to share that segment. The original podcast starts with FDA approvals then has an interview with Sony Exec Bob Ott on the future of video technology in entertainment and medicine that we did a the NAB broadcast meeting in Vegas.

Here’s an edit version with just the interview enjoy and share with your friends!

Standard Podcast [ 15:35 ] Play Now | Play in Popup | Download

Docinthemachine MedTech Podcast!

Posted on November 10, 2009 by Steven F. Palter, MD

Here’s the latest installment of the docinthemachine podcast. In this installment I review new FDA device approvals and then present an interview about HD technology for entertainment and medicine with Bob Ott (vice president of broadcast and professional audio/video products for Sony Electronics) recorded at the National Association of Broadcasters (NAB).

docinthemachine november 2009 [ 19:35 ] Play Now | Play in Popup | Download

First Ever Surgical Video in 4k Ultra High Def- Hollywood and Doctor Team Up for Revolutionary Projects on Future Visualization in Medicine

Posted on November 4, 2008 by Steven F. Palter, MD

our digital cinema control room- Sony controls

For the first time ever Hollywood’s digital vanguard has teamed up with a surgeon to use tomorrow’s cinematic tools to revolutionize visualization in the operating room. We successfully filmed produced and played a 4k digital cinema movie of a live human surgery. This was an amazing collaboration between a fertility surgeon, the AAGL endoscopy society, Sony Medical & Broadcast divisions, Red Digital Cinema Camera Company, FotoKem, Vincent Pace films, USC professor Richard Weinberg, and the NBA just to name a few! We also showed the largest medical 3d displays ever.

The AAGL reported on this groundbreaking session today . Further details are here and here.

Dual Sony SXRD 4k projectors in 3D scaffold

New cinematic technologies are transforming the film business today. The two major revolutionary developments are 1) ultrahigh definition “4k” technology – which brings resolution to 4 times that of HD and 2) realistic immersive high definition 3D. I set out to introduce these technologies to the medical world and to see if we could for the first time apply them to surgical practice- setting the goal to once again use technological innovation to improve our patient outcomes.

Virtually all of my surgery is endoscopic- performed through tiny telescopes and viewed on a TV monitor. In 2001 I performed the world’s firs HDTV surgery and demonstrated how increased resolution improved the surgeon’s visualization and performance of procedures. For those interested in the history of HDTV surgery and the details of its development further details of my work are here from the New York Times and here from Science Daily and here from MIT Technology Review. Over the past two years I refined this work with even better performing camera systems and this work was featured on 20/20 and on the National Geographic Channel’s first ever HD medical show – Inside the Living Body.

Hollywood is embracing its digital future by adopting (with $1 billion in financing and a follow-up deal by Sony) planned conversion of 20,000 theaters to ultrahigh definition 4k (4096x 2048) video. The revolutionary Red One camera is one of the few that can natively record this type of file. Having heard about it I went to vegas for NAB in April and saw with my own eyes the amazing realistic movies being made with it. While there I met with Ted Schilowitz, Red’s “Leader of the Rebellion” and was ecstatic with his desire to help with with a proof of concept- to film surgery with a 4k camera. I also partnered with Sony’s Medical and Broadcast Divisions who eagerly helped me set up a 4k and 3dHD digital cinema in an international medical conference hall in the Las Vegas Paris Hotel. The theme of my Keynote lecture was “Film and Medicine: From the Past to the Future- Through the 4th Wall.” In this, I traced how cinematic film technology has transformed the teaching and performance of surgery- and how the future will again have revolutionary changes in physician visualization. Our Hollywood partners allowed me to first show jaw-dropping pro-sports and theatrical demos of the technology.

Sony’s digital cinemas allow the next generation of film to be projected at these previously impossible resolutions. We used dual SXRD 4k projectors with linear polarizing filters for 3D HD then went to a single projector for the 4k. 4k footage has to be played off a dedicated media block digital server and we used an LMT-100 like a movie theater would and FotoKem converted the 4k files into a DCP digital cinema package that can be played. For 3d footage we played off an HD SR tape dual stream deck. I’ll post a follow-up explaination of each of these technologies in detail.

By increasing resolution to this level we allow the surgeon to be actually immersed in images that surpass the live surgical experience. The progress from regular surgical film technology is like comparing sitting in an HD home theater to watching a video on a cell phone.

Ultra high resolution digital cameras are transforming the art of cinema. Leading Hollywood directors such as Peter Jackson and Stephen Soderbergh today have just started filming the next generation of cinema blockbusters using cameras with “4K” resolution, four times the resolution of High Definition (HD) with 4096 lines of resolution to give audiences unprecedented realism. The Red One digital cinema camera is the at the forefront of the revolution. Director Soderberg previously described this technology as “This is the camera I’ve been waiting for my whole career: Red is going to change everything….Shooting with Red is like hearing The Beatles for the first time. Red sees the way I see.”

Amazingly, the surgeons in the conference were able to visualize the surgery they were watching better than if they had been in the operating room live. If it can transform the immersive experience of the movies with unprecedented realism wouldn’t you want that degree of vision in your surgeon’s hands? By combining unprecedented resolution and magnification the surgical images were beyond what a surgeon would have standing live in the operating room. Those in the audience predicted this technology would further revolutionize minimally invasive surgery as it becomes incorporated into the OR of the future.

We then projected the largest high definition (HD) 3d surgical images ever. Wearing glasses reminiscent of today’s 3d Hollywood blockbusters U23D and Journey to the Center of the Earth a standing-room only crowd of 1600 surgeons felt as if they could “reach out and join the operation”. These images were enabled by converting Sony’s ultrahigh definition and 3d theatrical systems to show medical footage in what the AAGL called the “Theater of Tomorrow.” A Vince Pace 3D demo reel showed where Hollywood is going as well with Gwen Stefani concert footage, underwater vistas, and movie clips.

Since laparoscopic surgery is performed by a surgeon watching on a video monitor making the images “better than live” may facilitate teaching and procedures in the future.

The 4K system, manufactured by RED Digital Cinema Camera Company, was used to film Jumper, Crossing the Line, and The Argentine. This recording represents its first use for medicine and biology in the world. In addition to a surgery we showed the first ever 4k microscopy images of single cell organisms ever recorded (from USC).

Stay tuned a for a series of posts coming up this week that will review the session, how we managed the collaborations, the difficulty in executing each of the technologies, and where I predict this is going in surgical visualization.

The Future of Medical Video: DITM Reports From NAB 2008

Posted on April 23, 2008 by Steven F. Palter, MD

I had the distinct pleasure of attending the 2008 National Association of Broadcasters (NAB) meeting last week in Las Vegas. As the foremost event for the TV, broadcast, and media industries this is the venue to see and explore the future of all things video and media.

The Floor of the Meeting and a Sea of Humanity- Why I Went

Being probably the only physician in a sea of 105,000 TV and media folks raises the inevitable question- why did I go?

Endoscopic surgery (laparoscopy hysteroscopy arthroscopy etc) all share the common use of video equipment. Since the late 70′s these procedures are performed as remote surgery looking through a thin telescope inserted into a body cavity and observed on a TV monitor.

The progress we make in medical video surgery is a direct trickle down of innovations from the broadcast arena. From the first CCD camera hooked to a laparoscope and suspended from the ceiling via a jerry-rigged boom to the first use of HDTV in the OR – broadcast and TV technology drives innovation in surgical video.

I thrive on researching new technology and then extrapolating new solutions to medical problems using these developments. This meeting provides the raw material for my creative process.

I was honored to accept invitations from several major broadcast, video, computer, and even surgical companies to attend the meeting, walk the floor with them, brainstorm new ways of helping patients with new devices and predict future needs and uses for technology in medicine.

Everybody kept asking me: What was the most important development I saw at the show? What future technology do I predict is poised to transform medicine?

Beyond the entire rooms filled with the latest newschoppers and remote satellite trucks

I was most impressed with the following technologies which have the potential to transform both consumer entertainment and medical devices- I will be posting further about each of these and what I saw (including a series of interviews):

Beyond HDTV- “ultra HD” 4k cameras and displays
3D video technology in SD and HD
OLED display technology

FDA Update: Ophthalmic Device Panel Meeting To Review LASIK Safety

Posted on April 11, 2008 by Steven F. Palter, MD

eyeball LASIK Safety Review

On April 25, 2008, the committee will discuss general issues concerning the experience and quality of life of patients who have had LASIK.

How Serious is the Problem? Are the Patients Going Blind?

Dr. Daniel Schultz, director of FDA’s Center for Devices and Radiological Health, said a number of concerns have been raised concerning patient satisfaction with LASIK.

Companies that could be affected by such a meeting include LASIK device makers such as Advanced Medical Optics Inc, as well as LASIK providers such as TLC Vision Corp and LCA-Vision Inc.

Obviously, it’s a technology that has caught on and is used very, very widely. And there have been questions raised in terms of … quality of life and what does it actually do for the patient as opposed to the technology itself,” Schultz told reporters. Shultz said the meeting will focus on the quality of patients’ lives after surgery.

Known complications from the procedure can include dry eyes, glare, double vision, an increased risk of corneal inflammation or infection, and blindness.

Patients Call for Ban: In July 2007, the FDA responded to petitions from an individual asking for a halt to the procedures and a withdrawal of their approval. The agency said the devices were safe and effective but advisory panel discussions “could complement” its other safety monitoring.

The Boston Globes writes on the adverse effect of the current economic downturn on Lasik numbers and the potential adverse effect of this meeting on Lasik businesses.

But the declining U.S. economy continues to impact the number of laser vision correction surgeries, and Biegelsen cautioned investors that the outcome of the upcoming FDA Lasik advisory committee meeting on April 25 will likely be neutral at best.

“A best case scenario is little-to-no negative headlines in the media and the panel endorses the long safety of Lasik and encourages further use of femtosecond lasers,” he wrote in a note to clients. “On the downside, negative media coverage of unhappy Lasik patients and a panel recommendation to narrow the indications could drive volume down further. We think the most important variable is the media coverage which tends to be negative.”

FDA Update: Ophthalmic Device Panel Meeting To Review Implantable Eye Telescope

Posted on April 11, 2008 by Steven F. Palter, MD

The FDA Ophthalmic Devices Panel will meet Apr 24, 08 8:30 AM – 5:00 PM & Apr 25, 08 8:30 AM – 5:00 PM in the Gaithersburg Holiday Inn, Ballroom – 2 Montgomery Village Ave. Gaithersburg , MD

On April 24, 2008, the committee will discuss, make recommendations, and vote on a premarket approval application, sponsored by VisionCare Technologies, Inc., for an implantable miniature telescope (IMT™). The IMT™, a visual prosthetic device, is indicated for monocular implant in patients with stable, moderate to profound central vision impairment due to bilateral central scotomas associated with end-stage macular degeneration with geographic atrophy or disciform scar, foveal involvement and cataract.

Since these diseases lead to central dysfunction of the retina the implantable telescope spreads the visual image over a larger area of normal retina.

The prosthetic telescope, together with the cornea, acts as a telephoto system to enlarge images 3X or 2.2X, depending on the device model used. The telephoto effect allows images in the central visual field (‘straight ahead vision’) to not be focused directly on the damaged macula, but over other healthy areas of the central and peripheral retina. This generally helps reduce the ‘blind spot’ impairing vision in patients with AMD

New Technique Sees Inside Blood Vessels in a Microsecond

Posted on April 8, 2008 by Steven F. Palter, MD

Technology Review is reporting on a new technology to look inside the tiniest spaces such as blood vessels in a microsecond. Up until now endoscopic surgery has been limited as engineers tried to shrink telescopes to ever smaller diameters shifting from glass lenses to fiberoptic scopes to newer technologies. You can read my brief history and overview of microendoscopy here.

The new scope is based on optical coherence tomography but now uses new mathematical image analysis. Read the full article if you are interested in the heavy technical foundations of the system.

Suffice it to say, the system is a sort of “ultrahigh resolution optical ultrasound” and the new modification allows it to process the signal so fast that it could be used inside blood vessels without needing to interrupt blood flow and flush out the blood. The players in this development are two compnaies – LightLab and CardioSpectra of Austin, TX. The latter company was recently purchased by Volcano, one of the leading manufacturers of IVUS products for $25M.

Example of an OCT image of a fingertip (standard old OCT system)

Basic Explanation of How the Foundation Technology of OCT Works from wiki

“OCT is a technique for obtaining sub-surface images of materials at a resolution equivalent to a low-power microscope. It is effectively ‘optical ultrasound’, imaging reflections from within tissue to provide cross-sectional images. It is attracting a great deal of interest in the medical community, because of its potential to provide images at a far higher resolution (better than 10 µm) than is possible with other imaging modalities such as MRI or ultrasound.”

Can Vision Testing Improve Surgical & Athletic Performance?

Posted on April 7, 2008 by Steven F. Palter, MD

Gizmodo is reporting today about how White Sox catcher A.J. Pierzynski said he improved his performance though the daily use of a Nintendo video game. This reminded me of research I did while at Yale on similar visual testing on surgeons. We found back then that for both athletes, and surgeons (especially endoscopic surgeons) visual acuity correlated with performance and both could be improved with exercises.

First the report on the catcher and the video game. Gizmodo writes:

We were just about to stuff Nintendo’s Flash Focus vision game into the snake oil file when White Sox catcher A.J. Pierzynski revealed he actually improved his hitting through daily use of the software. Like many of the White Sox batters last year, Pierzynski stunk, and to come around he played Flash Focus in the off season. Now, one week into the 2008 schedule, the hapless catcher has two home runs, a .529 average and a team-leading seven RBIs.

What is Flash Focus?: Flash Focus is a video game for the Nintendo DS. Basically it is a series of hand eye coordination exercises designed to improve overall visual acuity through vision “exercises”. A review of the game and screen shots are here.

The Concept of Optomeric Testing for Athletes and Surgeons: The idea behind optomic testing is two fold. First, there are innate differences in visual performance between people second, elite athletes perform significantly better on objective tests of visual performance and hand-eye coordination, and third, specific testing and exercises can identify areas of optical performance weakness and improve them- and performance.

A review of optometric testing in general for athletes can be read here:

The basic tests they mention are:

Visual acuity – right and left eye separately without an occluder
Stereopsis at distance – contour and random dot
Ocular dominance – independent of handedness
Letter contrast
Contrast sensitivity – at spatial frequencies known to be related to sport performance
Contrast sensitivity with glare
Reaction time
Reaction time with varying targets

Our experiments with surgeons- Some years ago whenI was at Yale My Partner David Olive and I conducted a lerge study of novice and expert endoscopic surgeons using the validated athletic optometric testing tools. We testing a very large group of trainees as well as experts at an international AAGL conference. We found similar patterns as with the athletes. Namely significant differerences existed between visual ability and eye hand coordination skills and these were related to measures of performance and skill. We also found that with exercise performance could improve.

New Monitoring System Approved by FDA- Potential for Future Robotic Diagnostics

Posted on April 2, 2008 by Steven F. Palter, MD

The FDA has just approved the FreeStyle Navigator Glucose Monitoring System – a glucose sensor that reports glucose values continuously for up to 120 hours. Here is a copy of the FDA PMA letter. This device is interesting to me since it works with a sensor inserted in either the abdomen or the back of the upper arm. The device then continuous provides glucose readings and updated glucose trend information for viewing and contains a built-in alarm that can be programmed to alert the user when results fall below pre-set values. Other similar devices have been approved that monitor for 7 days.

Potential for Future Robotic Diagnostics I have written before that I predict a whole new field of chip based biologic disease screening and monitoring in the future. This is another step to that result. Here a sensor is placed under the skin that measure blood sugar. In the future minitaturized chips could be placed in any body cavity or organ to sense any imaginable molecule.

Options for Future Diagnostics:

DNA based sensors screen for cancer metastasis or recurrances such as an intrabdomnal ovarian cancer detector.
Sensore that measure drug levels in target tissues – chemotherapy of course comes to mind
protein sensors that look for the earliest stages of disease development.

As personalized genetics becomes more widespread, we will identify individuals at particular risk for particular diseases before they occur. Since the genetic basis of these diseases will be known markers will likely exist. Implanted chip sensors could then be placed to sniff for these markers and wireless transmit the alarm- or even deliver a predetermined treatment agent- all before there is any external sign of the disease. First generation implantable devices such as this for blood sugar monitoring are lisated here (none yet available):

Here are links to some of the technology that will be involved for these future diagnostics including tiny sensors that transmit with RFID, smartdust sensors the size of a speck of dust or less, smart pills that travel through the body transmitting data and the concept of personalized genetic information based diagnostics and personalized genetics in general.

All approved continuouis blood sugar monitoring devices are here and a comparison from a patient site here:

As an aside – in terms of glucose monitoring the use of thse devices may come into question. An ongoing diabetes study called ACCORD was cut short in one treatment arm when it was shown that ultra tight strict blood sugar control in diabetics with heart disease actually WORSENED outcomes!

How Smartdust, Souveillance, Web 3.0, and Personalized Genetics Will Transform the Future of Medical Diagnostics

Posted on February 25, 2008 by Steven F. Palter, MD

There has been a flurry of debate in the military, industrial, and privacy sectors on “smartdust” and the concept of “souveillance” – but no one has yet realized this technology is poised to springboard into medicine and transform medical diagnostics. Here I wanted to give you an overview of what this idea is and why you should keep your eye on it.

First the general concept background:

“Smartdust” refers to micro devices (called motes) which are detection microchips each potentially the size of a speck of dust. These grains of sand however can automatically self-network. So far people have conceived of these low-power distributed sensing networks as having functions for climate control systems, entertainment devices and especially for big brother type surveillance systems.

Wikipedia wrote “the smartdust concept was introduced by Kristofer S. J. Pister (University of California) in 2001 , though similar ideas existed in science fiction before then. A recent review discusses various techniques to take smartdust in sensor networks beyond millimeter dimensions to the micrometre level. A typical application scenario is scattering a hundred of these sensors around a building or around a hospital to monitor temperature or humidity, track patient movements, or inform of disasters, such as earthquakes. In the military, they can perform as a remote sensor chip to track enemy movements, detect poisonous gas or radioactivity. The ease and low cost of such applications have raised privacy concerns.” Beyond web 2.0 vast networks of these real time sensors are once possible technology leap of the yet inknown web 3.0.

General concept – What is Souveillance?: is a term from Steve Mann that refers to “bottom up” surveillance using smart dust as opposed to “top down” big brother networks looking at us little people. Here instead activities are recorded from the “perspective of a participant in the activity, typically by way of small portable or wearable recording devices that often stream continuous live video to the Internet.” Remember the impact of the Rodney King video and of all the user generated video content on the web. Now fast forward to a world where a large segment or even a majority of the populice had real time streaming video devices on all the time (no we are not going to discuss the porn angle on this). This has also been called “inverse surveillance”.

Privacy advocates have been debating the merits or horrors of this type of sensor technology. I serve on the Scientific Advisory Board of the Lifeboat Foundation which is dedicated to protecting us from future technological threats through advocacy research and education. They have been having a heated debate on the “paradox of smart dust: we may not live without the greater security provided by smart dust, but many think they could not live with smart dust impinging on our privacy.’

Medical Implications: I have a vision that once this type of low power networked microsensor technology exists it will logically lead to medical sensor technology. Potential uses I see include:

mass screening for infectious disease or bioterror agents. Subjects walking into screening areas could be checked for signature molecules associated with infectious agents. Just as we have metal detectors and now have molecular signature detectors (the litle wipe test for explosives at the airport) we will have such biological screening techology.
The next step will be similar screening for disease states. Metabolomics is one such technology. Metabolomics is the study of the small-molecule metabolite byproducts left behind from cellular processes. In simple terms it’s like examining poop. The concept is that by measuring the collection of all the byproducts of the cells metabolism you can get a snapshot of the physiology of a cell or organism that translates to health. One such sensor is being developed as a breath sensor for disease. This could lead to Star Trek like medical sensors.
Similarly, such technology will lead to individual genetic screening for disease risk using chips that interact with the tiny bits of DNA we shed every time we touch something. Companies commercializing this approach also already exist and have products.
Taking a clue from smart dust we will then inject such sensors into our bodies where thy could circulate in the bloodstream or sit in the abdminal cavity silently sensing for disease, infectious agents, or the DNA or signature molecules of a cancer cell. Alternative chips could exist that sit and slowly release drugs when such cell reappear once a patient is diagnosed.

I will be writing more about the details of these concepts and devices being developed in future posts now that I have introducted the concepts. Let me know what you think!

docinthemachine

Transforming medicine and fertility with tomorrow’s technology

Category Archives: future vision