Microsoft produced this video as a demo of their vision of the future of healthcare.  I was delighted to see that they share so much with my vision of technologies that will be key transformers in the future of medicine.

Key technologies I predict will change health care that I see in the video include:

1. implantable glucose monitor sensors
2. display surface
3. wireless charger
4. gesture control
5. interactive smartwalls that become displays
6. RFID taggged medicine bottles
7. Digital examination tols interfacing with EMR’s
8. LCD opaque windows
9. Digital wallets
10. Biometrics

‘Duct tape is like the Force. It has a light side, a dark side, and it holds the universe together’ - Carl Zwanzig

My sci-fi writer wifeinthemachine being the creative type posted 2 funny videos - one is a mind-freak experiment where a couple of hundred people in NYC Grand Station simultaneously freeze and freak people out

the second is a song from India featured in the end credits of Inside Man that will get stuck in your brain- everyone loved the song but no one knew who or what it was.  Here it is- she calls it her writer muse-crack

“ Though sold as a game console, what will in fact enter the home is a Cell-based computer. ” - Ken Kutaragi

“Cell-based computers will revolutionize medical imaging” - Docinthemachine

The IBM Cell graphics processor at the heart of the PS3 is a remarkable chip.  Cell is shorthand for Cell Broadband Engine Architecture.  It has been described as “seemingly obscene computing capabilities for what will rapidly become a very low price.” 

A newer miniaturized lower power version has just been announced by ars technica that I predict will make it to medical video and VR processing.  I recently led a session on the use of VR in medicine where Andy Van Dam (VR pioneer , professor of computer science at Brown, and founder of Siggraph) and I spoke about the future of VR processing.  He predicted that the video grame industry hardware innovations will make the most dramatic strides and that this technology will then trickle down to VR due to its sheer massive computational power- beyond that of the old CAVEs of DARPA.

You may be unaware that this represent a new form of computer processing: 

The Cell concept was originally thought up by Sony Computer Entertainment inc. of Japan, for the PlayStation 3.  The genesis of the idea was in 1999 when Sony’s Ken Kutaragi  “Father of the PlayStation” was thinking about a computer which acted like Cells in a biological system.  A patent was applied for listing Masakazu Suzuoki and Takeshi Yamazaki as the inventors in 2002

The architecture as it exists today was the work of three companies: Sony, Toshiba and IBM.  Sony and Toshiba previously co-operated on the PlayStation 2 but this time the plan was more ambitious and went beyond chips for video games consoles.  The aim was to build a new general purpose processor for a computer.

In lay terms here is the muscle behind the processor:   

The setup of the Cell processor is like having a team of processors all working together on one chip to handle the large computational workload needed to run next-generation video games. In order to understand how the Cell processor works, it helps to look at each of the major parts that comprise this processor.

The “Processing Element” of the Cell is a 3.2-GHz PowerPC core equipped with 512 KB of L2 cache. The PowerPC core is a type of microprocessor similar to the one you would find running the Apple G5. It’s a powerful processor on its own and could easily run a computer by itself; but in the Cell, the PowerPC core is not the sole processor. Instead, it’s more of a “managing processor.” It delegates processing to the eight other processors on the chip, the Synergistic Processing Elements.

The computational workload comes in through the PowerPC core. The core then assesses the work that needs to be done, looks at what the SPEs are currently processing and decides how.

Watch out for our robot PS3 overloards.  This Chip has the potential to expand itself and distribute workloads over networks.  Don’t worry this is not some Singularity scenario where the chips start to think on their own.  Here is a review of the potnetial of the chip:

Chip giants such as Intel have already started working on dual-core chips, but Cell goes several steps further by giving processing units a measure of independence. Current multicore chips typically chop a single computing task into parts, which are distributed among processing units. Cell’s processing units–called “software cells”–can handle completely separate jobs.

“The software cells are designed to be kind of self-contained–they can kind of roam around,” Halfhill said.

Cells can even roam over a network, allowing the processor to perform a type of distributed or grid computing, an increasingly popular enterprise technique in which demanding tasks are divvied up among a gang of networked computers. A PlayStation 3 could borrow unused processing power from other consoles on a network, for example, to complete a demanding task such as delivering streaming video.

“The Cell architecture is designed to make grid computing almost universal,” Halfhill said. “It makes distributed processing part of the design. If you have several of these machines on a network, the work can be spread across a network.”

The cell design can allow cooperation between video devices:  “This architecture is not fixed, if you have a computer, PS3 and HDTV which have Cell processors they can co-operate on problems.  They’ve been talking about this sort of thing for years of course but the Cell is actually designed to do it.  According to IBM the Cell performs 10x faster than existing CPUs on many applications.  This may sound ludicrous but GPUs (Graphical Processors Units) already deliver similar or even higher sustained performance in many non-graphical applications.”

Medical uses:  We are at the cusp of a revolution due to the integration of computer video processing and surgical and radiological imaging.  Details of this concept of mine are here and a podcast here.  As we move ahead with virtual imaging and newer forms of optical processing it is the computational power of these kinds of chips that will be enabling.

Disclosure:  As I previously wrote, I was chosen to be a Sony Medical HD Luminary Site.  I receive no financial payment for this relationship which is only with Sony’s Medical division and is part of my medical research work on surgical tools and imaging.  Heck- I had to buy my PS3 at Best Buy just like anybody else. 

While at the Global Congress of Gyn Endoscopy I saw a really neat new device called LapCap designed to reduce the risk of entering the abdomen in laparoscopic surgery.

What is the problem- why is this needed? In laparoscopy the first step where the surgeon gains access the abdominal cavity is often a blind entry step.  There is a risk of injury to internal organs such as the bowel, bladder, or blood vessels.  While rare, these injuries can be severe or even life threatening.  If you want to read more about laparoscopic access and the risks of entry into the abdomen here is a link to an article I wrote for a Master’s Class in Gyn Surgery on this topic.

What is done in standard surgery: The most common method used by gyn surgeons is the insertion of a needle (the veress needle) which fills the abdomen with carbon dioxide gas (called insufflation).  Then a tool called a trocar is inserted into this space and the instruments go through this.  The needle insertion step is the most dangerous because it is the blind step.  various techniques are used including elevation of the abdominal wall to lift it off of and away from the underlying organs expecially the large vessels like the aorta nad vena cava.  The second common method  is the open or Hasson technique (invented by my good friend Gyn surgeon Harry Hasson) where the surgeon opens a small 10 mm incision and then inseted the trocar.  Injuries occur with all known techniques.

The New System:  The LapCap is a new system that puts a plastic dome on the abdomen attached to suction that then pulls a full thickness of the abdominal wall high up into it then allowing the veress needle to be inserted into a potentially larger and they claim safer spot.  The LapCap device received 510(k) approval from the FDA and is sold by Aragon Surgical.  It won an award from the SLS for new device innovation.  Here is a video from the company of it in operation.  I know surgeons who have used and were quite impressed and I will likley be involved with surgery using it soon.  I know that laparoscopic innovator and friend of mine Camran Nezhat is one of their advisors.  I will be speaking with him more about his experience.

Potential problems:  Two major issues- first statistical proof of demonstration of increased safety is nearly impossible for access devices such as this because the injury it might reduce so very rarely happens.  One study on this problem estimated that it would take a study of more than 200,000 people having the operation to adequately demonstrate reduced injuries statistically.  Second- one of the major risks for this sort of injury is when the bowel is adherent to the abdominal wall at the site of entry from previous surgery.  I would not expeect this device to help at all in this situation because the adherent bowel will be picked up with the abdominal wall.  However it is the vessel injuries that ar emost dangerous and this device might reduce the risk of those if it places the needle much farther away.

About the company:  They appear to have acquired the device via an acquisistion of starup Verisure. It was reported last month that they “raised $25 million in a Series B round of venture capital with the hopes of having five products on the market within two years.  In a statement on Monday, Jan. 7, Aragon said new investors Bay City Capital of San Francisco and Integral Capital Partners of Menlo Park, Calif., joined original investors Delphi Ventures and Onset Ventures, both of Menlo Park. They formed Palo Alto, Calif.-based Aragon in May 2005 to advance radio frequency technology developed at Stanford University Medical School for use in laproscopic and other surgeries. Bay City Capital was the lead investor in the most recent round.” 

I have spoken about the use of RFID for surgical and laproscopic procedures for several years and look forward to seeing where they go with it.  I know they also have tissue sealing and dividing technology and are part of the Delphi portfolio.  I had a phone call with Delphi last year as part of due diligence they were doing for another company and did think highly of them and their approach.  More details of the deal are here.

Chalk up another innovation to Ray Kurzweil America’s leading inventor (and Lifeboat Foundation Advisor along with me).  This one is a simple and elegant solution to help the visually impaired.

Developed in conjunction with the National Federation of the Blind, the device (The Kurzweil-National Federation of the Blind Reader) is a digital camera that can photograph any text or sign and then digitally OCR it and read it outload to the user!  Quite a simple concept.

Kurzweil recalls the invention of the First OCR Reader in 1974

“In 1974, computer programs that could recognize printed letters, called optical character recognition (OCR), were capable of handling only one or two specialized type styles. I founded Kurzweil Computer Products, Inc. that year to develop the first OCR program that could recognize any style of print, which we succeeded in doing later that year. So the question then became, ‘What is it good for?’ Like a lot of clever computer software, it was a solution in search of a problem…I had found the problem we were searching for—we could apply our ‘omni-font’ (any font) OCR technology to overcome this principal handicap of blindness.

New System is Portable  “The National Federation of the Blind (NFB) and Kurzweil Technologies, Inc. (KTI) have created the world’s first portable OCR device, that allows an ordinary page of text to be photographed and subsequently translated into voice. Over the last three decades there have been several computer-based solutions for translating OCR to voice, but none of them are portable.”

Click here for video of the device in action[wmv width=”375″ height=”211″]http://mfile3.akamai.com/12032/asf/kurzweil.download.akamai.com/12032/knfbr/CNN_Device_opens_the_world_for_blind.asf[/wmv]

I have previously written about the Army’s robotic prosthetic arm projects - run of course through DARPA.  You can see my posts and a video fest at Video Fest of Brain-Computer Links & Control. 

An equally amazing story is how the project has come to be- DARPA contacted Deam Kamen (and team at DEKA of Segway fame) and challenged him to create this amazing feat of technology. The NYT reports

Eighteen months ago Segway entrepreneur and serial inventor Dean Kamen received a visit from Anthony Tether, the electrical engineer who runs the Defense Advanced Research Projects Agency, the military’s research and development agency.

Mr. Tether had come to Mr. Kamen’s rural western Massachusetts workshop to persuade him to tackle a challenging engineering problem: a robotic arm that would make it possible for any of the 1,600 or more Iraq veteran amputees to resume a semblance of a normal life.

Mr. Kamen, who designed the two-wheeled Segway balancing transporter and several high tech wheel chairs, and who has a wealth of robotic engineering expertise, said that he initially thought the idea “was nuts.”

A more extensive review of the two parts of the project is at Wired’s Danger Room where Noah reports on the two phases of the program. 

Project 1 - the Holy Grail: Kuniholm and his fellow engineers at Johns Hopkins University’s Applied Physics Laboratory, or APL, are at work on the most ambitious prosthetics project in history. They seek the field’s holy grail — to build an artificial human arm that acts, looks and feels to its user like his native arm, and to do it with astonishing speed by the end of 2009.  (called Revolutionizing Prosthetics 2009)

Project 2- Hedge your bets:  The Kamen project: produce the best prosthetic arm possible with currently available technology

For now, both Deka and APL are based on cutting-edge myoelectric control systems pioneered by Todd Kuiken at the Rehabilitation Institute of Chicago, or RIC. Conventional myoelectric controls use electrodes on the surface of the skin to read muscle signals from some part of a user’s body unaffected by his amputation — his back for example — and pass the signal on to an artificial limb. The user twitches her back, and the limb moves in response.

A video has been released of the project’s progress so far in a demo. It was shown at the DARPA tech conference.  You can check it out here (sorry can’t get the youare.tv player to run in wordpress blog engine).

Another video is here

This year is the 50th anniversary of DARPA, or the Defense Advanced Research Projects Agency, the Pentagon research arm who turns science fiction fantasy into military reality.  DARPA conducts high-risk military research and in the process develops amazing medical technology.  To kick things off right at their 3-day DARPA tech conference (or official site here) they announced the upcoming deployment of the remote battlefield medic/surgeon Trauma Pod robotic system by 2009!  Surgical robotics was initially conceived by DARPA as remote battlefront or space surgical robots and this technology is now widely available in the DaVinci surgical robots. 

As Popular Mechanics reported from the Conference:  (skip to the end for links to videos)

the first portable, self-contained surgical robot will be deployed in the next two years. Brett Giroir, director of the research agency’s Defense Sciences Office also announced that the system, called Trauma Pod, has successfully “treated” a mannequin during a test, with no complications.

A single human will operate the robot remotely during surgery, but Trauma Pod will be able to perform a number of functions, such as fluid administration and surgical assistance, autonomously. The goal is to stabilize injured soldiers as quickly as possible, and previous Trauma Pod designs have included related systems that evacuate the patient. Giroir said that a prototype will be delivered to troops within two years.

Details of the System and its Use:

According to DSO Director Brett Giroir, the goal of the Trauma Pod is to conduct “emergency control surgery.” That means diagnosing and treating major trauma, focusing on airway management, head wounds and, as Giroir put it, “controlling uncontrollable bleeding.”

And while a surgeon will be controlling some of the Pod’s functions, such as the more invasive procedures, the system relies heavily on autonomous control. The robots in the Pod would insert breathing tubes and IVs, but the surgeon would direct the scalpel. Even during remote operation, auto-targeting systems will assist the surgeon, completing or fine-turning certain actions. “It’s not doing surgery the way a person is,” Giroir said. Instead of an exercise in advanced telepresence, the Trauma Pod is a synthesis of human judgement and robotic precision. Much of the surgeon’s input will be to tell the robots not to do something, such as inserting a breathing tube. Many of the systems are still up in the air, but Giroir expects the Pod to rely on CT scans for diagnosing trauma, and various surgical instruments that, as depicted in the video, the robots will literally grab out of a rack. It might incorporate technology from other programs, such as a device that triggers coagulation in a severed artery through high-intensity focused ultrasound.

Getting the patient off the battlefield and into a hospital is another matter. While the Pod is supposed to eventually meet certain size and weight restrictions, there are no plans yet to incorporate specific vehicles. Giroir does believe it will be compact enough to fit in the back of a Stryker vehicle, for example, and the experimental model that will be delivered in two years might still need to be trimmed down. The Trauma Pod is expected to be used by the Army initially, with possible, full-production deployment happening between 2011 and 2013. That’s a very rough estimate from Giroir, and much of the timing will depend on how quickly the system can be miniaturized, and whether it actually works.

Giroir was also excited about the Pod’s potential civilian use, for when trauma centers are often too far away to save a patient’s life. Local hospitals could stock a single Trauma Pod, and have a surgeon thousands of miles away assist in stabilizing the patient.

They say there are no video or photos available but here at docinthemachine I posted a report and videos of the systems concept and prototypes back last year.  You see the post and all the clips at Awesome Army Videos-Terminator 2025 Battlefield Surgery Built NOW!

For Those Who Want More DARPA Hi Tech Medical Information:  I have prevously written about DARPA and the medical offshoots of its research .  You can read about why DARPA can take the big risks in medical development private industry won’t in Risky Business:Why DARPA Does What Medical Industry Won’t.  Be sure to read Army Axing High-Tech Soldier of Tomorrow- MedTech Losses Predicted for all ofthemedicalbenefits of the Soldier of Tomorrow “Land Warrior” Program.  You can read about the history and future projects planned by DARPA in 2007 in DARPA Releases Strategic Plan 2007 and about it’s amazing array of projects in DARPA 2007 Pt2: Major Achievements, Future Plans, & Medical Benefits (including Newton’s Laws for Biology, Prosthetics, Biological Warfare Defense, and Real-Time Accurate Language Translation). 

Come back tomorrow for my next post reviewing the other robots they showcased…

My last post outlines the equipment I used to perform the world’s highest resolution endoscopic surgery.  Here I wanted to share what we had to do the record the footage at full HD resolution to get it to National Geographic for their HD feature “Inside the Living Body”.  Archiving HD surgical footage has been a significant shortcoming of many of the existing systems. 

First some historical background on archiving endoscopic surgery stills and videos.  Surgical endoscopy is archived mainly via still photos.  Surgeons keep a few shots of the main pathology seen or a few before and after shots.  Those who like to keep videos have mainly used simple consumer VHS (there is a huge medico-legal debate in the field if it is protective or dangerous to record all of your surgeries but that is another discussion).  You can imagine the nightmare of storage of a library of tapes that accumulate over the years!  For more than 10 years I have used S-VHS to get a slightly higher resolution capture.  I was limited since the OR systems best output was S-VHS.  For years (sadly more than 10 years) I have been trying to get the surgical equipment companies to put a firewire IEEE 1394 output on the camera units.  This would allow me to hook up either a mini-DV deck or digital disk based recorder that did not require analog capture and digitization.  They never went this route.  I wanted to use miniDV because of its higher resolution, ease of digital editing, and smaller form factor than VHS.  Remember back when this started there were not small affordable devices that could capture analog video on the fly and convert to digital as there are now. 

Next, the surgical video companies went to digital capture and archiving.  I loved the digital still capture that let me capture photos for lectures or the patient’s chart and burn it to a CD-R or DVD-R (actually when this started we were using horrible proprietary strange disk formats now extinct unreadable and sitting in my archives).  The captures are ok JPEG or TIFFs.  I wish they captured at higher resolution with less compression artifacts.  One problem I have seen in some ORs is that the machines are actually set for a low resolution capture which makes no sense at all.  They then added direct digital video capture and recording onto the same disks.  The quality of these captures is hugely variable between systems.  Some capture full motion 30 fps video at a decent resolution whereas others look like blocky jerky garbage.  Sadly, I usually have to jump into the capture system’s configuration menu before I start a case in many OR’s.  Just today I was working with a state-of-the-art system installed in one of my local hospitals just last week.  It was one of the newest HD systems.  In fact the manufacturer’s rep was in this morning doing an in-service and configuring the system for optimal use.  As usual, once in the configuration menu I found the still captures configured for the lowest resolution 640 x 480 BMP and the video on its lowest MPEG-1 setting.  As usual, configured for the lowest quality settings.

New HD Recording System we used:  Since we were operating in 1920 x 1080p for my HD project we needed something that could digitally record at this high resolution.  We chose the new Sony XDCAM HD Professional Disc Recorder.  This systems records 1080/59.94i, which Sony calls 1080 60i.   

Because our camera outputs true 1080p, in order to record we first have to scale the image down to 1080i using a Gefen DVI to HD SDI scaler.  The resultant 1080i signal is what is input to and recorded with the XDCAM HD deck to a proprietary Sony Professional Disc that is actually based on a Blu-ray disk (sorry won’t play on your PS3 - the disk is in a protective hard shell holder and recorded with a different format than consumer Blu-ray).

The XDCAM HD records HD input in MPEG2 HD, a Sony format.  This MPEG HD 1) is designed to yield high-quality video and audio recording and playback.   The MPEG HD 1) codec provides video compression compliant with the MPEG-2@HL standard.  It enables HD 4:2:0 digital compression recording in the 1080i (1,180 effective scanning lines, interlaced) format currently in use in many broadcast facilities.

There are three selectable video bitrates: HQ at 35 Mbps, SP at 25 Mbps, and LP at 18 Mbps.  The resultant “clip” is an MPEG HD file which bears the suffix .MXF.   

Another aspect of this recording medium is that it converts the input 16:9, 1920 x 1080 video to a 4:3 1440 x1080, then stretches the output back to 16×9 using non-square pixels (1:1.33).  this is similar to the way DV format handles wide screen in Standard Definition.

Next I had to get an exact copy of this disk to National Geographic in HD.  As Eric Portlow the video engineer wrote me “We can create a 1:1 copy of the original disc.  Because the Deck captures an MPEG HD file to disc and can be accessed as one would an external hard drive using File Access Mode (FAM) via an firewire interface, it is possible to create duplicate discs by importing all content from one disc to the computer, then exporting all files and directories to a blank disc.  The result is a lossless copy of the original. If you would like to have an exact duplicate of the original footage in XDCAM HD format, we could do this but it would require your having access to an XDCAM HD deck to view or access the file.

Our other option would have been to use HDCAM SR, DVCAM or DVCPRO

Stay tuned for the next post on HD in the OR hype vs reality

I’ve mentioned the National Geographic Special “The Living Body” and wanted to fill you in on why this is such an exciting project for me.  The show follows a woman from birth through life and death and traces the function of all of her bodily systems.  It will feature my surgical laparoscopy footage as the world’s first broadcast of surgery in HDTV.  To produce this I used a protype laparoscopy system that visualizes the highest resolution images ever seen of the human body. If you watch this show you’ll see endoscopic images on your living room HDTV set better than almost any surgeon has seen in the OR! 

I became involved when I received a call from the NG people and ther production team out of London.  They had heard of my work in developing HDTV endoscopic surgery and that I had performed the world’s first HDTV laparoscopy back in 2000.  This show was being produced in HDTV for braodcast in HD on the National Geographic HD channel (then downconverted to standard definition-SD for traditional broadcast and DVDs).  They were looking for some HD footage of internal organs. 

The original HD projects I did back in 2000 used a prototype 4-chip HD from JVC camera modified from a microscope camera for use in laparoscopic surgery.  At that time we had to use a w-vhs recorder as this was the only way to record the 1080i signal in HD. 

I became very excited at the prospect of helping them with this project since I had been testing a new endoscopic system that was unique in having the world’s highest reolution ever as well as the ability to record in full lossless HD using a new type of blu-ray HD DVD based recording system.

I recorded a series of shots for them of all the internal pelvic and abdominal organs and am delighted it made the final edit of the show.  You’ll see an egg developing inside an ovarian follicle, the bowel, liver and gall bladder, fallopian tubes, cervix and uterus.

My next post will review the technolgy between the HDTV system I used and some images from it.

Meet Jules, the most life-like robot you have ever seen. The robot is programmed with the most sophisticated artificial intelligence features.  It can recognize babies vs. adults, its creator and people he has ‘met’. 

video of “intelligent” android