From Navigation to Device Intelligence

Before I got involved with shape sensing, I spent years in electrophysiology, where knowing the location of a catheter inside the heart is fundamental to the procedure.

So when I was first introduced to fiber optic shape sensing, my mind immediately went to navigation. Here was a technology that could provide the continuous 3D shape of a catheter without relying on electromagnetic tracking or impedance measurements. I could immediately see the potential.

But after years of working with the technology, I don’t think navigation is the most interesting part anymore. Navigation answers an important question: where is the device?

The bigger opportunity, particularly in robotics, is understanding what the device is actually doing.

A long, flexible device doesn’t simply go where it’s intended to go. It bends and twists, encounters resistance, and can buckle or prolapse as it interacts with anatomy. What actually happens to the device isn’t necessarily apparent from the inputs to the robotic system or a tracked point at the distal tip.

That’s led me to think about shape sensing differently. It’s not simply another way to track a catheter. It’s a way to give a flexible device information about its own physical state. In robotics, that’s the beginning of something much more interesting: proprioception.

Knowing the input doesn’t tell you the result

When a physician operates a robotic system, the robot translates the physician’s input into mechanical actions: advancing, retracting, rotating or articulating the device. The system knows what its actuators are doing. What it doesn’t necessarily know is how the flexible device actually responded inside the body.

If an actuator pulls a steering wire to create a particular bend, for example, that doesn’t mean the catheter actually achieved the expected shape. There is anatomy in the way. The catheter may encounter resistance or bend differently than expected, and advancement at the proximal end doesn’t necessarily result in equivalent advancement at the distal end.

That’s a fundamental challenge with long, flexible tools. There can be a difference between the mechanical action applied at one end of the device and what the device actually does inside the patient.

The physician’s hands are part of the feedback loop

When physicians manipulate these devices manually, they don’t rely on vision alone. They use feel. A physician can feel when a device isn’t advancing normally or when it encounters resistance, and may intuitively rotate the device, pull back, change direction or try another maneuver. Experienced physicians make these adjustments constantly, often without consciously thinking through each individual movement.

When we put a robot between the physician and the device, some of that connection is lost. If the physician pushes a joystick forward and the robot simply translates that into forward actuator motion, we’ve changed the way the catheter is controlled, but we haven’t necessarily made the system smarter.

The opportunity is for the robot to have its own feedback about how the device is responding and use that information to help accomplish what the physician intended.

Knowing the tip isn’t the same as knowing the device

Tracking a distal tip is valuable, but tip position only tells part of the story. If the tip isn’t advancing, there could be many reasons why. The device could be bending somewhere along its length, buckling or prolapsing, or changing its configuration even though the distal position changes very little.

Knowing where the tip is doesn’t necessarily tell you what’s happening to the rest of the device.

This distinction is particularly important for flexible and continuum robots. Unlike a traditional rigid robot with discrete joints, a flexible instrument continuously changes shape, and its configuration can be difficult to infer from actuator inputs alone. This is one reason shape sensing is an active area of research in continuum robotics: sensing deformation provides information that can be used for state estimation and control.

Continuous shape sensing gives the system direct information about that configuration.

Start with stability

We don’t have to jump to autonomous surgery to see why that matters. Take stability: once a device reaches a target for a biopsy, therapy or another intervention, the job changes from getting there to staying there. If its configuration changes, the system can detect that change and potentially use the information as part of a control response.

This isn’t just theoretical. Intuitive’s Ion robotic bronchoscopy platform uses a fiber optic sensor to measure the full shape of its catheter hundreds of times per second. Intuitive combines those measurements with robotic control algorithms to help the catheter hold its position once it reaches a target.

That use is important because shape sensing isn’t simply creating another image for the physician to look at. The information can become feedback for the robot itself. The system can measure the resulting configuration of the device and use that information as part of its control loop.

Making the robot easier to use, not harder

Navigation is another example. A physician using a catheter manually does much more than push it forward. There is advancement, retraction, articulation and rotation, often happening together.

A robotic system shouldn’t require the physician to recreate every one of those hand movements through a joystick or a collection of controls. If we’re going to put a robot between the physician and the patient, the goal should be to make the procedure easier, not simply provide a different way of performing the same manual actions.

The more information the robot has about how its device is moving, the more it can potentially help. A physician should increasingly be able to tell the system what they want the device to accomplish while the robot handles more of the mechanics required to accomplish it. To do that well, the robot needs feedback about what is actually happening to the device.

From navigation to proprioception

This is why I increasingly think about shape sensing as a form of proprioception for flexible devices. Proprioception is our sense of the position and movement of our own body; we don’t need to watch our arm to know generally where it is or that it is moving.

A flexible surgical robot has no inherent equivalent. It knows the inputs to its actuation system, and it may have external imaging or a tracked point telling it something about where the device is. But those aren’t the same as continuously measuring the physical configuration of the device itself.

Shape sensing can provide part of that missing information. I emphasize part because it isn’t going to tell a robot everything it needs to know. There are other things we’d like an intelligent device to understand: force, contact, tissue characteristics, proximity to anatomy and other information specific to a procedure. No single sensor is going to provide all of that.

Over time, combining sensing technologies could give robotic systems a much richer understanding of what’s happening inside the patient. It also creates something we don’t really have when a physician manipulates a device by hand: a digital record of how the device actually behaved during the procedure. Eventually, data from many procedures could help robotic systems learn how devices respond to different situations and which maneuvers work.

That’s further down the road. Today, the opportunity is much more immediate: stability, smarter navigation, better feedback and the ability to begin closing the control loop.

Device intelligence

When I first encountered shape sensing, I saw a better way to navigate a catheter. I still think that’s valuable. But after spending years working with the technology and talking with companies developing the next generation of medical devices, I think the opportunity is much larger.

The future of medical devices is going to be increasingly intelligent, and intelligent devices need sensors that tell them what’s actually happening. For flexible surgical robots, that starts with understanding the device itself: how it bends, how it moves and what shape it is actually taking inside the patient.

The best robot won’t simply know what inputs went into moving its device. It will know how the device actually moved.

About the Author
Ryan DeBoer is CEO of The Shape Sensing Company and has spent two decades in the medical technology industry, including electrophysiology, where catheter localization and navigation are fundamental to clinical practice. Today, he works with medical device and robotics companies to integrate fiber optic shape sensing into the next generation of intelligent medical systems.