Patryk Filipiak, PhD, is a research scientist in the department of radiology at NYU Langone Health and its Center for Advanced Imaging Innovation and Research. Dr. Filipiak specializes in the development of neuroimaging methods and uses multiple modalities—including diffusion MRI, functional MRI, and other sensing techniques—to study brain connectivity in the presence of tumors. He holds a doctorate in computer science from University of Wrocław, Poland, and completed postdoctoral fellowships at Inria Sophia Antipolis, France, and NYU Grossman School of Medicine. Our conversation has been edited for clarity and length.
In broad strokes, how would you characterize the main area of your research at our center?
I’m most interested in how the brain’s structure and function changes due to pathology and how we can use this knowledge to help clinicians. I have particular interest in a type of brain tumors called glioma, because gliomas present important opportunities for tractography. One of the challenges in developing tractographic methods is that most of the time we don’t have validation that a visualized tract exists. But gliomas are often operated on using awake craniotomy, with the brain exposed while the patient is awake, and we can verify to a greater extent whether a tract exists.
What are some of the areas where advances in tractography may be especially useful to clinicians treating these kinds of conditions?
When a neurosurgeon is about to resect a tumor, they want to know how to do it safely—how to take the biggest margin possible without causing neurological deficits. That dilemma is slightly different for each patient, so a neurosurgeon wants to know what function is hidden in the area they’re considering resecting. Tractography can help visualize that, and we know that it’s generally accurate. The problem is that inaccuracies are greatest in proximity to pathology, which from the clinical perspective is the most crucial area.
High-grade gliomas grow very fast and the brain doesn’t have the time to compensate for the loss. Fast growth also causes a lot of damage around the tumor, with inflammation and vasogenic edema—an accumulation of liquid covering tissue that’s still functional but becomes a blind spot on MRI, especially diffusion MRI. That’s probably the biggest challenge for clinical tractography. This is an area I’m trying to make an improvement in and where better methods could help surgeons in the future.
And just to briefly define it here, tractography is an estimation of the location and direction of neural fibers based on the aggregate motion of water molecules sensed with diffusion MRI.
Yes, and this process gives structural information. What matters to neurosurgeons are the so-called eloquent fibers—fibers that are not just there but also in use. Our brains are very well equipped to deal with pathologies through brain plasticity. For example, when there’s a low-grade glioma—a relatively slowly growing tumor—the brain typically has time to compensate for the deficits that the tumor is causing by moving the function to another area. So, the structure doesn’t change that much, but the way the brain is using its structure—the functional connectivity—can be very dynamic.
Is that something that tractography on its own can shed light on?
Not by itself. One of the things I have particular interest in is exploring whether we can use all the modalities available in the current clinical practice to augment the information given by tractography in order to tell not only whether there is a fiber but also whether it’s likely an eloquent fiber.
What would be the other modalities that could potentially add up to this information?
The most obvious one—and one that has already been tested—is functional MRI. But what I believe we could add is microstructure modeling, because brain plasticity doesn’t only happen on the functional level; it also happens at the microstructural level. If it turns out that diffusion MRI is sensitive to these changes, I think that would be a significant breakthrough.
Can you talk about the probabilistic nature of tractography?
Actually, the tractography I work with the most is deterministic. Deterministic tractography always chooses the same directions whenever you run it.
But even in that case, the choice of direction is a statistical approximation, right?
I wouldn’t put it that way but I would agree that there are many factors, so the method is deterministic up to a point. However, this is different from a statistical approach, which is more like throwing dice every time you’re making a decision. I was actually a huge fan of probabilistic tractography, because I come from computer science and computer vision, so I like nice images, but when I confronted my images with clinicians, I realized that it’s a difficult conversation. Showing them tractography produced with a probabilistic approach required explanation, because what you’re looking at is not what we think is there but the distribution of what is there.
I guess most people would assume that a medical image is more directly representative of the underlying physical reality, whereas what you’re describing sounds more like a graphic representation of data.
Exactly—when you look at a map of, say, Manhattan, and you see a road, you want to trust that the road exists. If I show you a map where some roads are thick and some roads are thin, and the thin ones are not narrow but unlikely, it’s a different conversation.
A deterministic map wouldn’t have the thin lines, because they are low-probability lines, but every time you generate the map based on the same data, it would be the same. If you compare the two visually, you could say deterministic tractography doesn’t look as rich, but many people would say it’s more reliable.
Together with Steven Baete and colleagues, you have been working on making tractography more accurate in fibers crossing at shallow angles. Can you talk about what the crossing fiber problem is and why it’s important?
Crossing fibers are ubiquitous—they’re all over our brain. We have to find a way to reliably visualize them. Otherwise, we’re just showing a very simplistic image.
In your research with Dr. Baete and colleagues, you’ve developed a method able to reliably resolve fibers crossing at angles as shallow as 10 degrees, a significant improvement over alternatives reliable only to about 40-degree crossings. How were you able to do that and what kind of difference might that make?
It’s a paradigm shift. The concept behind our method has many fathers, including Fernando Boada and Steven Baete. Most of the time, when we reconstruct fibers, we follow the same steps: acquire diffusion MRI with as high accuracy as possible and use mathematical models to convert signal into orientation distribution functions (ODFs)—the statistical distribution of the most likely directions the water is diffusing in.
So, for unconstrained water an ODF would look like a sphere but for highly constrained water, the shape would resemble something like an hourglass.
You’re describing the 3D glyphs we use to represent ODFs. The problem is that often these glyphs are not very sharp. When you have an ODF that looks like a blob, you can’t tell whether there’s one fiber or two fibers crossing at a narrow angle. This is confusing for many reconstruction algorithms and has resulted in the famous 40-, 45-degree threshold below which things were too uncertain. The reason we’re facing this problem is that we’re looking at the blobs and trying to figure out their peaks.
But the statistical distribution is defined in a sphere, so by looking only at the maxima, we’re disregarding information outside the peaks. Fernando Boada had the insight that using the whole shape of ODFs could help us better understand what’s underneath. And with Steven Baete, they started implementing that through pattern matching, producing huge dictionaries of known configurations as ground truth, generating ODFs from those configurations, then using fingerprinting—which itself is not a new concept—to compare these dictionary items to data. It turned out that when you use entire shapes, you are more sensitive to tiny differences and able to distinguish between a single fiber and two fibers crossing at a narrow angle.
That gets at something you mentioned before, which is that when you’re developing tractographic methods, you rarely have access to biological ground truth. In this case, the research team has simulated a kind of truth in silico, but there’s still a relevant question of how that relates to biological truth. How do you determine that?
We have a bunch of reference methods we can use. They do not guarantee that our tractograms are correct but can give us good reasons to believe they’re not wrong, which is the one thing we can tell for sure.
The simplest approach is to do it in silico, because we can compare acquired data to the synthetic ground truth. But in our case, that wouldn’t have been fair. We tried using a more complex model for the input and a less complex model for the dictionary items, so at least we’d know they’re not the same, but that’s still fairly weak. Something else we did was collaborate with a manufacturer of diffusion phantoms who produced for us a bunch of crossing fibers at given angles, which we used for validation and comparisons. Yet another method were optic pathways in a mouse model. In primates the optic pathways split evenly between brain hemispheres—so it’s a coin-toss—but in rodents it’s 90 percent contralateral projections and 10 percent ipsilateral, and when we’re getting something close to that with tractography, we know we’re heading in the right direction.
Speaking of optic pathways, you and Dr. Baete and colleagues also played a role in the world’s first partial face and whole-eye transplant, performed in 2023 at NYU Langone. Can you talk about what you did and how that connects to your research?
That came about by serendipity. I was showing our results from optic pathways in mice to Walter Schneider, a professor at the University of Pittsburgh, and he told us that there’s someone at NYU Langone preparing for the first whole-eye transplant in the world who could use a longitudinal MRI study. That’s how we made the connection with Dr. Eduardo Rodriguez, who led the clinical team, and met Aaron James, the patient. At first, we didn’t know whether scans would be possible, so we started by imaging a phantom with the screws and plates the team was planning to use for the surgery, just to see how much artifacts all this metal would produce. The joke was that we were screwing up scans by scanning screws.
The artifacts were bad, but not bad enough to obfuscate the optic nerve. We scanned the patient once before surgery using the densest diffusion MRI sampling we could think of, and while the team waited for a donor, we were ready to potentially continue. And then Aaron got his transplant.
The imaging study focused on the connection of the transplanted eye to the patient’s optic nerve. What was the significance of that connection?
The goal was to ensure perfusion, just to make sure that there’s blood supply to the transplanted eye. There were no expectations regarding vision. But the anatomy of the optic nerve is relatively complex: there are axons, there’s vascular structure, and there’s meningeal sheath that surrounds the nerve. The surgical team had to connect all of these with the donor’s part. No one had ever done that before with a human optic nerve.
What did you learn from imaging that connection?
Our expectations were mostly to quantify microstructural changes in the optic nerve pre and post surgery. We were curious if we could probe them with diffusion MRI, to what extent the results would be consistent over time, and whether the findings would agree with ophthalmic examination. After the surgery, we knew from visual evoked potentials that stimuli were passing from the eye towards the brain, but we didn’t know whether the brain was receiving them or doing anything about them. The clinical team was particularly interested in functional MRI to see whether Aaron’s visual cortex was responding in any way. We designed studies that were simplifications of experimental paradigms for retinotopy—a very detailed mapping of the field of view onto the visual cortex—essentially, experiments with flashing lights, looking for activation.
At the beginning, we saw activation in the visual cortex that seemed to be caused by the stimulation of the transplanted eye. That just seemed too good to be true, so we started questioning ourselves and adding more validation. The hypothesis was that we were not covering the healthy eye well enough in experiments to prevent light from leaking in. And as we kept on adding extra layers of cover, we lost that response in the visual cortex. That was a lesson in humility. There were also constructive lessons, because we came up with a sampling scheme robust enough to withstand all the artifacts and noise and were able to propose a very simple microstructure model effective enough to obtain some useful information consistent with clinical indicators.
Before you came to NYU Langone, you did postdoctoral training at Inria Sophia Antipolis in France, working on diffusion MRI. But your PhD is in computer science. How did you go from computer science to applications in medical imaging?
There’s a part of me that’s still a computer scientist, but there’s another part that’s always been interested in medicine. I don’t have the right character to be a clinician, but I find the work clinicians do fascinating. At some point, I felt that I’d like to do something more useful than solving synthetic problems like the ones I worked on in my PhD, which was in dynamic optimization. I had an adventure with industry that I really enjoyed—doing computer-vision calibration for video cameras that monitor highway traffic in Germany—but still wanted to try and use computer science for medical applications. I started looking for a postdoc and came across Demian Wassermann at Sophia Antipolis in the south of France, who was hiring for a project on structural and effective brain connectivity. Demian thought he could use my experience in dynamic optimization to design a [diffusion] acquisition protocol, and we both took the risk.
That’s where you learned to work with MRI?
Yes, I had so much to catch up on. After my first year, Demian got a grant for a bigger project and moved to Paris. He had to leave what he had started in Sophia Antipolis because it was a collaboration with a hospital in Nice. His supervisor asked me whether I’d be up for taking over. Suddenly, my job was not only to process data but also to acquire it myself—this was electrocorticography, like EEG but placed directly on the brain. The first time the collaborating surgeon invited me to see the neurosurgery, I was terribly nervous. The surgical team was like a well-oiled mechanism; everything they did was perfectly choreographed. I didn’t really know where to stand. Eventually, I would have my table in the operating room with my laptop and cables to record the ECoG data. In all, I attended about twenty neurosurgeries.
So, you went from an intent to be closer to medicine to working inside an operating room. How did that affect your interest in medical research?
I thought I’d never be the same. The day after the first surgery, I was on a train to Paris, and I couldn’t think about anything else. It was a six-hour train ride, and the whole time I was just seeing the brain. One of the surprising things was that I had expected something bloody and gross but there was no blood at all—it was all suctioned off so that we didn’t even see it—and the brain was rather beautiful. A weird and in retrospect obvious thing was that the brain pulsates, too.
Speaking of beauty, tractography is known for colorful, vivid images, which are often the calling card of imaging research. Was the visual richness of tractography a part of the draw for you?
Of course. Tractography is a very nice introduction to medical imaging because the images are just so pretty—they’re art. I can see that my students enjoy them, too. Whenever I’m losing them in math, it’s good to show them some images and they perk up again.
How has being on the inside and working with the process by which tractograms are generated changed your relationship to their aesthetics?
It has become more important to me that the images are accurate. I remember the first time I showed the corticospinal tract reconstructed with our ODF fingerprinting method to Dimitris Placantonakis, a neurosurgeon at NYU Langone and our collaborator of many years. He said the image was spectacular. Hearing this word, ‘spectacular,’ from him was so rewarding. Making a clinician happy—that’s still my driver, my motivation.
Related Resources
A magnetic resonance fingerprinting method for tractography.