The New Age of Virtual Conferences

For decades, the scientific community has witnessed a growing trend towards online collaboration, publishing, and communication. The next natural step, started over the past decade, has been the emergence of virtual lectures, workshops, and conferences. My first virtual workshop took place back in about 2011 when I was asked to co-moderate a virtual session about 10 talks on MRI methods and neurophysiology. It was put on jointly by the International Society for Magnetic Resonance in Medicine (ISMRM) and the Organization for Human Brain Mapping (OHBM) and considered an innovative experiment at the time. I recall running it from a hotel room with spotty internet in Los Angeles as I was also participating in an in-person workshop at UCLA at the same time. It went smoothly, as the slides displayed well, speakers came through clearly, and, at the end of each talk, participants were able to ask questions by text which I could read to the presenter. It was easy, perhaps a bit awkward and new, but definitely worked and was clearly useful.

Since then, the virtual trend has picked up momentum. In the past couple of years, most talks that I attended at the NIH were streamed simultaneously using Webex. Recently, innovative use of twitter has allowed virtual conferences consisting of twitter feeds. An example of such twitter-based conferences is #BrainTC, which was started in 2017 and is now putting these on annually.

Using the idea started with #BrainTC, Aina Puce spearheaded OHBMEquinoX or OHBMx.  This “conference” took place on the Spring Equinox involving sequential tweets from speakers and presenters from around the world. It started in Asia and Australia and worked its way around with the sun during this first day of spring where the sun was directly above the equator and the entire planet had precisely the same number of hours of sunlight.

Recently, conferences with live streaming talks have been assembled in record time, with little cost overhead, providing a virtual conference experience to audiences numbering in the 1000’s at extremely low or even no registration cost. An outstanding recent example of a successful online conference is neuromatch.io. An insightful blog post summarized logistics of putting this on.

Today, the pandemic has thrown in-person conference planning, at least for the spring and summer of 2020, into chaos. The two societies with which I am most invested, ISMRM and OHBM, have taken different solutions to cancellations in their meetings. ISMRM has chosen to delay their meeting to August. ISMRM’s delay will hopefully be enough time for the current situation to return to normal, however, given the uncertainty of the precise timeline, even this delayed in-person meeting may have to be cancelled. OHBM has chosen to make this year’s conference virtual and are currently scrambling to organize it – aiming for the same start date in June that they had originally planned.

What we will see in June with OHBM will be a spectacular, ambitious, and extremely educational experiment. While we will be getting up to date on the science, most of us will also be having our first foray into a multi-day, highly attended, highly multi-faceted conference that was essentially organized in a couple of months.

Virtual conferences, now catalyzed by COVID-19 constraints, are here to stay. These are the very early days. Formats and capabilities of virtual conferences will be evolving for quite some time. Now is the time to experiment with everything, embracing all the available online technology as it evolves. Below is an incomplete list of the advantages, disadvantages, and challenges of virtual conferences, as I see them. 

What are the advantages of a virtual conference? 

1.         Low meeting cost. There is no overhead cost to rent a venue. Certainly, there are some costs in hosting websites however these are a fraction of the price of renting conference halls.

2.         No travel costs. No travel costs or time and energy are incurred for travel for the attendees and of course a corresponding reduction in carbon emissions from international travel. Virtual conferences allow an increased inclusivity to those who cannot afford to travel to conferences, potentially opening up access to a much more diverse audience – resulting in corresponding benefits to everyone.

3.         Flexibility. Because there is no huge venue cost the meeting can last as long or short as necessary and can take place for 2 hours a day or several hours interspersed throughout the day to accommodate those in other time zones. It can last the normal 4 or 5 days or can be extended for three weeks if necessary. There will likely be many discussions on what the optimal virtual conference timing and spacing should be. We are in the very early days here.

5.         Ease of access to information within the conference. With, hopefully, a well-designed website, session attendance can be obtained with a click of a finger. Poster viewing and discussing, once the logistics are fully worked out, might be efficient and quick. Ideally, the poster “browsing” experience will be preserved. Information on poster topics, speakers, and perhaps a large number of other metrics will be cross referenced and categorized such that it’s easy to plan a detailed schedule. One might even be able to explore a conference long after it is completed, selecting the most viewed talks and posters, something like searching articles using citations as a metric. Viewers might also be able to rate each talk or poster that they see, adding to usable information to search.

6.         Ease of preparation and presentation. You can present from your home and prepare up to the last minute in your home.

7.         Direct archival. It should be trivial to directly archive the talks and posters for future viewing, so that if one doesn’t need real-time interaction or misses the live feed, one can participate in the conference any time in the future at their own convenience. This is a huge advantage that is certainly also possible even for in-person conferences, but has not yet been achieved in a way that quite represents the conference itself. With a virtual conference, there can be a one-to-one conference “snapshot” preservation of precisely all the information contained in the conference as it’s already online and available.

What are the disadvantages of a virtual conference?

1.         Socialization. To me the biggest disadvantage is the lack of directly experiencing all the people. Science is a fundamentally human pursuit. We are all human, and what we communicate by our presence at a conference is much more than the science. It’s us, our story, our lives and context. I’ve made many good friends at conferences and look forward to seeing them and catching up every year. We have a shared sense of community that only comes from discussing something in front of a poster or over a beer or dinner. This is the juice of science. At our core we are all doing what we can towards trying to figure stuff out and creating interesting things. Here we get a chance to share it with others in real time and gauge their reaction and get their feedback in ways so much more meaningful than that provided virtually. One can also look at it in terms of information. There is so much information that is transferred during in-person meetings that simply cannot be conveyed with virtual meetings. These interactions are what makes the conference experience real, enjoyable, and memorable, which all feeds into the science.

2.         Audience experience. Related to 1, is the experience of being part of a massive collective audience. There is nothing like being in a packed auditorium of 2000 people as a leader of the field presents their latest work or their unique perspective. I recall the moment I first saw the first preliminary fMRI results presented by Tom Brady at ISMRM. My jaw dropped and I looked at Eric Wong, sitting next to me, in amazement. After the meeting, there was a group of scientists huddled in a circle outside the doors talking excitedly about the results. FMRI was launched into the world and everyone felt it and shared that experience. These are the experiences that are burnt into people’s memories and which fuel their excitement.

3.         No room for randomness. This could be built into a virtual conference, however at an in-person conference, one of the joys is to experience first-hand, the serendipitous experiences – the bit of randomness. Chance meetings of colleagues or passing by a poster that you didn’t anticipate. This randomness is everywhere at a conference venue perhaps more important than we realize. There may be clever ways to engineer a degree of randomness into a virtual conference experience, however.

4.         No travel. At least to me, one of the perks of science is the travel. Physically traveling to another lab, city, country, or continent is a deeply immersive experience that enriches our lives and perspectives. On a regular basis, while it can turn into a chore at times, is almost always worth it. The education and perspective that a scientist gets about our world community is immense and important.

5.         Distraction. Going to a conference is a commitment. The problem I always have when a conference is in my own city is that as much as I try to fully commit to it, I am only half there. The other half is attending to work, family, and the many other mundane and important things that rise up and demand my attention for no other reason than I am still here in my home and dealing with work. Going to a conference separates one from that life, as much as can be done in this connected world. Staying in a hotel or AirBnB is a mixed bag – sometimes delightful and sometimes uncomfortable. However, once at the conference, you are there. You assess your new surroundings, adapt, and figure out a slew of minor logistics. You immerse yourself in the conference experience, which is, on some level, rejuvenating – a break from the daily grind. A virtual conference is experienced from your home or office and can be filled with the distraction of your regular routine pulling you back. The information might be coming at you but the chances are that you are multi-tasking and interrupted. The engagement level during virtual sessions, and importantly, after the sessions are over, is less. Once you leave the virtual conference you are immediately surrounded by your regular routine. This lack of time away from work and home life I think is also a lost chance to ruminate and discuss new ideas outside of the regular context.

What are the challenges?

1.         Posters. Posters are the bread and butter of “real” conferences. I’m perhaps a bit old school in that I think that electronic posters presented at “real” conferences are absolutely awful. There’s no way to efficiently “scan” electronic posters as you are walking by the lineup of computer screens. You have to know what you’re looking for and commit fully to looking at it. There’s a visceral efficiency and pleasure of walking up and down the aisles of posters, scanning, pausing, and reading enough to get the gist, or stopping for extended times to dig in. Poster sessions are full of randomness and serendipity. We find interesting posters that we were not even looking for. Here we see colleagues and have opportunities to chat and discuss. Getting posters right in virtual conferences will likely be one of the biggest challenges. I might suggest creating a virtual poster hall with full, multi-panel posters as the key element of information. Even the difference between clicking on a title vs scrolling through the actual posters in full multi-panel glory will make a massive difference in the experience. These poster halls, with some thought, can be constructed for the attendee to search and browse. Poster presentations can be live with the attendee being present to give an overview or ask questions. This will require massive parallel streaming but can be done. An alternative is to have the posters up, a pre-recorded 3 minute audio presentation, and then a section for questions and answers – with the poster presenter being present live to answer in text questions that may arise and having the discussion text preserved with the poster for later viewing.

2.         Perspective. Keeping the navigational overhead low and whole meeting perspective high. With large meetings, there is a of course a massive amount of information that is transferred that no one individual can take in. Meetings like SFN, with 30K people, are overwhelming. OHBM and ISMRM, with 3K to 7K people, are also approaching this level. The key to making these meetings useful is creating a means by which the attendee can gain a perspective and develop a strategy for delving in. Simple to follow schedules with enough information but not too much, customized schedule-creation searches based on a wide rage of keywords and flags for overlap are necessary. The room for innovation and flexibility is likely higher at virtual conferences than at in-person conferences, as there are less constraints on temporal overlap. 

3.         Engagement. Fully engaging the listener is always a challenge, with a virtual conference it’s even more so. Sitting at a computer screen and listening to a talk can get tedious quickly. Ways to creatively engage the listener – real time feedback, questions to the audience, etc.. might be useful to try. Also, conveying effectively with clever graphics the size or relative interests of the audience might also be useful in creating this crowd experience.

4.         Socializing. Neuromatch.io included a socializing aspect to their conference. There might be separate rooms of specific scientific themes for free discussion, perhaps led by a moderator. There might also be simply rooms for completely theme-less socializing or discussion about any aspect of the meeting. Nothing will compare to real meetings in this regard, but there are some opportunities to potentially exploit the ease of accessing information about the meeting virtually to be used to enrich these social gatherings.

5.         Randomness. As I mentioned above, randomness and serendipity play a large role in making a meeting successful and worth attending. Defining a schedule and sticking to it is certainly one way of attacking a meeting, but others might want to randomly sample and browse and randomly run into people. It might be possible for this to be done in the meeting scheduling tool but designing opportunities for serendipity in the website experience itself should be given careful thought. One could decide on a time when they view random talks or posters or meet random people based on a range of keywords.

6.         Scalability. It would be useful to have virtual conferences constructed of scalable elements such as poster sessions, keynotes, discussion, proffered talks, that could start to become standardized to increase ease of access and familiarity across conferences of different sizes from 20 to 200,000 as it’s likely that virtual meeting sizes will vary more widely yet will be generally larger than “real” meetings.

7.         Costs vs. Charges? This will be of course determined on its own in a bottom up manner based on regular economic principles, however, in these early days, it’s useful to for meeting organizers to work through a set of principles of what to charge or if to make a profit at all. It is possible that if the web-elements of virtual meetings are open access, many of costs could disappear. However, for regular meetings of established societies there will be always be a need to support the administration to maintain the infrastructure.

Beyond Either-Or:

Once the unique advantages of virtual conferences are realized, I imagine that even as in-person conferences start up again, there will remain a virtual component, allowing a much higher number and wider range of participants. These conferences will perhaps simultaneously offer something to everyone – going well beyond simply keeping talks and posters archived for access – as is the current practice today.

While I have helped organize meetings for almost three decades, I have not yet been part of organizing a virtual meeting, so in this area, I don’t have much experience. I am certain that most thoughts expressed here have been thought through and discussed many times already. I welcome any discussion on points that I might have wrong or aspects I may have missed.

Virtual conferences are certainly going to be popping up at an increasing rate, throwing open a relatively unexplored wide open space for creativity with the new constraints and opportunities of this venue.  I am very much looking forward to seeing them evolve and grow – and helping as best I can in the process.

So I finally wrote a book…

One day, back in the mid 2010’s, feeling just a bit on top of my work duties, and more than a little ambitious, I decided that writing a book would be a worthwhile way to spend my extra time. I wanted to write an accessible book on fMRI, imbued with my own perspective of the field. Initially, I had thought of taking on the daunting task of writing a popular book on the story of fMRI – it’s origins and interesting developments (there are great stories there!) – but decided that I’ll put that off until my skill in that medium has improved. I approached Robert Prior from MIT Press to discuss the idea of a book on fMRI for audiences ranging from the interested beginner to expert. He liked it and, after about a couple of years of our trying to decide on the precise format, he approached me with he idea of making it part of the MIT Essential Knowledge Series. This is a series being put out by MIT Press containing relatively short “Handbooks” on a wide variety of topics with writing at the level of about a “Scientific American” article. Technical and accessible to anyone who has the interest but not overly technical or textbook dry – highly readable to people who want to get a good in depth summary of a topic or field from an expert. 


I agreed to give this a try. The challenge was that it had to be about 30K to 50K words and containing minimal figures with no color. The audience was tricky. I didn’t want to make it so simple as to present nuanced facts incorrectly and disgruntle my fellow experts, but I also didn’t want to have it go too much in depth on any particular issue thus leaving beginners wading through content that was not really enjoyable. My goal was to first describe the world of brain imaging that existed when fMRI was developed, and then outline some of the more interesting history from someone who lived it, all while giving essential facts about the technique itself. Later chapters deal with topics involving acquisition, paradigm design, processing and so forth – all while striving to keep the perspective broad and interesting. At the end of the book, I adopted a blog post as a chapter on the “26 controversies and challenges” of fMRI, adding a concluding perspective that while fMRI is mature, it still has more than its share of controversies and unknowns, and that these are in fact good things that keep the field moving along and advancing as they tend to focus and drive the efforts of many of the methodologists.

After all was done, I was satisfied with what I wrote and pleasantly surprised that my own unique perspective – that of someone who has been involved with the field since its inception – came clearly through. My goal, which I think I achieved, was incorporate as much insight into the book as possible, rather than just giving the facts. I am now in the early stages of attempting to write a book of the story of fMRI, perhaps adding perspective on where all this is eventually going, but for now I look forward to the feedback about this MIT Essential Knowledge Series book on fMRI. 


Some takeaway thoughts on my first major writing project since the composition of my Ph.D. thesis over 26 years ago. By the way, for those interested, my thesis can be downloaded from figshare: DOI and link below: 
 10.6084/m9.figshare.11711430. Peter Bandettini’s Ph.D. Thesis 1994.

  • I have to start by saying that these are just my reflections on the process and a few things that I found useful. I’m just a beginner when it comes to this, so take it all with a grain of salt.
  • Writing a book, similar to a chapter or paper or any large assignment, will never get done or started in any meaningful way unless it reaches the highest priority on your to-do list on a daily basis. If any of you are like me, you have these big projects that are always ranked at least 3 or lower on the list of priorities that we have for the day. We fully intend to get to them, but at the end of the day, they remain undone. It was only when I decided that writing will take precedence that I made any meaningful progress, so for the course of about 4 months, it was the first thing I worked on most days. 
  • This book took about 2 or so years longer to do than I anticipated. I had a few false starts and in the last year, had to re-write the book entirely. I was woefully behind deadline almost all the time. Thankfully Robert Prior was patient! I finally got into a regular rhythm – which is absolutely required – and made steady progress. 
  • It’s easy to lose track what you wrote in previous chapters and become repetitive. This is a unique and unanticipated problem that does not come up in papers or book chapters. So many chapters have some degree of thematic overlap (how does one easily separate acquisition strategies from contrast mechanisms or processing methods from paradigm designs?) When the chapters are written, there is so much content that one has to always go back to make sure information is not too repetitive. Some repetition is good, but too much of course is not good. 
  • It’s never perfect. I am not a perfectionist but, still, had to draw back on my impulses to continuously improve on what I wrote once it was all out on paper. With each read, I wanted to add something, when in fact, I needed to cut the content by 20K words. I had to eventually be satisfied that nothing is ever perfect and if it above a solid threshold, I needed to let go as there were diminishing returns. 
  • Getting words on paper (or computer screen) is the hard part, but should be done in the spirit of just plowing through. Editing written text – even badly written text – is much easier and more satisfying. 
  • Cutting is painful. On starting the book, I wondered how I was going to write so many words. On nearing completion of the first draft, I wondered how I was going to cut out so many words. I ended up eliminating three chapters all together. 
  • Every hour spend planning the book outline saves about 4 or more hours in writing…up to a point. It’s also good not to over plan as once you get into it, organizational changes will start cropping up naturally.
  • Writing this book revealed to myself where I have clear gaps and strengths. I learned a bit about my biases.  I know contrast mechanisms, pulse sequences, and all the interesting historical tidbits and major events very well. I have a solid sense of the issues, controversies, and importance of the advancements. While I’ve worked in processing and have a good intuition of good processing practices, I am not anywhere near a processing guru. I have to admit that I don’t really like statistics, although of course acknowledge their importance. Perhaps my Physicist bias comes through in this regard. I have the bias that if a result is dependent on the precise statistical model that is used it’s likely too small to be useful and not all that interesting. I’m learning to let go of that bias – especially in the age of Big Data. I’m a sucker for completely new and clever experimental designs – as esoteric as they may be – or a completely different way of looking at the data rather than a more “correct” way to look. My eyes glaze over when lectured on fixed effects or controlling for false positives. I crave fMRI results that jump out at me – that I can just see without statistics. I of course know better that  many if not most important fMRI results rely on good statistics, and for the method to be useful ultimately, it needs a solid foundation, grounded in proper models and statistics. That said, my feeling still is that we have not yet properly modeled the noise and the signal well enough to know what ground truth is. We should also remind ourselves that due to many sources of artifact, results may be statistically “correct” yet still not what we think we are seeing. Therefore, I did not dwell much on the details of the entire rapidly growing sphere of processing methods that much in the book. Rather I focussed on intuitively graspable and pretty basic processing concepts. I think I have a good sense of the strengths and weaknesses of fMRI and where it fits into wider fields of cognitive neuroscience and medicine, so throughout the book, my perspective on these contexts is provided. 
  • Overall, writing this book has helped refine and deepen my own perspective and appreciation on the field. It perhaps has also made me a slightly better communicator. Hopefully, I’ll have that popular book done in a year or so! 

Below is the preface to the book fMRI. I hope you will take a look and enjoy reading it when it comes out. Also, I welcome any feedback at all (good or bad). Writing directly to me via: bandettini@nih.gov will get my attention. 


Preface to FMRI:


In taking the first step and picking up this book, you may be wondering if this is just another book on fMRI (functional magnetic resonance imaging). To answer: This is not just another book on fMRI. While it contains all the basics and some of the more interesting advanced methods and concepts, it is imbued, for better or worse, with my unique perspective on the field. I was fortunate to be in the right place at the right time when fMRI first began. I was a graduate student at the Medical College of Wisconsin looking for a project. Thanks in large part to Eric Wong, my brilliant fellow graduate student who had just developed, for his own non-fMRI purposes, the hardware and pulse sequences essential to fMRI, and my co-advisors Scott Hinks and Jim Hyde who gave me quite a bit of latitude to find my own project, we were ready to perform fMRI before the first results were publicly presented by the Massachusetts General Hospital group on August 12, 1991, at the Society for Magnetic Resonance Meeting in San Francisco. After that meeting, I started doing fMRI, and in less than a month I saw my motor cortex light up when I tapped my fingers. As a graduate student, it was a mind-blowingly exciting time—to say the least. My PhD thesis was on fMRI contrast mechanisms, models, paradigms, and processing methods. I’ve been developing and using fMRI ever since. Since 1999, I have been at the National Institute of Mental Health, as chief of the Section on Functional Imaging Methods and director of the FunctionalMRI Core Facility that services over thirty principle investigators. This facility has grown to five scanners—one 7T and four 3Ts.


Thousands of researchers in the United States and elsewhere are fortunate that the National Institutes of Health (NIH) has provided generous support for fMRI development and applications continuously over the past quarter century. The technique has given us an unprecedented window into human brain activation and connectivity in healthy and clinical populations. However, fMRI still has quite a long way to go toward making impactful clinical inroads and yielding deep insights into the functional organization and computational mechanisms of the brain. It also has a long way to go from group comparisons to robust individual classifications.


The field is fortunate because in 1996, fMRI capability (high-speed gradients and time-series echo planar imaging) became available on standard clinical scanners. The thriving clinical MRI market supported and launched fMRI into its explosive adoption worldwide. Now an fMRI-capable scanner was in just about every hospital and likely had quite a bit of cheap free time for a research team to jump on late at night or on a weekend to put a subject in the scanner and have them view a flashing checkerboard or tap their fingers.


Many cognitive neuroscientists changed their career paths entirely in order to embrace this new noninvasive, relatively fast, sensitive, and whole-brain method for mapping human brain function. Clinicians took notice, as did neuroscientists working primarily with animal models using more invasive techniques. It looked like fMRI had potential. The blood oxygen level–dependent (BOLD) signal change was simply magic. It just worked—every time. That 5% signal change started revealing, at an explosive rate, what our brains were doing during an ever-growing variety and number of tasks and stimuli, and then during “rest.”


Since the exciting beginnings of fMRI, the field has grown in different ways. The acquisition and processing methods have become more sophisticated, standardized, and robust. The applications have moved from group comparisons where blobs were compared—simple cartography— to machine learning analysis of massive data sets that are able to draw out subtle individual differences in connectivity between individuals. In the end, it’s still cartography because we are far from looking at neuronal activity directly, but we are getting much better at gleaning ever more subtle and usefulinformation from the details of the spatial and temporal patterns of the signal change.  While things are getting more standardized and stable on one level, elsewhere there is a growing amount of innovation and creativity, especially in the realm of post-processing. The field is just starting to tap into the fields of machine learning, network science, and big data processing.


The perspective I bring to this book is similar to that of many who have been on the front lines of fMRI methodology research—testing new processing approaches and new pulse sequences, tweaking something here or there, trying to quantify the information and minimize the noise and variability, attempting to squeeze every last bit of interesting information from the time series—and still working to get rid of those large vessel effects! 


This book reflects my perspective of fMRI as a physicist and neuroscientist who is constantly thinking about how to make fMRI better—easier, more informative, and more powerful. I attempt to cover all the essential details fully but without getting bogged down in jargon and complex concepts. I talk about trade-offs—those between resolution and time and sensitivity, between field strength and image quality, between specificity and ease of use. 


I also dwell a bit on the major milestones—the start of resting state fMRI, the use and development of event-related fMRI, the ability to image columns and layers, the emergence of functional connectivity imaging and machine learning approaches—as reflecting on these is informative and entertaining. As a firsthand participant and witness to the emergence of these milestones, I aim to provide a nuanced historical context to match thescience.


A major part of fMRI is the challenge to activate the brain in just the right way so that functional information can be extracted by the appropriate processing approach against the backdrop of many imperfectly known sources of variability. My favorite papers are those with clever paradigm designs tailored to novel processing approaches that result in exciting findings that open up vistas of possibilities. Chapter 6 covers paradigm designs, and I keepthe content at a general level: after learning the basics of scanning and acquisition, learning the art of paradigm design is a fundamental part of doing fMRI well. Chapter 7 on fMRI processing ties in with chapter 6 and again, is kept at a general level in order to provide perspective and appreciation without going into too much detail.


Chapter 8 presents an overview of the controversies and challenges that have faced the field as it has advanced. I outline twenty-six of them, but there are many more. Functional MRI has had its share of misunderstandings, nonreproducible findings, and false starts. Many are not fully resolved. As someone who has dealt with all of these situations firsthand, I believe that they mark how the field progresses—one challenge, one controversy at a time. Someone makes a claim that catalyzes subsequent research, which then either confirms, advances, or nullifies it. This is a healthy process in such a dynamic research climate, helping to focus the field.


This book took me two years longer to write than I originally anticipated. I appreciate the patience of the publisher Robert Prior of MIT Press who was always very encouraging. I also thank my lab members for their constant stimulation, productivity, and positive perspective. Lastly, I want to thank my wife and three boys for putting up with my long blocks of time ensconced in my office at home, struggling to put words on the screen. I hope you enjoy this book. It offers a succinct overview of fMRI against the backdrop of how it began and has developed and—even more important—where it may be going.


The book “FMRI” can be purchased at MIT Press and Amazon, among other places:

MIT Press
Amazon


Starting a Podcast: NIMH Brain Experts Podcast

About a year or so ago, I was thinking of ways to improve NIMH outreach – to help show the world of non-scientists what NIMH-related researchers are doing. I wanted to not only convey the issues, insights, and implications of their work but also provide a glimpse into the world of clinical and basic brain research – to reveal the researchers themselves and what their day to day work looks like, what motivates and excites them, and what their challenges are. Initially, I was going to organize public lectures or a public forum, but the overall impact of this seemed limited. I wanted an easily accessible medium that also preserved the information for future access, so I decided to take the leap into podcasting. I love a good conversation and felt I was pretty good at asking good questions and keeping a conversation flowing. There have been so many great conversations that I have with my colleagues that I wish that I could have preserved and saved in some way. The podcast structure is slightly awkward (“interviewing” colleagues), and of course, there is always the pressure of not saying the wrong thing or not knowing some basic piece of information that I should know. I had and still have – for quite some time – much to learn with regard to perfecting this skill.

I decided to go through official NIMH channels to get this off the ground, and happily the people in the public relations department loved the idea. I had to provide them with two “pilot” episodes to make sure that it was all ok. Because the podcast was under the “official” NIMH label, I had to be careful not to say anything that could be misunderstood as an official NIMH position or at least I had to qualify any potentially controversial positions. Next were the logistics.


Before it started, I had to do a few things: pick an introduction musical piece and a graphic to show with the podcast. Also I had to pick a name for the podcast. I was introduced into the world of non-copyrighted music. I learned that there are many services out there that give you rights to a wide range of music for a flat fee. I used a website service: www.premiumbeat.com. I picked a tune that seemed thoughtful, energetic, and positive. As for the graphic, I chose an image that comes from a highly processed photo of a 3D printout of my own brain. It’s the image at the top of this post. Both the music and graphic were approved, and we finally arrived on a name “The Brain Experts” which pretty much what it was all about.


For in-person podcasts I use a multi-directional Yeti microphone and Quicktime on my Mac to record. This seems to work pretty well. I really should be making simultaneous backup recordings though – just in case IT decides to reboot my computer during a podcast. I purchased a muli-microphone & mixer setup to be used for future episodes. For remote podcasts, I use Zoom which has a super simple recording feature and has generally had the best performance of any videoconferencing software that I have used. I can also save only the audio files to a surprisingly small (much smaller than with Quicktime) file. Once the files are saved, it’s my responsibility to get them transcribed. There are many cheap and efficient transcription services out there. I also provide a separate introduction to the podcast and the guest – recorded at a separate time. Once the podcast and transcript are done, I send them to the public relations people, who do the editing and packaging.


The general format of the podcast is as follows: I interview the guest for about an hour and some of the interview is edited out – resulting in a podcast that is generally about 30 minutes in length. I wish it could be longer but the public relations people decided that 30 minutes was a good digestible time. I start with the guests’ backgrounds and how they got to where they are. I ask about what motivates them and what excites them. I then get into the science – the bulk of the podcast – bringing up recent work or perhaps discussing a current issue related to their own research. After that, I end by discussing any challenges they have going on, what their future plans are, and also if they had any advice to new researchers. I’ve been pleased that so far, no one has refused an offer to be on my podcast. I think most of gone well! I certainly learned quite a bit. Also, importantly, about a week before I interview the guests, I provide them with a rough outline of questions that I may ask and papers that I may want to discuss.


For the first four podcasts, I have chosen guests that I know pretty well: Francisco Pereira – an NIMH staff scientist heading up the Machine Learning Team that I started, Niko Kriegeskorte – a computational cognitive neuroscientist at Columbia University who was a former post doc of mine, Danny Pine – a Principle Investigator in the NIMH intramural program who has been a colleague of mine for almost 20 years, and Chris Baker – a Principle Investigator in  the NIMH intramural program who has been a co-PI with me in the Laboratory of Brain and Cognition at the NIMH for over a decade. Most recently, I interviewed Laura Lewis, from Boston University, who is working on some exciting advancements in fMRI methods that are near and dear to my heart. In the future I plan to branch out more to cover the broad landscape of brain assessment – beyond fMRI and imaging, however in these first few, I figured I would start in my comfort zone.


Brain research can be roughly categorized into: Understanding the brain, and Clinical applications. Of course, there is considerable overlap between the two, and the best research establishes a strong link between fundamental understanding and clinical implementation. Not all brain understanding leads directly to clinical applications as the growing field of artificial intelligence tries to glean organizational and functional insights from neural circuitry. The podcasts, while focused on a guest, each have a theme that is related to either of the above two categories. So far, Danny Pine has had a clinical focus – on the problem of how to make fMRI more clinically relevant in the context of psychiatric disorders, and Niko and Chris have had a more basic neuroscience focus. With Niko I focused on the sticky question of how relevant can fMRI be for informing mechanistic models of the brain. With Chris, we talked at length about the unique approach he takes to fMRI paradigm design and processing with regard to understanding visual processing and learning. Francisco straddled the two since machine learning methods promise to enhance both basic research and provide more powerful statistical tools for clinical implementation of fMRI.


In the future I plan to interview both intramural and extramural scientists covering the entire gamut of neuroscience topics. Podcasting is fascinating and exhausting. After each interview, I’m exhausted in that the level of “on” that I have to be is much higher in casual conversation. The research – even in areas that I know well – takes a bit of time, but is time well spent. Importantly, I try to not only glean over the topics, but dig for true insight into issues that we all are grappling with. The intended audience is broad: from the casual listener to the scientific colleague, so I try to guide the conversation to include something for everyone. The NIH agreed to 7 podcasts and it looks like they will wrap it up after the 7th due to the fact that they don’t have the personnel for the labor intensive editing and producing process, so it looks like I have one more to go. My last interview will be with Dr. Susan Amara, who is the director of the NIMH intramural program and will take place in December. I have other plans to continue podcasting, so stay tuned!

The podcasts can be found using most podcast apps: iTunes, Spotify, Castro, etc.. Just do a search for “NIMH Brain Experts Podcast.” 


The youtube versions of these can be found at https://www.youtube.com/playlist?list=PLV9WJDAawyhaMmciHR6SCwop-9BzsbsIl


The “official” posting of the first 6 podcasts can be found (with transcripts) here: 



Lastly, if you would like to be interviewed or know someone who you think would make a great guest, please give me an email at bandettini@nih.gov. I’m setting up my list now. The schedule is about one interview every three months.


We Don’t Need no Backprop

Companion post to: “Example Based Hebbian Learning may be sufficient to support Human Intelligence” on Biorxiv.

This dude learned in one example to do a backflip.

With the tremendous success of deep networks trained using backpropagation, it is natural to think that the brain might learn in a similar way. My guess is that backprop is actually much better at producing intelligence than the brain, and that brain learning is supported by much simpler mechanisms. We don’t go from Zero to super smart in hours, even for narrow tasks, as does AlphaZero. We spend most of our first 20 years slowly layering into our brains the distilled intelligence of human history, and now and then we might have a unique new idea. Backprop actually generates new intelligence very efficiently. It can discover and manipulates the huge dimensional manifolds or state spaces that describe games like go, and finds optimal mappings from input to output through these spaces with amazing speed. So what might the brain do if not backprop?

Continue reading “We Don’t Need no Backprop”

Twenty-Six Controversies and Challenges in fMRI

•Neurovascular Coupling•Draining Veins•Linearity•Pre-undershoot•Post-undershoot•Long duration•Mental Chronometry•Negative Signal Changes•Resting state source•Dead fish activation•Voodoo correlations•Global signal regression•Motion artifacts•The decoding signal•non-neuronal BOLD•relationship to other measures•contrast: spin-echo vs gradient-echo•contrast: SEEP contrast•contrast: diffusion changes•contrast: neuronal currents•contrast: NMR phase imaging•lie detection•correlation ≠ connection•clustering conundrum•reproducibility•dynamic connectivity changes


This will be a chapter in my upcoming book “Functional MRI” in the MIT Press Essential Knowledge Series 


Functional MRI is unique in that, in spite of being almost 30 years old as a method, it continues to progress in terms of sophistication of acquisition, hardware, processing, in our understanding of the signal itself. There has been no plateau in any of these areas. In fact, by looking at the literature, one gets the impression that this advancement is accelerating. Every new advance opens the potential range where it might have an impact, allowing new questions about the brain to be addressed.

In spite of its success – perhaps as a result of its success – it has had its share of controversies coincident with methods advancements, new observations, and novel applications. Some controversies have been more contentious than others. Over the years, I’ve been following these controversies and have at times performed research to resolve them or at least better understand them. A good controversy can help to move the field forward as it can focus and motivate groups of people to work on the issue itself, shedding a broader light on the field as these are overcome.

While a few of the controversies or issues of contention have been fully resolved, most remain to some degree unresolved. Understanding fMRI through its controversies allows a deeper appreciation for how the field advances as a whole – and how science really works – the false starts, the corrections, and the various claims made by those with potentially useful pulse sequences, processing methods, or applications. Below is the list of twenty-six major controversies in fMRI – in approximately chronological order.

#1: The Neurovascular coupling debate.

Since the late 1800’s, the general consensus, hypothesized by Roy and Sherrington in 1890 (1), was that activation-induced cerebral flow changes were driven by local changes in metabolic demand. In 1986, a publication by Fox et al. (2)challenged that view, demonstrating that with activation, localized blood flow seemingly increased beyond oxidative metabolic demand, suggesting an “uncoupling” of the hemodynamic response from metabolic demand during activation. Many, including Louis Sokolof, a well-established neurophysiologist at the National Institutes of Health, strongly debated the results. Fox nicely describes this period in history from his perspective in “The coupling controversy” (3).

I remember well, in the early days of fMRI, Dr. Sokolof standing up from the audience to debate Fox on several circumstances, arguing that the flow response should match the metabolic need and there should be no change in oxygenation. He argued that what we are seeing in fMRI is something other than an oxygenation change.

In the pre-fMRI days, I recall not knowing what direction the signal should go – as when I first laid eyes on the impactful video presented by Tom Brady during his plenary lecture on the future of MRI at the Society for Magnetic Resonance (SMR) Meeting in August of 1991, it was not clear from these time series movies of subtracted images the direction he performed the subtraction operation. Was it task minus rest or rest minus task? Did the signal go up or down with activation? I also remember very well, analyzing my first fMRI experiments, expecting to see a decrease in BOLD signal – as Ogawa, in an earlier paper(4), hypothesized that metabolic rate increases would lead to a decrease blood oxygenation thus a darkening of the BOLD signal during brain activation. Instead, all I saw were signal increases. It was Fox’s work that helped me to understand why the BOLD signal should increasewith activation. Flow goes up and oxygen delivery exceeds metabolic need, leading to an increase in blood oxygenation.

While models of neurovascular coupling have improved, we still do not understand the precise need for flow increases. First, we had the “watering the whole garden to feed one thirsty flower” hypothesis which suggested that flow increases matched metabolic need for one area but since vascular control was coarse, the abundant oxygenation was delivered to a wider area than was needed, causing the increase in oxygenation. We also had the “diffusion limited” model, where it was hypothesized that in order to deliver enough oxygen to the furthest neurons from the oxygen supplying vessels, an overabundance of oxygen was needed at the vessel itself since the decrease of oxygen as it diffused from the vessel to the furthest cell was described as an exponential.  This theory has fallen a bit from favor as the increases in CMRO2or the degree to which the diffusion of oxygen to tissue from blood is limited tend to be higher than physiologic measures. The alternative to the metabolic need hypothesis involves neurogenic “feed-forward” hypotheses – which still doesn’t get at the “why” of the flow response.

Currently, this is where the field stands. We know that the flow response is robust and consistent. We know that in active areas, oxygenation in healthy brains always increases, however we just don’t understand specifically why it’s necessary. Is it a neurogenic, metabolic, or some other mechanism to satisfy some evolutionary critical need that extends beyond simple need for more oxygen? We are still figuring that out. Nevertheless, it can be said that whatever the reason for this increase in flow, it is fundamentally important, as the BOLD response is stunningly consistent.

#2: The Draining Vein Effect

The question: “what about the draining veins?” I think was first posited at an early fMRI conference by Kamil Ugurbil of the University of Minnesota. Here and for the next several years he alerted the community to the observation that, especially at low field, draining veins are a problem as they smear and distort the fMRI signal change such that it’s hard to know specifically where the underlying neuronal activation is with a high degree of certainty. In the “20 years of fMRI: the science and the stories” special issue of NeuroImage, Ravi Menon writes a thorough narrative of the “Great brain vs vein” debate (5).  When the first fMRI papers were published, only one, by Ogawa et al. (6), was at high field – 4 Tesla – and relatively high resolution. In Ogawa’s paper, it was shown that there was a differentiation between veins (very hot spots) and capillaries (more diffuse weaker activation in grey matter). Ravi followed this up with another paper (7)using multi-echo imaging, to show that blood in veins had an intrinsically shorter T2* decay than gray matter at 4T and appeared as dark dots in T2* weighted structural images yet appeared as bright functional hot spots in the 4T functional images. Because of the low SNR and CNR at 1.5T, allowing only the strongest BOLD effects to be seen, and because models suggested that at low field strengths, large vessels contributed the most to the signal, the field worried that all fMRI was looking at was veins – at least at 1.5T.

The problem of large vein effects is prevalent using standard gradient-echo EPI – even at high fields. Simply put, the BOLD signal is directly proportional to the venous blood volume contribution in each voxel. If the venous blood volume is high – as with the case of a vein filling a voxel – then the potential for high BOLD changes is high if there is a blood oxygenation change in the area. At high field, indeed, there is not much intravascular signal left in T2* weighted Gradient-echo sequences, however, the extravascular effect of large vessels still exists.  Spin-echo sequences (sensitive to small compartments) still are sensitive to the susceptibility effects around intravascular red blood cells within large vessels – even at 7T where intravascular contribution is reduced. Even with some vein sensitivity, promising high resolution orientation column results have been produced at 7T using gradient-echo and spin-echo sequences (8). The use of arterial spin labeling has potential as a method insensitive to large veins, although the temporal efficiency, intrinsic sensitivity, and brain coverage limitations blunt its utility. Vascular Space Occupancy (VASO), a method sensitive to blood volume changes, has been shown to be exquisitely sensitive to blood volume changes in small vessels and capillaries. Preliminary results have shown clear layer dependent activation using VASO where other approaches have provided less clear delineation(9).

Methods have arisen to identify and mask large vein effects – including thresholding based on percent signal change (large veins tend to fill voxels and thus exhibit a larger fractional signal change), as well as temporal fluctuations (large veins are pulsatile thus exhibit more magnitude and phase noise). While these seem to be workable solutions, they have not been picked up and used extensively. With regard to using the phase variations as a regressor to eliminate pulsatile blood and tissue, perhaps the primary reason for this not being adopted is because standard scanners do not produce these images readily, thus users do not have easy access to this information.

The draining vein issue is least problematic at voxel sizes larger than 2mm, as at these resolutions, mostly the region of activation- as defined as >1 cm “blob” is used and interpreted. Other than enhancing the magnitude of activation, vein effects do not distort these “blobs” thus are typically of no concern for low resolution studies. In fact, the presence of veins helps to amplify the signal in these cases. Spatial smoothing and multi-subject averaging – still commonly practiced – also ameliorate vein effects as they tend to be averaged out as each subject has a spatially variant macroscopic venous structure.

The draining vein problem is most significant where details of high resolution fMRI maps need to be interpreted for understanding small and tortuous activation patterns in the context of layer and column level mapping. So far no fully effective method works at this resolution as the goal is not to mask the voxels containing veins since there may be useful capillary and therefore neuronal effects still within the voxel. We need to eliminate vein effects more effectively on the acquisition side.

#3: The linearity of the BOLD response

The BOLD response is complex and not fully understood. In the early days of fMRI, it was found that BOLD contrast was both linear and nonlinear.  The hemodynamic response tends to overestimate activation at very brief (<3 seconds) stimulus durations (10)or at very low stimulus intensities (11).  With stimuli that were of duration of 2 seconds or less, the response was much larger than predicted by a linear system. The reasons for these nonlinearities are still not fully understood, however for interpreting transient or weak activations relative to longer duration activation, a clear understanding of the nonlinearity of neurovascular coupling across all activation intensities and durations needs to be well established.

#4: The pre-undershoot

The fMRI pre-undershoot was first observed in the late 90’s. With activation, it was sometimes observed that the fMRI signal, in the first 0.5 second of stimulation, first deflected slightly downwards before it increased (12, 13). Only a few groups were able to replicate this finding in fMRI however it appears to be ubiquitous in optical imaging work on animal models. The hypothesized mechanism is that before the flow response has a chance to start, a more rapid increase in oxidative metabolic rate causes the blood to become transiently less oxygenated. This transient effect is then washed away by the large increase in flow that follows.

Animal studies have demonstrated that this pre-undershoot could be enhanced by decreasing blood pressure. Of the groups that have seen the effect in humans, a handful claim that it is more closely localized to the locations of “true” neuronal activation. These studies were all in the distant past (>15 years ago) and since then, very few papers have come out revisiting the study of the elusive pre-undershoot. It certainly may be that it exists, but the precise characteristics of the stimuli and physiologic state of the subject may be critically important to produce it.

While simulations of the hemodynamic response can readily reproduce this effect (14), the ability to robustly reproduce and modulate this effect experimentally in healthy humans has proven elusive. Until these experiments are possible, this effect remains incompletely understood and not fully characterized.

#5: The post-undershoot

In contrast to the pre-undershoot, the post-undershoot is ubiquitous and has been studied extensively (15). Similar to the pre-undershoot, its origins are still widely debated. The basic observation is that following brain activation, the BOLD signal decreases and then passes below baseline for up to 40 seconds. The hypothesized reasons for this include: 1) a perseveration of elevated blood volume causing the amount of deoxyhemoglobin to remain elevated even though oxygenation and flow are back to baseline levels, 2) a perseveration of an elevated oxidative metabolic rate causing the blood oxygenation to decrease below baseline levels as the flow and total blood volume have returned to baseline states, 3) a post stimulus decrease in flow below baseline levels. A decrease in flow with steady state blood volume and oxidative metabolic rate would cause a decrease in blood oxygenation. Papers have been published arguing for, and showing evidence suggesting each of these three hypotheses, so the mechanism of the post undershoot, as common as it is, stays unresolved. It has also been suggested that if it is perhaps due to a decrease in flow, then this decrease might indicate a refractory period where neuronal inhibition is taking place (16). For now, the physiologic underpinnings of the post-undershoot remain a mystery.

#6: Long duration stimulation effects

This is a controversy that has long been resolved (17). In the early days of fMRI, investigators were performing the basic tests to determine if the response was a reliable indicator of neuronal activity and one of the tests was to determine if, with long duration steady state neuronal activation, the BOLD response remains elevated. A study by Kruger et al came out suggesting that the BOLD response habituated after 5 minutes (18). A counter study came out showing that with a flashing checkerboard on for 25 minutes, the BOLD response and flow response (measured simultaneously) remained elevated (19). It was later concluded that the stimuli in the first study was leading to some degree of attentional and neuronal habituation and not a long duration change in the relationship between the level of BOLD and the level of neuronal activity. Therefore, it is now accepted that as long as neurons are firing, and as long as the brain is in a normal physiologic state, the fMRI signal will remain elevated for the entire duration of activation.

#7: Mental Chronometry with fMRI.

A major topic of study over the entire history of fMRI has been how much temporal resolution can be extracted from the fMRI signal. The fMRI response is relatively slow, taking about two seconds to start to increase and, with sustained activation, takes about 10 seconds to reach a steady-state “on” condition.  On cessation, it takes a bit longer to return to baseline – about 10 to 12 seconds, and has a long post-stimulus undershoot lasting up to 40 seconds. In addition to this slow response, it has been shown that the spatial distribution in delay is up to four seconds due to spatial variations in the brain vasculature.

Given this sluggishness and spatial variability of the hemodynamic response, it may initially seem that there wouldn’t be any hope in sub-second temporal resolution. However, the situation is more promising than one would expect. Basic simulations demonstrate that, assuming no spatial variability in the hemodynamic response, and given a typical BOLD response magnitude of 5% and a typical temporal standard deviation of about 1%, after 11 runs of 5 minutes each, a relative delay of 50 to 100ms could be discerned from one area to the next. However, the spatial variation in the hemodynamic response, is plus or minus 2 seconds depending on where one is looking in the brain and depends mostly on what aspect of the underlying vasculature is captured with each voxel. Large veins tend to have longer delays.

Several approaches have attempted to bypass or to calibrate the hemodynamic response. One way of bypassing the slow variable hemodynamic response problem is to modulate the timing of experiment, so the relative onset delays with task timing delays can be observed. This is using the understanding that in each voxel, the hemodynamic response is extremely well behaved and repeatable. Using this approach, relative delays of 500ms to 50 ms have been discerned (20-23). While these measures are not absolute, they are useful in discriminating which areas show delay modulations with specific tasks. This approach has been applied – to 500ms accuracy – to uncover the underlying dynamics and the relative timings of specific regions involved with word rotation and word recognition (23). Multivariate decoding approaches have been able to robustly resolve sub second (and sub-TR) relative delays in the hemodynamic response(24). By slowing down the paradigm itself, Formisano et al. (25)have been able to resolve absolute timing of mental operations down to the order of a second.  The fastest brain activation on-off rate that has been able to be resolved has recently been published by Lewis et al.(26)and is in the range of 0.75 Hz. While this is not mental chronometry in the strict sense of the term, it does indicate an upper limit at which high-speed changes in neuronal activation may be extracted.

#8: Negative signal changes.

Negative BOLD signal changes were mostly ignored for the first several years of fMRI as researchers did not know precisely how to interpret them. After several years, with a growing number of negative signal change observations, the issue arose in the field, spawning several hypotheses to explain them. One hypothesis invoked a “steal” effect, where active regions received an increase in blood flow at the expense of adjacent areas which would hypothetically experience a decrease in flow. If flow decreases from adjacent areas, these areas would exhibit a decrease in BOLD signal but not actually be “deactivated.” Another hypothesis was that these were areas that were more active during rest, thus becoming “deactivated” during a task as neuronal activity was re-allocated to other regions of the brain. A third was that they represented regions that were actively inhibited by the task. While in normal healthy subjects, the evidence for the steal effects is scant, the other hypotheses are clear possibilities. In fact, the default mode network was first reported as a network that showed consistent deactivation during most cognitive tasks(27). This network deactivation was also seen in the PET literature(28). A convincing demonstration of neuronal suppression associated with negative BOLD changes was carried out by Shmuel et al (29)showing simultaneous decreased neuronal spiking and decreased fMRI signal in a robustly deactivated ring of visual cortex surrounding activation to a ring annulus. These observations seem to point to the idea that the entire brain is tonically active and able to be inhibited by activity in other areas through several mechanisms. This inhibition is manifest as a decrease in BOLD.

#9: Sources of resting state signal fluctuations

Since the discovery that the resting state signal showed temporal correlations across functionally related regions in the brain, there has been an effort to determine their precise origin as well as their evolutionary purpose. The predominant frequency of these fluctuations is in the range of 0.1 Hz, which was eye opening to the neuroscience community since, previously, most had not considered that neuronally-meaningful fluctuations in brain activity occurred on such a slow time scale. The most popular model for the source of resting state fluctuations is that spontaneously activated regions induce fluctuations in the signal. As measured with EEG, MEG, or ECoG, these spontaneous spikes or local field potentials occur across a wide range of frequencies. When this rapidly changing signal is convolved with a hemodynamic response, the resulting fluctuations approximate the power spectrum of a typical BOLD time series. Direct measures using implanted electrodes combined with BOLD imaging show a temporal correspondence of BOLD fluctuations with spiking activity(30). Recent work with simultaneous calcium imaging – a more direct measure of neuronal activation – has also shown a close correspondence both spatially and temporally with BOLD fluctuations(31), thus strongly suggesting that these spatially and temporally correlated fluctuations are in fact, neuronal. Mention of these studies is only the tip of a very large iceberg of converging evidence that resting state fluctuations are in fact related to ongoing, synchronize, spontaneous neuronal activity.

While the basic neurovascular mechanisms behind resting state fluctuations may be understood to some degree, the mystery of the origins and purpose of these fluctuations remains. To complicate the question further, it appears that there are different types of correlated resting state fluctuations. Some are related to the task being performed. Some may be related to brain “state” or vigilance. Some are completely insensitive to task or vigilance state. It has been hypothesized that the spatially broad, global fluctuations may relate more closely to changes in vigilance or arousal. Some are perhaps specific to a subject, and relatively stable across task, brain state, or vigilance state – reflecting characteristics of an individual’s brain that may change only very slowly over time or with disease. A recent study suggests that resting state networks, as compared in extremely large data sets of many subjects, reveal clear correlations to demographics, life style, and behavior (32).

Regarding the source or purpose of the fluctuations, some models simply state it’s an epiphenomenon of a network at a state of criticality – ready to go into action. The networks have to be spontaneously active to be able to transition easily into engagement. The areas that are typically most engaged together are resultantly fluctuating together during “rest.” In a sense, resting state may be the brain continually priming itself in a readiness state for future engagement. Aside from this issue there is the issue of whether or not there are central “regulators” of resting state fluctuations. Do the resting state fluctuations arise from individual nodes of circuits simply firing on their own or is there a central hub that sends out spontaneous signals to these networks to fire. There has also been growing evidence suggesting that this activity represents more than just a subconscious priming.  The default mode network, for instance, has been shown to be central to cognitive activity as rumination and self-directed attention.

Work is ongoing in trying to determine if specific circuits have signature frequency profiles that might help to differentiate them. Work is also ongoing to determine what modulates resting state fluctuations. So far, it’s clear that brain activation tasks, vigilance state, lifestyle, demographics, disease and immediately previous task performance can have an effect. There are currently no definitive conclusions as to the deep origin and purpose of resting state fluctuations.

#10: Dead Fish (false positive) Activation.

In about 2006, a poster at the Organization for Human Brain Mapping was published by Craig Bennett, mostly as a joke (I think) – but with the intent to illustrate the shaky statistical ground that fMRI was standing on with regard to the multiple comparisons problem. This study was picked up by the popular media and went a bit viral. The study showed BOLD activation in a dead salmon’s brain. Here, it was clearly suggested that fMRI based on BOLD contrast has some problems if it shows activation where there should be none. In fact, it was a clear indication of false positives that can happen by chance even if the appropriate statistical tests are not used. The basic problem in creation of statistical maps is that the maps not appropriately normalized to “multiple comparisons.” It’s known, that purely by chance, if enough comparison is made (in this case it’s one comparison every voxel), then some voxels will appear to have significantly changed in signal intensity. Bonferroni corrections and false discovery rate corrections are almost always used and are all available in current statistical packages. Bonferroni is likely too conservative as each voxel is not fully independent. False discovery rate is perhaps closer to the appropriate test. When using these, false activations are minimized, however, they can still occur for other reasons. The structure of the brain and skull, having edges which can enhance any small motion or system instability, resulting in false positives. While this poster brought out a good point and was perhaps among the most cited fMRI works in the popular literature and blogs, it failed to convey a more nuanced and important message, that no matter what statistical test is used, the reality is that the signal and the noise are not fully understood, therefore all are actually approximations of truth, subject to errors. That said, a well-designed study with clear criteria and models of activation as well as appropriately conservative statistical tests, will minimize this false positive effect. In fact, it is likely that we are missing much of what is really going on by using over-simplified models of what we should expect of the fMRI signal.

A recent paper by Gonzalez-Castillo et al (33)showed that with a more open model of what to expect from brain activation, and 9 hours of averaging, nearly all grey matter becomes “active” in some manner. Does this mean that there is no null hypothesis? Likely not, but the true nature of the signal is still not fully understood, and both false positives and false negatives permeate the literature.

#11: Voodoo correlations and double dipping.

In about 2009, Vul et al published a paper that caused a small commotion in the field in that it identified a clear error in neuroimaging data analysis. It listed the papers – some quite high profile – that used this erroneous procedure that resulted in elevated correlations. The basic problem that was identified was that studies were performing circular analysis rather than pure selective analysis(34). A circular analysis is a form of selective analysis in which biases involved with the selection are not taken into account. Analysis is “selective” when a subset of data is first selected before performing secondary analysis on the selected data. This is otherwise known as “double dipping.” Because data always contain noise, the selected subset will never be determined by true effects only. Even if the data have no true effects at all, the selected data will show tendencies that they were selected for.

So, a simple solution to this problem is to analyze the selected regions using independent data (not data that were used to select the regions). Therefore, effects and not noise will replicate. Thus, the results will reflect actual effects without bias due to the influence of noise on the selection.

This was an example of an adroit group of statisticians helping to correct a problem as it was forming in fMRI. Since this paper, the number of papers published with this erroneous approach has sharply diminished.

#12: Global signal regression for time series cleanup

The global signal is obtained simply by averaging up the MRI signal intensity in every voxel of the brain for each time point. For resting state correlation analysis, global variations of the fMRI signal are often considered nuisance effects and are commonly removed(35, 36)by regression of the global signal against the fMRI time series. However the removal of global signal has been shown artifactually cause anticorrelated resting state networks in functional connectivity analyses (37). Before this was known, papers were published showing large anticorrelated networks in the brain and interpreted as large networks that were actively inhibited by the activation of another network. If global regression was not performed, these so called “anti-correlated” networks simply showed minimal correlation with – positive or negative – the spontaneously active network. Studies have shown that the removing the global signal not only induces negative correlation, but distorts positive correlations – leading to errors in interpretation (38).  Since then the field has mostly moved away from global signal regression. However, some groups use it as it does clean up the artifactual signal to some degree.

The global signal has been studied directly. Work has shown that it is a direct measure of vigilance as assessed by EEG (39). The monitoring of the global signal may be an effective way to ensure that when the resting state data is collected, subjects are in a similar vigilance state – which can have a strong influence on brain connectivity (40). In general, the neural correlates of the global signal change fluctuations are still not fully understood, however, it appears that the field has reached a consensus that, as a pre-processing step, the global signal should not be removed. Simply removing the global signal will not only induce temporal artifacts that look like interesting effects but will remove potentially useful and neuronally relevant signal.

#13: Motion Artifacts

Functional MRI is extremely sensitive to motion, particularly in voxels that have a large difference in signal intensity relative to adjacent voxels. Typical areas that manifest motion effects are edges, sinuses and ear canals where susceptibility dropout is common. In these areas, even a motion of a fraction of a voxel can induce a large fractional signal change, leading to incorrect results. Motion can be categorized into task correlated, slow, and pulsatile. Work has been performed over the past 27 years to develop methods to avoid or eliminate in acquisition or post processing, motion induced signal changes. In spite of this effort, motion is still a major challenge today. The most difficult kind of motion to eliminate is task-correlated motion that occurs when a subject tenses up or moves during a task or strains to see a display. Other types of motion include slow settling of the head during a scan, rotation of the head, swallowing, pulsation of blood and csf, and breathing induced motion-like effects.

Typical correction for motion is carried out by the use of motion regressors that are obtained by most image registration software. An a hoc method for dealing with motion – that can be quite effective – include visual inspection of the functional images and manually choosing time series signals that clearly contain motion effects that can then be regressed out or “orthogonalized.” Other approaches include image registration and time series “scrubbing.” Scrubbing involves automated detection of “outlier” images and eliminating them from analysis. Other ways of working around motion have included paradigm designs that involve brief tasks such that any motion from the task itself is able to be identified as a rapid change whereas a change in the hemodynamic response is slow, thus allowing the signals to be separable by their temporal signatures.

In recent years, an effort has been made to proactively reduce motion effects by tracking optical sensors positioned on the head, and then feeding the position information back to the imaging gradients such that the gradients themselves are slightly adjusted to maintain a constant head position through changing the location or orientation of the imaging volume. The primary company selling such capability is KinetiCor.

A more direct strategy for dealing with motion is the implementation of more effective methods to keep the head rigid and motionless. This approach has included bite bars and plastic moldable head casts. These have some effectiveness in some subjects but run the risk of being uncomfortable – resulting in abbreviated scanning times or worse, more motion due to active repositioning during the scan due to discomfort of the subject.

Aside from the problem of motion of the head, motion of the abdomen, without any concomitant head motion, can have an effect on the MRI signal. With each breath, the lungs fill with air, altering the difference in susceptibility between the signal in the chest cavity and the outside air. This alteration has an effect on the main magnetic field that can extend all the way into the brain, leading to breathing induced image distortions and signal dropout. This problem is increased at higher fields where the effects of susceptibility differences between tissue are enhanced. Possible solutions to this problem are direct measurement of the magnetic field in the proximity to the head using a “field camera” such as the one sold by the Swiss company called Skope, and then perhaps using these dynamically measured field perturbations as regressors in post processing or by feeding this signal to the gradients and shims prior to data collection in an attempt to compensate.

In resting state fMRI, motion is even more difficult to identify and remove as slow motion or breathing artifacts may have similar temporal signatures. Also, if there are systematic differences between intrinsic motion in specific groups such as children or those with Attention Deficit Disorder (ADD), then interpretation of group differences in resting state fMRI results is particularly problematic as the degree of motion can vary with the degree to which individuals suffer from these disorders.

In high resolution studies, specifically looking at small structures at the base of the brain, motion is a major confound as with each cardiac cycle, the base of the brain physically moves. Solutions to this have included cardiac gating and simple averaging. Gating is promising, however signal changes associated with the inevitably varying TR, which would vary with the cardiac cycle length, need to be accounted for by so far imperfect post processing approaches.

A novel approach to motion elimination has been with the use of multi-echo EPI, allowing the user to differentiate BOLD effects from non-BOLD effects based on the signal fit to a T2* change model.

Another motion artifact is that which arises from through plane movement. If the motion is such that a slightly different head position causes a slightly different slice to be excited (with an RF pulse), then some protons will not experience that RF pulse and will have magnetization that is no longer in equilibrium (achieved by a constant TR). If this happens, even if the slice position is corrected, there will be some residual non-equilibrium signal remaining that will take a few TR’s to get back into equilibrium. Methods have been put forward to correct for this, modeling the T1 effects of the tissue, however these can also eliminate the BOLD signal changes, so this problem still remains.

Lastly, apparent motion can also be caused by scanner drift. As the scanner is running, gradients and gradient amplifiers and RF coils can heat up, causing a drift in the magnetic field as well as the resonance frequency of the RF coils, causing a slow shifting of the image location and quality of image reconstruction. Most vendors have implemented software to account for this, but it is not an ideal solution. It would be better to have a scanner that does not have this instability to begin with.

In general, motion is still a problem in the field and one which every user still struggles with, however it is becoming better managed as we gain greater understanding of the sources of motion, the spatial and temporal signatures of motion artifacts, and the temporal and spatial properties of the BOLD signal itself.

There’s a payoff waiting once complete elimination of motion and non-neuronal physiologic fluctuations in solved. The upper temporal signal to noise of an fMRI time series is no higher than about 120/1 due to physiologic noise setting an upper limit. If this noise source were to be eliminated, then the temporal signal to noise ratio would only be limited by coil sensitivity and field strength, perhaps allowing fMRI time series SNR values to approach 1000/1. This improvement would transform the field.

#14: Basis of the decoding signal

Processing approaches have advanced beyond univariate processing that involves comparison of all signal changes against temporal model with the assumption that there is no spatial relationship between voxels or even blobs of activation. Instead, fine-grained voxel wise patterns of activation in fMRI have been shown to carry previously unappreciated information regarding specific brain activity associated with a region. Using multivariate models which compare the voxel wise pattern of activity associated with each stimulus, detailed information related to such discriminations as visual angle and face identity, as well as object category have been differentiated(41). A model to explain these voxel-specific signal change patterns hypothesizes that while each voxel is not small enough to capture the unique pool of neurons that are selectively activated by specific stimuli, the relative population of neurons that are active in a voxel causes a modulation in the signal that, considered with the array of other uniquely activated vowels, makes a pattern that, while having limited meaningful macroscopic topographic information, conveys information as to what activity the functional area is performing. The proposed mechanism of these multi-voxel signal changes of sub voxel activations, taken as a pattern, convey useful and unique functional  information. Another proposed mechanism for these changes is that there remain subtle macroscopic changes that occur.

One study has attempted to answer the question of whether the source of pattern effect mapping is multi-voxel or sub-voxel by testing the performance of pattern effect mapping as the activation maps were spatially smoothed (42). The hypothesis was that if the information were sub-voxel and varied from voxel to voxel, then the performance of the algorithm would diminish with spatial smoothing. If the information was distributed at a macroscopic level, smoothing would improve detection. This study showed that both voxel-wise and multi-voxel information contributed in different areas to the multivariate decoding success, thus while the decoding signal is robust, the origins, are, as with many other fMRI signals, complicated and varying.

#15: Signal change but no neuronal activity?

Several years ago, Sirotin and Das reported that they observed a hemodynamic response where no neuronal activity was present. They recorded simultaneously hemodynamic and electrical signals in an animal model during repeated and expected periodic stimuli. When the stimulus was removed when the animal was expecting it, then there was no neuronal firing but a hemodynamic response remained (43). Following this controversial observation, there were some papers that disputed their claim, suggesting that, in fact, there was, in fact, very subtle electrical activity still present. The hemodynamic response is known to consistently over-estimate neuronal activity for very low level or very brief stimuli. This study appears to be an example of just such an effect, despite what the authors claimed. While there was no clear conclusion to this controversy, the study was not replicated. In general, a claim such as this is extremely difficult to make as it is nearly impossible to show that that something doesn’t exist when one is simply unable to detect it.

#16: Curious relationships to other measures

Over the years the hemodynamic response magnitude has been compared with other neuronal measures such as EEG, PET, and MEG. These measures show comparable effects, yet at times, there have been puzzling discrepancies reported. One example is the following paper (44), comparing the visual checkerboard flashing rate dependence of BOLD and of MEG. At high spatialstimulus frequencies, the flashing visual checkerboard showed similar monotonic relationships between BOLD and MEG. At low spatial frequency, the difference between BOLD and MEG was profound – showing a much stronger BOLD signal than MEG signal. The reasons for this discrepancy are still not understood yet studies like these are important for revealing differences where it is otherwise easy to assume that they are measuring very similar effects.

#17: Contrast mechanisms: spin-echo vs gradient-echo.

Spin-echo sequences are known to be sensitive to susceptibility effects from small compartments and gradient-echo sequences are known to be sensitive to susceptibility effects from compartments of all sizes(45). From this observation it has been incorrectly inferred that spin-echo sequences are be sensitive to capillaries rather than large draining veins. The mistake in this inferences is that the blood within draining veins is not taken into account. Within draining veins are small compartments – red blood cells. So, while spin-echo sequences may be less sensitive to large vessel extravascular effects, they are still sensitive to the intravascular signal in vessels where the blood signal is present.

There have been methods proposed for removing the intravascular signal, such as the application of diffusion gradients that null out any rapidly moving spins. However, spin-echo sequences have lower BOLD contrast than gradient-echo by at least a factor of 2 to 4, and with the addition of diffusion weighting, the contrast almost completely disappears.

Another misunderstanding is that spin-echo EPI is truly a spin-echo. EPI takes time to form an image, having a long “readout” window – at least 20 ms whereas with spin-echo sequences there is only a moment when the echo forms. All other times, the signal is being refocused by gradient-echoes – as with T2* sensitive gradient echo imaging. Therefore, it is nearly impossible in EPI sequences to obtain a perfect spin-echo contrast.  Most of spin-echo EPI contrast is actually T2* weighted. “Pure” spin-echo contrast is where the readout window is isolated to just the echo – only obtainable with multi-shot sequences. However, even at high field strengths, spin-echo sequences are considerably less sensitive to BOLD contrast than gradient-echo sequences.

There is hope for “pure” spin-echo sequences at very high field might be effective in eliminating large vessel effects as, at high field, blood T2 rapidly shortens, and therefore the intravascular signal contributes minimally to the functional contrast. Spin-echo sequence have been used at 7T to visualize extremely fine functional activation structures such as ocular dominance and orientation columns (46, 47).

For these reasons, spin-echo sequences have not caught on for all but a small handful of fMRI studies at very high field. If high field is used and a short readout window is employed, then spin-echo sequences may be one of sequences of choice, along with the blood volume sensitive sequence, VASO – for spatial localization of activation.

#18: Contrast mechanisms: SEEP contrast.

Over the years, after tens of thousands of observations of the fMRI response and the testing of various pulse sequence parameters, some investigators have claimed that the signal does not always behave in a manner that is BOLD-like. One interesting example appeared about 15 years ago. Here Stroman et al.(48)was investigating the spinal cord and failed to find a clear BOLD signal. On using a T2-weighted sequence they claimed to see a signal change that did not show a TE dependence and was therefore not T2-based. Rather they claimed that it was based proton density changes – but not perfusion. It was also curiously most prevalent in the spinal cord.

The origin of this signal has not been completely untangled and SEEP contrast has disappeared from the current literature. Those investigating spinal cord activation have been quite successful using standard BOLD contrast.

#19: Contrast Mechanisms: Activation-Induced Diffusion Changes.

Denis LeBihan is a pioneer in fMRI and diffusion imaging. In the early 1990’s, he was an originator of diffusion tensor mapping(49). Later, he advanced the concept of intra-voxel incoherent motion (IVIM)(50, 51). The idea behind IVIM is that with extremely low levels of diffusion weighting, a pulse sequence may become selectively sensitized to randomly, slowly flowing blood rather than free water diffusion.  The pseudo-random capillary network supports blood flow patterns that may resemble, imperfectly, rapid diffusion, and thus could be imaged as rapid diffusion using gradients sensitized to this high diffusion rate of random flow rather than pure diffusion. This concept was advanced in the late 1980’s and excited the imaging community in that it suggested that, if MRI were sensitive to capillary perfusion, it could be sensitive to activation-induced changes in perfusion. This contrast mechanism, while theoretically sound, was unable to be clearly demonstrated in practice as relative blood volume is only 2% and sensitization of diffusion weighting to capillary perfusion also, unfortunately, sensitizes the sequences to CSF pulsation and motion in the brain.

Le Bihan emerged several years later with yet another potential functional contrast that is sensitive not to hemodynamics but, hypothetically to activation-induced cell swelling. He claimed that diffusion weighting revealed that on activation, measurable decreases in diffusion coefficient occur in the brain. The proposed mechanism is that with neuronal activation, active neurons swell, thus increasing the intracellular water content. High levels of diffusion weighting are used to detect subtle activation-induced shifts in neuronal water content. A shift in water distribution from extracellular space which has slightly higher diffusion coefficient, to intracellular space which slightly lower diffusion coefficient, would cause an increase in signal in a highly diffusion weighted sequences.

While LeBihan has published these findings(52, 53), the idea has not achieved wide acceptance. First, the effect itself is relatively small and noisy, if present at all. Secondly, there have been papers published demonstrating that the mechanism behind this signal is vascular rather than neuronal (54). Lastly, and perhaps most importantly, many groups, including my own, have tried this approach, coming up with null results. If the technique gives noisy and inconsistent results, it is likely not going to compete with BOLD, regardless of how selective it is to neuronal effects. Of course, it’s always worthwhile to work on advancing such methodology!

#20: Contrast mechanisms: Neuronal Current Imaging.

In the late 90’s it was proposed that MRI is theoretically directly sensitive to electric currents produced by neuronal activity. A current carried by a wire is known to set up magnetic fields around the wire. These magnetic fields when superimposed on the primary magnetic field of the scanner can cause NMR phase shifts or, if the wires are very small and randomly distributed, an NMR phase dispersion – or a signal attenuation. In the brain, dendrites and white matter tracts behave as wires that carry current. Basic models have calculated that the field in the vicinity of these fibers can be as high as 0.2 nT(55). MEG in fact, does detect these subtle magnetic fields as they fall off near the skull. At the skull surface the magnetic fields produced are on the order of 100 fT, inferring that at the source, they are on the order of 0.1 nT.

Over the last 15 years, work has continued around the world toward the goal of using MRI to detect neuronal currents. This work includes the attempts to observe rapid activation-induced phase shifts and magnitude shifts in susceptibility weighted sequences as the superimposed field distortion would cause either a phase shift or dephasing, depending on the geometry. Using these methods, no conclusive results have been reported in vivo. Other attempted methods include “Lorentz” imaging(56). Here the hypothesis is that when current is passing through a nerve within a magnetic field, a net torque is produced, causing the nerve to move just a small amount – potentially detectable by well-timed diffusion weighting. Again, no clear results have emerged. More recent approaches are based on the hypothesis that the longitudinal relaxation properties of spins may be affected if in resonance with intrinsic resonances in the brain such as alpha (10Hz) frequencies(57, 58). Spin-locking approaches that involve adiabatic pulses at these specific frequencies aim to observe neuronal activity based changes in the resonance of the NMR tissue. In such manner, maps of predominant oscillating frequency may be made. Again, such attempts have resulted in suggestive but not conclusive results.

Many are working on methods to detect neuronal currents directly, however, by most calculations, the effect is likely an order of magnitude to small. Coupled with the fact that physiologic noise and even BOLD contrast would tend to overwhelm neuronal current effects, the challenge remains daunting. Such approaches would require extreme acquisition speed (to detect transient effects that cascade over 200 ms throughout the brain), insensitivity to BOLD effects and physiologic noise, yet likely an order of magnitude higher sensitivity overall.

#21: Contrast mechanisms: NMR phase imaging.

The idea of observing phase changes rather than magnitude changes induced by oxygenation changes is not a new one. Back in the early 90’s it was shown that large vessels show a clear NMR phase change with activation. It was suggested that this approach may allow the clean separation of large vessel from tissue effects when interpreting BOLD. In general, it is known that large, relative to a voxel size, changes in susceptibility induce a net phase shift as well.

The concept of observing phase changes has been revisited. Studies have suggested that large activated regions, while mostly showing magnitude changes, may act as a single large susceptibility perturber and induce a large shift in phase. These studies suggest that use of both phase and magnitude information would boost both sensitivity and specificity. This approach has not yet caught on in the field. Perhaps one reason for it not catching on is that vendors typically only provide magnitude reconstruction of the scanner data. Most users simply don’t have access to NMR phase information. It may also turn out that any gains are only very small, thus making the extra effort in obtaining double the data and spending twice the time on pre and post processing not worth the effort.

#22: fMRI for Lie Detection

Over the past 20 years, fMRI has provided evidence that the brain of someone who is lying is active in distinctly different ways than that which is telling the truth. There is additional prefrontal and parietal activity with story fabrication(59). There have been papers that have demonstrated this effect in many different ways. In fact, there exist studies that show not only that lie detection is possible but extraction of the hidden knowledge of the truth is also possible using fMRI(60).

Because of this success, companies touting MRI-based lie detection services have cropped up (e.g. No Lie MRI— http://noliemri.com/and CEPHOS—http://www.cephoscorp.com/). The problem with this is that lie detection has never been fully tested in motivated and at times psychopathic criminals and negative results are more common than not. Inconclusive fMRI based lie detection results, if allowed in court, could potentially bias favor on the defense because, a negative or inconclusive result would provide suggestion that the individual is innocent.

The difference between research success and success or readiness for implementation in public use illustrates much of what the rest of fMRI faces. In real world applications, there are many variables which make fMRI nearly uninterpretable. Generalizing from group studies or highly controlled studies on motivated volunteers to individual studies in patients or criminals is extremely challenging to say the least.

Nevertheless, in spite of scientific, legal, ethical, operational, and social hurdles, machine learning and classification methods may ultimately prove capable in interpreting individual subject activation in association with lie detection with actual criminals or other real-world situations. There may in fact be regions of the brain or patterns of activity that tell truth from lies regardless of whether the person is a psychopathic hardened criminal or a motivated college student volunteer. No one has done those comparison studies yet.

#23: Does Correlation Imply Connectivity?

In resting state and sometimes task based fMRI, the correlation between voxels or regions is calculated. A growing trend is to replace the word correlation with connectivity. The assumption is that a high temporal correlation between disparate regions of the brain directly implies a high level of functional connectivity. It’s also assumed that any changein correlation between these regions implies that there is a corresponding change in connectivity. As a first approximation, these statements may be considered true, however there are many situations in which they are not true.

First, other processes can lead to a high correlation between disparate regions. Bulk movement, cardiac pulsation, and respiration are three primary sources of artifactual correlation. For the most part these are well dealt with – as motion is relatively easily identified and cardiac pulsation is at a distinct frequency (1Hz) that is fortunately far from resting state correlation frequencies (0.1 Hz). However aliased cardiac frequencies and respiration-induced correlations present a bit more of a challenge to remove. Respiration also can create signal changes that show T2* changes, so multi-echo sequences optimized to separate BOLD from non-BOLD effects would be less effective in removing respiration effects. Respiration is however identifiable by the fact that it is more spatially diffuse than correlations between distinct regions. This separation based on spatial pattern and spatial diffusivity is still extremely difficult to perform robustly and without error.

Modulations in correlation can occur for a number of reasons. First, if, when looking at a pair of regions that show correlation, if both signals contain noise (as they all do), an increase in the amplitude of one signal will naturally increase the correlation value, but no “real” increase in connectivity likely occurred. Likewise, if the noise in one or both of the signals is reduced, then, again, there will be a measurable reduction in correlation but likely no change in actual functional connectivity. If the relative latency or shape of one of the responses changes, then a change in correlation will occur without a change in connectivity perhaps. Let’s say an additional frequency was added from another oscillating signal that has nothing to do with the signal driving the “connectivity” between the two regions. If this happens, then again, the correlation between the two signals will be reduced without the connectivity between the regions being altered.

All of the above issues have not been addressed in any systematic manner as we are still in the early days of just figuring out the best ways to cleanly and efficiently extract correlation data. In the future, in order to make more meaningful interpretations of these signals, we will need to control for a the potentially confounding effects mentioned above.

#24: The clustering conundrum.

In 2016, a bombshell paper by Eklund et al (61)identified an error in most of the statistical fMRI packages, including SPM, FSL, and AFNI. Quoting a part of the abstract of that paper:

In theory, we should find 5% false positives (for a significance threshold of 5%), but instead we found that the most common software packages for fMRI analysis (SPM, FSL, AFNI) can result in false-positive rates of up to 70%. These results question the validity of a number of fMRI studies and may have a large impact on the interpretation of weakly significant neuroimaging results.”

The results showed that the packages correctly inferred statistical significance when considering independent voxels, however, when considering clusters of voxels as a single activation – as most activations are considered “clusters” or spatially correlated activations –  the estimations of cluster sizes, smoothness, or the statistical threshold for what constitutes a “significantly activated” cluster were incorrect, leading to incorrectly large clusters.

The implication of their paper was that for the past 15 years, these commonly used packages have overestimated activation extent. For well-designed studies with highly significant results, this would have virtually no effect on how results are interpreted. The same conclusions would be likely have been made. Most studies’ conclusions do not rely on absolute cluster size for their conclusions. Instead they draw conclusions based on the center of mass of activation, or whether a region was activated or not. Again, these studies would not be significantly affected.

Perhaps the greatest implications might be for pooled data in large data sets. If incorrectly large clusters are averaged across thousands of studies, then perhaps errors in interpretation of the extent of activation my crop up.

Unfortunately, the paper also went on to make a highly emotive statement: “These results question the validity of some 40,000 fMRI studies and may have a large impact on the interpretation of neuroimaging results.” Later in a correction, they removed the figure of 40,000 papers. Unfortunately, the media picked up on this figure with the sensational statement that most fMRI studies are “wrong,” which in itself is completely untrue by most definitions of the word “wrong.” The studies may have simply slightly overestimated the size of activation. If a paper relied on this error to reach a conclusion, it was naturally treading on thin statistical ice in the first place and would likely be in question anyway. Most activation (and most of the voxels in these clusters found) are highly significant.

After many popular articles raised concern in the public, the issue eventually died down. Most of the scientists in the field who understood the issue were completely unfazed as the slightly larger clusters did not have any influence on the conclusions drawn by their papers or papers that they relied on for guiding their work.

This brings up an issue related to statistical tests. All statistical tests involve estimates on the nature of the signal and noise, so, by definition are always somewhat “wrong.” They are close enough however. So much more goes into the proper interpretation of results. Most seasoned fMRI practitioners have gained experience in not over interpreting aspects of the results that are not extremely robust.

While the Eklund paper has performed a service to the field by alerting it to a highly propagated error, it also raised false concerns about the high-level of robustness and reproducibility of fMRI in drawing inferences about brain activation. Yes, brain activation extent has been slightly overestimated, but no, most of the papers produced using these analyses need to change in any manner, their conclusions.

#25: The issue of reproducibility

While fMRI is a robust and reproducible method, there is room for improvement. In science in general, it’s been suggested that up to 70% of published results have failed to be reproduced. The reasons for this high fraction may be in part related to what we mean by “successfully reproduced” as well as pressure to publish findings that push the limits of what may be a reasonable level of interpretation. Some might argue that this number is an indication of the health of the scientific process by which a study’s result either lives or dies based on whether it is replicated. If a study is not able to be replicated, the conclusions generally fade away from acceptance.

One researcher, Russ Poldrack, of Stanford is spearheading an effort to increase transparency and reproducibility of fMRI data as he heads the Stanford Center for Reproducible Science. The goal is to increase further the reproducibility of fMRI and therefore move the field towards a more full realization of its potential and less wasted work in the long run. He specifically is encouraging more replication papers, more shared data and code, more “data papers” contributing a valuable data set that may be re-analyzed, and an increased number of “registered studies” where the hypotheses and methods are stated up front before any data are collected. All of these approaches will make the entire process of doing science with fMRI much more transparent and able to be better leveraged by a growing number of researchers as they develop better processing methods or generate more penetrating hypotheses.

#26: Dynamic Connectivity Changes

The most recent “controversy” in the field of fMRI revolves around the interpretation of resting state scans. Typically, in the past, maps of resting state connectivity were created from entire time series lasting up to 30 minutes in length. The assumption was that brain connectivity remained somewhat constant over these long time periods. There is evidence that this assumption is not correct. In fact, it’s been shown that the connectivity profile of the brain changes fluidly over time. Today, fMRI practitioners now regard individual fMRI scans as rich 4D datasets with meaningful functional connectivity dynamics (dFC) requiring updated models able to accommodate this additional time-varying dimension. For example, individual scans today are often described in terms of a limited set of recurring, short-duration (tens of seconds), whole-brain FC configurations named FC states. Metrics describing their dwell times, ordering and frequency of transitions can then be used to quantify different aspects of empirically observed dFC. Many questions remain both about the etiology of empirically observed FC dynamics; as well as regarding the ability of models, such as FC states, to accurately capture behavioral, cognitive and clinically relevant dynamic phenomena.

Despite reports of dFC in resting humans, macaques and rodents, a consensus does not exist regarding the underlying meaning or significance of dFC while at rest. Those who hypothesize it to be neuronally relevant have explored resting dFC in the context of consciousness, development and clinical disorders. These studies have shown how the complexity of dFC decreases as consciousness levels decline; how  dynamic inter-regional interactions can be used to predict brain maturity, and how dFC derivatives (e.g., dwell times) can be diagnostically informative for conditions such as schizophrenia, mild cognitive impairment, and autism; to name a few.

Yet, many others have raised valid concerns regarding the ability of current dFC estimation methods to capture neuronally relevant dFC at rest. These concerns include lack of appropriate null models to discern real dynamics from sampling variability, improper pre-processing leading to spurious dynamics, and excessive temporal smoothing (a real concern for sliding window techniques used to estimate FC states) that hinder our ability to capture sharp and rapid transitions of interest (62). Finally, some have even stated that resting dFC is primarily a manifestation of sampling variability, residual head motion artifacts, and fluctuations in sleep state; and as such, mostly irrelevant.

One reason causing such discrepant views is that it is challenging to demonstrate the potential cognitive correlates of resting dFC, given the unconstrained cognitive nature of rest and scarcity of methods to infer the cognitive correlates of whole-brain FC patterns. When subjects are instructed to quietly rest, retrospective reports demonstrate that subjects often engage in a succession of self-paced cognitive processes including inner speech, musical experience, visual imagery, episodic memory recall, future planning, mental manipulation of numbers, and periods of heightened somatosensory sensations. Reconfigurations of FC patterns during rest could, to some extent, be a manifestation of this flow of covert self-paced cognition; even if other factors (e.g., random exploration of cognitive architectures, fluctuations in autonomic system activity and arousal levels, etc.) also contribute.

Research is ongoing in fMRI to determine the neural correlates of dynamic connectivity changes, to determine the best methods for extracting this rich data, as well as to determine how this information may be used to better understand ongoing cognition in healthy and clinical populations and individuals.

In conclusion, other interesting controversies also come to mind – such as BOLD activation in white matter, BOLD tensor mapping, the problem of reverse inference from fMRI data, the difference between mapping connectivity (ahem..correlation) vs mapping fMRI magnitude, inferring brain region causation from different brain regions during activation using fMRI, and other non-BOLD contrast mechanisms such as temperature or elasticity. Challenges also come to mind – including the robust extraction of individual similarities and differences from fMRI data, the problem of how to best parcellate brains, the creation of fMRI-based “biomarkers,” and potential utility of gradient coils.

The field of fMRI has advanced and further defined itself through many of these controversies. In my opinion, these indicate that the field is still growing and is quite robust and dynamic. Also, in the end, a consensus is usually reached – thus pushing the understanding forward. Controversies are healthy and welcomed. Let’s keep them coming!

References 

  1. Roy CS, Sherrington CS. On the regulation of the blood-supply of the brain. J Phsiol. 1890;11:85-108.
  2. Fox PT, Raichle ME. Focal physiological uncoupling of cerebral blood flow and oxidative metabolism during somatosensory stimulation in human subjects. Proc Natl Acad Sci USA. 1986;83:1140-4.
  3. Fox PT. The coupling controversy. NeuroImage. 2012;62(2):594-601.
  4. Ogawa S, Lee TM, Nayak AS, Glynn P. Oxygenation-Sensitive Contrast in Magnetic-Resonance Image of Rodent Brain at High Magnetic-Fields. Magnetic Resonance in Medicine. 1990;14(1):68-78.
  5. Menon RS. The great brain versus vein debate. NeuroImage. 2012;62(2):970-4.
  6. Ogawa S, Tank DW, Menon R, Ellermann JM, Kim SG, Merkle H, et al. Intrinsic Signal Changes Accompanying Sensory Stimulation – Functional Brain Mapping with Magnetic-Resonance-Imaging. Proceedings of the National Academy of Sciences of the United States of America. 1992;89(13):5951-5.
  7. Menon RS, Ogawa S, Tank DW, Ugurbil K. Tesla Gradient Recalled Echo Characteristics of Photic Stimulation-Induced Signal Changes in the Human Primary Visual-Cortex. Magnetic Resonance in Medicine. 1993;30(3):380-6.
  8. Yacoub E, Harel N, Uǧurbil K. High-field fMRI unveils orientation columns in humans. Proceedings of the National Academy of Sciences of the United States of America. 2008;105(30):10607-12.
  9. Huber L, Handwerker DA, Jangraw DC, Chen G, Hall A, Stuber C, et al. High-Resolution CBV-fMRI Allows Mapping of Laminar Activity and Connectivity of Cortical Input and Output in Human M1. Neuron. 2017;96(6):1253-63 e7.
  10. Birn RM, Bandettini PA. The effect of stimulus duty cycle and “off” duration on BOLD response linearity. NeuroImage. 2005;27(1):70-82.
  11. Logothetis NK, Pauls J, Augath M, Trinath T, Oeltermann A. Neurophysiological investigation of the basis of the fMRI signal. Nature. 2001;412(6843):150-7.
  12. Hu XP, Le TH, Ugurbil K. Evaluation of the early response in fMRI in individual subjects using short stimulus duration. Magnetic Resonance in Medicine. 1997;37(6):877-84.
  13. Hu X, Yacoub E. The story of the initial dip in fMRI. NeuroImage. 2012;62(2):1103-8.
  14. Buxton RB. Dynamic models of BOLD contrast. NeuroImage. 2012;62(2):953-61.
  15. van Zijl PC, Hua J, Lu H. The BOLD post-stimulus undershoot, one of the most debated issues in fMRI. NeuroImage. 2012;62(2):1092-102.
  16. Devor A, Tian P, Nishimura N, Teng IC, Hillman EM, Narayanan SN, et al. Suppressed neuronal activity and concurrent arteriolar vasoconstriction may explain negative blood oxygenation level-dependent signal. The Journal of neuroscience : the official journal of the Society for Neuroscience. 2007;27(16):4452-9.
  17. Krueger G, Granziera C. The history and role of long duration stimulation in fMRI. NeuroImage. 2012;62(2):1051-5.
  18. Frahm J, Kruger G, Merboldt KD, Kleinschmidt A. Dynamic uncoupling and recoupling of perfusion and oxidative metabolism during focal brain activation in man. Magnetic Resonance in Medicine. 1996;35(2):143-8.
  19. Bandettini PA, Kwong KK, Davis TL, Tootell RBH, Wong EC, Fox PT, et al. Characterization of cerebral blood oxygenation and flow changes during prolonged brain activation. Human brain mapping. 1997;5(2):93-109.
  20. Bandettini PA. The temporal resolution of Functional MRI. In: Moonen C, Bandettini P, editors. Functional MRI: Springer – Verlag; 1999. p. 205-20.
  21. Menon RS, Luknowsky DC, Gati JS. Mental chronometry using latency-resolved functional MRI. Proceedings of the National Academy of Sciences of the United States of America. 1998;95(18):10902-7.
  22. Menon RS, Gati JS, Goodyear BG, Luknowsky DC, Thomas CG. Spatial and temporal resolution of functional magnetic resonance imaging. Biochemistry and Cell Biology-Biochimie Et Biologie Cellulaire. 1998;76(2-3):560-71.
  23. Bellgowan PSF, Saad ZS, Bandettini PA. Understanding neural system dynamics through task modulation and measurement of functional MRI amplitude, latency, and width. Proceedings of the National Academy of Sciences of the United States of America. 2003;100(3):1415-9.
  24. Misaki M, Luh WM, Bandettini PA. Accurate decoding of sub-TR timing differences in stimulations of sub-voxel regions from multi-voxel response patterns. NeuroImage. 2013;66:623-33.
  25. Formisano E, Linden DEJ, Di Salle F, Trojano L, Esposito F, Sack AT, et al. Tracking the mind’s image in the brain I: Time-resolved fMRI during visuospatila mental imagery. Neuron. 2002;35(1):185-94.
  26. Lewis LD, Setsompop K, Rosen BR, Polimeni JR. Fast fMRI can detect oscillatory neural activity in humans. Proceedings of the National Academy of Sciences of the United States of America. 2016;113(43):E6679-E85.
  27. McKiernan K, D’Angelo B, Kucera-Thompson JK, Kaufman J, Binder J. Task-induced deactivation correlates with suspension of task-unrelated thoughts: An fMRI investigation. Journal of cognitive neuroscience. 2002:96-.
  28. Buckner RL. The serendipitous discovery of the brain’s default network. NeuroImage. 2012;62(2):1137-45.
  29. Shmuel A, Yacoub E, Pfeuffer J, Van de Moortele PF, Adriany G, Hu XP, et al. Sustained negative BOLD, blood flow and oxygen consumption response and its coupling to the positive response in the human brain. Neuron. 2002;36(6):1195-210.
  30. Shmuel A, Leopold D. Neuronal correlates of spontaneous fluctuations in fMRI signals in monkey visual cortex: implications for functional connectivity at rest. Human brain mapping. 2008;current issue.
  31. Ma Y, Shaik MA, Kozberg MG, Kim SH, Portes JP, Timerman D, et al. Resting-state hemodynamics are spatiotemporally coupled to synchronized and symmetric neural activity in excitatory neurons. Proceedings of the National Academy of Sciences of the United States of America. 2016;113(52):E8463-E71.
  32. Smith SM, Nichols TE, Vidaurre D, Winkler AM, Behrens TE, Glasser MF, et al. A positive-negative mode of population covariation links brain connectivity, demographics and behavior. Nature neuroscience. 2015;18(11):1565-7.
  33. Gonzalez-Castillo J, Saad ZS, Handwerker DA, Inati SJ, Brenowitz N, Bandettini PA. Whole-brain, time-locked activation with simple tasks revealed using massive averaging and model-free analysis. Proceedings of the National Academy of Sciences of the United States of America. 2012;109(14):5487-92.
  34. Vul E, Harris C, Winkielman P, Pashler H. Puzzlingly High Correlations in fMRI Studies of Emotion, Personality, and Social Cognition. Perspect Psychol Sci. 2009;4(3):274-90.
  35. Fox MD, Zhang D, Snyder AZ, Raichle ME. The global signal and observed anticorrelated resting state brain networks. Journal of neurophysiology. 2009;101(6):3270-83.
  36. Fox MD, Snyder AZ, Vincent JL, Corbetta M, Van Essen DC, Raichle ME. The human brain is intrinsically organized into dynamic, anticorrelated functional networks. Proceedings of the National Academy of Sciences of the United States of America. 2005;102(27):9673-8.
  37. Murphy K, Birn RM, Handwerker DA, Jones TB, Bandettini PA. The impact of global signal regression on resting state correlations: Are anti-correlated networks introduced? NeuroImage. 2009;44(3):893-905.
  38. Saad ZS, Gotts SJ, Murphy K, Chen G, Jo HJ, Martin A, et al. Trouble at rest: how correlation patterns and group differences become distorted after global signal regression. Brain connectivity. 2012;2(1):25-32.
  39. Wong CW, Olafsson V, Tal O, Liu TT. The amplitude of the resting-state fMRI global signal is related to EEG vigilance measures. NeuroImage. 2013;83:983-90.
  40. Liu TT, Nalci A, Falahpour M. The global signal in fMRI: Nuisance or Information? NeuroImage. 2017;150:213-29.
  41. Chen M, Han J, Hu X, Jiang X, Guo L, Liu T. Survey of encoding and decoding of visual stimulus via FMRI: an image analysis perspective. Brain imaging and behavior. 2014;8(1):7-23.
  42. Misaki M, Luh WM, Bandettini PA. The effect of spatial smoothing on fMRI decoding of columnar-level organization with linear support vector machine. Journal of Neuroscience Methods. 2013;212(2):355-61.
  43. Sirotin YB, Das A. Anticipatory haemodynamic signals in sensory cortex not predicted by local neuronal activity. Nature. 2009;457(7228):475-9.
  44. Muthukumaraswamy SD, Singh KD. Spatiotemporal frequency tuning of BOLD and gamma band MEG responses compared in primary visual cortex. NeuroImage. 2008.
  45. Bandettini PA, Wong EC, Jesmanowicz A, Hinks RS, Hyde JS. Spin-echo and gradient-echo EPI of human brain activation using BOLD contrast: A comparative study at 1.5 T. NMR in biomedicine. 1994;7(1-2):12-20.
  46. Yacoub E, Shmuel A, Pfeuffer J, Van De Moortele PF, Adriany G, Andersen P, et al. Imaging brain function in humans at 7 Tesla. Magnetic Resonance in Medicine. 2001;45(4):588-94.
  47. Duong TQ, Yacoub E, Adriany G, Hu X, Ugurbil K, Vaughan JT, et al. High-resolution, spin-echo BOLD, and CBF fMRI at 4 and 7 T. Magnetic Resonance in Medicine. 2002;48(4):589-93.
  48. Stroman PW, Krause V, Malisza KL, Frankenstein UN, Tomanek B. Extravascular proton-density changes as a Non-BOLD component of contrast in fMRI of the human spinal cord. Magnetic Resonance in Medicine. 2002;48(1):122-7.
  49. Douek P, Turner R, Pekar J, Patronas N, Le Bihan D. MR color mapping of myelin fiber orientation. Journal of computer assisted tomography. 1991;15(6):923-9.
  50. Le Bihan D, Breton E, Lallemand D, Grenier P, Cabanis E, Laval-Jeantet M. MR imaging of intravoxel incoherent motions: application to diffusion and perfusion in neurologic disorders. Radiology. 1986;161(2):401-7.
  51. Le Bihan D, Turner R, Moonen CT, Pekar J. Imaging of diffusion and microcirculation with gradient sensitization: design, strategy, and significance. Journal of magnetic resonance imaging : JMRI. 1991;1(1):7-28.
  52. Le Bihan D, Urayama SI, Aso T, Hanakawa T, Fukuyama H. Direct and fast detection of neuronal activation in the human brain with diffusion MRI. Proceedings of the National Academy of Sciences of the United States of America. 2006;103(21):8263-8.
  53. Kohno S, Sawamoto N, Urayama SI, Aso T, Aso K, Seiyama A, et al. Water-diffusion slowdown in the human visual cortex on visual stimulation precedes vascular responses. Journal of Cerebral Blood Flow and Metabolism. 2009;29(6):1197-207.
  54. Miller KL, Bulte DP, Devlin H, Robson MD, Wise RG, Woolrich MW, et al. Evidence for a vascular contribution to diffusion FMRI at high b value. Proceedings of the National Academy of Sciences of the United States of America. 2007;104(52):20967-72.
  55. Bandettini PA, Petridou N, Bodurka J. Direct detection of neuronal activity with MRI: Fantasy, possibility, or reality? Applied Magnetic Resonance. 2005;29(1):65-88.
  56. Truong TK, Avram A, Song AW. Lorentz effect imaging of ionic currents in solution. Journal of Magnetic Resonance. 2008;191(1):93-9.
  57. Buracas GT, Liu TT, Buxton RB, Frank LR, Wong EC. Imaging periodic currents using alternating balanced steady-state free precession. Magnetic resonance in medicine : official journal of the Society of Magnetic Resonance in Medicine / Society of Magnetic Resonance in Medicine. 2008;59(1):140-8.
  58. Witzel T, Lin FH, Rosen BR, Wald LL. Stimulus-induced Rotary Saturation (SIRS): a potential method for the detection of neuronal currents with MRI. NeuroImage. 2008;42(4):1357-65.
  59. Ofen N, Whitfield-Gabrieli S, Chai XJ, Schwarzlose RF, Gabrieli JD. Neural correlates of deception: lying about past events and personal beliefs. Social cognitive and affective neuroscience. 2017;12(1):116-27.
  60. Yang Z, Huang Z, Gonzalez-Castillo J, Dai R, Northoff G, Bandettini P. Using fMRI to decode true thoughts independent of intention to conceal. NeuroImage. 2014.
  61. Eklund A, Nichols TE, Knutsson H. Cluster failure: Why fMRI inferences for spatial extent have inflated false-positive rates. Proceedings of the National Academy of Sciences of the United States of America. 2016;113(28):7900-5.
  62. Shakil S, Lee CH, Keilholz SD. Evaluation of sliding window correlation performance for characterizing dynamic functional connectivity and brain states. Neuroimage. 2016;133:111-28.

If, how, and when fMRI goes clinical

 
 
 
This blog post was inspired by the twitter conversation that ensued after Chris Gorgoleski’s provocative tweet shown below. The link to the entire thread is  provided here.
 
Before I begin, I have to emphasize that while I am an NIH employee, my opinions in this blog are completely my own based on my own admittedly fMRI-biased perspective as an fMRI scientist for the past 28 years, and not in any way associated with my employer. I don’t have any official or unofficial influence on, or representation of, NIH policies. 
 
 
 
 
Back in 1991, the first fMRI signal changes were observed, ushering in a new era in human brain imaging that has reaped the benefits from its relatively high resolution, sensitive, fast, whole brain, and non-invasive assessment of brain activation at the systems level. With layer and columnar resolution fMRI currently producing promising results, it is starting to approach circuits level.  Functional MRI has filled a large temporal/spatial gap in our ability to non-invasively map human brain activity.  The appeal of fMRI has cut across disciplines – physics, engineering, physiology, psychology, statistics, computer science, and neuroscience to name a few, as the contrast needs to be better understood, the processing methods need to be developed, the pulse sequences need to be refined, the reliability needs to be improved, and ultimately the applications need to be realized. Neuroscientists and clinicians have applied fMRI to a wide range of questions regarding the functional organization and physiology of the brain and how they vary across clinical populations.
 
Because meaningful activation maps could be obtained from individual subjects (tap your fingers or shine a flickering checkerboard in your eyes, and the fMRI signal changes in the appropriate area in seconds – easily visible to the eye), the hope arose early on that this was a method that could be used clinically to complement prediction, diagnosis, and treatment of a wide range of neurologic and psychiatric pathologies. Sure, we can see motor cortex activation but can we differentiate, on an individual level, say, who is left handed vs right handed by comparing this activation? Perhaps group statistics might pull out a difference, but to assign an individual to one group (left handers) versus the other (right handers) with a level of certainty above 90% is a much more difficult problem. This type of problem encapsulates the essence of the difficulty associated with many hoped for clinical implementations of fMRI. Nevertheless, funding agencies embraced fMRI as it was generally accepted that its potential was high for shedding light on understanding the human brain and enhancing clinical treatment. Even with no clear clinical application, NIH embraced fMRI for its research potential. A sentence taken out of NIH’s mission statement is as follows:
 
“The mission of NIH is to seek fundamental knowledge about the nature and behavior of living systems and the application of that knowledge to enhance health, lengthen life, and reduce illness and disability.”
 
This clearly states the position that fundamental knowledge is important for clinical applications, even if the applications are not clearly defined. Functional MRI has certainly contributed to fundamental knowledge.
 
Over the years fMRI has grown in maturity as a tool for neuroscience research, substantially impacting the field, however, the clinical applications have not quite panned out. Pre-surgical mapping emerged as the only billable clinical application, obtaining a CPT (Current Procedural Terminology) code in 1997 – and even here it has not become the standard approach as it is being carried out in a relatively small number of hospitals worldwide.
 
There are several techniques that are being tested in the clinic. One promising example is a novel application and analysis of resting state fMRI that extracts the relative time shift of the fluctuations across the brain, is being tested and used in clinics in Germany and China. The basic idea is that in regions with compromised flow due to stroke, the temporal delay in a component of the BOLD based resting stater fluctuations is clearly visible. This is a method that may obviate the need for the current clinical practice of using Gd contrast agents in these patients as not only is the specificity outstanding but the sensitivity is comparable. 
 
Why the stalled clinical implementation of fMRI?
 
What are the reasons for this stalled clinical implementation? Let’s take a step back to look at why MRI, the precursor to fMRI by about a decade, has been so successful for clinical use. Using an array of available pulse sequences and corresponding structural contrasts, MRI can effectively be used to detect most tumors as well as most lesions associated with stroke and other types of trauma.  The lesions that are detectable are visible with minimal processing, allowing the Radiologist to simply view the image and make a diagnosis. The effective lesion or tumor contrast to noise ratio is high enough (at least above 10) such that detection is routine on a single subject basis by a trained Radiologist. 
 
Functional MRI on the other hand, requires several processing steps – all of which may influence the final result, as well as for the subject to either perform a task – or not (with resting state fMRI), and remain completely motionless as the threshold for motion is much more strict for fMRI. After processing, a map of activity or connectivity is created. These maps, typically color coded and superimposed on high resolution anatomic scans, show individual results with relatively high fidelity, but unfortunately, the difference between a functional map (from either a task or from resting state) of an individual with a pathology and that of a healthy volunteer, relative to the noise and variance among subjects, is too low for a visual assessment of a Radiologist or even for statistical reliability. There has also been the question of what task to use to highlight differences between normal controls and individuals with pathology. In resting state, there’s the issue of not being able to really know what the subject is doing – introducing further uncertainty.
 
In the case of presurgical mapping however, the fidelity of mapping the location of some functional regions (motor, somatosensory, visual, auditory, language) is high enough to allow the surgeon to identify and avoid these areas in individual subjects. However, even with presurgical mapping, the method is potentially confounded by compromised neurovascular coupling in the lesioned area, up to an hour of additional scanning, extreme sensitivity to motion (as mentioned, more than typical MRI scans), unique warping of echo planar images relative to structural scans causing misregistration, and, again, additional offline processing steps that add a degree of difficulty and uncertainty in functional localization. 
 
For the above reasons, fMRI has not caught on clinically even with presurgical mapping, as other more invasive approaches are arguably more precise, straightforward to implement, and less expensive. 
 
Now the question starts to loom, how much longer should clinically focused funding agencies need wait to see fruition before looking elsewhere? A large fraction of fMRI researchers including both those who develop the methods and those that apply them towards some neuroscience or clinical question generally maintain a belief that fMRI will become more clinically useful in the near or intermediate future. This position is not just a bluff or a vacuous promissory note by researchers willing to give proper lip service to a distant goal over the horizon. I think most of us get it – that we really want this all to pay off. It would be beneficial for many grants to include careful thinking on the steps that would be necessary to take the research to clinical practice.  Others think that health-focused funding agencies should start to actively look elsewhere for potential techniques that are more likely to achieve clinical traction in the near future. 
 
A current growth phase of fMRI
 
My own sense is that fMRI is in or rapidly approaching another major growth phase. New insights into brain organization are emerging at an increasing rate due to new and more sophisticated paradigms (real time fMRI, resting state fMRI, naturalistic viewing, fMRI adaptation), higher field strengths, better RF coils, and more specific and sensitive pulse sequences (blood volume sensitive imaging for layer specific fMRI), large multi-modal pooled data sets that allow world-wide access for data mining (Connectome project, UK Biobank, etc..), and perhaps most importantly, more sophisticated processing approaches (dynamic connectivity measures, cross subject correlation, machine learning, etc..). These advances have also enabled deeper insights into the functional organization of brains from individuals with psychiatric or neurologic disorders. Specifically, the use of Big Data in combination with machine learning or multivariate analysis in general, in combination with other modalities (genetics, EEG),  have started to generate potentially useful biomarkers that could be applied on individual subjects for disease diagnosis, prediction, and treatment.
 
Just one clinical application away
 
An second growth phase may be precipitated by one major clinical application that is more effective and perhaps even less expensive than the clinical practice that it replaces. Once this happens, I believe that the big scanner vendors (Siemens, GE, and Philips), and perhaps new companies will direct more attention to streamlining the basic implementation of fMRI in the clinic. Better hardware, pulse-sequence, subject interface devices, and processing methods will rapidly advance as economic incentives will supersede the influence of grant money in this context. Of the potential clinical applications mentioned below, it’s not clear which one will emerge first to break into clinical practice. 
 
For the past two decades, fMRI has benefited substantially from the success of MRI, as this has caused a proliferation of fMRI-ready scanners worldwide and has kept many costs down. Can you imagine how anemic the field of fMRI would be if MRI were not clinically useful? The substantially smaller research market of fMRI would have consisted of substandard and much more expensive scanners resulting in much slower advancement. Likewise, imagine what the field could look like if the fMRI market moved from research to clinical? The field would experience a transformation. Researchers would have immediate access to a wider variety of state of the art sequences that exist on only a handful of scanners today. Methodology including subject interface devices, processing pipelines would not only advance more rapidly, but be more standardized and quality-controlled across centers. The on-ramp to further clinical implementation would be much smoother. 
 
How long to wait?
 
So the question remains, how long should funding agencies wait to determine if fMRI will catch on clinically? Some feel that they’ve waited long enough. Others feel, as I do, that the increased focus of the field on fMRI towards individual assessment as well as layer specific fMRI will likely make clinical inroads and is really just getting started. I also believe that fMRI – in synergy with other modalities – is not anywhere close to realizing its full potential in revealing fundamental new insights into functional organization useful to both basic neuroscience and clinical practice. To stop or even reduce support of fMRI now would be tragic. 
 
Potential Clinical Applications of fMRI in the Immediate Future. 
 
What are the potential clinical applications and what specifically would be necessary to allow fMRI to be used on a day to day basic with patients? 
 
  1. Disorder/Disease Biomarkers: Large pooled data sets that also contain structural data, genetic data, and a slew of behavioral data are just starting to be mined with advanced processing methods. Already specific networks related to behavior, lifestyle, and genetic disorders have been discovered. The long term goal here is the creation of multivariate biomarkers that can be applied to individuals either to screen, diagnose, or guide treatment with an acceptable degree of certainty. There are perhaps hard limits to fMRI sensitivity and reliability, but if the number of meaningful dimensions of information from fMRI are increased, then the hope is that this massively multivariate data may allow highly sensitive and specific individual subject and/or patient differentiation based on resting state or activation information. 
  2. Biofeedback: It has been demonstrated that when presented in real time with useful fMRI activation-based feedback in real time on a specific aspect of their dynamic brain activity, subjects were able to alter and tune their activity. In many studies, this led to a change in an aspect of their behavior – touching on depression, phobias, and pain perception. The fMRI signal is still slow and noisy, however, of higher fidelity than other real time neuronal measures. Recently, simultaneous use of EEG has been proposed to enhance the effectiveness of real time fMRI feedback. This is still in its early stages, however, clinical trials are underway. 
  3. Localization for Neuromodulation: An emerging area of clinical treatment is that of neuromodulation by the use of methods to stimulate or interfere with brain activity in a targeted manner either invasively or non-invasively. Deep brain stimulation, TMS, tDCS, focused ultrasound, and more are currently being developed for clinical applications – alleviating depression, Parkinson’s disease, and other disorders. The placement and targeting of these interventions is critical to their success. I see fMRI a playing a significant role in providing functional localizers so that the efficacy of these neuromodulation approaches may be fully realized.
  4. Assessment of locked in patients:Recent studies have shown that fMRI is superior to EEG in assessing the brain health, activity, and function of locked in patients. In some instances fMRI activity was used as a means for communication. This approach has considerable potential to be used on a regular basis in a clinical setting as no other methods compare  – even in its early stages of implementation.
  5. Brain Metabolism/Neurovascular Coupling/Blood Oxygenation Assessment: While activation and connectivity studies dominate potential fMRI clinical applications, more fundamental physiologic  information obtained using simultaneous fMRI measures with the appropriate pulse sequence, such as a combined arterial spin-labelling (ASL) for perfusion, blood oxygenation level dependent (BOLD), and/or Vascular Space Occupancy (VASO) contrast for blood volume, during a stress such as breath-hold or CO2 inhalation – or even during normal breathing variations at rest, can provide insights into baseline blood oxygenation, neurovascular coupling, and even resting and activation-induced changes in Cerebral Metabolic Rate (CMRO2). All these provide potentially unique and useful information related to vascular patency and metabolic health of brain tissue – with potentially immediate clinical applications that may fill a niche between CT angiography, ultrasound, and positron emission tomography (PET). 
  6. Perfusion Deficit Detection using ASL: has been in existence as long as BOLD contrast and significant effort has been made to test it clinically. While the baseline perfusion information that it provides is comparable to that obtained with injected Gd contrast, its sensitivity is significantly lower, requiring a much longer acquisition time for averaging. This has slowed widespread clinical implementation.
  7. Perfusion Deficit Detection using resting state BOLD: This is perhaps the most promising of the possible clinical implementations of fMRI in the broadest interpretation of the name. Mapping the relative latencies of resting state BOLD fluctuations clearly reveals regions of flow deficit. This approach compares well to the clinically used approach of Gd contrast in terms of sensitivity and specificity. Creation of latency maps from BOLD fluctuations is also relatively straightforward and could be performed seamlessly and quickly in an automated manner. This approach is currently being implemented in a limited manner in hospitals in Germany and China. 
  8. Localization of seizure foci: The flip side of mapping regions for surgeons NOT to remove for pre surgical mapping applications is the mapping  of seizure generating tissue to provide surgeons with a target for removal. For certain types of seizure activity, the brain is constantly generating uniquely unusual activity, which translates into unique temporal signatures recorded with either EEG or resting state fMRI. Detection with EEG is much more easily and cheaply performed, but has less spatial precision fMRI. 
  9. Clinical Importance of Basic Neuroscience: Many would argue that the clinical importance of basic and cognitive neuroscience research, while not having a direct clinical application, has so many secondary and tertiary influences on the state of the art of clinical practice that this is in itself a sufficient justification for continued fMRI research funding by both basic science funding agencies as well as more clinically focused agencies.
Success? How to measure it – and on what time scale?
 
Getting back to the issue of funding. From my  perspective, there are two primary issues: 1. How to achieve a balance of short term and long term success. 2. How to even gauge the effectiveness of a funding initiative or of a specific funded project.
 
Clinical funding agencies generally fund basic research with the idea that clinical implementation is a long term goal that requires basic science groundwork to be established. If funding were only short term, many discoveries and new fruitful directions and opportunities would be missed. About 30 years ago, several notable large companies supported research of select employee/scientists that was more open ended. Examples are Varian (my Ph.D. co-advisor, Jim Hyde emerged from this renowned group) and famously, Bell Labs who allowed one of their scientists by the name of Seiji Ogawa to dabble in high field MRI – using hemoglobin as a potential contrast. Back then, companies seemed to have more latitude for open ended creative work but the culture seems to have shifted (with perhaps the exception of Google and the like). Today, MRI research by vendor employees has become more product focused and usually on short term problems. While this is an effective approach in many contexts, in my opinion, much of the creative potential of these employee scientists is lost on product development and troubleshooting.
 
Regarding the second issue of measures of success, this is an open problem that I believe vexes funding agencies and program officers around the word. Measures such as papers published or citations don’t really capture the essence of a successful new research direction. One has to gauge the entire field to determine the success of a new method. One also has to wait potentially decades to determine the true payoff. To the best of my knowledge, there are no clear objective or quantitative measures of funding success. Those deciding on the funding typically base their decisions on their own broad and deep knowledge of the field and advice from experts doing the research. Grant reviewers assess the quality of the proposals but the directors and program officers set the initiatives. It would be interesting and useful to develop more of a science for what general directions and what grants would be best to fund, looking back on what was funded and coming up with measures that can effectively predict “success.” This task might be a problem for the machine learning community.
 
What will it take for fMRI to be a clinical method? 
 
What it will take for fMRI to become a sought-after clinical method? To begin, a foundation of streamlined clinical testing needs to be established. At minimum, this will require a highly streamlined, patient/clinician – friendly protocol that collects fMRI data in real time (to allow for immediate identification of unacceptable motion, etc. so that the scans can be quickly cancelled and redone), an agreed upon processing pipeline that then collapses the salient information into a map or even a set of numbers that are both meaningful and easily understood by those making clinical decisions. Functional MRI subject interface devices need minimal setup time, and the protocol itself should take no longer than any other structural scan that is performed. Currently, no such highly integrated systems exist. With increased focus on better extraction and differentiation of individual information, clinical implementation will be a natural next step. I believe we just have to wait a bit, and no one really has a solid sense of whether or not fMRI will successfully penetrate clinical practice, but there’s a few things that can be done. 
 
Regarding utility and reliability, I think that currently, with our hardware, acquisition methods, noise reduction approaches, and other post processing methods, fMRI is not quite reliable or sensitive enough. One example is of how physiologic noise reduction can immensely improve the state of the art. Currently, physiologic noise sets an upper limit of about 120/1 on fMRI time series, no matter what the coil sensitivity or field strength is. If we were able to remove this physiologic noise, then the time series signal to noise ratio would be limited only by coil sensitivity – potentially increasing the time series signal to noise ratio by an order of magnitude. 
 
There are the large obstacles of cost effectiveness and clinical uniqueness. The cost/benefit has to place it above the competing clinical methods. Given the current rate that the field is making progress on individual assessment methods, my sense is that it will become reliable enough for a small but growing number of clinical applications. Which ones and when, I don’t think anyone knows, but I think at least one of the applications that I mentioned above will emerge within the next decade. Specifically, it appears that applications 5,6, and 7 which use fMRI to map physiology rather than function, and application 3 which is the use of fMRI activation as a functional localizer for neuromodulation, have the highest likelihood for clinical penetration. Approach 7, that of mapping resting state latencies and using these maps for perfusion deficit assessment, has the necessary ingredients for success: similar ease of implementation, sensitivity, and specificity to current approaches, and an added benefit of being less invasive than current clinical practice involving Gd injection. 
 
Funding the vendors
 
A ripe target for funding might be to the major scanner vendors or small businesses to create such a clinically viable platform that would be able to immediately implement and test the most promising basic science findings. At the moment, I feel that vendors are not devoting enough man hours to any major fMRI platform development, as there are no clearly profitable applications that exist in the short term. Catalyzing development along these lines by grants would enable more rapid clinical implementation and testing. As mentioned, once a clear clinical application is established, more vendor-funded fMRI development would then allocated by the vendors as it would translate into profit.
 
Other Suggestions
 
In the twitter conversation there were a few suggestions that emerged. One that is generally practiced but perhaps should be emphasized further, is, for those applying for grants from agencies where the mission is human health, more detail regarding how their research will lead to better clinical practice should be included. What are the steps needed? What clinical practice will be improved and how? What might be the timeline? I think that this approach should apply to a large fraction of these grant applications but for many, I don’t think that this should be a requirement as its generally accepted that the fallout of better understanding brain organization in health and disease can inform unexpected new avenues of clinical practice. One cannot and sometimes should not always connect the dots. There is a significant role of basic research – without an obvious or immediate clinical application – that is still beneficial to clinical practice in the long run. 
 
Fund more tool development, implementation, and streamlining. One gap that I see in some of the funding opportunities is that of taking a potentially useful tool and making it work in regular clinical practice. This could be either before or after the clinical trials stage. I think that funding more nuts and bolts research and development – scaling up a tool from concept to general practice – should have a larger role as often this gap is prohibitively wide.
 
Fund infrastructure creation for data, tool, and model sharing and testing. In recent years, the creation of large, curated, mine-able databases has shown to be effective in accelerating, among other things, methods development research and discovery science as well as transparency and reproducibility. One can imagine other useful infrastructures created for computational model sharing, cross modality data pooling, tool testing and development, and generally integrating the vast disconnected body of scientific literature in neuroscience. As a concrete example, I’m often struck with how disconnected the information is at a typical Society for Neuroscience Meeting. Attendees are quickly overwhelmed with the information. If there was some structure, perhaps organized by high priority open questions or models that need to be tested, that the diverse findings could be linked with, this would go a long way towards increasing the focus of the community, identifying research opportunities, and pointing out clear gaps in our understanding. 
 
Funding for fMRI is well worth it. 
 
My response to those who feel that fMRI funding should be cut is to of course welcome them to provide viable alternatives. Perhaps there are new directions out there that need more focus. I think that most in the field of neuroimaging – as well as those outside – would agree however that fMRI has not only established its place as a formidable tool in neuroscience and clinically directed research, it is a technique that has revolutionized much of cognitive neuroscience. It’s also clear that we are currently in the midst of a wave of innovation in everything from pulse sequence design to multi-modal integration to processing methods. The field is advancing surprisingly well. It is making a growing number of clear contributions to neuroscience research and will eventually make inroads, one way or another, into clinical practice. 
 
 
 
 
 
 
 
 
 
 
 
 
 

#CCNeuro asks: “How can we find out how the brain works?”

The organizers of the upcoming conference Cognitive Computational Neuroscience (#CCNeuro) have done a very cool thing ahead of the meeting. They asked their keynote speakers the same set of 5 questions, and posted their responses on the conference blog.

The first of these questions is “How can we find out how the brain works?”. In addition to recommending reading the insightful responses of the speakers, I offer here my own unsolicited suggestion.

A common theme among the responses is the difficulty posed by the complexity of the brain and the extraordinary expanse of scales across which it is organized.

The most direct approach to this challenge may be to focus on the development of recording technologies to measure neural activity that more and more densely span the scales until ultimately the entire set of neural connections and synaptic weights is known. At that point the system would be known but not understood.

In the machine learning world, this condition (known but not understood) is just upon us with AlphaGo and other deep networks. While it has not been proven that AlphaGo works like a brain, it seems close enough that it would be silly not to use as a testbed for any theory that tries to penetrate the complexity of the brain a system that has human level performance in a complex task, is perfectly and noiselessly known, and was designed to learn specifically because we could not make it successful by programming it to execute known algorithms (contrast Watson).

Perhaps the most typical conceptual approach to understanding the brain is based on the idea (hope) that the brain is modular in some fashion, and that models of lower scale objects such as cortical columns may encapsulate their function with sufficiently few parameters that the models can be built up hierarchically and arrive at a global model whose complexity is in some way still humanly understandable, whatever that means.

I think that modularity, or something effectively like modularity is necessary in order to distill understanding from the complexity. However, the ‘modularity’ that must be exploited in understanding the brain will likely need to be at a higher level of abstraction than spatially contiguous structures such as columns, built up into larger structures. The idea of brain networks that can be overlapping is already such an abstraction, but considering the density of long range connections witnessed by the volume of our white matter, the distributed nature of representations, and the intricate coding that occurs at the individual neuron level, it is likely that the concept of overlapping networks will be necessary all the way down to the neuron, and that the brain is like an extremely fine sparse sieve of information flow, with structure at all levels, rather than a finite set of building blocks with countable interactions.

Review of “Incognito: The Secret Lives of the Brain” by David Eagleman

Most of our brain activity is not conscious –  from processes that maintain our basic physiology to those that determine how we catch a baseball and play a piano well. Further, these unconscious processes include those that influence our basic perceptions of the world. Our opinions and deepest held beliefs – those that we prefer to feel that our conscious mind completely determines –  are shaped largely by unconscious processes. The book, “Incognito: Secret Lives of the Brain” by David Eagleman, is an engaging account of those processes – packed with practical and interesting examples and insight. Eagleman is not only a neuroscientist, but an extremely clear and engaging writer. His writing, completely accessible to the non expert, is filled with solid neuroscience, packaged in a way that not only provides interesting information, but also builds perspective. It’s the first book that I’ve encountered that delves deeply into this particular subject. We mostly think of our brains as generating conscious thought, but, as he explains it’s just the small tip of the iceberg.  

Continue reading “Review of “Incognito: The Secret Lives of the Brain” by David Eagleman”

Mini Book Review: “Explaining the Brain,” by Carl Craver

Explaining the Brain” is a 2007 book by Carl Craver, who applies philosophical principles to comment on the current state of neuroscience. This is my first and only exposure to the philosophy of science, so my viewpoint is very naive, but here are some main points from the book that I found insightful.

The book starts by making a distinction between two broad goals in neuroscience: explanation, which is concerned with how the brain works; and control, which is concerned with practical things like diagnosis, repair, and augmentation of the brain. In my previous post on this blog, I tried to highlight that same distinction. This book focuses on explanation, which is essentially defined as the ability to fully describe the mechanisms by which a system operates.

A major emphasis is on the question of what it takes to establish a mechanism, and the notion of causality is integral to this question.

Continue reading “Mini Book Review: “Explaining the Brain,” by Carl Craver”

The Wearable Tech + Digital Health Conference at Stanford University

The future of healthcare both small and big. It’s big data, machine learning, and massive amounts of data coming from tiny robust devices or phone apps of individuals. It’s individualized medicine – not only for patients who need care but for healthy individuals. The data will come from devices that will become ever more ubiquitous – stickers on skin, tattoos, clothing, contact lenses, and more.  This conference, organized by Applysci, and held on Feb 7 and 8, 2017 at Stanford University, involved a slate of some of the most creative, ambitious, and successful people in the digital health industry. I was both mesmerized and inspired. 

I decided to venture outside my comfort zone of fMRI and brain imaging conferences to get a glimpse of the future of wearable technology and digital health by attending this conference. The speakers were mostly academics who have started companies related to their particular area of expertise. Others were solidly in industry or government. Some were quite famous and others were just getting started. All were great communicators – many having night jobs as writers. My goal for being here was to see how these innovations could complement fMRI – or vise versa.  Were there new directions to go, strategies to consider, or experiments to try? What were the neural correlates of expanding one’s “umwelt?” – a fascinating concept elegantly described by one of the speakers, David Engleman.   

On a personal level, I just love this stuff. I feel that use of the right data can truly provide insight into so many aspects of an individual’s health, fitness, and overall well-being, and can be used for prediction and classification. There’s so much untapped data that can be measured and understood on an individual level.  

Many talks were focussed on flexible, pliable, wearable, and implantable devices that can measure, among other things, hemodynamics, neuronal activity, sweat content, sweat rate, body heat, solar radiation, body motion, heart rate, heart rate variability, skin conductance, blood pressure, electrocardiogram measures, then communicate this to the user and the cloud – all for analysis, feedback, and diagnosis. Other talks were on the next generation of brain analysis and imaging techniques. Others focussed on brain computer interfaces to allow for wired and wireless prosthetic interfacing. Frankly, the talks at this conference were almost all stunning. The prevailing theme that ran through each talk could be summarized as: In five or so years, not much will happen, but in ten to fifteen years, brace yourselves. The world will change! Technophiles see this future as a huge leap forward – as information will be more accessible and usable – reducing the cost of healthcare and, in some contexts – bypassing clinicians altogether and increasing the well-being of a very large fraction of the population. Others may see a dystopia wrought with the inevitable ethical issues of who can use and control the data.   

Below are abbreviated notes, highlights, and personal thoughts from each of the talks that I attended. I don’t talk about the speakers themselves as they are easily googled – and most are more or less famous. I focus simply on what the highlights were for me. 

Continue reading “The Wearable Tech + Digital Health Conference at Stanford University”