Here we highlight collaborative projects supported by the Consortium

Application of the hierarchical bootstrap to multi-level data in neuroscience
Saravanan V, Berman GJ, Sober SJ
Neurons, Behavior, Data Analysis and Theory (NBDT) July 21 (2020)

Neuroscience experiments yield a dizzying array of data types, including anatomical, electrophysiological, optical, genetic, and behavioral datasets. What neuroscience datasets typically share, however, is a hierarchical structure, as when we collect repeated measurements from multiple subjects or across different experimental conditions.

For example, we might analyze electrophysiological recordings of 100 neurons from each of 5 animals in response to 2 different stimuli, or count dendritic spines on 100 neurons in 5 animals treated with one drug and 5 animals treated with another drug. Because of this hierarchy, the individual measurements of neural activity or spine number are not independent, since both the activity and spine data are more likely to be similar within individual subjects than between them.

The common practice of analyzing hierarchical data with statistical methods that assume - and in fact rely on - independent data sampling often results in errors. As an illustration, in the hypothetical datasets described above

If we treat all measurements for a given stimulus or drug condition as independent, we will massively inflate our false-positive (type 1 error) rate.

If we average within each animal to compensate for cross-animal differences, we will massively inflate our false-negative (type 2 error) rate.

The “hierarchical bootstrap” (Efron and Tibshirani, 1994) is a robust and easy-to-implement approach that accounts for multi-level data structure when computing statistical uncertainty. Although widely used in other fields, the hierarchical bootstrap is seldom applied to neuroscience datasets. Our paper presents a guide to applying this technique to neuroscience data and provides examples of performing the hierarchical bootstrap on both synthetic neural population activity and real-world behavioral datasets from songbirds and fruit flies.