Lecture 3:  Maximum entropy models for biological networks

 

Most of the interesting things that happen in living organisms result from networks of interactions, whether among neurons in the brain, genes in a single cell, or amino acids in single protein molecule.  Especially in the context of neural networks, there is a long tradition of using ideas from statistical physics to think about the emergence of collective behavior from the microscopic interactions, with the hope that this functional collective behavior will be robust (universal?) to our ignorance of many details in these systems.  In the past decade or so, new experimental techniques have made it possible to monitor the activity of many biological networks much more completely, and the availability of these data has made the problems of analysis much more urgent: given what the new techniques can measure, can we extract a global picture of the network dynamics?  In this lecture I'll show how an old idea, the maximum entropy construction, can be used to attack this problem.  What is most exciting is that this construction provides a path directly from real data to the classical models of statistical mechanics.  I'll describe in detail how this works for a network of neurons in the retina as it responds to complex, naturalistic inputs, where the relevant model is exactly the Ising model with pairwise, frustrated interactions.   Remarkably, the data suggest that the system is poised very close to a critical point. I'll try to highlight some open theoretical questions in this field, as well as making connections to other systems.  Again, I hope we'll see the outlines of how common theoretical ideas can unify our understanding of diverse systems.  

 

Note: For the moment I just have references with some categories.  I hope to add some text as a guide!

 

 

Weak pairwise correlations imply strongly correlated network states in a neural population.  E Schneidman, MJ Berry II, R Segev & W Bialek, Nature 440, 1007-1012 (2006); q–bio.NC/0512013.

 

Ising models for networks of real neurons. G Tkacik, E Schneidman, MJ Berry II & W Bialek, q–bio.NC/0611072 (2006).

 

Other work on neurons

 

The structure of multi-neuron firing patterns in primate retina.  J Shlens, GD Field, JL Gauthier, MI Grivich, D Petrusca, A Sher, AM Litke & EJ Chichilnisky, J Neurosci  26, 8254-8266 (2006).

 

A maximum entropy model applied to spatial and temporal correlations from cortical networks in vitro. A Tang, D Jackson, J Hobbs, W Chen, JL Smith, H Patel, A Prieto, D Petrusca, MI Grivich, A Sher, P Hottowy, W Dabrowski, AM Litke & JM Beggs, J Neurosci  28, 505-518 (2008).

 

Connecting to other kinds of networks

 

Using the principle of entropy maximization to infer genetic interaction networks from gene expression patterns. TR Lezon, JR Banavar, M Cieplak, A MAritan & NV Federoff, Proc NatŐl Acad Sci (USA)  103, 19033-19038 (2006).

 

Rediscovering the power of pairwise interactions.  W Bialek & R Ranganathan, arXiv.org:0712.4397 [q–bio.QM] (2007).

 

Maximum entropy approach for deducing amino acid interactions in proteins.  F Seno, A Trovato, JR Banavar & A Maritan.  Phys Rev Lett  100, 078102 (2008).

 

Toward a statistical mechanics of four letter words.  GJ Stephens & W Bialek, arXiv.org:0801.0253 [q–bio.NC] (2008).

 

 

Algorithmic issues

 

Faster solutions of the inverse pairwise Ising problem.  T Broderick, M Dudik, G Tkacik, RE Schapire & W Bialek, arXiv:0712.2437 [q–bio.QM] (2007).

 

Constraint satisfaction problems and neural networks:  A statistical physics perspective.  M Mezard & T Mora, arXiv.org:0803.3061 [q–bio.NC] (2008)

 

Some of the technology for multi-neuron recording

 

Multi-neuronal signals from the retina: acquisition and analysis.  M Meister, J Pine & DA Baylor.  J Neurosci Meth 51, 95-106 (1994).

 

What does the eye tell the brain?  Development of a system for the large-scale recording of retinal output activity.  AM Litke, N Bezayiff, EJ Chichilnisky, W Cunningham, W Dabrowski, AA Grillo, M Grivich, P Grybos, P Hottowy, S Kachiguine, RS Kalmar, K Mathieson, D Petrusca, M Rahman & A Sher, IEEE Trans Nucl Sci  51, 1434-1440 (2004).

 

Recording spikes from a large fraction of the ganglion cells in a retinal patch.  R Segev, J Goodhouse, J Puchalla & MJ Berry II, Nature Neurosci 7, 1155-1162 (2004).