Quantifying the multi-scale performance of network inference algorithms

Chris J. Oates; Richard Amos; Simon E.F. Spencer

doi:10.1515/sagmb-2014-0012

Published by De Gruyter August 5, 2014

Quantifying the multi-scale performance of network inference algorithms

Chris J. Oates , Richard Amos and Simon E.F. Spencer

From the journal Statistical Applications in Genetics and Molecular Biology

https://doi.org/10.1515/sagmb-2014-0012

Showing a limited preview of this publication:

Abstract

Graphical models are widely used to study complex multivariate biological systems. Network inference algorithms aim to reverse-engineer such models from noisy experimental data. It is common to assess such algorithms using techniques from classifier analysis. These metrics, based on ability to correctly infer individual edges, possess a number of appealing features including invariance to rank-preserving transformation. However, regulation in biological systems occurs on multiple scales and existing metrics do not take into account the correctness of higher-order network structure. In this paper novel performance scores are presented that share the appealing properties of existing scores, whilst capturing ability to uncover regulation on multiple scales. Theoretical results confirm that performance of a network inference algorithm depends crucially on the scale at which inferences are to be made; in particular strong local performance does not guarantee accurate reconstruction of higher-order topology. Applying these scores to a large corpus of data from the DREAM5 challenge, we undertake a data-driven assessment of estimator performance. We find that the “wisdom of crowds” network, that demonstrated superior local performance in the DREAM5 challenge, is also among the best performing methodologies for inference of regulation on multiple length scales.

Keywords: multi-scale scores; network inference; performance assessment

Corresponding author: Chris J. Oates, Department of Statistics, Zeeman Building, University of Warwick, Gibbet Hill Road, Coventry, CV4 7AL, UK, e-mail: c.oates@warwick.ac.uk

Acknowledgments

The authors are grateful to the Associate Editor and three anonymous reviewers whose comments greatly improved this manuscript. In addition the authors thank Gustavo Stolovitzky and Sach Mukherjee for their feedback at an earlier stage of this research. This work was supported by the Centre for Research in Statistical Methodology grant (EPSRC EP/D002060/1). CJO was involved in designing the DREAM8 Challenge. CJO, RA, SEFS were not involved in the DREAM5 Challenge.

Appendix A: Proofs

Proof of Theorem 1.S-optimality of the benchmark network G follows immediately from the definition of each score. To prove nonuniqueness we begin by considering MSS1. For this, take G to be the network A→B→C→A and take G′ to be the network A←B←C←A; then both networks entail the same descendancy relationships. It follows from (D3; consistency) that both G and G′ maximise SMSS1ROC and SMSS1PR, demonstrating that the maximiser will be non-unique in general. The argument for MSS2 is analogous, using the same pair of networks.

Proof of Theorem 2. (a) Given ε>0. We proceed by constructing a sequence of pairs (G, Ĝ) indexed by p, the number of vertices, such that the scores associated with Ĝ end simultaneously to the required limits as p→∞. Define a data-generating network G on p vertices by the edge set

(9)E(G)={(1, i) : 2≤i≤p−1}{(i,p) : 2≤i≤p−1} (9)

and consider an unweighted network Ĝ with topology E(Ĝ)=E(G)∪{(p, 1)}. i.e., Ĝ differs to G in the addition of a single edge (p, 1). The ROC curve corresponding to Ĝ is defined by the three points

(10){(0,0), (FPR,TPR), (1,1)} (10)

so that by linear interpolation the area under the curve is 12(1+TPR−FPR). The PR curve is similarly defined by

(11){(0,0), (TPR, PPV), (1, Tp(p−1))}. (11)

Here T is the number of true examples; for local scores this is E(G), for MSS1 this is #{(i, j):G(i→j)≠∅, i≠j} and for MSS2 this is ∑_i,je_{i, j}. To compute the area under this curve we must use nonlinear interpolation (Goadrich et al., 2004). Specifically, interpolation between the two points A, B with true/false positive values (TP_A, FP_A), (TP_B, FP_B) respectively we create new points for each of TP_A+1, TP_A+2, …, TP_B–1, increasing the false positives for each new point by (FP_B–FP_A)/(TP_B–TP_A). By direct calculation we have that the area under this interpolated curve is

(12)TP2T(TP+FP)+1T∑j=1T−TP(TP+j)(T−TP)(TP+FP)(T−TP)+j[p(p−1)−TP−FP]. (12)

Fix ε>0. For (i), since Ĝ differs to G by just a single edge we have TP=2(p–2), FP=1, T=2(p–2), TPR=1, FPR=1(p−1)(p−2),PPV=2p−42p−3. It is then easily checked that

(13)SlocalROC=12(1+1−1(p−1)(p−2))→1 as p→∞, (13)

(14)SlocalPR=[2(p−2)]2[2(p−2)][2(p−2)+1]→1 as p→∞. (14)

Thus ∃P1∈ℕ:∀p≥P1, SlocalROC>1−e , SlocalPR>1−e.

For (ii-iii) and MSS1 note that Ĝ(i→j)≠∅ for all i≠j, so that Ĝ predicts every possible descendancy. Thus TP=2 p–3, FP=p²–3p+3, T=2p–3, TPR=1, FPR=1, PPV=2p−3p(p−1) and

(15)SMSS1ROC=12(1+1−1)→12 as p→∞, (15)

(16)SMSS1PR=[2p−3]2[2p−3][(2p−3)+(p2−3p+3)]→0 as p→∞. (16)

Thus ∃P2∈ℕ:∀p≥P2, SMSS1ROC<12+e, SMSS1PR<e. For MSS2 note that the estimated effects are e^=1p×p whereas the benchmark effects are given in the sparse matrix

(17)e=[11…111(p−2)−1⋱⋮(p−2)−11] (17)

leading to TP =2p, FP =p(p–2), T=2p, TPR =1, FPR =1, PPV=2p and

(18)SMSS2ROC=12(1+1−1)→12 as p→∞, (18)

(19)SMSS2PR=[2p]2[2p][2p+p(p−2)]→0 as p→∞. (19)

Thus ∃P3∈ℕ:∀p≥P3, SMSS2ROC<12+e, SMSS2PR<e.

Taking p>max{P₁, P₂, P₃}, we have shown that (G, Ĝ) satisfy the conclusions of the theorem.

(b) The converse result is proved similarly, taking G to be the cycle 1→2→…→p→1 and Ĝ to be the reverse 1←2←…←p←1.□

Proof of Theorem 3. Let P=(p₀, …, p_m) be a path from p₀ to p_m. Define the intermediate vertices in the path to be the vertices p₁, …, p_m–1. For each pair of vertices i and j we wish to calculate τ_{i, j}, the largest threshold τ for which i and j are connected by a path in H^Γ.

For k=0, …, p, let τi, j(k) represent the largest value of the threshold τ for which H^Γ contains a path from i to j with intermediate vertices in the set (1, …, k). From this definition, we have τi, j(0)=H, the weighted adjacency matrix and τi,j=τi,j(p). Next we show that for k=1, …, p,

(20)τi, j(k)=max{τi,j(k−1), min{τi,k(k−1), τk,j(k−1)}}. (20)

To see this, note that τi, j(k)=τi, j(k−1) unless there is a more strongly connected path which goes through k. Such a path must first connect i to k, and then connect k to j, with both paths involving intermediate vertices in {1, …, k–1}. This connection is broken in H^Γ once the threshold τ is raised above the minimum of these two connections, that is when τ>min{τi, k(k−1), τk,j(k−1)}.

The conclusion of the Theorem follows by iterative application of Equation 20.□

Remarks on the proof of Theorem 3: To make the algorithm more memory efficient, we note that it is not necessary to store more than O(p2) values for τ at any point. This stems from the fact that each iteration is calculated only from the results of the previous one, reducing the necessary storage to 2p². This can be improved further by noting that if i=k or j=k then k cannot be an intermediate vertex in a path connecting i to j and hence τi, j(k)=τi, j(k−1). These 2k–1 index values are the only elements needed to calculate the kth iteration, and they themselves do not need to be updated. Hence, the remaining elements in the matrix are needed only for their own update and can be safely over-written. A corollary to this is that the i and j loops in the algorithm can be completed in any order or even in parallel. Note that this algorithm works whether or not the weight matrix H is allowed to contain loops. Irrespective of whether loops are permitted or not, τ_{i, i} represents the largest value of the edge threshold for which node i is contained in a cycle.

For Theorem 4 we require a series of Lemmas:

Lemma 1The effects

(21)eij=δij+∑P∈H(i→j)∏k=1ℓ(P)1|H(•,pk)| (21)

are well-defined and satisfy 0≤e_ij≤1.

Proof. We show the sum in Equation 21 converges absolutely. Since all terms are non-negative, it is sufficient to prove that, for each i≠j such that H(i→j)≠∅, the sum is bounded above. Consider the partial sums

(22)eij(n)=∑P∈H(i→j):ℓ(P)≤n∏k=1ℓ(P)1|H(•, pk)|. (22)

If Equation 21 diverges, so must limn→∞eij(n) since all paths are of finite length, so it is sufficient to prove eij(n) is bounded above by one as n→∞. We proceed inductively, with base case

(23)eij(1)={1|H(•,j)|:i∈H(•,j)0:otherwise}≤1. (23)

Suppose eij(n−1)≤1. Then for i∉H(•, j) we have

(24)eij(n)=∑k∈H(•, j)1|H(•, j)|eik(n−1) (24)

since a path P of length m can be decomposed into a path from p₀=i to p_m–1=k and an edge from p_m–1=k to p_m=j. Therefore eij(n)≤∑k∈H(•, j)1|H(•,j)|=1 as required. The case where i∈H(•, j) is similar, leading to

(25)eij(n)=∑k∈H(•, j)\{i}1|H(•, j)|eik(n−1) (25)

from which we again conclude eij(n)≤∑k∈H(•, j)1|H(•, j)|=1. Therefore eij(n)≤1 for all n by induction.□

Lemma 2Effects, as defined in Equation 21, satisfy (i) e_ij=δ_ijfor all i, j s.t. H(i→j)=∅, (ii)eij=∑k∈H(•, j)1|H(•,j)|eikfor all i, j s.t. H(i→j)≠∅.

Proof of Lemma 2. The first statement follows immediately from Equation 21. For the second statement, consider two cases: Firstly, for i≠j such that i∉H(•, j) we have

(26)∑k∈H(•, j)1|H(•, j)|eik=∑k∈H(•, j)1|H(•, j)|∑P∈H(i→k)∏l=1ℓ(P)1|H(•, pl)| (26)

(27)=∑P∈H(i→j)∏l=1ℓ(P)1|H(•, pl)|=eij. (27)

Note that absolute convergence (Lemma 1) ensures the manipulation of infinite sums is valid. The proof for i∈H(•, j) is similar but requires slightly more care:

(28)∑k∈H(•, j)1|H(•, j)|eik=∑k∈H(•, j)1|H(•, j)|{δik+∑P∈H(i→k)∏l=1ℓ(P)1|H(•, pl)|} (28)

(29)=1|H(•,j)|+∑k≠i1|H(•,j)|∑P∈H(i→k)∏l=1ℓ(P)1|H(•,pl)| (29)

(30)=∑P∈H(i→j)pℓ(P)−1=i∏l=1ℓ(P)1|H(•,pl)|+∑P∈H(i→j)pℓ(P)−1≠i∏l=1ℓ(P)1|H(•,pl)| (30)

(31)=∑P∈H(i→j)∏l=1ℓ(P)1|H(•,pl)|=eij. (31)

□

Lemma 3Define a matrixM⁽ⁱ⁾ by (i)Mim(i)=δim, (ii) for H(i→k)≠∅ and m∈H(•, k), setMkm(i)=1|H(•,k)|, (iii)Mkm(i)=0 otherwise. Then the conclusions of Lemma 2 are equivalent to the following: For each i, the vectorvwhere v_k=e_ik satisfiesM⁽ⁱ⁾v=v.

Proof. Equivalence may be verified directly.□

Lemma 4SupposevsatisfiesM⁽ⁱ⁾v=v. Then for all k we must have∣v_k∣≤v_i∣.

Proof. Suppose not. Then there exists k such that (i) ∣v_k∣>v_i∣ and (ii) ∣v_k∣ is maximal. Without loss of generality H(i→k)≠∅ since otherwise vk=∑jMkj(i)vj=∑j0vj=0. Then ∣v_m∣=∣v_k∣ for all m∈H(•, k), since vk=∑mMkm(i)vm=∑m∈H(•,k)1|H(•,k)|vm and ∑m∈H(•,k)1|H(•,k)|=1. By induction on a path from i to K we arrive at ∣v_i∣=∣v_k∣, contradicting (i).□

Lemma 5There exists a unique solution toM⁽ⁱ⁾v=vwith v_i=1.

Proof.Existence: The ith row of M⁽i⁾ contains only zeros, except for the ith entry which is equal to 1. Hence the characteristic polynomial contains a root (eigenvalue) λ=1 with corresponding eigenvector v˜≠0. From Lemma 4 we must have v_i≠0 (else |v˜j|≤|v˜i|=0 for all j, implying that v˜=0). Taking v=v˜/vi proves existence.

Uniqueness: Suppose v and v′ both satisfy the criteria. Then v″=v–v′ is also an eigenvector of M⁽i⁾ with unit eigenvalue. From Lemma 4 we have |v″k|≤|v″i|=0 for all k, implying that v=v′.□

Proof of Theorem 4. Together Lemmas 1–5 prove that the effect matrix e={e_ij} exists and is unique. Moreover, the ith row e_i of e may be computed as the unique eigenvector of M⁽i⁾ corresponding to unit eigenvalue.

For moderate dimensional matrices M⁽i⁾, eigenvectors may be calculated using any valid algorithm. One such algorithm, suitable for high dimensional problems, is the “power iteration” method. Specifically we compute the leading eigenvector v of M⁽i⁾ as

(32)ei=limn→∞ v(n), v(n)=M(i)v(n−1)||M(i)v(n−1)||, vk(0)=δik. (32)

Efficient computation in this setting is reviewed in Berkhin (2005).□

References

Banerjee, A. and J. Jost (2009): “Graph spectra as a systematic tool in computational biology,” Discrete Appl. Math., 157, 2425–2431.Search in Google Scholar

Berkhin, P. (2005): “A survey on pagerank computing,” Internet Mathematics, 2, 73–120.10.1080/15427951.2005.10129098Search in Google Scholar

Breitkreutz, D., L. Hlatky, E. Rietmanc and J. A. Tuszynski (2012): “Molecular signaling network complexity is correlated with cancer patient survivability,” Proc. Natl. Acad. Sci. USA 109, 9209–9212.10.1073/pnas.1201416109Search in Google Scholar PubMed PubMed Central

Cantone, I., L. Marucci, F. Iorio, M. A. Ricci, V. Belcastro, M. Bansal, S. Santini, M. di Bernardo, D. di Bernardo and M. P. Cosma (2009): “A yeast synthetic network for in vivo assessment of reverse-engineering and modeling approaches,” Cell 137, 172–181.10.1016/j.cell.2009.01.055Search in Google Scholar PubMed

Chuang, H. Y., E. Lee and T. Ideker (2007): “Network-based classification of breast cancer metastasis,” Mol. Syst. Biol. 3, 140.Search in Google Scholar

Dash, D. (2003): “Caveats for causal reasoning with equilibrium models,” PhD thesis, Intelligent Systems Program, University of Pittsburgh.Search in Google Scholar

Davis, J. and M. Goadrich (2006): “The relationship between Precision-Recall and ROC curves.” Proceedings of the 23rd International Conference on Machine Learning. ACM Press, New York, pp. 233–240.10.1145/1143844.1143874Search in Google Scholar

De Jongh, M. and M. J. Druzdzel (2009): “A comparison of structural distance measures for causal Bayesian network models.” In: Klopotek, M., Przepiorkowski, A., Wierzchon, S.T., Trojanowski, K. (Eds.), Recent Advances in Intelligent Information Systems, Academic Publishing House EXIT, Warsaw, pp. 443–456.Search in Google Scholar

Drummond, C. and R. C. Holte (2004): “What ROC curves can’t do (and cost curves can).” Proc. 1st Workshop ROC Analysis in AI, pp. 19–26.Search in Google Scholar

Fawcett, T. (2006): “An introduction to ROC analysis,” Pattern Recogn, 27, 861–874.10.1016/j.patrec.2005.10.010Search in Google Scholar

Feiglin, A., A. Hacohen, A. Sarusi, J. Fisher, R. Unger and Y. Ofran (2012): “Static network structure can be used to model the phenotypic effects of perturbations in regulatory networks,” Bioinformatics, 28, 2811–2818.10.1093/bioinformatics/bts517Search in Google Scholar PubMed

Feizi, S., D. Marbach, M. Médard and M. Kellis (2013): “Network deconvolution as a general method to distinguish direct dependencies in networks,” Nat. Biotechnol., 31, 726–733.Search in Google Scholar

Filosi, M., R. Visintainer, S. Riccadonna, G. Jurman and C. Furlanello (2014): “Stability indicators in network reconstruction,” PloS One, 9, e89815.10.1371/journal.pone.0089815Search in Google Scholar PubMed PubMed Central

Goadrich, M., L. Oliphant and J. Shavlik (2004): “Learning ensembles of first-order clauses for recall-precision curves: A case study in biomedical information extraction.” Proceedings of the 14th International Conference on Inductive Logic Programming (ILP).10.1007/978-3-540-30109-7_11Search in Google Scholar

Heiser, L. M., A. Sadanandam, W. L. Kuo, S. C. Benz, T. C. Goldstein, S. Ng, W. J. Gibb, N. J. Wang, S. Ziyad, F. Tong, N. Bayani, Z, Hu, J. I. Billig, A. Dueregger, S. Lewis, L. Jakkula, J. E. Korkola, S. Durinck, F. Pepin, Y. Guan, E. Purdom, P. Neuvial, H. Bengtsson, K. W. Wood, P. G. Smith, L. T. Vassilev, B. T. Hennessy, J. Greshock, K. E. Bachman, M. A. Hardwicke, J. W. Park, L. J. Marton, D. M. Wolf, E. A. Collisson, R. M. Neve, G. B. Mills, T. P. Speed, H. S. Feiler, R. F. Wooster, D. Haussler, J. M. Stuart, J. W. Gray and P. T. Spellman (2012): “Subtype and pathway specific responses to anticancer compounds in breast cancer,” Proc. Natl. Acad. Sci. USA, 109, 2724–2729.10.1073/pnas.1018854108Search in Google Scholar PubMed PubMed Central

Hill, S. M., Y. Lu, J. Molina, L. M. Heiser, P. T. Spellman, T. P. Speed, J. W. Gray, G. B. Mills and S. Mukherjee (2012): “Bayesian inference of signaling network topology in a cancer cell line,” Bioinformatics, 28, 2804–2810.10.1093/bioinformatics/bts514Search in Google Scholar PubMed PubMed Central

Iwasaki, Y. and H. A. Simon (1994): “Causality and model abstraction,” Artif. Intell., 67, 143–194.Search in Google Scholar

Jeong, H., B. Tombor, R. Albert, Z. N. Oltvai and A. -L. Barabási (2000): “The large-scale organization of metabolic networks,” Nature, 407, 651–654.10.1038/35036627Search in Google Scholar PubMed

Johannes, M., J. C. Brase, H. Fröhlich, S. Gade, M. Gehrmann, M. Fälth, H. Sültmann and T. Beissbarth (2010): “Integration of pathway knowledge into a reweighted recursive feature elimination approach for risk stratification of cancer patients,” Bioinformatics, 26, 2136–2144.10.1093/bioinformatics/btq345Search in Google Scholar PubMed

Jurman, G., S. Riccadonna, R. Visintainer and C. Furlanello (2011a): “Biological network comparison via Ipsen-Mikhailov distance,” arXiv, 1109.0220.Search in Google Scholar

Jurman, G., R. Visintainer and C. Furlanello (2011b): “An introduction to spectral distances in networks,” Fr. Art. In. 226, 227–234.Search in Google Scholar

Lakhina, A., K. Papagiannaki, M. Crovella, C. Diot, E. D. Kolaczyk and N. Taft (2004): “Structural analysis of network traffic flows.” Proceedings of International Conference on Measurement and Modeling of Computer Systems, ACM, New York, USA, pp. 61–72.10.1145/1005686.1005697Search in Google Scholar

Maathuis, M. H., D. Colombo, M. Kalisch and P. Bühlmann (2010): “Predicting causal effects in large-scale systems from observational data,” Nat. Methods, 7, 247–248.Search in Google Scholar

Marbach, D., T. Schaffter, C. Mattiussi and D. Floreano (2009): “Generating realistic in silico gene networks for performance assessment of reverse engineering methods,” J. Comp. Biol., 16, 229–239.Search in Google Scholar

Marbach, D., R. J. Prill, T. Schaffter, C. Mattiussi, D. Floreano and G. Stolovitzky (2010): “Revealing strengths and weaknesses of methods for gene network inference,” Proc. Natl. Acad. Sci. USA, 107, 6286–6291.10.1073/pnas.0913357107Search in Google Scholar PubMed PubMed Central

Marbach, D., J. C. Costello, R. Küffner, N. M. Vega, R. J. Prill, D. M. Camacho, K. R. Allison, M. Kellis, J. J. Collins and G. Stolovitzky (2012): “Wisdom of crowds for robust gene network inference,” Nat. Methods, 9, 796–804.Search in Google Scholar

Milenković, T., F. Ioannis, L. Michael and N. Pržulj (2009): “Optimized null models for protein structure networks,” PLoS One, 4, e5967.10.1371/journal.pone.0005967Search in Google Scholar PubMed PubMed Central

Morrison, J. L., R. Breitling, D. J. Higham and D. R. Gilbert (2005): “GeneRank: using search engine technology for the analysis of microarray experiments,” BMC Bioinformatics, 6, 233.10.1186/1471-2105-6-233Search in Google Scholar PubMed PubMed Central

Nelander, S., W. Wang, B. Nilsson, Q. B. She, C. Pratilas, N. Rosen, P. Gennemark and C. Sander (2008): “Models from experiments: combinatorial drug perturbations of cancer cells,” Mol. Syst. Biol., 4, 216.Search in Google Scholar

Oates, C. and S. Mukherjee (2012): “Network inference and biological dynamics,” Ann. Appl. Stat., 6, 1209–1235.Search in Google Scholar

Oates, C.J., J. Korkola, J. W. Gray and S. Mukherjee (2014a): “Joint estimation of multiple related biological networks,” Ann. Appl. Stat., to appear.10.1214/14-AOAS761Search in Google Scholar

Oates, C. J., F. Dondelinger, N. Bayani, J. Korola, J. W. Gray and S. Mukherjee (2014b): “Causal network inference using biochemical kinetics,” Bioinformatics, to appear.10.1093/bioinformatics/btu452Search in Google Scholar PubMed PubMed Central

Page, L., S. Brin, R. Motwani and T. Winograd (1999): “The pagerank citation ranking: bringing order to the web,” Technical Report, Stanford InfoLab.Search in Google Scholar

Pearl, J. (2000): Causality: models, reasoning, and inference. Cambridge University Press, Cambridge, UK.Search in Google Scholar

Peters, J. and P. Bühlmann (2013): “Structural Intervention Distance (SID) for evaluating causal graphs,” arXiv, 1306.1043.Search in Google Scholar

Prill, R. J., D. Marbach, J. Saez-Rodriguez, P. K. Sorger, L. G. Alexopoulos, X. Xue, N. D. Clarke, G. Altan-Bonnet and G. Stolovitzky (2010): “Towards a rigorous assessment of systems biology models: The DREAM3 challenges,” PLoS One, 5, e9202.10.1371/journal.pone.0009202Search in Google Scholar PubMed PubMed Central

Prill, R. J., J. Saez-Rodriguez, L. G. Alexopoulos, P. K. Sorger and G. Stolovitzky (2011): “Crowdsourcing network inference: the DREAM predictive signaling network challenge,” Sci. Signal, 4(189), mr7.10.1126/scisignal.2002212Search in Google Scholar PubMed PubMed Central

Scutari, M. and R. Nagarajan (2013): “Identifying significant edges in graphical models of molecular networks,” Artif. Intell. Med., 57, 207–217.Search in Google Scholar

Shrivastava, A. and P. Li (2014): “A new space for comparing graphs,” arXiv, 1404.4644.Search in Google Scholar

Simon, N. and R. Tibshirani (2012): Comment on “Detecting novel associations in large datasets” by Reshef et al. Technical Report, Stanford, CA, USA.Search in Google Scholar

Städler, N. and S. Mukherjee (2014): “Multivariate gene-set testing based on graphical models,” Biostatistics, to appear.10.1093/biostatistics/kxu027Search in Google Scholar PubMed

Tong, A. H. Y., G. Lesage, G. D. Bade, H. Ding, H. Xu, X. Xin, J. Young, G. F. Berriz, R. L. Brost, M. Chang, Y. Q. Chen, X. Cheng, G. Chua, H. Friesen, D. S. Goldberg, J. Haynes, C. Humphries, G. He, S. Hussein, L. Ke, N. Krogan, Z. Li, J. N. Levinson, H. Lu, P. Ménard, C. Munyana, A. B. Parsons, O. Ryan, R. Tonikian, T. Roberts, A. M. Sdicu, J. Shapiro, B. Sheikh, B. Suter, S. L. Wong, L. V. Zhang, H. Zhu, C. G. Burd, S. Munro, C. Sander, J. Rine, J. Greenblatt, M. Peter, A. Bretscher, G. Bell, F. P. Roth, G. W. Brown, B. Andrews, H. Bussey and C. Boone (2004): “Global mapping of the yeast genetic interaction network,” Science, 303, 808–813.10.1126/science.1091317Search in Google Scholar PubMed

Wang, C., J. Xuan, L. Chen, P. Zhao, Y. Wang, R. Clarke and E. Hoffman (2008): “Motif-directed network component analysis for regulatory network inference,” BMC Bioinformatics, 9(Suppl. 1), S21.Search in Google Scholar

Warshall, S. (1962): “A theorem on Boolean matrices,” J. ACM., 9, 11–12.Search in Google Scholar

Weile, J., K. James, J. Hallinan, S. J. Cockell, P. Lord, A. Wipat and D. Wilkinson (2012): “Bayesian integration of networks without gold standards,” Bioinformatics, 28, 1495–1500.10.1093/bioinformatics/bts154Search in Google Scholar PubMed PubMed Central

Winter, C., G. Kristiansen, S. Kersting, J. Roy, D. Aust, T. Knösel, P. Rümmele, B. Jahnke, V. Hentrich, F. Rückert, M. Niedergethmann, W. Weichert, M. Bahra, H.J. Schlitt, U. Settmacher, H. Friess, M. Büchler, H.-D. Saeger, M. Schroeder, C. Pilarsky, R. Grützmann (2012): “Google goes cancer: improving outcome prediction for cancer patients by network-based ranking of marker genes,” PLoS Comput. Biol., 8, e1002511.Search in Google Scholar

Yates, P. D. and N. D. Mukhopadhyay (2013): “An inferential framework for biological network hypothesis tests,” BMC Bioinformatics, 14, 94.10.1186/1471-2105-14-94Search in Google Scholar PubMed PubMed Central

Supplemental Material

The online version of this article (DOI: 10.1515/sagmb-2014-0012) offers supplementary material, available to authorized users.

Published Online: 2014-8-5

Published in Print: 2014-10-1

Quantifying the multi-scale performance of network inference algorithms

Abstract

Acknowledgments

Appendix A: Proofs

References

Supplemental Material

Journal and Issue

Articles in the same Issue