Abstract
We study the complexity of learning arbitrary Boolean functions of n variables by membership queries, if at most r variables are relevant. Problems of this type have important applications in fault searching, e.g. logical circuit testing and generalized group testing. Previous literature concentrates on special classes of such Boolean functions and considers only adaptive strategies. First we give a straightforward adaptive algorithm using O(r2r log n) queries, but actually, most queries are asked nonadaptively. This leads to the problem of purely nonadaptive learning. We give a graph-theoretic characterization of nonadaptive learning families, called r-wise bipartite connected families. By the probabilistic method we show the existence of such families of size O(r2r log n + r 22r). This implies that nonadaptive attribute-efficient learning is not essentially more expensive than adaptive learning. We also sketch an explicit pseudopolynomial construction, though with a slightly worse bound. It uses the common derandomization technique of small-biased k-independent sample spaces. For the special case r = 2, we get roughly 2.275 log n adaptive queries, which is fairly close to the obvious lower bound of 2 log n. For the class of monotone functions, we prove that the optimal query number O(2r + r log n) can be already achieved in O(r) stages. On the other hand, Ω(2r log n) is a lower bound on nonadaptive queries.
Article PDF
Similar content being viewed by others
References
Almuallim, H. & Dietterich, T.G. (1994). Learning Boolean concepts in the presence of many irrelevant features. Artificial Intelligence, 69, 279–305.
Alon, N., Bruck, J., Naor, J., Naor, M., & Roth, R. (1992a). Construction of asymptotically good, low-rate errorcorrecting codes through pseudorandom graphs. IEEE Transactions on Information Theory, 38, 509–516.
Alon, N., Goldreich, O., Hăstad, J., & Peralta, R. (1992b). Simple constructions of almost k-wise independent random variables. Random Structures and Algortihms, 3, 289–304; Ibid. (1993) 4, 119–120.
Althöfer, I. & Triesch, E. (1993). Edge search in graphs and hypergraphs of bounded rank. Discrete Mathematics, 115, 1–9.
Angluin, D. (1987). Queries and concept learning. Machine Learning, 2, 319–342.
Balding, D. J. & Torney, D. C. (1995). A comparative survey of non-adaptive pooling designs. In Genetic mapping and DNA sequencing (IMA volumes in mathematics and its applications) (pp. 133–155). Springer.
Balding, D. J. & Torney, D. C. (1996). Optimal pooling designs with error detection. Journal of Combinatorial Theory A, 74, 131–140.
Beimel, A., Geller, F., & Kushilevitz, E. (1998). The query complexity of finding local minima in the lattice. In Proceedings of the 11th Conference on Computational Learning Theory (COLT) (pp. 294–302). ACM Press.
Ben-David, S., Kushilevitz, E., & Mansour, Y. (1997). Online learning versus offline learning. Machine Learning, 29, 45–63.
Blum, A. (1992). Learning Boolean functions in an infinite attribute space. Machine Learning, 9, 373–386.
Blum, A., Hellerstein, L., & Littlestone, N. (1995). Learning in the presence of finitely or infinitely many irrelevant attributes. Journal of Computer and System Sciences, 50, 32–40.
Bshouty, N. H. (1995). Exact learning Boolean functions via the monotone theory. Information and Computation, 123, 146–153.
Bshouty, N. H. & Cleve, R. (1992). On the exact learning of formulas in parallel. In Proceedings of the 33th IEEE Foundations of Computer Science (FOCS) (pp. 513–522). IEEE Press.
Bshouty, N. H. & Hellerstein, L. (1996). Attribute-efficient learning in query and mistake-bound models. In Proceedings of the 9th Conference on Computational Lerning Theory (COLT) (pp. 235–243). ACM Press.
Clausen, M., Dress, A., Grabmeier, J., & Karpinski, M. (1991). On zero-testing and interpolation of k-sparse multivariate polynomials over finite fields. Theoretical Computer Science, 84, 151–164.
Colbourn, C. J. & Dinitz, J. H. (1996). The CRC Handbook of Combinatorial Designs. CRC Press.
Damaschke, P. (1994). Atight upper bound for group testing in graphs. Discrete Applied Mathematics, 48, 101–109.
Damaschke, P. (1997). The algorithmic complexity of chemical threshold testing. In Lecture Notes in Computer Science, Vol. 1203: Proceedings of the 3rd Italian Conference on Algorithms and Complexity (CIAC) (pp. 205–216). Springer.
Damaschke, P. (1998a). Achip search problem on binary numbers. In Lecture Notes in Computer Science, Vol. 1380: Proceedings of the 3rd Latin American Symposium on Theoretical Informatics (LATIN) (pp. 11–22). Springer.
Damaschke, P. (1998b). Comutational aspects of parallel attribute-efficient learning. In Lecture Notes in Artificial Intelligence, Vol. 1501: Proceedings of the 9th International Workshop on Algorithmic Learning Theory (ALT) (pp. 103–111). Springer.
Damaschke, P. (1998c). Randomized group testing for mutually obscuring defectives. Information Processing Letters, 67, 131–135.
De Bonis, A., Gargano, L., & Vaccaro, U. (1998). Improved algorithms for chemical threshold testing problems. In Lecture Notes in Computer Science, Vol. 1449: Proceedings of the 4th Conference on Computing and Combinatorics (COCOON) (pp. 127–136). Springer.
De Bonis, A. & Vaccaro, U. (1998). Improved algorithms for group testing with inhibitors. Information Processing Letters, 67, 57–64.
Dhagat, A. & Hellerstein, L. (1994). PAC learning with irrelevant attributes. In Proceedings of the 35th IEEE Foundations of Computer Science (FOCS) (pp. 64–74). IEEE Press.
Du, D. Z. & Hwang, F. K. (1993). Combinatorial Group Testing and its Applications. World Scientific.
Farach, M., Kannan, S., Knill, E., & Muthukrishnan, S. (1997). Group testing problems in experimental molecular biology. In Proceedings of Compression and Complexity of Sequences (pp. 357–367). IEEE Computer Society.
Fischer, P., Klasner, N., & Wegener, I. (1999). On the cut-off point for combinatorial group testing. Discrete Applied Mathematics, 91, 83–92.
Goldman, S. A. & Sloan, R. H. (1994). The power of self-directed learning. Machine Learning, 14, 271–294.
Hofmeister, T. (1999). An application of codes to attribute-efficient learning. In Lecture Notes in Artificial Intelligence, Vol. 1572: Proceedings of 5th European Conference on Computational Learning Theory (EuroCOLT) (pp. 101–110). Springer.
Khardon, R. & Roth, D. (1996). Reasoning with models. Artificial Intelligence, 87, 187–213.
Kivinen, J., Mannila, H., & Ukkonen, E. (1992). Learning hierarchical rule sets. In Proceedings of the 5th Conference on Computational Learning Theory (COLT) (pp. 37–44). ACM Press.
Kleitman, D. J. & Spencer, J. H. (1973). Families of k-independent sets. Discrete Mathematics, 6, 255–262.
Knill, E. (1995). Lower bounds for identifying subset members with subset queries. In Proceedings of the 6th ACM-SIAM Symposium on Discrete Algorithms (SODA) (pp. 369–377).
Littlestone, N. (1988). Learning quickly when irrelevant attributes abound: A new linear-threshold algorithm. Machine Learning, 2, 285–318.
Macula, A. & Reuter, G. Simplified searching for two defects. Journal of Statistical Planning and Inference. To appear.
Motwani, R. & Raghavan, P. (1995). Randomized Algorithms. Cambridge University Press.
Naor, J. & Naor, M. (1993). Small-bias probability spaces: Efficient constructions and applications. SIAM Journal on Computing, 22, 838–856.
Naor, M., Schulman, L. J., & Srinivasan, A. (1995). Splitters and near-optimal derandomization. In Proceedings of the 36th IEEE Foundations of Computer Science (FOCS) (pp. 182–191). IEEE Press.
Roth, R. M. & Benedek, G. M. (1991). Interpolation and approximation of sparse multivariate polynomials over GF(2). SIAM Journal on Computing, 20, 291–314.
Seroussi, G. & Bshouty, N. H. (1988). Vector sets for exhaustive testing of logic circuits. IEEE Transactions on Information Theory, 34, 513–522.
Triesch, E. (1996). A group testing problem for hypergraphs of bounded rank. Discrete Applied Mathematics, 66, 185–188.
Uehara, R., Tsuchida, K., & Wegener, I. (1997). Optimal attribute-efficient learning of disjunction, parity, and threshold functions. In Lecture Notes in Artificial Intelligence, Vol. 1208: Proceedings of the 3rd European Conference on Computational Learning Theory (EuroCOLT)(pp. 171–184). Springer.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Damaschke, P. Adaptive Versus Nonadaptive Attribute-Efficient Learning. Machine Learning 41, 197–215 (2000). https://doi.org/10.1023/A:1007616604496
Issue Date:
DOI: https://doi.org/10.1023/A:1007616604496