1 Introduction

Feature selection is a problem which arises naturally in a number of applications, and, in particular, in biomedical tasks, where the number of parameters is potentially very high but just a small subset of them is informative. In the past decades a number of model selection methods have been proposed, including methods for group and hierarchical feature selection, which are supposed to reveal some structure of underlying data. An important issue is the stability of feature selection methods. A result of model selection is very sensitive to the samples used during feature extraction and to a learning method. Moreover, the power of statistical feature selection methods was hardly studied. Recently, [1] considered 32 state-of-the-art feature selection approaches, however, a simple univariate t-test appeared to reach the highest stability and a very reasonable performance.

It has been observed that functions captured by different feature sets can be very similar, despite a very low degree of overlapping between these feature sets. In our contribution, we tackle this complex problem of stable feature selection on the functional level. We hope that our results will provide biologists with some intuition on functional classes of features, i.e., on functional classes of biological parameters.

Our research is motivated by real rich high-dimensional biomedical applications, in particular, by challenges of quantitative metagenomics and transcriptomics. In quantitative metagenomics we study the collective genome of the micro-organisms inhabiting human body, and, since recently, it has become feasible to measure the abundance of bacterial species. Scientists doing pre-clinical research are often interested to find groups of bacterial species associated with a particular phenomenon. So, e.g., in [2] the challenge is to associate groups of bacterial species with the development of obesity. Another source of data, transcriptomics considers the complete set of RNA transcripts produced by the genome at different time points, and allows scientists to analyze various phenomena in various tissues. The metagenomics and transciptomics applications are extremely high-dimensional, and to extract the most significant biomarkers, some efficient feature selection is needed. The number of features can be very high but at the same time the features are structured in unobserved functional categories of comparable predictive performance, i.e., there are subsets of features that encode the same biological function and therefore perform equally.

It is known [3, 4] that there exist strong correlations or similarities between genes which can be examined, for instance, with an automated annotation tool. In our work, we consider both stability on the level of features and on the level of their functions.

To select the most pertinent subsets of parameters and to reach the stability on the functional level, we propose to apply sparse zero-sum games. We introduce a thresholding stochastic bandit which efficiently ranks sets of features. The idea to use multi-armed bandits to carry out ranking, is not new. To our knowledge, the first attempt to rank with a bandit algorithm was done by [5], and it was applied to Web documents ranking; [6] applied the ranking to digital libraries. An approach which is in some sense similar to ours was proposed by [7], where the problem of feature selection is formalized as a reinforcement learning problem (a one-player game), and the learning procedure relies on the Monte-Carlo tree search.

One of the most classical games considered by the game theory, is the prisoner’s dilemma which allows to investigate pairwise cooperations. In its original form, there are two players in the game, and the players can either cooperate or defect. Both players get the reward R if they cooperate and they get the punishment P if both defect. If one player defects and another cooperates, the first one obtains the temptation T, and the second the payoff S, where T > R > P > S. The game’s strong Nash equilibrium is the situation where both players defect, i.e. the situation from which each player could only do worth by changing its move.

Various extensions of the prisoner’s dilemma have been proposed. In particular, the problem how to promote the evolution of cooperation has been studied. For instance, it has been shown that in a spatial social dilemma game, where players can be represented as nodes on a grid, cooperators can profit through forming clusters of cooperators which are protected from exploitation of defectors [10]. In [11] it has been shown that it is important to define the highest reputation neighbors as strategy donors to promote cooperation. Such clusters of collaboration promoters are robust, and protect efficiently against defectors.

Another important factor which influences cooperative behavior is inertia. [12] demonstrated that cooperation with small inertia will be easily invaded by defectors, and with large inertia the cooperation level will hardly change since the players are not likely to change their strategies. The study [12] discusses an intermediate inertia level which leads to an optimal cooperation.

Public good games come from experimental economics. Players decide how many of their private tokens should be put into a public pool. Then the “public good” is multiplied by a factor and is evenly divided among all players. The group’s reward is maximized when all players contribute all their tokens. However, the Nash equilibrium is zero contributions by players.

A spatial version of the game is defined on a grid where each agent is a node, has several neighbors, and plays simultaneously with all the neighbors. As a result, one observes spatial clusters of cooperation and defection. It was noticed that changing existing links between neighbors can be beneficial for the cooperation, as well as growth of the network has been observed [13]. An aspiration-induced version of the game was recently proposed [14] where a player would cut the link with the neighbor if his payoff from the group centered on this neighbor does not exceed an aspiration level. In this case, the player will connect to another randomly selected player. It has been shown that there exists an aspiration level which induces an optimal cooperation.

In spatial public good games it is usually assumed that the investment of all players is identical. [15] considers the case where players are heterogeneous, what is much more realistic. The study concludes that heterogeneous investments can promote cooperation.

To our knowledge, our contribution is the first attempt to see the problem of group feature selection as a sparse matrix game. Without loss of generality, a stochastic bandit used in our experiments, is the EXP3 algorithm [8, 9]. We illustrate by our experiments that the proposed framework achieves both a stability and performance which are not worse, and in some cases are better, than ones of the state-of-the-art methods.

The novel thresholding bandit learns a sparse probability distribution on sets of features. This is identical to estimating sparse Nash equilibria. Finding Nash equilibria of games in strategic form, is a challenging task, especially if games are large-scale. In our contribution we consider bimatrix (two-player) zero-sum games, and refer to them, for short, as “matrix games”. The proposed approach combines both advantages: compact mixed equilibria and fast computation. We illustrate both by artificial and real biomedial applications that the introduced framework can be applied for the biomarker discovery from high-dimensional data.

This paper is organized as follows. Section 2 discusses the sparsity issues in large games, and how to induce sparsity on mixed Nash equilibria. In Section 3 we present the thresholding stochastic bandit for model selection. We provide some theoretical analysis of the introduced method in Section 3.1. In Section 4 we discuss some similarity and stability measures which we use to evaluate the feature selection methods. In Section 5 we show the results of our experiments on simulated and real data sets. Concluding remarks and perspectives close the paper.

2 Sparsity in Zero-Sum Matrix Games

In this section, we consider matrix games and the state-of-the-art methods to find Nash and ϵ−Nash equilibria of these games.

We are interested in particular in the situation where the mixed strategies played by agents are sparse. The intuition behind such a sparse mixed strategy is as follows. Although the number of all possible pure strategies can be very big or even infinite, the number of reasonable moves can be quite small. Speaking more formally, sparse mixed strategies are related to games with limited randomness: “to play a game” means that a player samples a pure strategy what entails randomness. In real life, due to computational issues, the randomness is limited or not available. [16].

[17] proposed an algorithm to find sparse Nash equilibria which was a building block for Monte-Carlo Tree Search, and the main goal was to accelerate bandits. The very recent work of [18] introduces an algorithm which is similar to [17] and is based on the rounding solution of the EXP3 what leads to better theoretical bounds.

A zero-sum matrix game is usually defined by a payoff matrix M. Let k₁ be a number of pure strategies of player 1, and k₂ be a number of pure strategies of player 2. Without loss of generality, let the values of the payoff matrix be in the interval [0, 1]. When player 1 chooses action i ∈ ⟦1, k₁⟧ and player 2 chooses action j ∈ ⟦1, k₂⟧, then player 1 gets reward M_{i, j} and player 2 gets reward −M_{i, j}.

Mixed strategies x and y of player 1 and player 2 are probability distributions on their pure strategies. A best response x of player 1 to the mixed strategy y of player 2 is a strategy which maximizes the expected payoff xMy of player 1. Similarly, a best response of player 2 maximizes his/her expected gain.

A Nash equilibrium [19] is a pair of mixed strategies that are best responses to each other. More formally, a Nash equilibrium is a pair (x*, y*) with and such that (1) where the sum of the elements of x* and of y* equals one; this will be implicitly assumed in this paper for maxima or minima over mixed strategies.

An ϵ-Nash equilibrium [20] is an approximation (x, y) of a Nash equilibrium such that (2) This reflects the idea that players do not intend to change their strategies, since the amount of gain they could get by choosing other strategies is very small. Every Nash equilibrium is surrounded by a region of ϵ-Nash, and that is why ϵ-Nash equilibria always exist [20] for any ϵ > 0.

Matrix games can be solved (in the sense that an exact Nash equilibrium can be found) by linear programming (LP) [21]. Linear programming can itself be solved polynomially, with exponent 3.5. Therefore, finding the Nash equilibrium of a matrix game is polynomial. However, this result has two weaknesses:

• the exponent 3.5 is not negligible;
• this result holds for deterministic matrix games only, in which playing a given pair (i, j) of strategies always leads to the same result; the case of a random payoff (independently and identically drawn from a payoff distribution ) is not covered.

One of the earliest approaches to find Nash equilibria is an algorithm called fictitious play [22]. It is an iterative method which is simple to implement, however, it is assumed that the players know their payoff matrices. If a game is large and it is not feasible to enumerate all possible strategies, and therefore, store the payoff matrices, the application of the fictitious play in its original form is not possible.

This is why an extensive literature, including bandits algorithms, is devoted to how to find ϵ-Nash equilibria, with exponent 1 (as a function of K = max(k₁, k₂), within logarithmic factor), with a limited dependency in ϵ (exponent 2).

It has been shown in [23] that one can find an ϵ-Nash equilibrium of a matrix game in time O(K log(K)/ϵ²) with probability at least . In particular, [23] has shown that sublinear complexity makes sense for the problem under consideration: we do not have to read the matrix entirely to solve the game approximately. This implies that we measure the complexity on a machine with two crucial components:

• random-access memory, so that we can sample any coefficient of the matrix without reading all elements of the matrix;
• and equipped with a random generator, so that we can randomly shuffle the indices of the strategies (otherwise, there are counter-examples to the complexities, and we will see that randomization is a necessary tool for sublinear complexities in matrix games).

This idea was extended in [24], showing that the same can be done with stochastic problems, i.e., cases where the result of a game is stochastic as a function of the actions: for a pair (i, j) of actions (action i for the first player, action j for the second player), the reward is drawn in [0, 1] according to some probability distribution (independently at each trial), with expectation M_{i, j}.

In a number of games, there are many unreasonable strategies, which humans would not even consider. Under an unreasonable strategy we mean a strategy which contain suboptimal actions. The number of reasonable strategies is much smaller than the complete set of pure strategies. There are two main reasons for using sparsity in matrix games:

1. find a sparse mixed strategy, so that we can manipulate pure strategies in a computationally efficient manner;
2. find a computationally efficient approach to estimate a Nash equilibrium.

It has been shown by [25] and [26] that an impressive sparsity can be expected from matrix games without any assumption on the matrix. It is possible to find a subset of ⌈log(K)/ϵ²⌉ strategies (for each player) which is an ϵ-approximate Nash equilibrium. The existence of a deterministic algorithm with running polynomial time to find this subset has been proved by [27].

Therefore, we can answer the first question above concerning sparse pure strategies, but not the second one. If we test all subsets of this size to find the solution, the cost is O(K^O(log(K))) [25]. This cost is higher than the cost of LP for exact solution.

Recently [17, 28], and [29] proposed purification and thresholding techniques to cope with large games. These methods belong to post-processing approaches. The idea of the purification is to choose a pure strategy with highest probability and to play it with probability 1. If there are several pure strategies with the same high probability value, they are played with equal probability. The thresholding of [28] and [29] is less extreme than the purification. Only probabilities of pure strategies which are less than some threshold are put to zero. Our thresholding algorithm is similar to this idea.

Another avenue to increase the computational speed is to use the tree structure. The idea that sequences of choices instead of pure strategies can be used, has been introduced by [30]. The sequences of choices are efficiently represented as a tree. The number of these sequences is linear and not exponential in the size of the game tree. The same idea has been exploited in [31] and [32]. [32] has proposed to find Nash equilibria by a variation of the Lemke-Howson algorithm [33].

However, removing non-dominant strategies can have some unpredictable effects. It has been illustrated on the example of Poker game [34] that space reduction in large games is not obvious, and that a solution for a small game cannot be always easily extrapolated to the initial large game. That is why it is important to control how suboptimal the approximate solutions are.

3 Large-Scale Problems: Sparsity via Thresholding Bandits

A Nash equilibrium defined above as Eq (1) is a pair of mixed strategies where non of the players expects a better reward if he changes unilaterally his strategy. Note that it is possible to have several Nash equilibria in a game, however, the value of the game (x*)^T My* does not change.

Our goal is to find sparse mixed strategies for two players, and to do it efficiently. In this section, we propose an approach which finds compact Nash equilibria. In the following, we consider an ϵ−Nash equilibrium which is defined as a pair (x, y) (3) (4)

In our contribution, we do not assume that the matrix M is sparse. On the contrary, we suppose that the Nash equilibria and the ϵ-Nash equilibria have a very compact support size.

The intuition behind the novel approach is based on that sparsity can be induced via elimination of those strategies, whose estimated probability values are not significant after the number of rounds t: (5) where x_i is a i-coordinate of the probability distribution, is a corresponding (sparse) approximation, and parameter α ∈ (0, 1). The thresholding algorithm is summarized in Algorithm 1. Note that x_i is associated with a subset of features, and not with one feature.

The probability distribution x can be estimated by a bandit algorithm such as INF (implicitly normalized forecasters) or EXP3 (exponential exploration-exploitation) algorithm (see [8] and [24]). A stochastic bandit algorithm, for example, the EXP3 algorithm, drafted as Algorithm 2, can be used to rank the sets of features. In our experiments in Section 5 we will use the EXP3 algorithm.

Algorithm 1 THRESHOLDING-BANDIT: remove strategies with low weight

Find an approximate solution (x, y) of the zero-sum matrix game (after t iterations)

for all pure strategies x_i, y_j do

 if x_i < t^α/t then

  

 end if

 if y_j < t^α/t then

  

 end if

end for

Consider (x′, y′) to be the approximate Nash equilibrium

Optionally: use the bandit algorithm again, on strategies with non-zero coefficients in x′, y′ only

Below, we consider the theoretical analysis of the proposed thresholding bandit.

4 Similarity and Stability Measures

To compare the outcomes of different feature selection methods, we adopt two measures. We also estimate the stability on the functional level. The first measure we apply is a similarity measure [3, 4]. The I-overlap similarity measure [4] between two sets of features 𝓕₁ and 𝓕₂ is computed as the number of variables that are present in both sets, ∣𝓕₁ ∩ 𝓕₂∣. The value is normalized in such a way that the final result is in the interval [0;1], where the normalization factor is ∣𝓕₁ ∪ 𝓕₂∣. The I-overlap is computed by averaging over all pairwise comparisons.

The second measure we test, is a stability measure [35, 36] called relative weighted consistency CW_rel. This measure is subset-size-unbiased. Let us carry out feature selection k times and obtain 𝓢 = {𝓢₁,…,𝓢_ω} subsets of selected features from ω experiments. is the sum of cardinalities of sub-sets 𝓢_i and F_f is the number of times the variable f ∈ 𝓕 was observed in 𝓢. The CW measure is defined as follows (6) The CW_rel measure is obtained by adjusting the CW value by its minimal and maximal values: (7)

As we have already mentioned, biological functions expressed by different gene sets can be similar although the overlap between these sets is very low. To compare the functional overlap of the genes sets, we refer to the Gene Ontology (GO), and we apply the CW_rel measure on the functional level.

5 Experiments

In this section, we demonstrate the results of our experiments on two artificial data sets, and on a real complex original biomedical application.

A “small n, large p” scenario is simulated in Section 5.1, where we demonstrate the accuracy and stability of the thresholding bandits in the case where the number of observations is very small compared to the number of parameters. In Section 5.2 we consider the performance of the proposed method on an artificial data where features belong to functional categories. Section 5.3 illustrates our results on a real biomedical challenge which includes several heterogeneous data types.

[1] has recently performed an extensive comparison of stability of more than thirty feature selection methods. The conclusion of the study was that the univariate t-test is the most stable method. Therefore, we compare our results to the univariate t-test.

On the artificial data, we show that the accuracy and stability of the proposed thresholding bandit is higher than one of the t-test for a case where the number of observations is much less than the number of parameters (Section 5.1). In Section 5.2 we show on a simulated data set, that the thresholding bandit is efficient for subsets feature ranking where features belong to functional categories. Finally, in Section 5.3 we consider a real high-dimensional biomedical application, and illustrate a high accuracy and stability of the introduced method compared to such approaches as t-test, SVM, and a stochastic bandit without thresholding.

6 Conclusion

Biomarker selection is a crucial problem in biomedical high-dimensional applications. To find relationships between biomarkers, and to define functional families of features, is another problem related to feature selection. We have introduced a feature selection approach based on a sparse stochastic bandit, and we compared its predictive performance and stability to several state-of-the-art methods which are reported to be the most accurate and stable. We considered the newly introduced feature selection approach from cross-validation error rate and stability viewpoints, and our experiments on artificial and real data illustrate that the proposed approach is competitive. We also hope that functional classes of features learned from experiments, will provide additional knowledge and give rise to new hypotheses for biologists and physicians doing pre-clinical research.

Acknowledgments

The clinical work was supported by Agence Nationale de la Recherche (ANR MICRO-Obes), KOT-Ceprodi and the association Fondation Coeur et Arteres. All ethical agreement are obtained. This work is also part of the European Union’s Seventh Framework Program under grant agreement HEALTH-F4-2012-305312 (Metacardis project).

MicroObese Consortium list of contributors: Aurélie Cotillard, Sean P. Kennedy, Edi Prifti, Nicolas Pons, Emmanuelle Le Chatelier, Mathieu Almeida, Benoit Quinquis, Florence Levenez, Nathalie Galleron, Jean-Michel Batto, Joel Doré, Stanislav Dusko Ehrlich, Catherine Juste, Tomas de Wouters.