A simple approach for adapting continuous load balancing processes to discrete settings

Abstract

We consider the neighbourhood load balancing problem. Given a network of processors and an arbitrary distribution of tasks over the network, the goal is to balance load by exchanging tasks between neighbours. In the continuous model, tasks can be arbitrarily divided and perfectly balanced state can always be reached. This is not possible in the discrete model where tasks are non-divisible. In this paper we consider the problem in a very general setting, where the tasks can have arbitrary weights and the nodes can have different speeds. Given a continuous load balancing algorithm that balances the load perfectly in $T$ rounds, we convert the algorithm into a discrete version. This new algorithm is deterministic and balances the load in $T$ rounds so that the difference between the average and the maximum load is at most $2d\cdot w_{\max }$, where d is the maximum degree of the network and $w_{\max }$ is the maximum weight of any task. For general graphs, these bounds are asymptotically lower compared to the previous results. The proposed conversion scheme can be applied to a wide class of continuous processes, including first and second order diffusion, dimension exchange, and random matching processes. For the case of identical tasks, we present a randomized version of our algorithm that balances the load up to a discrepancy of $\mathscr {O}(\sqrt{d \log n})$ provided that the initial load on every node is large enough.

Appendices

Appendix 1: Derivation of round equation in SOS

Here we show that setting $y_{i,j}^{{}}(t)$ based on Eq. (4) leads to round equation of Eq. (5). We use [n] to denote $\{1,\dots ,n\}$:

$$\begin{aligned} x_{i}^{{}}(t+1)&= x_{i}^{{}}(t) + \sum _{j \in [n]} ( y_{j,i}^{{}}(t)-y_{i,j}^{{}}(t) )\\&= (1-\beta +\beta )\cdot x_{i}^{{}}(t) \\&\quad + \sum _{j \in [n]} \left( (\beta -1)\cdot y_{j,i}^{{}}(t-1) + \beta \cdot P_{j,i}\cdot x_{j}^{{}}(t) \right) \\&\quad - \sum _{j \in [n]} \left( (\beta -1)\cdot y_{i,j}^{{}}(t-1) + \beta \cdot P_{i,j}\cdot x_{i}^{{}}(t) \right) \\&= (1-\beta )\cdot \left( x_{i}^{{}}(t) -\sum _{j \in [n]} (y_{j,i}^{{}}(t-1)-y_{i,j}^{{}}(t-1))\right) \\&\quad +\beta \cdot x_{i}^{{}}(t) -\beta \cdot \sum _{j \in [n]} x_{i}^{{}}(t) \cdot P_{i,j} \\&\quad +\beta \cdot \sum _{j \in [n]} x_{j}^{{}}(t) \cdot P_{j,i}\\&= (1-\beta )\cdot x_{i}^{{}}(t-1) +\beta \cdot x_{i}^{{}}(t) \\&\quad -\beta \cdot x_{i}^{{}}(t) \cdot \sum _{j \in [n]} P_{i,j} +\beta \cdot \sum _{j \in [n]} x_{j}^{{}}(t) \cdot P_{j,i}\\&= (1-\beta )\cdot x_{i}^{{}}(t-1)+\beta \cdot \sum _{j \in [n]} x_{j}^{{}}(t) \cdot P_{i,j} \end{aligned}$$

where the last equation holds since we have $\sum _{j \in [n]} P_{i,j} =1$. Translating to the matrix form, this yields the desired round equation.

Appendix 2: Hoeffding’s bound—adaptation for randomized rounding

Lemma 12

Let $X_1,\dots , X_k$ be k independent random variables, and $p_1,\dots ,p_k$ be constants where for all $i, 0<| p_i | <1$. Suppose $X_i$ is $p_i-1$ with probability $p_i$ and $p_i$ otherwise. If we define the random variable $X = \sum _{1\leqslant i\leqslant k} X_i$, then we have for any $\delta > 0$ that

Proof

We first note that for each i, we have

$$\begin{aligned} {\mathbf {Ex}}\left[ X_i\right] = (p_i-1)\cdot p_i + p_i\cdot (1-p_i) = 0; \end{aligned}$$

Hence, by the linearity of expectation we get ${\mathbf {Ex}}\left[ X\right] = \sum _{1\leqslant i\leqslant k}{\mathbf {Ex}}\left[ X_i\right]$. Also, for each $i, p_i-1\leqslant X_i \leqslant p_i$. Therefore, we can apply the Hoeffding’s bound [30] to get , as required. $\square$

Appendix 3: Comparison tables

See Tables 1 and 2.

Table 1 Final max-min discrepancy of our algorithms compared to other discrete diffusion processes for different graph classes

Table 2 Final max-min discrepancy of our algorithms compared to other discrete processes in the matching model