Noise Stable Halfspaces are Close to Very Small Juntas

: Bourgain [Bou02] showed that any noise stable Boolean function f can be well-approximated by a junta. In this note we give an exponential sharpening of the parameters of Bourgain’s result under the additional assumption that f is a halfspace.


Introduction
There is a sequence of results [NS94,Fri98,Bou02] in the theory of Boolean functions which share the following general flavor: if the Fourier spectrum of a Boolean function f is concentrated on lowdegree coefficients, then f must be close to a junta (a function that depends only on a small number of its input variables).Bourgain's junta theorem [Bou02] is the most recent and strongest of these results; roughly speaking, it says that if a Boolean function f has low noise sensitivity then f must be close to a junta.See Section 1.1 for definitions and a precise statement of Bourgain's theorem.(Subsequently [KS] generalized Bourgain's result to product distributions, albeit with somewhat weaker parameters.More recently [KO12] gave a sharpening in the parameters of Bourgain's theorem; see Section 1.1.) The parameters in the statement of Bourgain's theorem are essentially the best possible for general Boolean functions, in the sense that the n-variable Majority function almost (but not quite) satisfies the premise of the theorem -its noise sensitivity is only slightly higher than the bound required by the theorem -but is very far from any junta.It is interesting, though, to consider whether quantitative improvements of the theorem are possible for restricted classes of Boolean functions; this is what we do in this paper, by considering the special case when f is a halfspace.In [DS09] a quantitatively stronger version of an earlier "junta theorem" due to Friedgut [Fri98] was proved for the special case of halfspaces, and it was asked whether a similarly strengthened version of Bourgain's theorem held for halfspaces as well.Intuitively, any halfspace which has noise sensitivity lower than that of Majority should be "quite unlike Majority" and thus could reasonably be expected to depend on few variables; our result makes this intuition precise.
In this note we show that halfspaces do indeed satisfy a junta-type theorem which is similar to Bourgain's but with exponentially better parameters.Our main result shows that if f is a halfspace which (unlike the Majority function) satisfies a noise sensitivity bound similar to the one in Bourgain's original theorem, then f must be close to a junta of exponentially smaller size than is guaranteed by the original theorem.Our proof does not follow either the approach of Bourgain or of [DS09] but instead is a case analysis based on the value of a structural parameter known as the "critical index" [Ser07, DGJ + 10, OS11] of the halfspace.
A function f : {−1, 1} n →{−1, 1} is said to be a "junta on J ⊆ [n]" if f only depends on the coordinates in J .We say that f is a J-junta, 0 ≤ J ≤ n, if it is a junta on some set of cardinality at most J.
Definition 1 (Noise sensitivity).The noise sensitivity of a Boolean function f : {−1, 1} n →{−1, 1} at noise rate is defined as where x is uniformly distributed and y is obtained from x by flipping each bit of x independently with probability .

Theorem 2 ([Bou02], [KO12]
).There exists a universal constant C > 0 such that the following holds.Fix Discussion.Bourgain's paper had a somewhat stronger assumption on the noise sensitivity, in particular NS (f ) ≤ (δ √ ) 1+o(1) for an unspecified function in the o(1).Subsequently Khot  A halfspace, or linear threshold function (henceforth simply referred to as an LTF), over {−1, 1} n is a Boolean function f : {−1, 1} n →{−1, 1} of the form f (x) = sign( n i=1 w i x i − θ), where w 1 , . . ., w n , θ ∈ R. The function sign(z) takes value 1 if z ≥ 0 and takes value −1 if z < 0; the values w 1 , . . ., w n are the weights of f and θ is the threshold.LTFs have been intensively studied for decades in many different fields such as machine learning and computational learning theory, computational complexity, and voting theory and the theory of social choice.
Our main result, given below, is a strengthening of Bourgain's theorem that applies to the special case of halfspaces: Theorem 3 (Main Result).There exists a universal constant C > 0 such that the following holds.Fix f : {−1, 1} n →{−1, 1} to be any LTF and , δ sufficiently small.If

Comparison with Previous Work
In comparing Theorem 3 with Bourgain's junta theorem (Theorem 2), it should of course be emphasized that Theorem 3 applies only to LTFs while Theorem 2 applies to any Boolean function.When Theorem 3 does apply it requires a slightly stronger bound on the noise sensitivity in terms of δ, namely as much as δ (2− )/(1− ) versus essentially δ, but the resulting junta size bound of Theorem 3 is exponentially smaller, both as a function of and of δ, than the bound of Theorem 2.
Our Theorem 3 directly implies a result that is qualitatively similar to, but somewhat quantitatively weaker than, Theorem 4. To see this, given an LTF f set = C 2 δ 4 /Inf(f ) 2 .Then, using the well-known fact that NS (f ) ≤ • Inf(f ), we get that so by Theorem 3 we have that f is δ-close to a junta over many variables.(It should be noted that this bound does not give a meaningful result for LTFs unless Inf(f ) n 1/4 , whereas the original result of [DS09] gives a meaningful bound as soon as Inf(f ) n 1/2 , which is the largest possible value for LTFs.)On the other hand, we observe that Theorem 3 can sometimes give much stronger quantitative bounds for LTFs than Theorem 4. To see this, consider the LTF f : {−1, 1} n+(log n)/10 →{−1, 1}, (The constant "1/10" is chosen solely for concreteness here; any other constant in (0, 1/2) would do as well.)Observing that f (x, y) = 1 if and only if both x 1 = • • • = x (log n)/10 = 1 and Maj(y 1 , . . ., y n ) = 1, it is easy to verify that Inf(f ) = Θ(n 0.4 ).Hence taking δ to be (say) 1/1000, Theorem 4 only implies that f is δ-close to a junta over O(n 0.8 ) many variables, which is quite a poor bound on junta size.In contrast, Theorem 3 gives a much sharper bound; taking = Θ(1) and recalling that NS (f ) ≤ 2 Pr[f = 1] = O(n −1/10 ), we may apply Theorem 3 to obtain that f is δ-close to an O(1)-junta.

Probabilistic Facts
We require some basic probability results including the standard additive Hoeffding bound (see e.g., [DP09]): Theorem 5. Let X 1 , . . ., X n be independent random variables such that for each j ∈ The Berry-Esséen theorem (see e.g., [Fel68]) gives explicit error bounds for the Central Limit Theorem: Theorem 6. (Berry-Esséen) Let X 1 , . . ., X n be independent random variables satisfying E[X i ] = 0 An easy consequence of the Berry-Esséen theorem is the following fact, which says that a τ -regular linear form behaves approximately like a Gaussian up to error O(τ ): We say that two real-valued random variables X, Y are ρ-correlated if E[XY ] = ρ.We will need the following generalization of Fact 8 which is a corollary of the two-dimensional Berry-Esséen theorem (see e.g., Theorem 68 in [MORS10]).
Theorem 9. Let w = (w 1 , . . ., w n ) be a τ -regular vector in R n with w 2 = 1.Let (x, y) be a pair of ρ-correlated n-bit binary strings, i.e., a draw of (x, y) is obtained by drawing x uniformly from {−1, 1} n and independently for each i choosing where (X, Y ) is a pair of ρ-correlated standard Gaussians.

Fourier Basics over {−1, 1} n
We consider functions f : {−1, 1} n →R (though we often focus on Boolean-valued functions which map to {−1, 1}), and we view the inputs x to f as being distributed according to the uniform distribution.The set of such functions forms a 2 n -dimensional inner product space with inner product given by f, g = E[f (x)g(x)].The set of functions (χ S ) S⊆[n] defined by χ S (x) = i∈S x i forms a complete orthonormal basis for this space.We will often simply write x S for i∈S x i .
Given a function f : {−1, 1} n →R we define its Fourier coefficients by f (S) x S ], and we have that f (x) = S f (S)x S .
As an easy consequence of orthonormality we have Plancherel's identity f, g = S f (S) g(S), which has as a special case Parseval's identity, E[f (x) 2 ] = S f (S) 2 .From this it follows that for every f : {−1, 1} n →{−1, 1} we have S f (S) 2 = 1.It is well-known and easy to show that the noise sensitivity of f can be expressed as a function of its Fourier spectrum as follows NS (f ) = 3 Proof of Theorem 3 Fix , δ sufficiently small.Let We start by observing that for δ 1 1− < √ the desired statement follows easily; indeed, under the assumption of the theorem f is δ-close to a constant function.This is formalized in the following simple claim which holds for any Boolean function: Claim 10.Let f : {−1, 1} n →{−1, 1} be any Boolean function and 0 < δ Proof.For any Boolean function we have where the equality follows from Parseval's identity.Therefore, we can write 1− / 1/2 ≤ δ where the first inequality follows from the assumed upper bound on the noise sensitivity and the second uses the assumption that δ It follows that f is δ-close to sign( f (∅)) and this completes the proof.
Using the above lemma, for the rest of the proof we can assume that δ , where we assume, without loss of generality, that i w 2 i = 1 and The proof proceeds by case analysis based on the value of the -critical index of the vector w, which we now define.
Definition 11 (critical index, [Ser07]).We define the τ -critical index (τ ) of a vector w ∈ R n as the smallest index i ∈ [n] for which |w i | ≤ τ • σ i .If this inequality does not hold for any i ∈ [n], we define (τ ) = ∞.
The case analysis is essentially the same as the one used in [Ser07, DGJ + 10].Let def = ( ) be the -critical index of f .We fix a parameter for an appropriately large value of the constant in the Θ(•).If = 1, then the linear form behaves like a Gaussian and must be either biased or noise sensitive.In Lemma 12, we show that such an f is either δ-close to constant or has noise sensitivity Ω(δ ). (See Case I below.)If > L, then previous results [Ser07] establish that f is δ-close to a junta.(See Case III.) Finally, for 1 < < L, we consider taking random restrictions to the variables before the critical index.If a (1 − δ)-fraction of these restrictions result in subfunctions which are very biased, then f must be 3δ-close to a junta over the first L variables.Otherwise, a δ-fraction of the restrictions result in regular LTFs which are not very biased, and we can apply the results from Case I to show that the noise sensitivity of f must be too large to satisfy the conditions of Theorem 3. We show this in Lemma 16, Case II.Our requirement on the noise sensitivity in Theorem 3, which is probably stronger than optimal, comes from the analysis of this case.
We now proceed to consider each of these three cases formally.Case I: [ = 1, i.e., the vector w is -regular.]In this case we show that f is δ-close to a constant function.The argument proceeds as follows: ) contradicting the assumption of the theorem.Hence, |E[f ]| ≥ 1−δ, i.e., f is δ-close to a constant.Our main lemma in this section establishes the intuitive fact that a regular LTF that is not-too-biased towards a constant function has high noise sensitivity.
Case I follows easily from the above lemma.Suppose that p ≤ δ.Then the function f is δ-close to a constant.Otherwise, the lemma implies that NS (f ) = Ω(δ ).This contradicts our assumed upper bound on NS (f ) from the statement of the main theorem.
The proof of Lemma 12 proceeds by first establishing the analogous statement in Gaussian space (Lemma 13 below) and then using invariance to transfer the statement to the Boolean setting.
We start by giving a lower bound on the Gaussian noise sensitivity of any LTF as a function of the noise rate and the threshold of the LTF.The following lemma is classical for θ = 0. We were not able to find an explicit reference for arbitrary θ, so we give a proof for the sake of completeness.
Proof.Let X and Y be ρ-correlated standard Gaussians.As is well known, (X, Y ) can be generated as follows where Z 1 and Z 2 are independent standard Gaussians.For the random variables X − θ and Y − θ we can write 1+ρ and consider the 2-dimensional random vector T = (−Z 2 + αθ, Z 1 − θ).Note that T is orthogonal to the vector (Z 1 − θ, Z 2 − αθ).
We now observe that Pr[sign(X − θ) = sign(Y − θ)] = Pr[T "splits" vectors (1, 0) and (ρ, 1 − ρ 2 )] We refer to Figure 1 for the rest of the proof.Let R be the region between the horizontal axis (the line spanned by (1, 0)) and the line spanned by the vector (ρ, 1 − ρ 2 ).The RHS of the above equation is equal to the probability mass of R under a 2-dimensional unit variance Gaussian centered at (αθ, −θ).We estimate the Gaussian integral restricted to the region by considering points at distance r ≥ r 0 from (αθ, −θ).Using polar coordinates to compute the integral, we obtain: Pr[T "splits" vectors (1, 0) and (ρ, 1 The angles β(r), γ(r) are illustrated in Figure 1, and r 0 is the distance of the point (αθ, −θ) from the origin, i.e., where the second equality follows from the definition of α.To compute (1), we need the following claim: Claim 14.For all r > r 0 , it holds that (β − γ)(r) = arccos(ρ).
Therefore, the RHS of (1) can be written as follows: where the last equality follows from (2).This concludes the proof of Lemma 13.
We are now ready to give the proof of Lemma 12.
From this it follows that e −θ 2 /2 = Θ p log(1/ p) and (4) yields (3).It remains to get the desired bound on p. Assume that θ ≥ 0; for θ < 0 the argument is symmetric.First, it is easy to see that where Φ(θ) Since θ ≥ 0, we have Φ(θ) ≤ 1/2, hence p = 2 Φ(θ).The desired bound on p now follows from the following elementary fact: Fact 15.For all θ ≥ 0, it holds Φ(θ) = Θ( e −θ 2 /2 |θ|+1 ).We now turn to the Boolean setting to finish the proof of Lemma 12. Let f = sign(w • x − θ) be a Boolean -regular LTF (where without loss of generality w 2 = 1) that has |E[f ]| = 1 − p.We use (3) and invariance to prove the lemma.In particular, we have the following sequence of inequalities: = Ω( p = Ω(p where (5) follows from Theorem 9 and ( 6) is an application of (3).To see (7), note that, by Fact 8 (a corollary of the Berry-Esséen theorem) we get that p ≈ p, and hence This completes the proof of the lemma.
Case II: [1 < ≤ L.] In this case, we show that f is δ-close to an -junta.Consider the partition of the set [n] into a set of head variables H = [ ] and a set of tail variables T = [n] \ H. Let us write H(x H ) to denote w H • x H and T (x T ) to denote w T • x T , the linear forms corresponding to the head and the tail.
The argument proceeds as follows: If a non-trivial fraction of restrictions to the head variables lead to a not-too-biased LTF, we show that the original LTF has high noise sensitivity contradicting the assumption of the theorem.On the other hand, if most restrictions to the head lead to a substantially biased LTF, we argue that the original LTF is close to a junta over the head coordinates.
Let ρ ∈ {−1, 1} |H| denote an assignment to the head coordinates and f ρ be the corresponding restriction of f .Note that for any restriction ρ of the head variables the resulting f ρ is anregular LTF (with a threshold of H(ρ) − θ).Formally, we consider two sub-cases depending on the distribution of |E[f ρ ]| for a random choice of ρ.

Case IIa: [This case corresponds to Pr
That is, at least a δ fraction of restrictions to the head variables result in a "not-too-biased" LTF.Since each of these restricted subfunctions has high noise-sensitivity, we can show that the overall noise-sensitivity is also somewhat high.This intuitive claim is quantified in the following lemma.
Lemma 16.Let , δ be sufficiently small values that satisfy δ 2 ≥ √ .Let the -critical index of Therefore, in Case IIa we reach a contradiction.To prove the above lemma, we need the following claim, whch implies that if a noticeable fraction of restrictions to a Boolean function have high noise sensitivity, then so does the original function.
Proof.The following elementary fact will be useful for the proof: By linearity of expectation and Fact 18 we get that On the other hand, we have: Combining equations 8 and 9, we obtain Using the above claim we can prove Lemma 16.Since f is δ-close to g and g is 2δ-close to sign(h) (a junta over H), this completes the proof.
This completes Case II.
Case III: [ > L].In this case, we merely observe that f is δ-close to an L-junta.This follows immediately from the arguments in [Ser07, DGJ + 10].In particular, and Naor (see Theorem 4.3 of [KN06]) optimized the parameters of Bourgain's proof providing an explicit dependence.The aforementioned tight quantitative statement follows from the recent work of Kindler and O'Donnell [KO12].It is a slight strengthening of Corollary 3.21 in their paper, whose proof is very similar to the proof of the latter.The essential difference is that one needs to use Theorem 3.2 of [KO12] instead of Theorem 3.19 in the proof [O'D13].

Figure 1 :
Figure 1: Illustration of the integration region for Lemma 13.