**1. Examples **

The example showing that the optimum value of an SDP might not be achieved is this: subject to

For this to be positive-semidefinite, we need . But then the optimum value of the SDP is (it is the infimum of the values can take), but it not achieved by any feasible solution.

Here is an example where the primal is feasible and finite, but the dual is infeasible.

The optimum is , achieved by for . But the dual seeks to maximize subject to

But this is infeasible, since psd matrices cannot have non-zero entries in the same row/column as a zero diagonal entry.

**2. Proofs of Some Useful Facts **

We used several facts about matrices in lecture today, without proof. Here are some of the proofs. Recall that . For symmetric (and hence for psd) matrices, .

*Proof:* .

Lemma 1For a symmetric matrix , is psd if and only if for all psd .

*Proof:*

One direction is easy: if is not psd, then there exists for which . But is psd, which shows that for some psd .

In the other direction, let be psd. We claim that , defined by , is also psd. (This matrix is called the *Schur-Hadamard product* of and .) Then

by the definition of psd-ness of . To see the claim: since , there exist random variables such that . Similarly, let for r.v.s . Moreover, we can take the ‘s to be independent of the ‘s. So if we define the random variables , then

and we are done. (Note we used independence of the ‘s and ‘s to make the product of expectations the expectation of the product.)

This clean random variable-based proof is from this blog post. One can also show the following claim. (I don’t know of a slick r.v. based proof—anyone see one?) BTW, the linear-algebraic proof also gives an alternate proof of the above Lemma 1.

Lemma 2For (i.e., it is positive definite), for all psd , .

*Proof:* Let’s write as where is orthonormal, and is the diagonal matrix containing ‘s eigenvalues (which are all positive, because .

Let , and hence . Note that is psd: indeed, . So all of ‘s diagonal entries are non-negative. Moreover, since , not all of ‘s diagonal entries can be zero (else, by ‘s psd-ness, it would be zero). Finally,

Since and for all , and for some , this sum is strictly positive.

*Proof:* Clearly if then .

For the other direction, we use the ideas (and notation) from Lemma 1. Again take the Schur-Hadamard product defined by . Then is also psd, and hence for random variables . Then

If this quantity is zero, then the random variable must be zero with probability . Now

so is identically zero.

**3. Eigenvalues Seen Differently, or Old Friends Again **

We saw that finding the maximum eigenvalue of a symmetric matrix can be formulated as:

and its dual

The dual can be reinterpreted as follows: recall that means we can find reals and unit vectors such that . By the fact that ‘s are unit vectors, , and the trace of this matrix is then . But by our constraints, , so .

Rewriting in this language, is the maximum of

such that the ‘s are unit vectors, and . But for any such solution, just choose the vector among these that maximizes ; that is at least as good as the average, right? Hence,

which is the standard variational definition of the maximum eigenvalue of .

## Leave a Reply