Lecture #5 clarification

Kevin asked in class: in the “geometric” proof of duality we talked about, did we need the fact that existed a BFS, i.e., that there were ${n}$ tight constraints? (Recall we were considering the LP ${\max\{ c^\top x \mid Ax \leq b\}}$ ; since we were not talking about equational form LPs, the optimum could be achieved at a non-BFS.) He was right: we don’t. Here’s a proof.

Pick an optimal feasible point for the LP, call this point ${x^*}$ . Let ${a_i^\top x \leq b_i}$ for ${i \in I}$ be all the constraints tight at ${x^*}$ —there may be ${n}$ linearly independent constraints in there, or there may be fewer, we don’t care. Now the claim is that the objective function vector ${c}$ is contained in the cone ${K = \{ x \mid x = \sum_{i \in I} \lambda_i a_i, \lambda_i \geq 0 \}}$ generated by these vectors ${\{a_i\}_{i \in I}}$ . If we prove this claim, we’re done–the rest of the argument follows that in lecture.

(Indeed, this was precisely where we were being hand-wavy earlier, even when we assumed ${x^*}$ was a BFS. So this also answers Jamie’s request.)

Ok, back to business at hand. Suppose ${c}$ does not lie in the cone ${K}$ generated by the vectors ${\{a_i\}_{i \in I}}$ . Then there must be a separating hyperplane between ${c}$ and ${K}$ : i.e., there exists a vector ${d \in {\mathbb R}^n}$ such that ${a_i^\top d \leq 0}$ for all ${i \in I}$ , but ${c^\top d > 0}$ . So now consider the point ${z = x^* + \epsilon d}$ for some tiny ${\epsilon > 0}$ . Note the following:

We claim that for small enough ${\epsilon > 0}$ , the point ${z}$ satisfies the constraints ${Ax \leq b}$ . Consider ${a_j^\top x \leq b}$ for ${j \not \in I}$ : since this constraint was not tight, we won’t violate it if we choose ${\epsilon}$ small enough. And for ${a_j^\top x \leq b}$ with ${j \in I}$ ? See, ${a_j^\top z = a_j^\top x + \epsilon\, a_j^\top d = b + \epsilon\, a_j^\top d \leq b}$ , since ${\epsilon > 0}$ and ${a_j^\top d \leq 0}$ .
What about the objective function value? ${c^\top z = c^\top x^* + \epsilon\, c^\top d > c^\top x^*}$ .