**Semidefinite
and Second Order Cone Programming**

16:711.530

Fall 2012

**Instructor:**Farid Alizadeh**Office:**RUTCOR room 105**Phone:**732-445-4857**e-mail:**alizadeh@rutcor.rutgers.edu**Office hours:**Mondays any time

This course is a seminar covering semidefinite, Second order and related optimization problems. For more detailed description read the course announcement.

Each student is required to be responsible for a lecture, take
careful notes and transcribe them by into LaTeX. Then the Latex
document should be e-mailed to me **before the Sunday following the
lecture.** I will edit the scribe notes and post them in in this
page. For preparing your lecture notes use this
template. Also make sure that all vectors are represented by
boldface letters e.g **b **for vector b. Use the file myDefs.sty
for useful predefined commands.

Also each student is required to either read a current research paper, or finish a (possibly computational project) and present it in class in a 30-40 minute talk.

Lecture
Notes: last updated on

Here are lecture notes as scribed by you. I have done minimal editing. Be careful, there may be errors in these notes.

Lecture01.tex Lecture01.pdf
Scribe **Gyorgy Matyasfalv**i

Lecture02.tex
Lecture02.pdf
Scribe **Wang Yao**

Lecture03.tex
Lecture03.pdf Scribe **Marta
Cavaleiro**

Lecture04.tex
Lecture04.pdf Scribe: **Emre
Yamangil **

Lecture05.tex
Lecture05.pdf Scribe: **Ritwik
Mitra**

Lecture06.tex
Lecture06.pdf Scribe:
**Svetlana Soloveva**

Lecture07.tex
Lecture07.pdf Scribe:
**Deniz Seyed Eskandani**

Lecture08.tex
Lecture08.pdf Scribe: **Ai
Kagawa**

Lecture09.tex
Lecture09.pdf Scribe: **Shunqiao
Sun**

Lecture10.tex
Lecture10.pdf Scribe: **Joonhee
Lee**

Lecture11.tex
Lecture11.pdf Scribe: **Jingnan
Fan**

Presentations

First
order and augmented lagrangian methods, by **Wang
Yao**

Eigenvalue
decomposition: by **Gyorgy
Matyasfalv**i

Matrix
Completion: by **Marta
Cavaleiro**

Rank
Minimization:
by
**Rirwik
Mitra**

Computer
Vision: by **Svetlana
Soloveva**

Robust
Optimization: by **Ai
Kagawa**

Applications
of Roust Optimization in Signal Processing and Beam Forming: by
**Shunqiao
Sun**

The
S-Procedure: by **Jingnan
Fan**

Shape
Constrained Estimation: by **Joonhee
Lee**

The
Theta Function: by **Emre
Yamangil**

Comparison
of Lovasz-Schrijver, Sherali-Adams and Lassere Hierarchies: by
**Deniz
Seyed Eskandani**

Possible topics for in-class presentation:

Each student should choose a topic, find one or more papers on that topic and present a survey in class.

Also each student is responsible for scribing his/her own
presentation. The scribe notes for presentation should be handed in
to me **before** the day of presentation itself.

Student presentations will start approximately in November.

Students may choose one of the following topics or propose another topic to me.

Suggested topics:

Robust optimization, and the possibility of expressing robust problems as SDP or SOCP

Various extensions of the Goemans-Williamson approximation of the MAX-CUT and MAX-2SAT problems including Nesterov's generalization to non-convex or binary quadratic programming

Survey of positive and sum-of-squares polynomials. This topic can be pursued in different sub-topics, for example: Expression of combinatorial optimization problems as positive polynomial problems and their relaxations to sum-of-square problems (Lasserre's and Parillo's papers), nonnegative polynomials and moment problems over intervals(Karlin-Studden, Nesterov, Parrilo), gap between positive and sum-of-square multivariate polynomial. Another direction is review of real algebraic geometry, Schmudgen's theorem, Putinar's lemma, and applications,

Applications in Engineering and finance: for example signal processing, portfolio optimization, statistical design of experiments, matrix completion problems and applications

Applications in control theory: The S procedure, Lypunov operator, Youla's Theorem and applications, KYP Lemma and connections to SDP.

Students suggestion: If you have ideas about project or presentation, let me know.

Reading list:

Following is a list of useful references to read. More may be added as needed:

The following book contains a collection of papers on all aspects semidefinite programming and is probably the most useful source (See me to borrow a copy if you wish)

@book {MR1778223, TITLE = {Handbook of semidefinite programming}, SERIES = {International Series in Operations Research \& Management Science, 27}, EDITOR = {Wolkowicz, Henry and Saigal, Romesh and Vandenberghe, Lieven}, NOTE = {Theory, algorithms, and applications}, PUBLISHER = {Kluwer Academic Publishers}, ADDRESS = {Boston, MA}, YEAR = {2000}, PAGES = {xxviii+654}, ISBN = {0-7923-7771-0}, MRCLASS = {90-00 (90C22)}, MRNUMBER = {MR1778223 (2001k:90001)}, MRREVIEWER = {K. G. Murty}, }

Alizadeh's paper on semidefinite programming

@article {MR1315703, AUTHOR = {Alizadeh, Farid}, TITLE = {Interior point methods in semidefinite programming with applications to combinatorial optimization}, JOURNAL = {SIAM J. Optim.}, FJOURNAL = {SIAM Journal on Optimization}, VOLUME = {5}, YEAR = {1995}, NUMBER = {1}, PAGES = {13--51}, ISSN = {1052-6234}, CODEN = {SJOPE8}, MRCLASS = {90C27}, MRNUMBER = {MR1315703 (95k:90065)}, MRREVIEWER = {Edward J. Anderson}, }

This survey paper contains various applications of semidefinite programming

@article {MR1379041, AUTHOR = {Vandenberghe, Lieven and Boyd, Stephen}, TITLE = {Semidefinite programming}, JOURNAL = {SIAM Rev.}, FJOURNAL = {SIAM Review. A Publication of the Society for Industrial and Applied Mathematics}, VOLUME = {38}, YEAR = {1996}, NUMBER = {1}, PAGES = {49--95}, ISSN = {0036-1445}, CODEN = {SIREAD}, MRCLASS = {90-02 (90C25)}, MRNUMBER = {MR1379041 (96m:90005)}, MRREVIEWER = {Henry Wolkowicz}, }

This paper is a survey of second order cone programming

@article {MR1971381, AUTHOR = {Alizadeh, F. and Goldfarb, D.}, TITLE = {Second-order cone programming}, NOTE = {ISMP 2000, Part 3 (Atlanta, GA)}, JOURNAL = {Math. Program.}, FJOURNAL = {Mathematical Programming. A Publication of the Mathematical Programming Society}, VOLUME = {95}, YEAR = {2003}, NUMBER = {1, Ser. B}, PAGES = {3--51}, ISSN = {0025-5610}, MRCLASS = {90C08 (90C25)}, MRNUMBER = {MR1971381 (2004j:90060)}, }

This paper surveys applications of SOCP especially in engineering

@article {MR1655138, AUTHOR = {Lobo, Miguel Sousa and Vandenberghe, Lieven and Boyd, Stephen and Lebret, Herv{\'e}}, TITLE = {Applications of second-order cone programming}, NOTE = {ILAS Symposium on Fast Algorithms for Control, Signals and Image Processing (Winnipeg, MB, 1997)}, JOURNAL = {Linear Algebra Appl.}, FJOURNAL = {Linear Algebra and its Applications}, VOLUME = {284}, YEAR = {1998}, NUMBER = {1-3}, PAGES = {193--228}, ISSN = {0024-3795}, CODEN = {LAAPAW}, MRCLASS = {90C08 (90C20 90C90 93B40)}, MRNUMBER = {MR1655138 (99k:90103)}, MRREVIEWER = {Jos F. Sturm}, }

The original paper of Michel Goemans and David Williamson on MAX-CUT and MAX-2SAT problems (available from MathSciNet from Rutgers library):

@article {MR1412228, AUTHOR = {Goemans, Michel X. and Williamson, David P.}, TITLE = {Improved approximation algorithms for maximum cut and satisfiability problems using semidefinite programming}, JOURNAL = {J. Assoc. Comput. Mach.}, FJOURNAL = {Journal of the Association for Computing Machinery}, VOLUME = {42}, YEAR = {1995}, NUMBER = {6}, PAGES = {1115--1145}, ISSN = {0004-5411}, CODEN = {JACOAH}, MRCLASS = {90C27 (68Q25 90C35)}, MRNUMBER = {MR1412228 (97g:90108)}, MRREVIEWER = {Nikolay N. Ivanov}, }

A survey paper by Goemans on apoplications of SDP in combinatorial optimization

@article {MR1464765, AUTHOR = {Goemans, Michel X.}, TITLE = {Semidefinite programming in combinatorial optimization}, NOTE = {Lectures on mathematical programming (ismp97) (Lausanne, 1997)}, JOURNAL = {Math. Programming}, FJOURNAL = {Mathematical Programming}, VOLUME = {79}, YEAR = {1997}, NUMBER = {1-3, Ser. B}, PAGES = {143--161}, ISSN = {0025-5610}, CODEN = {MHPGA4}, MRCLASS = {90C27 (90-02)}, MRNUMBER = {MR1464765 (98g:90028)}, MRREVIEWER = {D. de Werra}, }

Another survey of applications of SDP in combinatorial optimization by Lova'sz

Nesterov's paper on SDP relaxations of boolean quadratic problems and other nonconvex quadratic problems (available from MathSciNet from Rutgers library)

@article {MR1618100, AUTHOR = {Nesterov, Yu.}, TITLE = {Semidefinite relaxation and nonconvex quadratic optimization}, JOURNAL = {Optim. Methods Softw.}, FJOURNAL = {Optimization Methods and Software}, VOLUME = {9}, YEAR = {1998}, NUMBER = {1-3}, PAGES = {141--160}, ISSN = {1055-6788}, CODEN = {OMSOE2}, MRCLASS = {90C20 (90C26)}, MRNUMBER = {MR1618100 (99a:90144)}, }

Nesterov's survey on alternative to Goemans-Williamson SDP relaxations

@incollection {MR1778235, AUTHOR = {Nesterov, Yuri and Wolkowicz, Henry and Ye, Yinyu}, TITLE = {Semidefinite programming relaxations of nonconvex quadratic optimization}, BOOKTITLE = {Handbook of semidefinite programming}, SERIES = {Internat. Ser. Oper. Res. Management Sci.}, VOLUME = {27}, PAGES = {361--419}, PUBLISHER = {Kluwer Acad. Publ.}, ADDRESS = {Boston, MA}, YEAR = {2000}, MRCLASS = {90C22 (90C26)}, MRNUMBER = {MR1778235}, }

This book by Karlin and Studden is the main reference on one-variable nonnegative functions and corresponding moment problems (available at Rutgers Math library, or see me to borrow my copy)

@book {MR0204922, AUTHOR = {Karlin, Samuel and Studden, William J.}, TITLE = {Tchebycheff systems: {W}ith applications in analysis and statistics}, SERIES = {Pure and Applied Mathematics, Vol. XV}, PUBLISHER = {Interscience Publishers John Wiley \& Sons, New York-London-Sydney}, YEAR = {1966}, PAGES = {xviii+586}, MRCLASS = {41.00}, MRNUMBER = {MR0204922 (34 \#4757)}, MRREVIEWER = {G. G. Lorentz}, }

Nesterov's paper on SDP representation of sum-of-squares functions

@incollection {MR1748764, AUTHOR = {Nesterov, Yurii}, TITLE = {Squared functional systems and optimization problems}, BOOKTITLE = {High performance optimization}, SERIES = {Appl. Optim.}, VOLUME = {33}, PAGES = {405--440}, PUBLISHER = {Kluwer Acad. Publ.}, ADDRESS = {Dordrecht}, YEAR = {2000}, MRCLASS = {90C22 (12D15 90C34 90C51)}, MRNUMBER = {MR1748764 (2001b:90063)}, MRREVIEWER = {Jos F. Sturm}, }

Paper of Lasserre on global optimization via sum-of-square polynomials (available form MathSciNet form Rutgers library servers)

@article {MR1814045, AUTHOR = {Lasserre, Jean B.}, TITLE = {Global optimization with polynomials and the problem of moments}, JOURNAL = {SIAM J. Optim.}, FJOURNAL = {SIAM Journal on Optimization}, VOLUME = {11}, YEAR = {2000/01}, NUMBER = {3}, PAGES = {796--817 (electronic)}, ISSN = {1052-6234}, MRCLASS = {90C26 (90C22)}, MRNUMBER = {MR1814045 (2002b:90054)}, MRREVIEWER = {Jos F. Sturm}, }

Paper of Parrilo on SDP and sum-of-square polynomials (available form MathSciNet form Rutgers library servers)

@article {MR1993050, AUTHOR = {Parrilo, Pablo A.}, TITLE = {Semidefinite programming relaxations for semialgebraic problems}, NOTE = {Algebraic and geometric methods in discrete optimization}, JOURNAL = {Math. Program.}, FJOURNAL = {Mathematical Programming. A Publication of the Mathematical Programming Society}, VOLUME = {96}, YEAR = {2003}, NUMBER = {2, Ser. B}, PAGES = {293--320}, ISSN = {0025-5610}, MRCLASS = {90C22 (13J30 14P10 90C10)}, MRNUMBER = {MR1993050 (2004g:90075)}, MRREVIEWER = {Franz Rendl}, }

A survey on Robust optimization (available form MathSciNet form Rutgers library servers)

@article {MR2218121, AUTHOR = {Ben-Tal, Aharon and Boyd, Stephen and Nemirovski, Arkadi}, TITLE = {Extending scope of robust optimization: comprehensive robust counterparts of uncertain problems}, JOURNAL = {Math. Program.}, FJOURNAL = {Mathematical Programming. A Publication of the Mathematical Programming Society}, VOLUME = {107}, YEAR = {2006}, NUMBER = {1-2, Ser. B}, PAGES = {63--89}, ISSN = {0025-5610}, MRCLASS = {90C31 (90C05 90C25 90C34 93C55)}, MRNUMBER = {MR2218121 (2006m:90169)}, MRREVIEWER = {Oliver Stein}, }

El-Ghaoui's power-point presentation on robust optimization

Nemirovski's slides of talk at RUTCOR on SDP-representation