CVX featured in new optimization textbook

textbookIt is always uplifting to see one’s work cited in a scientific publication. How much more of a thrill it is, then, to see it incorporated into a textbook, and presumably into courses that rely upon the book!

For years, the textbook Convex Optimization by Stephen Boyd and Lieven Vandenberghe has been used in courses around the world, and many have incorporated CVX into their assignments and exams. On October 27, a new book was released that should do the same: Introduction to Nonlinear Optimization: Theory, Algorithms, and Applications with MATLAB by Amir Beck, Associate Professor of Industrial Engineering and Management at the Technion.

From the book’s web page:

This book provides the foundations of the theory of nonlinear optimization as well as some related algorithms and presents a variety of applications from diverse areas of applied sciences. The author combines three pillars of optimization—theoretical and algorithmic foundation, familiarity with various applications, and the ability to apply the theory and algorithms on actual problems—and rigorously and gradually builds the connection between theory, algorithms, applications, and implementation.

Chapter 8, a sample of which is provided on the site, Prof. Beck provides a surprisingly comprehensive introduction to CVX, offers an number of examples of complete CVX models, and includes several exercises calling upon readers to build their own CVX models.

It really is a wonderful source of encouragement that CVX is trusted enough to be incorporated into teaching. You’ve made our day, Prof. Beck, thank you!

CVX on Octave: coming soon—see a live demo!

Important note: CVX is not compatible with the Octave 3.8.1 or earlier. Please do not try to install it—you will waste your time! But read on for some good news.

One of the more common questions we have been asked over the years is why CVX cannot run on Octave, a free alternative to Matlab. Since Octave aims to allow Matlab packages to run with little or no modification, it is certainly a good question! Unfortunately, it simply is not possible—yet.

A little history

We have been talking about Octave support since the “embedded modeling language” design of CVX was first conceived in early 2005. Implementing this design relies on quite a few of Matlab’s more esoteric language features. At the time, those features were not present in Octave; and some of them are still missing today.

When version 3.8.0 of Octave was released this past December, it was very clear that the “CVX feature gap” was significantly reduced, so I began working with the Octave source code myself. I submitted several patches that would close the gap completely, allowing CVX to run on Octave. Many of these patches have already been accepted into Octave 3.8.1, while others have been accepted into the official development tree for a future release. A few others are pending, but the leader of the Octave project has been very helpful, and shares my confidence that the official Octave releases will soon support CVX.

A live demo!

UPDATE: I’m afraid this live demo is no longer active. We’ll look into ways to resurrect a live demo soon.

Would you like some proof? How about this: the folks at Terminal.com have put together an on-line, web-based demo of Octave running CVX. Click here to go right to the CVX demo page; click the “Run” button, and it will launch a web-based interface to a cloned virtual machine running the software. You can try the example library, enter your own models, even plot results.

Please keep in mind that this is a patched version of Octave that is not available to the general public. (Please do not ask for it.) But it is running the very same version of CVX that you can download today. And of course, if the Terminal.com virtual machine is sufficient for your purposes, then you can indeed work with CVX on Octave now!

Porting the solvers

Of course, CVX is useless without solvers! Fortunately, work had already been done to make SeDuMi compatible with Octave, and I have worked to improve upon that. I’ve also ported SDPT3 to Octave as well. The updated versions of these solvers are now available on GitHub, so you do not have to wait for CVX to use them. In addition, I have added support for GLPK, the GNU Linear Programming Kit, in CVX. We will not be bundling GLPK with CVX; but since it is included with most Octave installations, we do not need to! (Note: CVX’s connections to commercial solvers, including Gurobi and MOSEK, will remain limited to Matlab, for both technical and contractual reasons.)

The news is good.

CVX is ready! And Octave will be ready soon, too. Please watch our web site, our Google+ page, or our Twitter feed for the announcement. We’re very pleased to be able to offer CVX on a free computation platform for the first time.

What’s next? Well, we hope to bring CVX to other computational platforms as well. Clearly that takes a lot more work that porting it to another MATLAB-compatible platform. But it is work we have already started…

New demo video for CVX

As of February 20, over 10000 students are enrolled in CVX101, Stanford University’s online course in convex optimization offered by Professor Stephen Boyd and colleagues. This course has been taught using Stanford’s internal video distance learning platform for years, so it is well suited for the OpenEdX MOOC platform.

Our Matlab package CVX is being used throughout the course. The Mathworks is providing time-limited Matlab licenses to all enrollees for the duration of the course.

To provide enrollees with a gentle introduction to CVX, Prof. Boyd recorded the following video. If you’re not yet a CVX user yourself, you might find it useful as well!

TFOCS is now fully open source

We are pleased to announce that the authors of TFOCS, in conjunction with copyright holders CVX Research and the California Institute of Technology, have agreed to release TFOCS under a permissive 3-Clause BSD license. It is our sincere hope that moving to a permissive open source license will encourage wider use and community-driven improvements.

As part of our shift to this new license, we have moved the source repository for TFOCS to GitHub. From this site, users can download a versioned release, clone the repository for more frequent updates, or create their own fork for customization.

Community support for TFOCS is available on CVX Forum, a question and answer site modeled after the popular StackExchange family of sites. The forum serves users of both TFOCS and the namesake package CVX. For more dedicated assistance, paid support contracts are available from the packages’ authors.

For more details, please visit the TFOCS home page. The full text of the license is available on the download page as well as in the file LICENSE in the package itself.

Bundled versions of Gurobi and MOSEK now available with CVX

One of the key features added to CVX 2.0 is the ability to connect to the commercial solvers Gurobi and MOSEK. Both solvers are highly respected in the optimization community, and we have found that they work very well with CVX. These solvers have also enabled us to support mixed-integer modeling.

This week we have taken a step to further simplify the process of using CVX with these solvers by bundling them with CVX itself. Of course, if you already use CVX with these solvers, nothing has changed—you can continue to connect CVX to your existing Gurobi and MOSEK installations. But now it is even easier for new users of CVX to use our modeling framework with these excellent solvers.

Some technical details:

We have improved our installation instructions to cover the details of installing a CVX Professional License, and have created separate sections describing how to install and use Gurobi and/or MOSEK with CVX.

Because these bundles are a new offering for us, we have decided to keep the “beta” version designation for just a bit longer, in case this wider release reveals some installation issues. But we have many users happily using Gurobi and MOSEK with CVX already, so we encourage you download CVX 2.0 now.

We are grateful to both Gurobi Optimizaton and MOSEK ApS for working closely with us to ensure that the use of their solvers with CVX is as seamless and straightforward as possible.

CVX 2.0 updated: Gurobi 5.1, MOSEK 7 with SDP support

We have been hard at work to exit the beta period for CVX 2.0. We’re almost there! However, due to some recent developments in the solver arena, we have decided to release a new build of CVX 2.0 beta today. We invite you to download a copy. Feel free to install this beta on top of your current version of CVX 2.0, but do re-run cvx_setup once you have done so.

Ordinarily, an update to a solver Gurobi or MOSEK would not require a new release of CVX. Gurobi 5.1 was released on January 8th, and the previous version of CVX 2.0 beta worked perfectly with that new release. If you’re content with your current version of CVX, there is no need to download the new build.

Mosek 7 adds support for semidefinite programs (SDPs), however, and this required some code changes to the Mosek solver shim. Previously, users who required an SDP-capable solver had to rely on the SDPT3 and SeDuMi solvers bundled with CVX. We are very pleased that now our users will have another new, commercially supported option for such problems now.

If you want to try the beta version of Mosek 7 now, head over to the beta section of their web site. There you can download a copy of MOSEK for your preferred platform, as well as a trial license that expires on March 1, 2013. Once you’ve installed MOSEK 7.0, add its Matlab MEX directory to your MATLAB path, and re-run cvx_setup.

Both Gurobi and MOSEK are excellent solvers, and the companies behind them have been wonderful to work with. It is a privilege for us to be able to support their use with CVX. In fact, we have some exciting new developments brewing on that front that we hope to share very soon. Stay tuned…

CVX 2.0 beta updated; academic licenses available

A new version of CVX 2.0 beta has been posted to the web site. Please go get it. We’ve fixed a handful of bugs, but most importantly, we’ve completed the licensing engine. As with the previous beta, this version comes with a license for CVX Professional that expires on November 30, 2012—a little over three weeks away. After that date, you’ll need a license to use the CVX Professional features.

If you are an academic user, you can obtain a license key at no charge. Just visit our Academic license request page and fill out the online form. An active university email address is required, as well as the username and host ID of the computer you are using. If everything checks out, you should receive your license key within minutes. Please note that these license keys require this latest beta release to operate (build 883 or later).

Commercial users will need to purchase a CVX Professional license from us to continue using Gurobi and MOSEK with CVX after the conclusion of the beta period. Feel free to send us an email at sales@cvxr.com for a quote.

Lastly, let us take this opportunity to promote the CVX Forum! This is a community-driven Q&A site where you can find answers to a variety of common usage questions posed by users of CVX. If you have a question that hasn’t yet been asked, of course, you can ask it. And if you have an answer that others may find useful, you can provide it, too. We truly hope that over time the CVX Forum will become a rich database of expertise in disciplined convex programming, CVX, and TFOCS.

CVX 2.0 beta period extended; new license key posted

We’re hard at work getting ready for the full release of CVX 2.0. We have incorporated some great feedback from our users already, including bug fixes submitted to our help desk and issues raised on our Q&A site.

We are committed to offering academic licenses at no charge—and with as little inconvenience as possible. I have been hard at work building the infrastructure to support that. But it looks like that work will not be finished by the time the beta period ends in two weeks—on October 31, 2012. For that reason, I have created a new beta license key that extends the beta period another month—to November 30, 2012.

To grab the new license key, click on this link to be taken to our help desk site. There you will find a link to download the new license key, as well as instructions for installing it.

Remember, a license key is required only if you need to use CVX with commercial solvers. If you are content with SDPT3 or SeDuMi, a license is unnecessary.

EDIT: We’ve posted a new version of CVX 2.0 beta on the download page that includes the license key, as well as a couple of minor fixes.

A new support option: the CVX Forum

CVX has reached a download pace of over 10,000 downloads per year! The acceptance of CVX in research settings has been amazing—and challenging. Supporting all of our users via email, as we have done for the last 6 years, is simply no longer feasible.

Of course, if you have found a bug in CVX, we want to hear about it; see our support page for information on submitting bug reports. CVX is on a rapid-release cycle, so a reported bug can be often fixed and made available to users in just a few short days.

But in fact, bug reports represent only a fraction of our total support effort. Most of our time is spent helping people to become effective users of CVX. Sometimes, this involves answering a usage question that is frequently asked, but not readily included in standard documentation. In other cases, it involves offering advice on how to implement a particular model found in a paper or application.

If only we were experts in the myriad applications in which that CVX has been applied. Alas, we are not; but our users are! It is clear to us that what is needed is an online knowledgebase and a community-driven Q&A forum—a place where the valuable experience of our large user community can be collected, organized, and shared.

That is the purpose of our new creation, the CVX Forum.

If you have visited sites such as Stack Overflow or OR Exchange, then the CVX Forum should look very familiar. Sites such as these have proven to be extremely successful at providing community-based support on a variety of topics. Through a novel voting system, the site self-organizes so that common questions and their most helpful answers are easy to find, creating a rich knowledgebase that no user manual can match. The CVX Forum will be a great place to discuss (and vote upon!) feature requests as well.

We encourage all CVX users to join us at the CVX Forum and join in a communal effort to improve the state of disciplined convex programming. Frankly, some of our users are better qualified to answer some of the questions we receive than we are! We hope you will prove us right.

Mixed-integer support in CVX 2.0

Gurobi and MOSEK, the first commercial solvers we have connected to CVX, support mixed-integer models: models with one or more variables constrained to assume integer or even binary (0/1) values. Such variables can be used not just in linear programs, but nonlinear convex programs as well.

We have decided to introduce the ability to specify integer and binary variables into CVX models as well. To use this feature, simply add the integer or binary keyword to the relevant variable declaration; for example:

variable x(10) integer;
variable y(3,3) binary;

Models with integer and binary variables must still obey all of the same disciplined convex programming rules that CVX enforces for continuous models. For this reason, we are calling these models mixed-integer disciplined convex programs, or MIDCPs.

Danger! Danger!

Mixed-integer DCPs are not convex.

As with mixed-integer linear and quadratic programs (MILPs/MIQPs), MIDCPs are theoretically intractable. In practice, your results will vary. Solving them requires the combination of a continuous optimization algorithm, such as an interior-point method, and an exhaustive search, such as a branch-and-bound. In some cases, your model will be solved globally after only a few branches. In other cases, the solver will fail to find a global solution in any reasonable amount of time.

Furthermore, solvers for MIDCPs are less mature than those devoted specifically to MILPs/MIQPs. Most of the developments that have improved the performance of MILP solvers map logically to MIDCPs, but MIDCPs present new challenges. We hypothesize that solver performance will be more sensitive to problem size with MIDCPs than with MILPs, and that the best results will come with models that make only sparing use of integer variables.

Non-convex constraints as MIDCP constraints

One of the more interesting consequences of MIDCP support is that, in limited cases, a nonconvex model can be transformed in to an MIDCP. For instance, suppose x is a vector variable, and l and u are constant vectors. If you were to specify the constraint

l <= abs(x) <= u;

in CVX, you would get the dreaded Disciplined convex programming error: message. But in fact, this constraint can be represented as an MIDCP by introducing a vector of binary variables:

varaible xb(size(x)) binary;
l - xb .* ( l + u ) <= x <= u - xb .* ( l + u );

For a nonlinear example, consider these constraints:

max(norm(x),norm(y)) <= 2;
min(norm(x),norm(y)) <= 1;

The first constraint is convex, the second is not. But together they can be specified in an MIDCP:

binary variable nb;
norm(x) <= 1 + nb;
norm(y) <= 2 - nb;

We emphasize that the ability to convert nonconvex constraints to MIDCP form is the exception rather than the rule. For instance, the constraint

1 <= norm(x) <= 2;

looks very similar to our first example, but it cannot be converted to an MIDCP. Furthermore, CVX will not perform these conversions for you; you will have to identify and perform them yourself. Do not ignore convexity during the modeling process, assuming that you will be able to convert your model to an MIDCP after the fact!

Solver support

Neither of CVX’s free solvers supports MIDCPs. Gurobi and MOSEK are currently the only choices for these problem types. Academic users will be able to use either solver (or both!) at no charge. Commercial users can use them by purchasing a CVX Professional License.

We hope to add support for a free mixed-integer solver in a future version of CVX. We are currently studying the use of GLPK, the GNU Linear Programming Kit. Of course, as the name implies, GLPK will only support models that can be converted to LPs or MILPs. If a reliable, free solver that supports both integer variables and conic constraints becomes available, we will certainly consider it; please contact us!

Can a mixed-integer problem truly be disciplined?

Among those of you who know us well, there might be some surprise at our decision to include mixed-integer support. After all, we have been, and remain, very strong advocates for a deliberate approach to convex programming. Indeed, we coined the term disciplined convex programming to describe our philosophy: that the best way to exploit convexity is not to search for it after the model is constructed, but rather to make convexity an explicit goal of the modeling process itself.

We hold that a proper use of MIDCPs still requires a disciplined approach to model building. As we said above, non-convex models representable as MIDCPs will be the exception, not the rule; it is still necessary to build models with convexity as a goal, not an artifact. The performance limits of MIDCP solvers will tend to limit the use of integer and binary variables in CVX models.

There is no doubt that the addition of MIDCPs to CVX is an experiment for us. It will also be an experiment for our users and our solver partners as well. We look forward to seeing the new applications that arise when modelers can add modest numbers of integer constraints to their models. And we look forward to seeing the algorithms for mixed-integer convex programming improve as a suite of “difficult” test problems accumulates.