Open Journal of Mathematical Optimization
Combinatorial Robust Optimization with Decision-Dependent Information Discovery and Polyhedral Uncertainty
Given a nominal combinatorial optimization problem, we consider a robust two-stages variant with polyhedral cost uncertainty, called DDID. In the first stage, DDID selects a subset of uncertain cost coefficients to be observed, and in the second-stage, DDID selects a solution to the nominal problem, where the remaining cost coefficients are still uncertain. Given a compact linear programming formulation for the nominal problem, we provide a mixed-integer linear programming (MILP) formulation for DDID. The MILP is compact if the number of constraints describing the uncertainty polytope other than lower and upper bounds is constant. In that case, the formulation leads to polynomial-time algorithms for DDID when the number of possible observations is polynomially bounded. We extend this formulation to more general nominal problems through column generation and constraint generation algorithms. We illustrate our reformulations and algorithms numerically on the selection problem, the orienteering problem, and the spanning tree problem.
Special Issue on Optimization at the Second Level
Iterative Linear Quadratic Optimization for Nonlinear Control: Differentiable Programming Algorithmic Templates
Differentiable programming entails the implementation of mathematical functions to enable access to, typically, their first order information by automatic differentiation. We consider how nonlinear control algorithms based on linear and quadratic approximations of the objective can serve as a basis to expand the capabilities of differentiable programming by integrating efficient implementations of second order oracles through the graph of computations associated with the dynamical structure of nonlinear control problems. We present the implementation of a gradient descent, a Gauss-Newton method, a Newton method, differential dynamic programming approaches with linear quadratic or quadratic approximations, various line-search strategies, and regularized variants of nonlinear control algorithms from a common differentiable programming perspective rather than a linear algebraic viewpoint as previously taken. We derive the computational complexities of all algorithms and present sufficient optimality conditions. We compare the algorithms on several nonlinear control benchmarks, such as autonomous car racing using a bicycle model of a car. The algorithms are coded in a differentiable programming language in a publicly available package.
First order algorithms for computing linear and polyhedral estimates
It was recently shown in [7,9] that ``properly built'' linear and polyhedral estimates nearly attain minimax accuracy bounds in the problem of recovery of unknown signal from noisy observations of linear images of the signal when the signal set is an ellitope. However, design of nearly optimal estimates relies upon solving semidefinite optimization problems with matrix variables, what puts the synthesis of such estimates beyond the rich of the standard Interior Point algorithms of semidefinite optimization even for moderate size recovery problems. Our goal is to develop First Order Optimization algorithms for the computationally efficient design of linear and polyhedral estimates. In this paper we (a) explain how to eliminate matrix variables, thus reducing dramatically the design dimension when passing from Interior Point to First Order optimization algorithms and (2) develop and analyse a dedicated algorithm of the latter type---Composite Truncated Level method.
first-order methods of semidefinite optimization
statistical applications of semidefinite optimization
Optimizing transient gas network control for challenging real-world instances using MIP-based heuristics
Optimizing the transient control of gas networks is a highly challenging task. The corresponding model incorporates the combinatorial complexity of determining the settings for the many active elements as well as the non-linear and non-convex nature of the physical and technical principles of gas transport. In this paper, we present the latest improvements of our ongoing work to solve this problem for real-world, large-scale problem instances: By adjusting our mixed-integer non-linear programming model regarding the gas compression capabilities in the network, we reflect the technical limits of the underlying units more accurately while maintaining a similar overall model size. In addition, we introduce a new algorithmic approach that is based on splitting the complexity of the problem by first finding assignments for discrete variables and then determining the continuous variables as locally optimal solution of the corresponding non-linear program. For the first task, we design multiple different heuristics based on concepts for general time-expanded optimization problems that find solutions by solving a sequence of sub-problems defined on reduced time horizons. To demonstrate the competitiveness of our approach, we test our algorithm on particularly challenging historic demand scenarios. The results show that high-quality solutions are obtained reliably within short solving times, making the algorithm well-suited to be applied at the core of time-critical industrial applications.
The Binary Linearization Complexity of Pseudo-Boolean Functions
We consider the problem of linearizing a pseudo-Boolean function f : {0,1}n → R by means of k Boolean functions. Such a linearization yields an integer linear programming formulation with only k auxiliary variables. This motivates the definition of the linarization complexity of f as the minimum such k. Our theoretical contributions are the proof that random polynomials almost surely have a high linearization complexity and characterizations of its value in case we do or do not restrict the set of admissible Boolean functions. The practical relevance is shown by devising and evaluating integer linear programming models of two such linearizations for the low auto-correlation binary sequences problem. Still, many problems around this new concept remain open.
Tight analyses for subgradient descent I
Consider the problem of minimizing functions that are Lipschitz and convex, but not necessarily differentiable. We construct a function from this class for which the Tth iterate of subgradient descent has error Omega(log(T)/sqrt(T)). This matches a known upper bound of O(log(T)/sqrt(T)). We prove analogous results for functions that are additionally strongly convex. There exists such a function for which the error of the Tth iterate of subgradient descent has error Omega(log(T)/T), matching a known upper bound of O(log(T)/T). These results resolve a question posed by Shamir (2012).
Quadratic minimization: from conjugate gradient to an adaptive Heavy-ball method with Polyak step-sizes
In this work, we propose an adaptive variation on the classical Heavy-ball method for convex quadratic minimization. The adaptivity crucially relies on so-called “Polyak step-sizes”, which consists of using the knowledge of the optimal value of the optimization problem at hand instead of problem parameters such as a few eigenvalues of the Hessian of the problem. This method happens to also be equivalent to a variation of the classical conjugate gradient method, and thereby inherits many of its attractive features, including its finite-time convergence, instance optimality, and its worst-case convergence rates. The classical gradient method with Polyak step-sizes is known to behave very well in situations in which it can be used, and the question of whether incorporating momentum in this method is possible and can improve the method itself appeared to be open. We provide a definitive answer to this question for minimizing convex quadratic functions, an arguably necessary first step for developing such methods in more general setups.
An Interior Proximal Gradient Method for Nonconvex Optimization
We consider structured minimization problems subject to smooth inequality constraints and present a flexible algorithm that combines interior point (IP) and proximal gradient schemes. While traditional IP methods cannot cope with nonsmooth objective functions and proximal algorithms cannot handle complicated constraints, their combined usage is shown to successfully compensate the respective shortcomings. We provide a theoretical characterization of the algorithm and its asymptotic properties, deriving convergence results for fully nonconvex problems, thus bridging the gap with previous works that successfully addressed the convex case. Our interior proximal gradient algorithm benefits from warm starting, generates strictly feasible iterates with decreasing objective value, and returns after finitely many iterations a primal-dual pair approximately satisfying suitable optimality conditions.
Safe-screening rules for atomic-norm regularization
Safe-screening rules are algorithmic techniques meant to detect and safely discard unneeded variables during the solution process with the aim of accelerating computation. These techniques have been shown to be effective for one-norm regularized problems. This paper generalizes the safe-screening rule proposed by Ndiaye et al.~[J. Mach. Learn. Res., 2017] to various optimization problem formulations with atomic-norm regularization. For nuclear-norm regularized problems in particular, it is shown that the proposed generalized safe-screening rule cannot safely reduce the problem size. In that case, approximation guarantees are developed that allows for reducing the problem size.
Cycle-based formulations in Distance Geometry
The distance geometry problem asks to find a realization of a given simple edge-weighted graph in a Euclidean space of given dimension $K$, where the edges are realized as straight segments of lengths equal (or as close as possible) to the edge weights. The problem is often modelled as a mathematical programming formulation involving decision variables that determine the position of the vertices in the given Euclidean space. Solution algorithms are generally constructed using local or global nonlinear optimization techniques. We present a new modelling technique for this problem where, instead of deciding vertex positions, the formulations decide the length of the segments representing the edges in each cycle in the graph, projected in every dimension. We propose an exact formulation and a relaxation based on a Eulerian cycle. We then compare computational results from protein conformation instances obtained with stochastic global optimization techniques on the new cycle-based formulation and on the existing edge-based formulation. While edge-based formulations take less time to reach termination, cycle-based formulations are generally better on solution quality measures.
Adaptive Cut Selection in Mixed-Integer Linear Programming
Cut selection is a subroutine used in all modern mixed-integer linear programming solvers with the goal of selecting a subset of generated cuts that induce optimal solver performance. These solvers have millions of parameter combinations, and so are excellent candidates for parameter tuning. Cut selection scoring rules are usually weighted sums of different measurements, where the weights are parameters. We present a parametric family of mixed-integer linear programs together with infinitely many family-wide valid cuts. Some of these cuts can induce integer optimal solutions directly after being applied, while others fail to do so even if an infinite amount are applied. We show for a specific cut selection rule, that any finite grid search of the parameter space will always miss all parameter values, which select integer optimal inducing cuts in an infinite amount of our problems. We propose a variation on the design of existing graph convolutional neural networks, adapting them to learn cut selection rule parameters. We present a reinforcement learning framework for selecting cuts, and train our design using said framework over MIPLIB 2017. Our framework and design show that adaptive cut selection does substantially improve performance over a diverse set of instances, but that finding a single function describing such a rule is difficult. Code for reproducing all experiments is available at https://github.com/Opt-Mucca/Adaptive-Cutsel-MILP.
cut selection; mixed-integer linear programming; reinforcement learning
A Hölderian backtracking method for min-max and min-min problems
We present a new algorithm to solve min-max or min-min problems out of the convex world. We use rigidity assumptions, ubiquitous in learning, making our method applicable to many optimization problems. Our approach takes advantage of hidden regularity properties and allows us to devise a simple algorithm of ridge type. An original feature of our method is to come with automatic step-size adaptation which departs from the usual overly cautious backtracking methods. In a general framework, we provide convergence theoretical guarantees and rates. We apply our findings on simple GAN problems obtaining promising numerical results.
A Note on Robust Combinatorial Optimization with Generalized Interval Uncertainty
In this paper, we consider a robust combinatorial optimization problem with uncertain weights and propose an uncertainty set that generalizes interval uncertainty by imposing lower and upper bounds on deviations of subsets of items. We prove that if the number of such subsets is fixed and the family of these subsets is laminar, then the robust combinatorial optimization problem can be solved by solving a fixed number of deterministic problems. This result generalizes a previous similar result for the case where the family of these subsets is a partition of the set of items.
robust combinatorial optimization, interval uncertainty, budgeted uncertainty, complexity
Inertial-relaxed splitting for composite monotone inclusions
In a similar spirit of the extension of the proximal point method developed by Alves et al. [2], we propose in this work an Inertial-Relaxed primal-dual splitting method to address the problem of decomposing the minimization of the sum of three convex functions, one of them being smooth, and considering a general coupling subspace. A unified setting is formalized and applied to different average maps whose corresponding fixed points are related to the solutions of the inclusion problem associated with our extended model. An interesting feature of the resulting algorithms we have designed is that they present two distinct versions with a Gauss-Seidel or a Jacobi flavor, extending in that sense former proximal ADMM methods, both including inertial and relaxation parameters. Finally we show computational experiments on a class of the fused LASSO instances of medium size.
Operator splitting methods, Convex composite optimization
The continuous quadrant penalty formulation of logical constraints
Could continuous optimization address efficiently logical constraints? We propose a continuous-optimization alternative to the usual discrete-optimization (big-M and complementary) formulations of logical constraints that leads to an effective practical method. Based on the simple idea of guiding the search of a continuous-optimization method towards the parts of the domain where the logical constraint is satisfied, we introduce a smooth penalty-function formulation of logical constraints, and related theoretical results. The effectiveness of this formulation of logical constraints, that allows the direct use of state-of-the-art continuous optimization solvers, is demonstrated on an aircraft conflict avoidance application.
Quadratic error bound of the smoothed gap and the restarted averaged primal-dual hybrid gradient
We study the linear convergence of the primal-dual hybrid gradient method. After a review of current analyses, we show that they do not explain properly the behavior of the algorithm, even on the most simple problems. We thus introduce the quadratic error bound of the smoothed gap, a new regularity assumption that holds for a wide class of optimization problems. Equipped with this tool, we manage to prove tighter convergence rates. Then, we show that averaging and restarting the primal-dual hybrid gradient allows us to leverage better the regularity constant. Numerical experiments on linear and quadratic programs, ridge regression and image denoising illustrate the findings of the paper.
Computing directional stationary points for a class of nonconvex and nonsmooth optimization problems via Frank-Wolfe-like algorithms
Frank and Wolfe's celebrated conditional gradient method is a well-known tool for solving smooth optimization problems for which minimizing a linear function over the feasible set is computationally cheap. However, when the objective function is nonsmooth, the method may fail to compute a stationary point. In this work, we show that the Frank-Wolfe algorithm can be employed to compute Clarke-stationary points for the class of nonconvex and nonsmooth optimization problems consisting of minimizing upper-$C^{1,\alpha}$ functions over convex and compact sets. Under more restrictive assumptions, either on the objective function or the feasible set, we propose new variants of the algorithm having stronger stationarity guarantees, namely directional stationarity and even local optimality.
AC Optimal Power Flow: a Conic Programming relaxation and an iterative MILP scheme for Global Optimization
We address the issue of computing a global minimizer of the AC Optimal Power Flow problem. We introduce valid inequalities to strengthen the Semidefinite Programming relaxation, yielding a novel Conic Programming relaxation. Leveraging these Conic Programming constraints, we dynamically generate Mixed-Integer Linear Programming (MILP) relaxations, whose solutions asymptotically converge to global minimizers of the AC Optimal Power Flow problem. We apply this iterative MILP scheme on the IEEE PES PGLib benchmark and compare the results with two recent Global Optimization approaches.
Equivariant Perturbation in Gomory and Johnson’s Infinite Group Problem VII
In this self-contained paper, we present a theory of the piecewise linear minimal valid functions for the 1-row Gomory–Johnson infinite group problem. The non-extreme minimal valid functions are those that admit effective perturbations. We give a precise description of the space of these perturbations as a direct sum of certain finite- and infinite-dimensional subspaces. The infinite-dimensional subspaces have partial symmetries; to describe them, we develop a theory of inverse semigroups of partial bijections, interacting with the functional equations satisfied by the perturbations. Our paper provides the foundation for grid-free algorithms for the Gomory–Johnson model, in particular for testing extremality of piecewise linear functions whose breakpoints are rational numbers with huge denominators.
Tight computationally efficient approximation of matrix norms with applications
We address the problems of computing operator norms of matrices induced by given norms on the argument and the image space. It is known that aside of a fistful of "solvable cases," most notably, the case when both given norms are Euclidean, computing operator norm of a matrix is NP-hard. We specify rather general families of norms on the argument and the images space ("ellitopic" and "co-ellitopic", respectively) allowing for reasonably tight computationally efficient upper-bounding of the associated operator norms. We extend these results to bounding "robust operator norm of uncertain matrix with box uncertainty," that is, the maximum of operator norms of matrices representable as a linear combination, with coefficients of magnitude $\leq1$, of a collection of given matrices. Finally, we consider some applications of norm bounding, in particular, (1) computationally efficient synthesis of affine non-anticipative finite-horizon control of discrete time linear dynamical systems under bounds on the peak-to-peak gains, (2) signal recovery with uncertainties in sensing matrix, and (3) identification of parameters of time invariant discrete time linear dynamical systems via noisy observations of states and inputs on a given time horizon, in the case of "uncertain-but-bounded" noise varying in a box.
Frameworks and Results in Distributionally Robust Optimization
The concepts of risk-aversion, chance-constrained optimization, and robust optimization have developed significantly overthe last decade. The statistical learning community has also witnessed a rapid theoretical and applied growth by relying on these concepts. A modeling framework, called distributionally robust optimization (DRO), has recently received significant attention in both the operations research and statistical learning communities. This paper surveys main concepts and contributions to DRO, and its relationships with robust optimization, risk-aversion, chance-constrained optimization, and function regularization. Various approaches to model the distributional ambiguity and their calibrationare discussed. It also describes main solution techniques used to the solve the resulting optimization problems.
Screening for a Reweighted Penalized Conditional Gradient Method
The conditional gradient method (CGM) is widely used in large-scale sparse convex optimization, having a low per iteration computational cost for structured sparse regularizers and a greedy approach to collecting nonzeros. We explore the sparsity acquiring properties of a general penalized CGM (P-CGM) for convex regularizers and a reweighted penalized CGM (RP-CGM) for nonconvex regularizers, replacing the usual convex constraints with gauge-inspired penalties. This generalization does not increase the per-iteration complexity noticeably. Without assuming bounded iterates or using line search, we show O(1/t) convergence of the gap of each subproblem, which measures distance to a stationary point. We couple this with a screening rule which is safe in the convex case, converging to the true support at a rate O(1/(δ2)) where δ ≥ 0 measures how close the problem is to degeneracy. In the nonconvex case the screening rule converges to the true support in a finite number of iterations, but is not necessarily safe in the intermediate iterates. In our experiments, we verify the consistency of the method and adjust the aggressiveness of the screening rule by tuning the concavity of the regularizer.
Characterizations for Stability of Error Bounds for Convex Inequalities Constraint Systems
In this paper, we mainly study error bounds for a single convex inequality and semi-infinite convex constraint systems, and give characterizations for stability of error bounds via directional derivatives. For a single convex inequality, it is proved that the stability of local error bounds under small perturbations is essentially equivalent to the non-zero minimun of the directional derivative at a reference point over the sphere, and the stability of global error bounds is proved to be equivalent to the strictly positive infimum of the absolute directional derivatives, at all points in the boundary of the solution set, over the sphere as well as some mild constraint qualification. When these results are applied to semi-infinite convex constraint systems, characterizations for the stability of local and global error bounds under small perturbations are also provided. In particular such stability of error bounds is proved to only require that all component functions in semi-infinite convex constraint systems have the same linear perturbation. Our work demonstrates that verifying the stability of error bounds for convex inequalities constraint systems is, to some degree, equivalent to solving the convex optimization (defined by directional derivatives) over the sphere.
Local and global error bounds, Stability, Convex inequality, Semi-infinite convex constraint systems, Directional derivative
A note on approximate accelerated forward-backward methods with absolute and relative errors, and possibly strongly convex objectives
In this short note, we provide a simple version of an accelerated forward-backward method (a.k.a. Nesterov's accelerated proximal gradient method) possibly relying on approximate proximal operators and allowing to exploit strong convexity of the objective function. The method supports both relative and absolute errors, and its behavior is illustrated on a set of standard numerical experiments. Using the same developments, we further provide a version of the accelerated proximal hybrid extragradient method of Monteiro and Svaiter (2013) possibly exploiting strong convexity of the objective function.
Convergence Rates of Gradient Methods for Convex Optimization in the Space of Measures
We study the convergence rate of Bregman gradient methods for convex optimization in the space of measures on a $d$-dimensional manifold. Under basic regularity assumptions, we show that the suboptimality gap at iteration k is in O(log(k)k-1) for multiplicative updates, while it is in O(k-q/(d+q)) for additive updates for some q in {1,2,4} determined by the structure of the objective function. Our flexible proof strategy, based on approximation arguments, allows to painlessly cover all Bregman Proximal Gradient Methods (PGM) and their acceleration (APGM) under various geometries such as the hyperbolic entropy and Lp divergences. We also prove the tightness of our analysis with matching lower bounds and confirm the theoretical results with numerical experiments on low dimensional problems. Note that all these optimization methods must additionally pay the computational cost of discretization, which can be exponential in d.
The Referenced Vertex Ordering Problem: Theory, Applications, and Solution Methods
We introduce the referenced vertex ordering problem (REVORDER) as a combinatorial decision problem generalizing several vertex ordering problems that already appeared in the scientific literature under different guises. In other words, REVORDER is a generic problem with several possible extensions corresponding to various real-life applications. Previous works show that REVORDER has a polynomial complexity, whereas its optimization version, denoted in our work as MIN REVORDER, is NP-hard. We give a survey of methods and algorithms that can be applied to the solution of MIN REVORDER, and we develop a new enumeration scheme for its solution. Our theoretical analysis of this scheme yields several pruning techniques aimed at the reduction of the number of enumeration nodes. We then discuss how upper and lower bounds can be computed during the enumeration to design a branch-and-bound algorithm. Finally, we validate the branch-and-bound by conducting a large set of computational experiments on instances coming from various real-life applications. Our results highlight that the newly introduced pruning techniques allow for major reductions of computational cost, with a constant solution quality. In result, our branch-and-bound outperforms other existing solution methods: among 180 instances with 60 vertices, it solves 179 instances to optimality whereas the best existing method is only able to solve 109 of them. Moreover, our tests show that our algorithm can solve medium-scale instances including up to 500 vertices, which opens the perspective of handling new real-life problems. Our implementation of the branch-and-bound algorithm, together with all instances we have used, is publicly available on GitLab.
Trading off 1-norm and sparsity against rank for linear models using mathematical optimization
The M-P (Moore-Penrose) pseudoinverse has as a key application the computation of least-squares solutions of inconsistent systems of linear equations. Irrespective of whether a given input matrix is sparse, its M-P pseudoinverse can be dense, potentially leading to high computational burden, The M-P (Moore-Penrose) pseudoinverse has as a key application the computation of least-squares solutions of inconsistent systems of linear equations. Irrespective of whether a given input matrix is sparse, its M-P pseudoinverse can be dense, potentially leading to high computational burden, especially when we are dealing with high-dimensional matrices. The M-P pseudoinverse is uniquely characterized by four properties, but only two of them need to be satisfied for the computation of least-squares solutions. Fampa and Lee (2018) and Xu, Fampa, Lee, and Ponte (2019) propose local-search procedures to construct sparse block-structured generalized inverses that satisfy the two key M-P properties, plus one more (the so-called reflexive property). That additional M-P property is equivalent to imposing a minimum-rank condition on the generalized inverse. (Vector) 1-norm minimization is used to induce sparsity and, importantly, to keep the magnitudes of entries under control for the generalized-inverses constructed. Here, we investigate the trade-off between low 1-norm and low rank for generalized inverses that can be used in the computation of least-squares solutions. We propose several algorithmic approaches that start from a $1$-norm minimizing generalized inverse that satisfies the two key M-P properties, and gradually decrease its rank, by iteratively imposing the reflexive property. The algorithms iterate until the generalized inverse has the least possible rank. During the iterations, we produce intermediate solutions, trading off low 1-norm (and typically high sparsity) against low rank.
Moore-Penrose pseudoinverse, generalized inverse, linear model, least squares, sparse optimization, low rank
Exact methods for the unrelated parallel machine problem
We study methods for exact solution of the unrelated parallel machine problem (UPMP). Our original application arises in an automotive assembly process where a robot load balancing problem needs to be solved. This involves the solutions of several UPMPs, which proved hard for a general-purpose MILP solver since the makespan objective induces weak LP relaxation bounds. To improve these bounds and to enable the solution of larger instances, we propose two alternative methods: first, a reformulation of the MILP model based on split disjunction, which emulates a branch-and-bound rule different from variable branching; second, a branch-and-bound method based on a Lagrangian relaxation with NP-hard subproblems. For this second method we prove the zero duality gap property holds when a pair of machines is considered. Both methods yield significant reductions in computing time when applied to general UPMP instances, as compared to several state-of-the-art methods.
Robust Combinatorial Optimization with Locally Budgeted Uncertainty
Budgeted uncertainty sets have been established as a major influence on uncertainty modeling for robust optimization problems. A drawback of such sets is that the budget constraint only restricts the global amount of cost increase that can be distributed by an adversary. Local restrictions, while being important for many applications, cannot be modeled this way. We introduce a new variant of budgeted uncertainty sets, called locally budgeted uncertainty. In this setting, the uncertain parameters are partitioned, such that a classic budgeted uncertainty set applies to each part of the partition, called region. In a theoretical analysis, we show that the robust counterpart of such problems for a constant number of regions remains solvable in polynomial time, if the underlying nominal problem can be solved in polynomial time as well. If the number of regions is unbounded, we show that the robust selection problem remains solvable in polynomial time, while also providing hardness results for other combinatorial problems. In computational experiments using both random and real-world data, we show that using locally budgeted uncertainty sets can have considerable advantages over classic budgeted uncertainty sets.
Gradient formulae for probability functions depending on a heterogenous family of constraints
Probability functions measure the degree of satisfaction of certain constraints that are impacted by decisions and uncertainty. Such functions appear in probability or chance constraints ensuring that the degree of satisfaction is sufficiently high. These constraints have become a very popular modelling tool and are indeed intuitively easy to understand. Optimization problems involving probability constraints have thus arisen in many sectors of the industry, such as in the energy sector. Finding efficient solution methodology is important and first order information of probability functions play a key role therein. In this work we are motivated by probability functions measuring the degree of satisfaction of a potentially heterogenous family of constraints. We suggest a framework wherein each individual such constraint can be analyzed structurally. Our framework then allows us to establish formulae for the generalized subdifferential of the probability function itself.
Stochastic optimization; probabilistic constraints ; chance constraints ; generalized gradients