Open Journal of Mathematical Optimization
type: channel
language: en
publication name: Open Journal of Mathematical Optimization
issn: 2777-5860
items: Theoretical analysis of the randomized subspace regularized Newton method for non-convex optimization
items: One to beat them all: “RYU” – a unifying framework for the construction of safe balls
items: A Multi-Objective Trust-Region Filter Method for Nonlinear Constraints using Inexact Gradients
items: Tikhonov Regularization for Stochastic Convex Optimization]{Tikhonov Regularization for Stochastic Non-Smooth Convex Optimization in Hilbert Spaces
items: Learning structured approximations of combinatorial optimization problems.
items: Good and Fast Row-Sparse ah--Symmetric Reflexive Generalized Inverses
items: Network Localization and Multi-Dimensional Scaling: Escaping Saddles and a Local Optimality Condition
items: A Stochastic Objective-Function-Free Adaptive Regularization Method with Optimal Complexity
items: Connections and Reformulations between Robust and Bilevel Optimization
items: The Robust Bilevel Selection Problem
items: Grid is Good
items: An Exponentially Converging Particle Method for the Mixed Nash Equilibrium of Continuous Games
items: Combinatorial Robust Optimization with Decision-Dependent Information Discovery and Polyhedral Uncertainty
items: Iterative Linear Quadratic Optimization for Nonlinear Control: Differentiable Programming Algorithmic Templates
items: First order algorithms for computing linear and polyhedral estimates
items: Optimizing transient gas network control for challenging real-world instances using MIP-based heuristics
items: The Binary Linearization Complexity of Pseudo-Boolean Functions
items: Tight analyses for subgradient descent I
items: Quadratic minimization: from conjugate gradient to an adaptive Heavy-ball method with Polyak step-sizes
items: An Interior Proximal Gradient Method for Nonconvex Optimization
items: Safe-screening rules for atomic-norm regularization
items: A Hölderian backtracking method for min-max and min-min problems
items: Cycle-based formulations in Distance Geometry
items: Adaptive Cut Selection in Mixed-Integer Linear Programming
items: A Note on Robust Combinatorial Optimization with Generalized Interval Uncertainty
items: The continuous quadrant penalty formulation of logical constraints
items: Quadratic error bound of the smoothed gap and the restarted averaged primal-dual hybrid gradient
items: Inertial-relaxed splitting for composite monotone inclusions
items: Computing directional stationary points for a class of nonconvex and nonsmooth optimization problems via Frank-Wolfe-like algorithms
items: Characterizations for Stability of Error Bounds for Convex Inequalities Constraint Systems

Theoretical analysis of the randomized subspace regularized Newton method for non-convex optimization
type: item
description: While there already exist randomized subspace Newton methods that restrict the search direction to a random subspace for a convex function, we propose a randomized subspace regularized Newton method for a non-convex function {and more generally we investigate thoroughly, for the first time, the local convergence rate of the randomized subspace Newton method}. In our proposed algorithm, we use a modified Hessian of the function restricted to some random subspace so that, with high probability, the function value decreases at each iteration, even when the objective function is non-convex. We show that our method has global convergence under appropriate assumptions and its convergence rate is the same as that of the full regularized Newton method. %We also prove that Furthermore, we obtain a local linear convergence rate under some additional assumptions, and prove that this rate is the best we can hope, in general, when using a random subspace. We furthermore prove that if the Hessian, at the local optimum, is rank deficient then super-linear convergence holds.
subject: Continuous Optimization
  topic: section
subject: random projection
  topic: keywords
subject: newton method
  topic: keywords
subject: non-convex optimization
  topic: keywords
subject: local-convergence
  topic: keywords
creator: Terunari Fuji
creator: Pierre-Louis POIRION
creator: Akik Takeda
rights: Copyright (c) 2026 Open Journal of Mathematical Optimization
date: 2026-02-23
publication date: 2026-02-23
doi: 10.5802/ojmo.46

One to beat them all: “RYU” – a unifying framework for the construction of safe balls
type: item
description: In this paper, we present a new framework, called “RYU”, for constructing “safe” regions – specifically, bounded sets that are guaranteed to contain the dual solution of a target optimization problem. Our framework applies to the standard case where the objective function is composed of two components: a closed, proper, convex function with Lipschitz-smooth gradient and another closed, proper, convex function. We show that the RYU framework not only encompasses but also improves upon the state-of-the-art methods proposed over the past decade for this class of optimization problems.
subject: Continuous Optimization
  topic: section
subject: convex optimization
  topic: keywords
subject: screening rules
  topic: keywords
subject: safe regions
  topic: keywords
creator: Thu-Le Tran
creator: Clément Elvira
creator: Hong-Phuong Dang
creator: Cédric Herzet
rights: Copyright (c) 2026 Open Journal of Mathematical Optimization
date: 2026-02-23
publication date: 2026-02-23
doi: 10.5802/ojmo.45

A Multi-Objective Trust-Region Filter Method for Nonlinear Constraints using Inexact Gradients
type: item
description: In this article, we build on previous work to present an optimization algorithm for non-linearly constrained multi-objective optimization problems. The algorithm combines a surrogate-assisted derivative-free trust-region approach with the filter method known from single-objective optimization. Instead of the true objective and constraint functions, so-called fully linear models are employed, and we show how to deal with the gradient inexactness in the composite step setting, adapted from single-objective optimization as well. Under standard assumptions, we prove convergence of a subset of iterates to a quasi-stationary point and, if constraint qualifications hold, then the limit point is also a KKT-point of the multi-objective problem.
subject: Continuous Optimization
  topic: section
subject: multi-objective optimization; nonlinear optimization; derivative-free optimization; trust-region methods; surrogate modeling
  topic: keywords
creator: Manuel Bastian Berkemeier
creator: Sebastian Peitz
rights: Copyright (c) 2026 Open Journal of Mathematical Optimization
date: 2026-02-23
publication date: 2026-02-23

Tikhonov Regularization for Stochastic Convex Optimization]{Tikhonov Regularization for Stochastic Non-Smooth Convex Optimization in Hilbert Spaces
type: item
description: To solve convex optimization problems with a noisy gradient input, we analyze the global behavior of subgradient-like flows under stochastic errors. The objective function is composite, being equal to the sum of two convex functions, one being differentiable and the other potentially non-smooth. We then use stochastic differential inclusions where the drift term is minus the subgradient of the objective function, and the diffusion term is either bounded or square-integrable. In this context, under Lipschitz's continuity of the differentiable term and a growth condition of the non-smooth term, our first main result shows almost sure weak convergence of the trajectory process towards a minimizer of the objective function. Then, using Tikhonov regularization with a properly tuned vanishing parameter, we can obtain almost sure strong convergence of the trajectory towards the minimum norm solution. We find an explicit tuning of this parameter when our objective function satisfies a local error-bound inequality. We also provide a comprehensive complexity analysis by establishing several new pointwise and ergodic convergence rates in expectation for the convex, strongly convex, and Lojasiewicz case.
subject: Continuous Optimization
  topic: section
subject: Stochastic Optimization
  topic: keywords
subject: inertial gradient system
  topic: keywords
subject: convex optimization
  topic: keywords
subject: non-smooth optimization
  topic: keywords
subject: stochastic differential equation
  topic: keywords
subject: stochastic differential inclusion
  topic: keywords
subject: Tikhonov regularization
  topic: keywords
subject: error bound inequality
  topic: keywords
subject: Lojasiewicz inequality
  topic: keywords
subject: KL inequality
  topic: keywords
subject: Convergence rate
  topic: keywords
subject: Asymptotic behavior
  topic: keywords
creator: Rodrigo Maulen-Soto
rights: Copyright (c) 2026 Open Journal of Mathematical Optimization
date: 2026-02-23
publication date: 2026-02-23
doi: 10.5802/ojmo.44

Learning structured approximations of combinatorial optimization problems.
type: item
description: Machine learning pipelines that include a combinatorial optimization layer can give surprisingly efficient heuristics for difficult combinatorial optimization problems. Three questions remain open: which architecture should be used, how should the parameters of the machine learning model be learned, and what performance guarantees can we expect from the resulting algorithms? Following the intuitions of geometric deep learning, we explain why equivariant layers should be used when designing such pipelines, and illustrate how to build such layers on routing, scheduling, and network design applications. We introduce a learning approach that enables to learn such pipelines when the training set contains only instances of the difficult optimization problem and not their optimal solutions, and show its numerical performance on our three applications. Finally, using tools from statistical learning theory, we prove a theorem showing the convergence speed of the estimator. As a corollary, we obtain that, if an approximation algorithm can be encoded by the pipeline for some parametrization, then the learned pipeline will retain the approximation ratio guarantee. On our network design problem, our machine learning pipeline has the approximation ratio guarantee of the best approximation algorithm known and the numerical efficiency of the best heuristic.
subject: Discrete Optimization
  topic: section
subject: Combinatorial Optimization
  topic: keywords
subject: Machine Learning
  topic: keywords
subject: Approximation Algorithm
  topic: keywords
creator: Axel Parmentier
rights: Copyright (c) 2026 Open Journal of Mathematical Optimization
date: 2025-07-08
publication date: 2025-07-08
doi: 10.5802/ojmo.43

Good and Fast Row-Sparse ah--Symmetric Reflexive Generalized Inverses
type: item
description: We present several algorithms aimed at constructing sparse and structured sparse (row-sparse) generalized inverses, with application to the efficient computation of least-squares solutions, for inconsistent systems of linear equations, in the setting of multiple right-hand sides and a rank-deficient constraint matrix. Leveraging our earlier formulations to minimize the 1- and 2,1-norms of generalized inverses that satisfy important properties of the Moore-Penrose pseudoinverse, we develop efficient and scalable ADMM algorithms to address these norm-minimization problems and to limit the number of nonzero rows in the solution. We establish a 2,1-norm approximation result for a local-search procedure that was originally designed for 1-norm minimization, and we compare the ADMM algorithms with the local-search procedure and with general-purpose optimization solvers.
subject: Computational aspects and applications
  topic: section
subject: Moore-Penrose properties, generalized inverse, sparse optimization, norm minimization, least squares, local search, convex optimization, ADMM
  topic: keywords
creator: Jon Lee
creator: Marcia Fampa
creator: Gabriel Ponte
creator: Luze Xu
rights: Copyright (c) 2026 Open Journal of Mathematical Optimization
date: 2026-02-23
publication date: 2026-02-23

Network Localization and Multi-Dimensional Scaling: Escaping Saddles and a Local Optimality Condition
type: item
description: In this paper, we focus on a class of problems characterized by solving a set of non-linear composite norm equations, where each equation represents a measurement that approximates a norm of a linear transformation. These problems manifest as nonlinear least squares minimization problems, which are characterized by their non-convex and non-smooth nature, presenting challenges in finding (locally) optimal solutions. While existing optimization algorithms mostly concentrate on finding critical points of the associated least squares objective function, these functions often possess multiple non-global local minima and saddle points. These problems find wide applications in various domains, and within this category we focus our attention on two challenging problems: Wireless Sensor Network Localization and Multi-Dimensional Scaling. We establish that non-differentiable points correspond to maximum or saddle points, and we provide a constructive approach to determine descent directions at these points. Leveraging this, we propose a straightforward procedure to escape non-differentiable saddle points that is applicable in either centralized or distributed computational setting. Furthermore, we develop a necessary condition for differentiable points to be locally optimal, by exploiting the structure of the objective function of these problems.
subject: Continuous Optimization
  topic: section
subject: Nonconvex optimization
  topic: keywords
subject: escaping saddle points
  topic: keywords
subject: sensor network localization
  topic: keywords
subject: optimality condition
  topic: keywords
creator: Shoham Sabach
creator: Eyal Gur
rights: Copyright (c) 2026 Open Journal of Mathematical Optimization
date: 2026-02-23
publication date: 2026-02-23
doi: 10.5802/ojmo.42

A Stochastic Objective-Function-Free Adaptive Regularization Method with Optimal Complexity
type: item
description: A fully stochastic second-order adaptive-regularization method for unconstrained nonconvex optimization is presented which never computes the objective-function value, but yet achieves the optimal $\mathcal{O}(\epsilon^{-3/2})$ complexity bound for finding first-order critical points. The method is noise-tolerant and the inexactness conditions required for convergence depend on the history of past steps. Applications to cases where derivative evaluation is inexact and to minimization of finite sums by sampling are discussed. Numerical experiments on large binary classification problems illustrate the potential of the new method.
subject: Continuous Optimization
  topic: section
subject: stochastic optimization, adaptive regularization methods, evaluation complexity, Objective-Function-Free-Optimization (OFFO), nonconvex optimization.
  topic: keywords
creator: Serge Gratton
creator: Sadok Jerad
creator: Philippe Louis Toint
rights: Copyright (c) 2026 Open Journal of Mathematical Optimization
date: 2025-05-16
publication date: 2025-05-16
doi: 10.5802/ojmo.41

Connections and Reformulations between Robust and Bilevel Optimization
type: item
description: Robust and bilevel optimization share the common feature that they involve a certain multilevel structure. Hence, although they model something rather different when used in practice, they seem to have a similar mathematical structure. In this paper, we analyze the connections between different types of robust problems (strictly robust problems with and without decision-dependence of their uncertainty sets, min-max-regret problems, and two-stage robust problems) as well as of bilevel problems (optimistic problems, pessimistic problems, and robust bilevel problems). It turns out that bilevel optimization seems to be more general in the sense that for most types of robust problems, one can find proper reformulations as bilevel problems but not necessarily the other way around. We hope that these results pave the way for a stronger connection between the two fields - in particular to use both theory and algorithms from one field in the other and vice versa.
subject: Special Issue on Optimization at the Second Level
  topic: section
subject: Bilevel optimization, Robust optimization, Reformulations
  topic: keywords
creator: Marc Goerigk
creator: Jannis Kurtz
creator: Martin Schmidt
creator: Johannes Thürauf
rights: Copyright (c) 2026 Open Journal of Mathematical Optimization
date: 2025-01-16
publication date: 2025-01-16
doi: 10.5802/ojmo.38

The Robust Bilevel Selection Problem
type: item
description: In bilevel optimization problems, two players, the leader and the follower, make their decisions in a hierarchy, and both decisions influence each other. Usually one assumes that both players have full knowledge also of the other player's data. In a more realistic model, uncertainty can be quantified, e.g., using the robust optimization approach: We assume that the leader does not know the follower's objective function precisely, but only knows an uncertainty set of potential follower's objectives, and her aim is to optimize the worst case of the corresponding scenarios. Now the question arises how the computational complexity of bilevel optimization problems changes under the additional complications of this type of uncertainty. We make a further step towards answering this question by examining an easy bilevel problem. In the Bilevel Selection Problem, the leader and the follower each select some items, while a common number of items to select in total is given, and each of the two players minimizes the total costs of the selected items, according to different sets of item costs. We show that this problem can be solved in polynomial time without uncertainty and then investigate the complexity of its robust version. If the item sets controlled by the leader and by the follower are disjoint, it can still be solved in polynomial time in case of discrete uncertainty, interval uncertainty, and discrete uncorrelated uncertainty, using ideas from [5]. Otherwise, we show that the Robust Bilevel Selection Problem becomes NP-hard, even for discrete uncertainty. We present a 2-approximation algorithm and exact exponential-time approaches for this setting, including an algorithm that runs in polynomial time if the number of scenarios is a constant. Furthermore, we investigate variants of the Bievel Selection Problem where one or both of the two decision makers take a continuous decision. One variant leads to an example of a bilevel optimization problem whose optimum value may…
subject: Special Issue on Optimization at the Second Level
  topic: section
subject: Bilevel Optimization
  topic: keywords
subject: Robust Optimization
  topic: keywords
subject: Combinatorial Optimization
  topic: keywords
subject: Selection Problem
  topic: keywords
subject: Computational Complexity
  topic: keywords
creator: Dorothee Henke
rights: Copyright (c) 2026 Open Journal of Mathematical Optimization
date: 2025-04-28
publication date: 2025-04-28
doi: 10.5802/ojmo.40

Grid is Good
type: item
description: We propose an adaptive refinement algorithm to solve total variation regularized measure optimization problems. The method iteratively constructs dyadic partitions of the unit cube based on i) the resolution of discretized dual problems and ii) on the detection of cells containing points that violate the dual constraints. The detection is based on upper-bounds on the dual certificate, in the spirit of branch-and-bound methods. The interest of this approach is that it avoids the use of heuristic approaches to find the maximizers of dual certificates. We prove the convergence of this approach under mild hypotheses and a linear convergence rate under additional non-degeneracy assumptions. These results are confirmed by simple numerical experiments.
subject: Continuous Optimization
  topic: section
subject: Total variation
  topic: keywords
subject: measure spaces
  topic: keywords
subject: Frank-Wolfe
  topic: keywords
creator: Axel Flinth
creator: Frédéric de Gournay
creator: Pierre Weiss
rights: Copyright (c) 2026 Open Journal of Mathematical Optimization
date: 2026-02-23
publication date: 2026-02-23
doi: 10.5802/ojmo.39

An Exponentially Converging Particle Method for the Mixed Nash Equilibrium of Continuous Games
type: item
description: We consider the problem of computing mixed Nash equilibria of two-player zero-sum games with continuous sets of pure strategies and with first-order access to the payoff function. This problem arises for example in game-theory-inspired machine learning applications, such as distributionally-robust learning. In those applications, the strategy sets are high-dimensional and thus methods based on discretisation cannot tractably return high-accuracy solutions. In this paper, we introduce and analyze a particle-based method that enjoys guaranteed local convergence for this problem. This method consists in parametrizing the mixed strategies as atomic measures and applying proximal point updates to both the atoms' weights and positions. It can be interpreted as a time-implicit discretization of the "interacting" Wasserstein-Fisher-Rao gradient flow. We prove that, under non-degeneracy assumptions, this method converges at an exponential rate to the exact mixed Nash equilibrium from any initialization satisfying a natural notion of closeness to optimality. We illustrate our results with numerical experiments and discuss applications to max-margin and distributionally-robust classification using two-layer neural networks, where our method has a natural interpretation as a simultaneous training of the network's weights and of the adversarial distribution.
subject: Continuous Optimization
  topic: section
subject: particle methods
  topic: keywords
subject: mixed Nash equilibrium
  topic: keywords
subject: min-max optimization
  topic: keywords
subject: optimization on measures
  topic: keywords
subject: Wasserstein
  topic: keywords
subject: Fisher-Rao
  topic: keywords
subject: mirror prox
  topic: keywords
creator: Guillaume Wang
creator: Lénaïc Chizat
rights: Copyright (c) 2026 Open Journal of Mathematical Optimization
date: 2025-01-13
publication date: 2025-01-13
doi: 10.5802/ojmo.37

Combinatorial Robust Optimization with Decision-Dependent Information Discovery and Polyhedral Uncertainty
type: item
description: Given a nominal combinatorial optimization problem, we consider a robust two-stages variant with polyhedral cost uncertainty, called DDID. In the first stage, DDID selects a subset of uncertain cost coefficients to be observed, and in the second-stage, DDID selects a solution to the nominal problem, where the remaining cost coefficients are still uncertain. Given a compact linear programming formulation for the nominal problem, we provide a mixed-integer linear programming (MILP) formulation for DDID. The MILP is compact if the number of constraints describing the uncertainty polytope other than lower and upper bounds is constant. In that case, the formulation leads to polynomial-time algorithms for DDID when the number of possible observations is polynomially bounded. We extend this formulation to more general nominal problems through column generation and constraint generation algorithms. We illustrate our reformulations and algorithms numerically on the selection problem, the orienteering problem, and the spanning tree problem.
subject: Special Issue on Optimization at the Second Level
  topic: section
subject: robust combinatorial optimization
  topic: keywords
subject: compact formulations
  topic: keywords
subject: column generation
  topic: keywords
subject: cutting plane
  topic: keywords
creator: Michaël Poss
rights: Copyright (c)
date: 2024-09-05
publication date: 2024-09-05
doi: 10.5802/ojmo.33

Iterative Linear Quadratic Optimization for Nonlinear Control: Differentiable Programming Algorithmic Templates
type: item
description: Differentiable programming entails the implementation of mathematical functions to enable access to, typically, their first order information by automatic differentiation. We consider how nonlinear control algorithms based on linear and quadratic approximations of the objective can serve as a basis to expand the capabilities of differentiable programming by integrating efficient implementations of second order oracles through the graph of computations associated with the dynamical structure of nonlinear control problems. We present the implementation of a gradient descent, a Gauss-Newton method, a Newton method, differential dynamic programming approaches with linear quadratic or quadratic approximations, various line-search strategies, and regularized variants of nonlinear control algorithms from a common differentiable programming perspective rather than a linear algebraic viewpoint as previously taken. We derive the computational complexities of all algorithms and present sufficient optimality conditions. We compare the algorithms on several nonlinear control benchmarks, such as autonomous car racing using a bicycle model of a car. The algorithms are coded in a differentiable programming language in a publicly available package.
subject: Computational aspects and applications
  topic: section
subject: Differentiable Programming
  topic: keywords
subject: Nonlinear Discrete Time Control
  topic: keywords
subject: Newton
  topic: keywords
subject: Gauss-Newton
  topic: keywords
subject: Differentiable Dynamic Programming
  topic: keywords
creator: Vincent Roulet
creator: Siddhartha Srinivasa
creator: Maryam Fazel
creator: Zaid Harchaoui
rights: Copyright (c)
date: 2024-11-04
publication date: 2024-11-04
doi: 10.5802/ojmo.32

First order algorithms for computing linear and polyhedral estimates
type: item
description: It was recently shown in [7,9] that ``properly built'' linear and polyhedral estimates nearly attain minimax accuracy bounds in the problem of recovery of unknown signal from noisy observations of linear images of the signal when the signal set is an ellitope. However, design of nearly optimal estimates relies upon solving semidefinite optimization problems with matrix variables, what puts the synthesis of such estimates beyond the rich of the standard Interior Point algorithms of semidefinite optimization even for moderate size recovery problems. Our goal is to develop First Order Optimization algorithms for the computationally efficient design of linear and polyhedral estimates. In this paper we (a) explain how to eliminate matrix variables, thus reducing dramatically the design dimension when passing from Interior Point to First Order optimization algorithms and (2) develop and analyse a dedicated algorithm of the latter type---Composite Truncated Level method.
subject: Computational aspects and applications
  topic: section
subject: first-order methods of semidefinite optimization
  topic: keywords
subject: statistical applications of semidefinite optimization
  topic: keywords
creator: Anatoli Juditsky
creator: Arkadi Nemirovski
creator: Yannis Bekri
rights: Copyright (c)
date: 2024-10-30
publication date: 2024-10-30
doi: 10.5802/ojmo.35

Optimizing transient gas network control for challenging real-world instances using MIP-based heuristics
type: item
description: Optimizing the transient control of gas networks is a highly challenging task. The corresponding model incorporates the combinatorial complexity of determining the settings for the many active elements as well as the non-linear and non-convex nature of the physical and technical principles of gas transport. In this paper, we present the latest improvements of our ongoing work to solve this problem for real-world, large-scale problem instances: By adjusting our mixed-integer non-linear programming model regarding the gas compression capabilities in the network, we reflect the technical limits of the underlying units more accurately while maintaining a similar overall model size. In addition, we introduce a new algorithmic approach that is based on splitting the complexity of the problem by first finding assignments for discrete variables and then determining the continuous variables as locally optimal solution of the corresponding non-linear program. For the first task, we design multiple different heuristics based on concepts for general time-expanded optimization problems that find solutions by solving a sequence of sub-problems defined on reduced time horizons. To demonstrate the competitiveness of our approach, we test our algorithm on particularly challenging historic demand scenarios. The results show that high-quality solutions are obtained reliably within short solving times, making the algorithm well-suited to be applied at the core of time-critical industrial applications.
subject: Computational aspects and applications
  topic: section
subject: Transient Gas Network Optimization
  topic: keywords
subject: Sequential Mixed-Integer Programming
  topic: keywords
subject: Rolling Horizon Heuristic
  topic: keywords
subject: Aggregated Horizon Heuristic
  topic: keywords
subject: Real-World Historic Instances
  topic: keywords
subject: Industry-Ready
  topic: keywords
creator: Felix Hennings
creator: Kai Hoppmann-Baum
creator: Janina Zittel
rights: Copyright (c)
date: 2024-04-11
publication date: 2024-04-11
doi: 10.5802/ojmo.29

The Binary Linearization Complexity of Pseudo-Boolean Functions
type: item
description: We consider the problem of linearizing a pseudo-Boolean function f : {0,1}n → R by means of k Boolean functions. Such a linearization yields an integer linear programming formulation with only k auxiliary variables. This motivates the definition of the linarization complexity of f as the minimum such k. Our theoretical contributions are the proof that random polynomials almost surely have a high linearization complexity and characterizations of its value in case we do or do not restrict the set of admissible Boolean functions. The practical relevance is shown by devising and evaluating integer linear programming models of two such linearizations for the low auto-correlation binary sequences problem. Still, many problems around this new concept remain open.
subject: Discrete Optimization
  topic: section
subject: Pseudo-Boolean optimization
  topic: keywords
subject: multilinear optimization
  topic: keywords
creator: Matthias Walter
rights: Copyright (c)
date: 2024-10-22
publication date: 2024-10-22
doi: 10.5802/ojmo.34

Tight analyses for subgradient descent I
type: item
description: Consider the problem of minimizing functions that are Lipschitz and convex, but not necessarily differentiable. We construct a function from this class for which the Tth iterate of subgradient descent has error Omega(log(T)/sqrt(T)). This matches a known upper bound of O(log(T)/sqrt(T)). We prove analogous results for functions that are additionally strongly convex. There exists such a function for which the error of the Tth iterate of subgradient descent has error Omega(log(T)/T), matching a known upper bound of O(log(T)/T). These results resolve a question posed by Shamir (2012).
subject: Continuous Optimization
  topic: section
subject: Gradient descent
  topic: keywords
subject: Non-smooth optimization
  topic: keywords
subject: Lower bounds
  topic: keywords
creator: Nicholas Harvey
creator: Chris Liaw
creator: Sikander Randhawa
rights: Copyright (c)
date: 2024-10-22
publication date: 2024-10-22
doi: 10.5802/ojmo.31

Quadratic minimization: from conjugate gradient to an adaptive Heavy-ball method with Polyak step-sizes
type: item
description: In this work, we propose an adaptive variation on the classical Heavy-ball method for convex quadratic minimization. The adaptivity crucially relies on so-called “Polyak step-sizes”, which consists of using the knowledge of the optimal value of the optimization problem at hand instead of problem parameters such as a few eigenvalues of the Hessian of the problem. This method happens to also be equivalent to a variation of the classical conjugate gradient method, and thereby inherits many of its attractive features, including its finite-time convergence, instance optimality, and its worst-case convergence rates. The classical gradient method with Polyak step-sizes is known to behave very well in situations in which it can be used, and the question of whether incorporating momentum in this method is possible and can improve the method itself appeared to be open. We provide a definitive answer to this question for minimizing convex quadratic functions, an arguably necessary first step for developing such methods in more general setups.
subject: Continuous Optimization
  topic: section
subject: Optimization
  topic: keywords
subject: Quadratic
  topic: keywords
subject: Conjugate gradient
  topic: keywords
subject: Heavy-ball
  topic: keywords
subject: Polyak step-sizes
  topic: keywords
subject: Optimality
  topic: keywords
creator: Baptiste Goujaud
creator: Adrien Taylor
creator: Aymeric Dieuleveut
rights: Copyright (c)
date: 2024-10-22
publication date: 2024-10-22
doi: 10.5802/ojmo.36

An Interior Proximal Gradient Method for Nonconvex Optimization
type: item
description: We consider structured minimization problems subject to smooth inequality constraints and present a flexible algorithm that combines interior point (IP) and proximal gradient schemes. While traditional IP methods cannot cope with nonsmooth objective functions and proximal algorithms cannot handle complicated constraints, their combined usage is shown to successfully compensate the respective shortcomings. We provide a theoretical characterization of the algorithm and its asymptotic properties, deriving convergence results for fully nonconvex problems, thus bridging the gap with previous works that successfully addressed the convex case. Our interior proximal gradient algorithm benefits from warm starting, generates strictly feasible iterates with decreasing objective value, and returns after finitely many iterations a primal-dual pair approximately satisfying suitable optimality conditions.
subject: Continuous Optimization
  topic: section
subject: interior point methods
  topic: keywords
subject: proximal algorithms
  topic: keywords
subject: nonsmooth nonconvex optimization
  topic: keywords
subject: locally Lipschitz gradient
  topic: keywords
subject: barrier problems
  topic: keywords
creator: Alberto De Marchi
creator: Andreas Themelis
rights: Copyright (c)
date: 2024-07-04
publication date: 2024-07-04
doi: 10.5802/ojmo.30

Safe-screening rules for atomic-norm regularization
type: item
description: Safe-screening rules are algorithmic techniques meant to detect and safely discard unneeded variables during the solution process with the aim of accelerating computation. These techniques have been shown to be effective for one-norm regularized problems. This paper generalizes the safe-screening rule proposed by Ndiaye et al.~[J. Mach. Learn. Res., 2017] to various optimization problem formulations with atomic-norm regularization. For nuclear-norm regularized problems in particular, it is shown that the proposed generalized safe-screening rule cannot safely reduce the problem size. In that case, approximation guarantees are developed that allows for reducing the problem size.
subject: Continuous Optimization
  topic: section
subject: sparse optimization
  topic: keywords
subject: atomic norms
  topic: keywords
subject: convex optimization
  topic: keywords
creator: Zhenan Fan
creator: Huang Fang
creator: Michael P Friedlander
rights: Copyright (c)
date: 2024-04-15
publication date: 2024-04-15
doi: 10.5802/ojmo.27

A Hölderian backtracking method for min-max and min-min problems
type: item
description: We present a new algorithm to solve min-max or min-min problems out of the convex world. We use rigidity assumptions, ubiquitous in learning, making our method applicable to many optimization problems. Our approach takes advantage of hidden regularity properties and allows us to devise a simple algorithm of ridge type. An original feature of our method is to come with automatic step-size adaptation which departs from the usual overly cautious backtracking methods. In a general framework, we provide convergence theoretical guarantees and rates. We apply our findings on simple GAN problems obtaining promising numerical results.
subject: Continuous Optimization
  topic: section
subject: Ridge method
  topic: keywords
subject: Gradient method
  topic: keywords
subject: Gans
  topic: keywords
subject: Holderian gradient
  topic: keywords
creator: Jérôme Bolte
creator: Edouard Pauwels
creator: Lilian Glaudin
creator: Mathieu Serrurier
rights: Copyright (c)
date: 2023-12-20
publication date: 2023-12-20
doi: 10.5802/ojmo.24

Cycle-based formulations in Distance Geometry
type: item
description: The distance geometry problem asks to find a realization of a given simple edge-weighted graph in a Euclidean space of given dimension $K$, where the edges are realized as straight segments of lengths equal (or as close as possible) to the edge weights. The problem is often modelled as a mathematical programming formulation involving decision variables that determine the position of the vertices in the given Euclidean space. Solution algorithms are generally constructed using local or global nonlinear optimization techniques. We present a new modelling technique for this problem where, instead of deciding vertex positions, the formulations decide the length of the segments representing the edges in each cycle in the graph, projected in every dimension. We propose an exact formulation and a relaxation based on a Eulerian cycle. We then compare computational results from protein conformation instances obtained with stochastic global optimization techniques on the new cycle-based formulation and on the existing edge-based formulation. While edge-based formulations take less time to reach termination, cycle-based formulations are generally better on solution quality measures.
subject: Discrete Optimization
  topic: section
subject: Mathematical Programming
  topic: keywords
subject: cycle basis
  topic: keywords
subject: protein conformation
  topic: keywords
creator: Leo Liberti
creator: Gabriele Iommazzo
creator: Carlile Lavor
creator: Nelson Maculan
rights: Copyright (c)
date: 2026-03-19
publication date: 2026-03-19
doi: 10.5802/ojmo.18

Adaptive Cut Selection in Mixed-Integer Linear Programming
type: item
description: Cut selection is a subroutine used in all modern mixed-integer linear programming solvers with the goal of selecting a subset of generated cuts that induce optimal solver performance. These solvers have millions of parameter combinations, and so are excellent candidates for parameter tuning. Cut selection scoring rules are usually weighted sums of different measurements, where the weights are parameters. We present a parametric family of mixed-integer linear programs together with infinitely many family-wide valid cuts. Some of these cuts can induce integer optimal solutions directly after being applied, while others fail to do so even if an infinite amount are applied. We show for a specific cut selection rule, that any finite grid search of the parameter space will always miss all parameter values, which select integer optimal inducing cuts in an infinite amount of our problems. We propose a variation on the design of existing graph convolutional neural networks, adapting them to learn cut selection rule parameters. We present a reinforcement learning framework for selecting cuts, and train our design using said framework over MIPLIB 2017. Our framework and design show that adaptive cut selection does substantially improve performance over a diverse set of instances, but that finding a single function describing such a rule is difficult. Code for reproducing all experiments is available at https://github.com/Opt-Mucca/Adaptive-Cutsel-MILP.
subject: Discrete Optimization
  topic: section
subject: cut selection; mixed-integer linear programming; reinforcement learning
  topic: keywords
creator: Mark Turner
creator: Thorsten Koch
creator: Felipe Serrano
creator: Michael Winkler
rights: Copyright (c)
date: 2023-07-20
publication date: 2023-07-20
doi: 10.5802/ojmo.25

A Note on Robust Combinatorial Optimization with Generalized Interval Uncertainty
type: item
description: In this paper, we consider a robust combinatorial optimization problem with uncertain weights and propose an uncertainty set that generalizes interval uncertainty by imposing lower and upper bounds on deviations of subsets of items. We prove that if the number of such subsets is fixed and the family of these subsets is laminar, then the robust combinatorial optimization problem can be solved by solving a fixed number of deterministic problems. This result generalizes a previous similar result for the case where the family of these subsets is a partition of the set of items.
subject: Optimization under Uncertainty
  topic: section
subject: robust combinatorial optimization, interval uncertainty, budgeted uncertainty, complexity
  topic: keywords
creator: Hande Yaman
rights: Copyright (c)
date: 2023-05-22
publication date: 2023-05-22
doi: 10.5802/ojmo.23

The continuous quadrant penalty formulation of logical constraints
type: item
description: Could continuous optimization address efficiently logical constraints? We propose a continuous-optimization alternative to the usual discrete-optimization (big-M and complementary) formulations of logical constraints that leads to an effective practical method. Based on the simple idea of guiding the search of a continuous-optimization method towards the parts of the domain where the logical constraint is satisfied, we introduce a smooth penalty-function formulation of logical constraints, and related theoretical results. The effectiveness of this formulation of logical constraints, that allows the direct use of state-of-the-art continuous optimization solvers, is demonstrated on an aircraft conflict avoidance application.
subject: Continuous Optimization
  topic: section
subject: Logical constraints
  topic: keywords
subject: penalty function
  topic: keywords
subject: continuous optimization
  topic: keywords
subject: aircraft conflicts
  topic: keywords
creator: Sonia Cafieri
creator: Andrew R. Conn
creator: Marcel Mongeau
rights: Copyright (c)
date: 2023-09-18
publication date: 2023-09-18
doi: 10.5802/ojmo.28

Quadratic error bound of the smoothed gap and the restarted averaged primal-dual hybrid gradient
type: item
description: We study the linear convergence of the primal-dual hybrid gradient method. After a review of current analyses, we show that they do not explain properly the behavior of the algorithm, even on the most simple problems. We thus introduce the quadratic error bound of the smoothed gap, a new regularity assumption that holds for a wide class of optimization problems. Equipped with this tool, we manage to prove tighter convergence rates. Then, we show that averaging and restarting the primal-dual hybrid gradient allows us to leverage better the regularity constant. Numerical experiments on linear and quadratic programs, ridge regression and image denoising illustrate the findings of the paper.
subject: Continuous Optimization
  topic: section
subject: linear convergence
  topic: keywords
subject: primal-dual algorithm
  topic: keywords
subject: error bound
  topic: keywords
subject: restart
  topic: keywords
creator: Olivier Fercoq
rights: Copyright (c)
date: 2023-08-01
publication date: 2023-08-01
doi: 10.5802/ojmo.26

Inertial-relaxed splitting for composite monotone inclusions
type: item
description: In a similar spirit of the extension of the proximal point method developed by Alves et al. [2], we propose in this work an Inertial-Relaxed primal-dual splitting method to address the problem of decomposing the minimization of the sum of three convex functions, one of them being smooth, and considering a general coupling subspace. A unified setting is formalized and applied to different average maps whose corresponding fixed points are related to the solutions of the inclusion problem associated with our extended model. An interesting feature of the resulting algorithms we have designed is that they present two distinct versions with a Gauss-Seidel or a Jacobi flavor, extending in that sense former proximal ADMM methods, both including inertial and relaxation parameters. Finally we show computational experiments on a class of the fused LASSO instances of medium size.
subject: Continuous Optimization
  topic: section
subject: Operator splitting methods, Convex composite optimization
  topic: keywords
creator: Philippe Mahey
creator: Ernesto O. Albornoz
creator: Eladio Ocaña Anaya
rights: Copyright (c)
date: 2026-03-19
publication date: 2026-03-19
doi: 10.5802/ojmo.22

Computing directional stationary points for a class of nonconvex and nonsmooth optimization problems via Frank-Wolfe-like algorithms
type: item
description: Frank and Wolfe's celebrated conditional gradient method is a well-known tool for solving smooth optimization problems for which minimizing a linear function over the feasible set is computationally cheap. However, when the objective function is nonsmooth, the method may fail to compute a stationary point. In this work, we show that the Frank-Wolfe algorithm can be employed to compute Clarke-stationary points for the class of nonconvex and nonsmooth optimization problems consisting of minimizing upper-$C^{1,\alpha}$ functions over convex and compact sets. Under more restrictive assumptions, either on the objective function or the feasible set, we propose new variants of the algorithm having stronger stationarity guarantees, namely directional stationarity and even local optimality.
subject: Continuous Optimization
  topic: section
subject: Nonsmooth Optimization
  topic: keywords
subject: Nonconvex Optimization
  topic: keywords
creator: Welington de Oliveira
rights: Copyright (c)
date: 2026-03-19
publication date: 2026-03-19
doi: 10.5802/ojmo.21

Characterizations for Stability of Error Bounds for Convex Inequalities Constraint Systems
type: item
description: In this paper, we mainly study error bounds for a single convex inequality and semi-infinite convex constraint systems, and give characterizations for stability of error bounds via directional derivatives. For a single convex inequality, it is proved that the stability of local error bounds under small perturbations is essentially equivalent to the non-zero minimun of the directional derivative at a reference point over the sphere, and the stability of global error bounds is proved to be equivalent to the strictly positive infimum of the absolute directional derivatives, at all points in the boundary of the solution set, over the sphere as well as some mild constraint qualification. When these results are applied to semi-infinite convex constraint systems, characterizations for the stability of local and global error bounds under small perturbations are also provided. In particular such stability of error bounds is proved to only require that all component functions in semi-infinite convex constraint systems have the same linear perturbation. Our work demonstrates that verifying the stability of error bounds for convex inequalities constraint systems is, to some degree, equivalent to solving the convex optimization (defined by directional derivatives) over the sphere.
subject: Continuous Optimization
  topic: section
subject: Local and global error bounds, Stability, Convex inequality, Semi-infinite convex constraint systems, Directional derivative
  topic: keywords
creator: Michel A THERA
creator: Zhou Wei
creator: Jen-Chih Yao
rights: Copyright (c)
date: 2022-06-17
publication date: 2022-06-17
doi: 10.5802/ojmo.13