Multi-Objective Trust-Region Filter Method for Nonlinear Constraints using Inexact Gradients

Manuel Berkemeier; Sebastian Peitz

doi:10.5802/ojmo.47

Manuel Berkemeier ¹; Sebastian Peitz ¹

¹ Department of Computer Science, TU Dortmund University, Germany

Open Journal of Mathematical Optimization, Volume 6 (2025), article no. 11, 36 p.

Abstract

In this article, we build on previous work to present an optimization algorithm for non-linearly constrained multi-objective optimization problems. The algorithm combines a surrogate-assisted derivative-free trust-region approach with the filter method known from single-objective optimization. Instead of the true objective and constraint functions, so-called fully linear models are employed, and we show how to deal with the gradient inexactness in the composite step setting, adapted from single-objective optimization as well. Under standard assumptions, we prove convergence of a subset of iterates to a quasi-stationary point and, if constraint qualifications hold, then the limit point is also a KKT-point of the multi-objective problem.

Received: 2023-01-20
Revised: 2024-10-31
Accepted: 2025-11-10
Published online: 2025-12-08

DOI: 10.5802/ojmo.47

Keywords: Multi-Objective Optimization, Multiobjective Optimization, Nonlinear Optimization, Derivative-Free Optimization, Trust-Region Method, Surrogate Models, Filter Method

Author's affiliations:

Manuel Berkemeier ¹; Sebastian Peitz ¹

¹ Department of Computer Science, TU Dortmund University, Germany

License:

CC-BY 4.0

Copyrights: The authors retain unrestricted copyrights and publishing rights

@article{OJMO_2025__6__A11_0,
     author = {Manuel Berkemeier and Sebastian Peitz},
     title = {Multi-Objective {Trust-Region} {Filter} {Method} for {Nonlinear} {Constraints} using {Inexact} {Gradients}},
     journal = {Open Journal of Mathematical Optimization},
     eid = {11},
     pages = {1--36},
     year = {2025},
     publisher = {Universit\'e de Montpellier},
     volume = {6},
     doi = {10.5802/ojmo.47},
     language = {en},
     url = {https://ojmo.centre-mersenne.org/articles/10.5802/ojmo.47/}
}

TY  - JOUR
AU  - Manuel Berkemeier
AU  - Sebastian Peitz
TI  - Multi-Objective Trust-Region Filter Method for Nonlinear Constraints using Inexact Gradients
JO  - Open Journal of Mathematical Optimization
PY  - 2025
SP  - 1
EP  - 36
VL  - 6
PB  - Université de Montpellier
UR  - https://ojmo.centre-mersenne.org/articles/10.5802/ojmo.47/
DO  - 10.5802/ojmo.47
LA  - en
ID  - OJMO_2025__6__A11_0
ER  -

%0 Journal Article
%A Manuel Berkemeier
%A Sebastian Peitz
%T Multi-Objective Trust-Region Filter Method for Nonlinear Constraints using Inexact Gradients
%J Open Journal of Mathematical Optimization
%D 2025
%P 1-36
%V 6
%I Université de Montpellier
%U https://ojmo.centre-mersenne.org/articles/10.5802/ojmo.47/
%R 10.5802/ojmo.47
%G en
%F OJMO_2025__6__A11_0

Manuel Berkemeier; Sebastian Peitz. Multi-Objective Trust-Region Filter Method for Nonlinear Constraints using Inexact Gradients. Open Journal of Mathematical Optimization, Volume 6 (2025), article  no. 11, 36 p.. doi: 10.5802/ojmo.47

References
Cited by

[1] Isabela Albuquerque; Joao Monteiro; Thang Doan; Breandan Considine; Tiago Falk; Ioannis Mitliagkas Multi-Objective Training of Generative Adversarial Networks with Multiple Discriminators, Proceedings of the 36th International Conference on Machine Learning (Proceedings of Machine Learning Research), Volume 97, PMLR (2019), pp. 202-211

[2] Hédy Attouch; Guillaume Garrigos; Xavier Goudou A Dynamic Gradient Approach to Pareto Optimization with Nonsmooth Convex Objective Functions, J. Math. Anal. Appl., Volume 422 (2015) no. 1, pp. 741-771 | DOI | Zbl

[3] Charles Audet; Jean Bigeon; Dominique Cartier; Sébastien Le Digabel; Ludovic Salomon Performance Indicators in Multiobjective Optimization, Eur. J. Oper. Res., Volume 292 (2020) no. 2, pp. 397-422 | DOI | Zbl | MR

[4] Charles Audet; Gilles Savard; Walid Zghal A Mesh Adaptive Direct Search Algorithm for Multiobjective Optimization, Eur. J. Oper. Res., Volume 204 (2010) no. 3, pp. 545-556 | DOI | MR | Zbl

[5] Manuel B. Berkemeier CoMPrOMISE.Jl Github Repository, 2024 (https://github.com/manuelbb-upb/Compromise.jl)

[6] Manuel B. Berkemeier; Sebastian Peitz Derivative-Free Multiobjective Trust Region Descent Method Using Radial Basis Function Surrogate Models, Math. Comput. Appl., Volume 26 (2021) no. 2, 31, 37 pages | DOI

[7] Jean Bigeon; Sébastien Le Digabel; Ludovic Salomon DMulti-MADS: Mesh Adaptive Direct Multisearch for Bound-Constrained Blackbox Multiobjective Optimization, Comput. Optim. Appl., Volume 79 (2021) no. 2, pp. 301-338 | DOI | MR | Zbl

[8] Jean Bigeon; Sébastien Le Digabel; Ludovic Salomon Handling of Constraints in Multiobjective Blackbox Optimization, Comput. Optim. Appl., Volume 89 (2024) no. 1, pp. 69-113 | MR | DOI | Zbl

[9] Carmo P. Brás; Ana Luísa Custódio On the Use of Polynomial Models in Multiobjective Directional Direct Search, Comput. Optim. Appl., Volume 77 (2020) no. 3, pp. 897-918 | DOI | MR | Zbl

[10] Tinkle Chugh; Karthik Sindhya; Jussi Hakanen; Kaisa Miettinen A Survey on Handling Computationally Expensive Multiobjective Optimization Problems with Evolutionary Algorithms, Soft Computing, Volume 23 (2019) no. 9, pp. 3137-3166 | DOI

[11] Guido Cocchi; Giampaolo Liuzzi; Stefano Lucidi; Marco Sciandrone On the Convergence of Steepest Descent Methods for Multiobjective Optimization, Comput. Optim. Appl., Volume 77 (2020) no. 1, pp. 1-27 | DOI | MR | Zbl

[12] Guido Cocchi; Giampaolo Liuzzi; Alessandra Papini; Marco Sciandrone An Implicit Filtering Algorithm for Derivative-Free Multiobjective Optimization with Box Constraints, Comput. Optim. Appl., Volume 69 (2018) no. 2, pp. 267-296 | DOI | MR | Zbl

[13] Carlos A. Coello Coello; David. A. van Veldhuizen; Gary B. Lamont Evolutionary Algorithms for Solving Multi-Objective Problems, Genetic and Evolutionary Computation, Springer, 2013 | MR

[14] Andrew R. Conn; Nicholas I. M. Gould; Philippe Louis Toint Trust-Region Methods, MPS/SIAM Series on Optimization, Society for Industrial and Applied Mathematics, 2000 | DOI | MR | Zbl

[15] Andrew R. Conn; Nick Gould; Annick Sartenaer; Philippe Louis Toint Global Convergence of a Class of Trust Region Algorithms for Optimization Using Inexact Projections on Convex Constraints, SIAM J. Optim., Volume 3 (1993) no. 1, pp. 164-221 | DOI | MR | Zbl

[16] Andrew R. Conn; Katya Scheinberg; Luís N. Vicente Introduction to Derivative-Free Optimization, MPS/SIAM Series on Optimization, Society for Industrial and Applied Mathematics/Mathematical Programming Society, 2009 no. 8 | DOI | MR

[17] Ana Luísa Custódio; José Firmino Aguilar Madeira; A. Ismael F. Vaz; Luís N. Vicente Direct Multisearch for Multiobjective Optimization, SIAM J. Optim., Volume 21 (2011) no. 3, pp. 1109-1140 | DOI | MR | Zbl

[18] Simon Danisch; Julius Krumbiegel Makie.Jl: Flexible High-Performance Data Visualization for Julia, J. Open Source Softw., Volume 6 (2021) no. 65, 3349, 5 pages | DOI

[19] Kalyanmoy Deb; Amrit Pratap; Sameer Agarwal; T. Meyarivan A Fast and Elitist Multiobjective Genetic Algorithm: NSGA-II, IEEE Trans. Evol. Comput., Volume 6 (2002) no. 2, pp. 182-197 | DOI

[20] Kalyanmoy Deb; Proteek Chandan Roy; Rayan Hussein Surrogate Modeling Approaches for Multiobjective Optimization: Methods, Taxonomy, and Results, Math. Comput. Appl., Volume 26 (2020) no. 1, 5, 27 pages | DOI

[21] Sander Dedoncker; Wim Desmet; Frank Naets An Adaptive Direct Multisearch Method for Black-Box Multi-Objective Optimization, Optim. Eng., Volume 23 (2022) no. 3, pp. 1411-1437 | DOI | MR | Zbl

[22] Elizabeth D. Dolan; Jorge J. Moré Benchmarking Optimization Software with Performance Profiles, Math. Program., Volume 91 (2002) no. 2, pp. 201-213 | DOI | MR | Zbl

[23] Luis Mauricio Graña Drummond; Benar Fux Svaiter A Steepest Descent Method for Vector Optimization, J. Comput. Appl. Math., Volume 175 (2005) no. 2, pp. 395-414 | DOI | Zbl

[24] John P. Eason A Trust Region Filter Algorithm for Surrogate-based Optimization, Ph. D. Thesis, Carnegie Mellon University (2018)

[25] John P. Eason; Lorenz T. Biegler A Trust Region Filter Method for Glass Box/Black Box Optimization, AIChE J., Volume 62 (2016) no. 9, pp. 3124-3136 | DOI

[26] Nélida E. Echebest; Maria L. Schuverdt; R. P. Vignau An Inexact Restoration Derivative-Free Filter Method for Nonlinear Programming, Comput. Appl. Math., Volume 36 (2017) no. 1, pp. 693-718 | DOI | MR | Zbl

[27] Matthias Ehrgott Multicriteria Optimization, Springer, 2005 | MR | Zbl

[28] Gabriele Eichfelder Adaptive Scalarization Methods in Multiobjective Optimization, Vector Optimization, Springer, 2008 | DOI | MR | Zbl

[29] Gabriele Eichfelder Twenty Years of Continuous Multiobjective Optimization in the Twenty-First Century, EURO J. Comput. Optim., Volume 9 (2021), 100014 | DOI | MR | Zbl

[30] Priscila S. Ferreira; Elizabeth W. Karas; Mael Sachine; Francisco N. C. Sobral Global Convergence of a Derivative-Free Inexact Restoration Filter Algorithm for Nonlinear Programming, Optimization, Volume 66 (2017) no. 2, pp. 271-292 | DOI | MR | Zbl

[31] Anthony V. Fiacco Introduction to Sensitivity and Stability Analysis in Nonlinear Programming, Mathematics in Science and Engineering, 165, Elsevier, 1983 | DOI | MR

[32] Roger Fletcher; Nicholas I. M. Gould; Sven Leyffer; Philippe Louis Toint; Andreas Wächter Global Convergence of a Trust-Region SQP-Filter Algorithm for General Nonlinear Programming, SIAM J. Optim., Volume 13 (2002) no. 3, pp. 635-659 | DOI

[33] Jörg Fliege; Benar Fux Svaiter Steepest Descent Methods for Multicriteria Optimization, Math. Methods Oper. Res., Volume 51 (2000) no. 3, pp. 479-494 | DOI | MR

[34] Jörg Fliege; A. Ismael F. Vaz A Method for Constrained Multiobjective Optimization Based on SQP Techniques, SIAM J. Optim., Volume 26 (2016) no. 4, pp. 2091-2119 | DOI | MR

[35] Ellen H. Fukuda; Luis Mauricio Graña Drummond A Survey on Multiobjective Descent Methods, Pesqui. Oper., Volume 34 (2014) no. 3, pp. 585-620 | DOI

[36] Sebastian R. Garreis Optimal Control under Uncertainty: Theory and Numerical Solution with Low-Rank Tensors, Ph. D. Thesis, Technische Universität München (2019) (https://mediatum.ub.tum.de/1452538)

[37] Bennet Gebken; Sebastian Peitz An Efficient Descent Method for Locally Lipschitz Multiobjective Optimization Problems, J. Optim. Theory Appl., Volume 188 (2021) no. 3, pp. 696-723 | DOI | MR

[38] Bennet Gebken; Sebastian Peitz; Michael Dellnitz A Descent Method for Equality and Inequality Constrained Multiobjective Optimization Problems, Numerical and Evolutionary Optimization – NEO 2017 (Studies in Computational Intelligence), Volume 785, Springer (2018), pp. 29-61 | DOI

[39] Bennet Gebken; Sebastian Peitz; Michael Dellnitz On the Hierarchical Structure of Pareto Critical Sets, AIP Conf. Proc., Volume 2070 (2019), 020041 (Proceedings LEGO – 14th International Global Optimization Workshop) | DOI

[40] Clóvis C. Gonzaga; Elizabeth W. Karas; Márcia Vanti A Globally Convergent Filter Method for Nonlinear Programming, SIAM J. Optim., Volume 14 (2004) no. 3, pp. 646-669 | DOI | MR

[41] Nicholas I. M. Gould; Sven Leyffer; Philippe Louis Toint A Multidimensional Filter Algorithm for Nonlinear Equations and Nonlinear Least-Squares, SIAM J. Optim., Volume 15 (2004) no. 1, pp. 17-38 | DOI | MR

[42] Claus Hillermeier Nonlinear Multiobjective Optimization: A Generalized Homotopy Approach, Springer, 2001 | DOI | MR

[43] Qi Huangfu; J. A. Julian Hall Parallelizing the Dual Revised Simplex Method, Math. Program. Comput., Volume 10 (2018) no. 1, pp. 119-142 | DOI | MR

[44] Johannes Jahn Vector Optimization: Theory, Applications, and Extensions, Springer, 2011 | DOI | MR

[45] Steven G. Johnson The NLopt Nonlinear-Optimization Package, 2007 (https://github.com/stevengj/nlopt)

[46] Harold W. Kuhn; Albert W. Tucker Nonlinear Programming, Proceedings of the Second Berkeley Symposium on Mathematical Statistics and Probability, University of California Press (2014), pp. 481-492

[47] Matteo Lapucci; Pierluigi Mansueto Improved Front Steepest Descent for Multi-Objective Optimization, Oper. Res. Lett., Volume 51 (2023) no. 3, pp. 242-247 | DOI | MR

[48] Matteo Lapucci; Pierluigi Mansueto; Davide Pucci Effective Front-Descent Algorithms with Convergence Guarantees (2024) | arXiv

[49] J. G. Lin Maximal Vectors and Multi-Objective Optimization, J. Optim. Theory Appl., Volume 18 (1976) no. 1, pp. 41-64 | DOI | MR

[50] Giampaolo Liuzzi; Stefano Lucidi; Francesco Rinaldi A Derivative-Free Approach to Constrained Multiobjective Nonsmooth Optimization, SIAM J. Optim., Volume 26 (2016) no. 4, pp. 2744-2774 | DOI | MR

[51] L. R. Lucambio Pérez; Leandro F. Prudente Nonlinear Conjugate Gradient Methods for Vector Optimization, SIAM J. Optim., Volume 28 (2018) no. 3, pp. 2690-2720 | DOI | MR

[52] Zhongwei Ma Evolutionary Constrained Multiobjective Optimization: Test Suite Construction and Performance Comparisons, IEEE Trans. Evol. Comput., Volume 23 (2019) no. 6, pp. 972-986

[53] Adanay Martín; Oliver Schütze Pareto Tracer: A Predictor–Corrector Method for Multi-Objective Optimization Problems, Eng. Optim., Volume 50 (2018) no. 3, pp. 516-536 | DOI | MR

[54] José Mario Martínez; Benar Fux Svaiter A Practical Optimality Condition Without Constraint Qualifications for Nonlinear Programming, J. Optim. Theory Appl., Volume 118 (2003) no. 1, pp. 117-133 | DOI | MR

[55] Ioan Maruşciac On Fritz John Type Optimality Criterion in Multi-Objective Optimization, Math., Rev. Anal. Numér. Théor. Approximation, Anal. Numér. Théor. Approximation, Volume 11 (1982) no. 1-2, pp. 109-114 | MR

[56] Kaisa Miettinen Nonlinear Multiobjective Optimization., Springer, 2013 | MR

[57] Aboozar Mohammadi; Ana Luísa Custódio A Trust-Region Approach for Computing Pareto Fronts in Multiobjective Optimization, Comput. Optim. Appl., Volume 87 (2024) no. 1, pp. 149-179 | DOI | MR

[58] Aboozar Mohammadi; Davood Hajinezhad; Alfredo Garcia A Trust-Region Approach for Computing Pareto Fronts in Multiobjective Derivative-Free Optimization, Optim. Lett., Volume 19 (2025) no. 2, pp. 233-266 | DOI | MR

[59] Hiro Mukai Algorithms for Multicriterion Optimization, IEEE Trans. Autom. Control, Volume 25 (1980) no. 2, pp. 177-186 | DOI | MR

[60] Aviv Navon; Aviv Shamsian; Gal Chechik; Ethan Fetaya Learning the Pareto Front with Hypernetworks (2021) (International Conference on Learning Representations) | arXiv

[61] Sebastian Peitz; Michael Dellnitz Gradient-Based Multiobjective Optimization with Uncertainties, NEO 2016 (Studies in Computational Intelligence), Volume 731, Springer, 2018, pp. 159-182 | DOI

[62] Sebastian Peitz; Michael Dellnitz A Survey of Recent Trends in Multiobjective Optimal Control—Surrogate Models, Feedback Control and Objective Reduction, Math. Comput. Appl., Volume 23 (2018) no. 2, 30, 33 pages | DOI | MR

[63] Michael James David Powell A Direct Search Optimization Method That Models the Objective and Constraint Functions by Linear Interpolation, Advances in Optimization and Numerical Analysis (Susana Gomez; Jean-Pierre Hennart, eds.), Springer, 1994, pp. 51-67 | DOI

[64] Shaojian Qu; Mark Goh; Bing Liang Trust Region Methods for Solving Multiobjective Optimisation, Optim. Methods Softw., Volume 28 (2013) no. 4, pp. 796-811 | DOI | MR

[65] Jong-Hyun Ryu; Sujin Kim A Derivative-Free Trust-Region Method for Biobjective Optimization, SIAM J. Optim., Volume 24 (2014) no. 1, pp. 334-362 | DOI | MR

[66] Oliver Schütze; Oliver Cuate; Adanay Martín; Sebastian Peitz; Michael Dellnitz Pareto Explorer: A Global/Local Exploration Tool for Many-Objective Optimization Problems, Eng. Optim., Volume 52 (2020) no. 5, pp. 832-855 | DOI | MR

[67] Ozan Sener; Vladlen Koltun Multi-Task Learning as Multi-Objective Optimization (2019) | arXiv

[68] Everton J. Silva; Ana Luísa Custódio An Inexact Restoration Direct Multisearch Filter Approach to Multiobjective Constrained Derivative-Free Optimization, Optim. Methods Softw., Volume 40 (2024) no. 2, pp. 406-432 | DOI | Zbl | MR

[69] Jana Thomann; Gabriele Eichfelder A Trust-Region Algorithm for Heterogeneous Multiobjective Optimization, SIAM J. Optim., Volume 29 (2019) no. 2, pp. 1017-1047 | DOI | MR

[70] Kely D. V. Villacorta; Paulo R. Oliveira; Antoine Soubeyran A Trust-Region Method for Unconstrained Multiobjective Problems with Applications in Satisficing Processes, J. Optim. Theory Appl., Volume 160 (2014) no. 3, pp. 865-889 | DOI | MR

[71] Andreas Wächter; Lorenz T. Biegler Line Search Filter Methods for Nonlinear Programming: Motivation and Global Convergence, SIAM J. Optim., Volume 16 (2005) no. 1, pp. 1-31 | DOI | MR

[72] Andrea Walther; Lorenz T. Biegler On an Inexact Trust-Region SQP-filter Method for Constrained Nonlinear Optimization, Comput. Optim. Appl., Volume 63 (2016) no. 3, pp. 613-638 | MR | DOI

[73] Roger Jean-Baptiste Wets On the Continuity of the Value of a Linear Program and of Related Polyhedral-Valued Multifunctions, Mathematical Programming Studies, Springer, 1985, pp. 14-29 | DOI | MR

[74] Stefan M. Wild Derivative-Free Optimization Algorithms For Computationally Expensive Functions, Ph. D. Thesis, Cornell University (2009)

[75] Stefan M. Wild; Rommel G. Regis; Christine A. Shoemaker ORBIT: Optimization by Radial Basis Function Interpolation in Trust-Regions, SIAM J. Sci. Comput., Volume 30 (2008) no. 6, pp. 3197-3219 | DOI | MR

[76] Outi Wilppu; Napsu Karmitsa; M. Mäkelä New Multiple Subgradient Descent Bundle Method for Nonsmooth Multiobjective Optimization (2014) (Turku Centre for Computer Science)

Cited by Sources: