On the Determination of Lagrange Multipliers for a Weighted LASSO Problem Using Geometric and Convex Analysis Techniques

Gianluca Giacchi; Bastien Milani; Benedetta Franceschiello

doi:10.1007/s00245-023-10096-0

On the Determination of Lagrange Multipliers for a Weighted LASSO Problem Using Geometric and Convex Analysis Techniques

Appl Math Optim. 2024;89(2):31. doi: 10.1007/s00245-023-10096-0. Epub 2024 Jan 17.

Authors

Gianluca Giacchi^{1

2

3

4}, Bastien Milani³, Benedetta Franceschiello^{2

4}

Affiliations

¹ Università di Bologna, Dipartimento di Matematica, Piazza di Porta San Donato 5, 40126 Bologna, Italy.
² Institute of Systems Engineering, School of Engineering, HES-SO Valais-Wallis, Rue de l'Industrie 23, 1950 Sion, Switzerland.
³ Lausanne University Hospital and University of Lausanne, Lausanne, Department of Diagnostic and Interventional Radiology, Rue du Bugnon 46, Lausanne, 1011 Switzerland.
⁴ The Sense Innovation and Research Center, Avenue de Provence 82 1007, Lausanne and Ch. de l'Agasse 5, 1950 Sion, Switzerland.

Abstract

Compressed Sensing (CS) encompasses a broad array of theoretical and applied techniques for recovering signals, given partial knowledge of their coefficients, cf. Candés (C. R. Acad. Sci. Paris, Ser. I 346, 589-592 (2008)), Candés et al. (IEEE Trans. Inf. Theo (2006)), Donoho (IEEE Trans. Inf. Theo. 52(4), (2006)), Donoho et al. (IEEE Trans. Inf. Theo. 52(1), (2006)). Its applications span various fields, including mathematics, physics, engineering, and several medical sciences, cf. Adcock and Hansen (Compressive Imaging: Structure, Sampling, Learning, p. 2021), Berk et al. (2019 13th International conference on Sampling Theory and Applications (SampTA) pp. 1-5. IEEE (2019)), Brady et al. (Opt. Express 17(15), 13040-13049 (2009)), Chan (Terahertz imaging with compressive sensing. Rice University, USA (2010)), Correa et al. (2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) pp. 7789-7793 (2014, May) IEEE), Gao et al. (Nature 516(7529), 74-77 (2014)), Liu and Kang (Opt. Express 18(21), 22010-22019 (2010)), McEwen and Wiaux (Mon. Notices Royal Astron. Soc. 413(2), 1318-1332 (2011)), Marim et al. (Opt. Lett. 35(6), 871-873 (2010)), Yu and Wang (Phys. Med. Biol. 54(9), 2791 (2009)), Yu and Wang (Phys. Med. Biol. 54(9), 2791 (2009)). Motivated by our interest in the mathematics behind Magnetic Resonance Imaging (MRI) and CS, we employ convex analysis techniques to analytically determine equivalents of Lagrange multipliers for optimization problems with inequality constraints, specifically a weighted LASSO with voxel-wise weighting. We investigate this problem under assumptions on the fidelity term ${(A x - b)}_{2}^{2}$ , either concerning the sign of its gradient or orthogonality-like conditions of its matrix. To be more precise, we either require the sign of each coordinate of $2 {(A x - b)}^{T} A$ to be fixed within a rectangular neighborhood of the origin, with the side lengths of the rectangle dependent on the constraints, or we assume $A^{T} A$ to be diagonal. The objective of this work is to explore the relationship between Lagrange multipliers and the constraints of a weighted variant of LASSO, specifically in the mentioned cases where this relationship can be computed explicitly. As they scale the regularization terms of the weighted LASSO, Lagrange multipliers serve as tuning parameters for the weighted LASSO, prompting the question of their potential effective use as tuning parameters in applications like MR image reconstruction and denoising. This work represents an initial step in this direction.

Keywords: Compressed Sensing; Convex optimization; LASSO; Lagrange duality; MRI; Tuning parameters.