{"id":16500,"date":"2019-12-10T14:33:31","date_gmt":"2019-12-10T19:33:31","guid":{"rendered":"https:\/\/bangla.salearningschool.com\/recent-posts\/overview-on-optimization-concepts-from-the-internet\/"},"modified":"2020-02-08T09:42:48","modified_gmt":"2020-02-08T14:42:48","slug":"overview-on-optimization-concepts-from-the-internet","status":"publish","type":"post","link":"http:\/\/bangla.sitestree.com\/?p=16500","title":{"rendered":"Overview on optimization concepts: From the Internet"},"content":{"rendered":"<p><strong>Optimization Concepts:<\/strong><\/p>\n<p><strong>Convex sets:<\/strong><br \/>\n&quot;A convex set is a set of points such that, given any two points A, B in that set, the line AB joining them lies entirely within that set. Intuitively, this means that the set is connected (so that you can pass between any two points without leaving the set) and has no dents in its perimeter.<br \/>\nConvexity\/What is a convex set? &#8211; Wikibooks, open books for &#8230;<br \/>\n<a href=\"https:\/\/en.wikibooks.org\">https:\/\/en.wikibooks.org<\/a> \u203a wiki \u203a Convexity \u203a What_is_a_convex_set?&quot;<\/p>\n<p><strong>Convex functions:<\/strong><br \/>\n&quot;A convex function is a real-valued function defined on an interval with the property that its epigraph (the set of points on or above the graph of the function) is a convex set. Convex minimization is a subfield of optimization that studies the problem of minimizing convex functions over convex sets.<br \/>\nConvex set &#8211; Wikipedia<br \/>\n<a href=\"https:\/\/en.wikipedia.org\">https:\/\/en.wikipedia.org<\/a> \u203a wiki \u203a Convex_set&quot;<\/p>\n<p><strong>Optimization problems:<\/strong><br \/>\nInteresting simple optimization problems and solutions:<br \/>\n<a href=\"http:\/\/tutorial.math.lamar.edu\/Classes\/CalcI\/Optimization.aspx\">http:\/\/tutorial.math.lamar.edu\/Classes\/CalcI\/Optimization.aspx<\/a><br \/>\nMore Simple Optimization Problems and Solutions:<br \/>\n<a href=\"https:\/\/www.khanacademy.org\/search?page_search_query=Optimization%20problems%20(calculus)\">https:\/\/www.khanacademy.org\/search?page_search_query=Optimization%20problems%20(calculus)<\/a><\/p>\n<p><strong>Basics of convex analysis:<\/strong><br \/>\n<a href=\"https:\/\/en.wikipedia.org\/wiki\/Convex_analysis\">https:\/\/en.wikipedia.org\/wiki\/Convex_analysis<\/a><br \/>\nA good overview: <a href=\"http:\/\/eceweb.ucsd.edu\/~gert\/ECE273\/CvxOptTutPaper.pdf\">http:\/\/eceweb.ucsd.edu\/~gert\/ECE273\/CvxOptTutPaper.pdf<\/a><\/p>\n<p><strong>least-squares:<\/strong><br \/>\n&quot;The least squares method is a statistical procedure to find the best fit for a set of data points by minimizing the sum of the offsets or residuals of points from the plotted curve. Least squares regression is used to predict the behavior of dependent variables.Sep 2, 2019<br \/>\nLeast Squares Method Definition &#8211; Investopedia<br \/>\n<a href=\"https:\/\/www.investopedia.com\">https:\/\/www.investopedia.com<\/a> \u203a terms \u203a least-squares-method&quot;<\/p>\n<p>&quot;minimizing the sum of the squares of the residuals made in the results of every single equation.&quot;<br \/>\n&quot;The most important application is in data fitting. The best fit in the least-squares sense minimizes the sum of squared residuals (a residual being: the difference between an observed value, and the fitted value provided by a model).&quot;<br \/>\n&quot;Least-squares problems fall into two categories: linear or ordinary least squares and nonlinear least squares, depending on whether or not the residuals are linear in all unknowns. The linear least-squares problem occurs in statistical regression analysis; it has a closed-form solution. The nonlinear problem is usually solved by iterative refinement; at each iteration the system is approximated by a linear one, and thus the core calculation is similar in both cases.<\/p>\n<p>Polynomial least squares describes the variance in a prediction of the dependent variable as a function of the independent variable and the deviations from the fitted curve.<\/p>\n<p>When the observations come from an exponential family and mild conditions are satisfied, least-squares estimates and maximum-likelihood estimates are identical.[1] The method of least squares can also be derived as a method of moments estimator.<\/p>\n<p>The following discussion is mostly presented in terms of linear functions but the use of least squares is valid and practical for more general families of functions. Also, by iteratively applying local quadratic approximation to the likelihood (through the Fisher information), the least-squares method may be used to fit a generalized linear model.&quot;<br \/>\n<a href=\"https:\/\/en.wikipedia.org\/wiki\/Least_squares\">https:\/\/en.wikipedia.org\/wiki\/Least_squares<\/a><\/p>\n<p><strong>linear and quadratic programs:<\/strong><br \/>\n&quot;A linear programming (LP) problem is one in which the objective and all of the constraints are linear functions of the decision variables.&quot;<br \/>\n&quot;A quadratic programming (QP) problem has an objective which is a quadratic function of the decision variables, and constraints which are all linear functions of the variables.&quot;<br \/>\n&quot;LP problems are usually solved via the Simplex method. &quot;<br \/>\n&quot;An alternative to the Simplex method, called the Interior Point or Newton-Barrier method, was developed by Karmarkar in 1984. Also in the last decade, this method has been dramatically enhanced with advanced linear algebra methods so that it is often competitive with the Simplex method, especially on very large problems.&quot;<br \/>\n&quot;Since a QP problem is a special case of a smooth nonlinear problem, it can be solved by a smooth nonlinear optimization method such as the GRG or SQP method. However, a faster and more reliable way to solve a QP problem is to use an extension of the Simplex method or an extension of the Interior Point or Barrier method.&quot;<br \/>\n<a href=\"https:\/\/www.solver.com\/optimization-problem-types-linear-and-quadratic-programming\">https:\/\/www.solver.com\/optimization-problem-types-linear-and-quadratic-programming<\/a><br \/>\nQuadratic programming<br \/>\n<a href=\"https:\/\/optimization.mccormick.northwestern.edu\/index.php\/Quadratic_programming\">https:\/\/optimization.mccormick.northwestern.edu\/index.php\/Quadratic_programming<\/a><\/p>\n<p><strong>semidefinite programming:<\/strong><br \/>\n&quot;Semidefinite programming &#8211; Wikipedia<br \/>\n<a href=\"https:\/\/en.wikipedia.org\">https:\/\/en.wikipedia.org<\/a> \u203a wiki \u203a Semidefinite_programming<br \/>\nSemidefinite programming (SDP) is a subfield of convex optimization concerned with the optimization of a linear objective function (a user-specified function that the user wants to minimize or maximize) over the intersection of the cone of positive semidefinite matrices with an affine space, i.e., a spectrahedron.<br \/>\n\u200eMotivation and definition \u00b7 \u200eDuality theory \u00b7 \u200eExamples \u00b7 \u200eAlgorithms&quot;<\/p>\n<p><strong>&quot;Semidefinite Programming<\/strong><br \/>\n<a href=\"https:\/\/web.stanford.edu\">https:\/\/web.stanford.edu<\/a> \u203a ~boyd \u203a papers \u203a sdp<br \/>\nIn semidefinite programming we minimize a linear function subject to the constraint that an affine combination of symmetric matrices is positive semidefinite. Such a constraint is nonlinear and nonsmooth, but convex, so positive definite programs are convex optimization problems.&quot;<\/p>\n<p><a href=\"https:\/\/ocw.mit.edu\/courses\/electrical-engineering-and-computer-science\/6-251j-introduction-to-mathematical-programming-fall-2009\/readings\/MIT6_251JF09_SDP.pdf\">https:\/\/ocw.mit.edu\/courses\/electrical-engineering-and-computer-science\/6-251j-introduction-to-mathematical-programming-fall-2009\/readings\/MIT6_251JF09_SDP.pdf<\/a><\/p>\n<p><strong>minimax:<\/strong><\/p>\n<p>&quot;Minimax &#8211; Wikipedia<br \/>\n<a href=\"https:\/\/en.wikipedia.org\">https:\/\/en.wikipedia.org<\/a> \u203a wiki \u203a Minimax<br \/>\nMinimax (sometimes MinMax, MM or saddle point) is a decision rule used in artificial intelligence, decision theory, game theory, statistics and philosophy for minimizing the possible loss for a worst case (maximum loss) scenario. When dealing with gains, it is referred to as &quot;maximin&quot;\u2014to maximize the minimum gain.&quot;&quot;<\/p>\n<p><strong>duality theory:<\/strong><br \/>\n&quot;In mathematical optimization theory, duality or the duality principle is the principle that optimization problems may be viewed from either of two perspectives, the primal problem or the dual problem. The solution to the dual problem provides a lower bound to the solution of the primal (minimization) problem.<br \/>\nDuality (optimization) &#8211; Wikipedia<br \/>\n<a href=\"https:\/\/en.wikipedia.org\">https:\/\/en.wikipedia.org<\/a> \u203a wiki \u203a Duality_(optimization)&quot;<br \/>\n&quot;Usually the term &quot;dual problem&quot; refers to the Lagrangian dual problem but other dual problems are used \u2013 for example, the Wolfe dual problem and the Fenchel dual problem. The Lagrangian dual problem is obtained by forming the Lagrangian of a minimization problem by using nonnegative Lagrange multipliers to add the constraints to the objective function, and then solving for the primal variable values that minimize the original objective function. This solution gives the primal variables as functions of the Lagrange multipliers, which are called dual variables, so that the new problem is to maximize the objective function with respect to the dual variables under the derived constraints on the dual variables (including at least the nonnegativity constraints).&quot;<\/p>\n<p><strong>theorems of alternative:<\/strong><br \/>\n&quot;Farkas&#8217; lemma belongs to a class of statements called &quot;theorems of the alternative&quot;: a theorem stating that exactly one of two systems has a solution.<br \/>\nFarkas&#8217; lemma &#8211; Wikipedia<br \/>\n<a href=\"https:\/\/en.wikipedia.org\">https:\/\/en.wikipedia.org<\/a> \u203a wiki \u203a Farkas&#8217;_lemma&quot;<\/p>\n<p>&quot;In layman&#8217;s terms, a Theorem of the Alternative is a theorem which states that given two conditions, one of the two conditions is true. It further states that if one of those conditions fails to be true, then the other condition must be true.May 8, 1991&quot;<br \/>\n<a href=\"http:\/\/digitalcommons.iwu.edu\/cgi\/viewcontent.cgi?article=1000&amp;context=math_honproj\">http:\/\/digitalcommons.iwu.edu\/cgi\/viewcontent.cgi?article=1000&amp;context=math_honproj<\/a><\/p>\n<p>theorems of alternative applications; : <a href=\"https:\/\/link.springer.com\/article\/10.1007\/BF00939083\">https:\/\/link.springer.com\/article\/10.1007\/BF00939083<\/a><\/p>\n<p><strong>interior-point methods:<\/strong><br \/>\n&quot;Interior-point methods (also referred to as barrier methods or IPMs) are a certain class of algorithms that solve linear and nonlinear convex optimization problems.&quot;&quot;<br \/>\n<a href=\"https:\/\/en.wikipedia.org\/wiki\/Interior-point_method\">https:\/\/en.wikipedia.org\/wiki\/Interior-point_method<\/a><\/p>\n<p><strong>Applications of signal processing:<\/strong><br \/>\n&quot;Applications of DSP include audio signal processing, audio compression, digital image processing, video compression, speech processing, speech recognition, digital communications, digital synthesizers, radar, sonar, financial signal processing, seismology and biomedicine.<br \/>\nDigital signal processing &#8211; Wikipedia<br \/>\n<a href=\"https:\/\/en.wikipedia.org\">https:\/\/en.wikipedia.org<\/a> \u203a wiki \u203a Digital_signal_processing&quot;<\/p>\n<p><strong>Applications of optimization to signal processing:<\/strong><br \/>\n&quot;Convex optimization has been used in signal processing for a long time, to choose coefficients for use in fast (linear) algorithms, such as in filter or array design; more recently, it has been used to carry out (nonlinear) processing on the signal itself.&quot;<br \/>\n<a href=\"https:\/\/web.stanford.edu\/~boyd\/papers\/rt_cvx_sig_proc.html\">https:\/\/web.stanford.edu\/~boyd\/papers\/rt_cvx_sig_proc.html<\/a><\/p>\n<p>Kinect Audio Signal Optimization:<br \/>\n<a href=\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2016\/02\/ivantash-optimization_methods_and_their_applications_in_dsp.pdf\">https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2016\/02\/ivantash-optimization_methods_and_their_applications_in_dsp.pdf<\/a><\/p>\n<p><strong>statistics and machine learning:<\/strong><br \/>\n&quot;The Actual Difference Between Statistics and Machine Learning<br \/>\n<a href=\"https:\/\/towardsdatascience.com\">https:\/\/towardsdatascience.com<\/a> \u203a the-actual-difference-between-statistics-an&#8230;<br \/>\nMar 24, 2019 &#8211; \u201cThe major difference between machine learning and statistics is their purpose. Machine learning models are designed to make the most accurate predictions possible. Statistical models are designed for inference about the relationships between variables.\u201d &#8230; Statistics is the mathematical study of data.&quot;<\/p>\n<p>Machine Learning vs Statistics &#8211; KDnuggets<br \/>\n<a href=\"https:\/\/www.kdnuggets.com\">https:\/\/www.kdnuggets.com<\/a> \u203a 2016\/11 \u203a machine-learning-vs-statistics<br \/>\nMachine learning is all about predictions, supervised learning, and unsupervised learning, while statistics is about sample, population, and hypotheses. But are &#8230;<br \/>\n<a href=\"https:\/\/www.kdnuggets.com\/2016\/11\/machine-learning-vs-statistics.html\">https:\/\/www.kdnuggets.com\/2016\/11\/machine-learning-vs-statistics.html<\/a><\/p>\n<p>The Close Relationship Between Applied Statistics and Machine Learning<br \/>\n<a href=\"https:\/\/machinelearningmastery.com\/relationship-between-applied-statistics-and-machine-learning\/\">https:\/\/machinelearningmastery.com\/relationship-between-applied-statistics-and-machine-learning\/<\/a><\/p>\n<p><strong>Control and mechanical engineering:<\/strong><br \/>\n&quot;Control engineering is the engineering discipline that focuses on the modeling of a diverse range of dynamic systems (e.g. mechanical systems) and the design of controllers that will cause these systems to behave in the desired manner. &#8230; In most cases, control engineers utilize feedback when designing control systems.&quot; <a href=\"https:\/\/en.wikipedia.org\/wiki\/Control_engineering\">https:\/\/en.wikipedia.org\/wiki\/Control_engineering<\/a><\/p>\n<p><strong>Digital and analog circuit design:<\/strong><br \/>\n&quot;With the advent of logic synthesis, one of the biggest challenges faced by the electronic design automation (EDA) industry was to find the best netlist representation of the given design description. While two-level logic optimization had long existed in the form of the Quine\u2013McCluskey algorithm, later followed by the Espresso heuristic logic minimizer, the rapidly improving chip densities, and the wide adoption of HDLs for circuit description, formalized the logic optimization domain as it exists today.&quot; <a href=\"https:\/\/en.wikipedia.org\/wiki\/Logic_optimization\">https:\/\/en.wikipedia.org\/wiki\/Logic_optimization<\/a><\/p>\n<p>The analysis and optimization algorithms of the electronic circuits design<br \/>\n<a href=\"https:\/\/www.researchgate.net\/publication\/269211254_The_analysis_and_optimization_algorithms_of_the_electronic_circuits_design\">https:\/\/www.researchgate.net\/publication\/269211254_The_analysis_and_optimization_algorithms_of_the_electronic_circuits_design<\/a><\/p>\n<p><strong>Optimization Methods in Finance<\/strong><br \/>\n<a href=\"http:\/\/web.math.ku.dk\/~rolf\/CT_FinOpt.pdf\">http:\/\/web.math.ku.dk\/~rolf\/CT_FinOpt.pdf<\/a><\/p>\n<p>Optimization Models and Methods with Applications in Finance: <a href=\"http:\/\/www.bcamath.org\/documentos_public\/courses\/Nogales_2012-13_02_18-22.pdf\">http:\/\/www.bcamath.org\/documentos_public\/courses\/Nogales_2012-13_02_18-22.pdf<\/a><\/p>\n<p>Optimization for financial engineering: a special issue: <a href=\"https:\/\/link.springer.com\/article\/10.1007\/s11081-017-9358-1\">https:\/\/link.springer.com\/article\/10.1007\/s11081-017-9358-1<\/a><\/p>\n<p>&#8212;-<\/p>\n<p><em><strong>Sayed Ahmed<\/strong><br \/>\n<\/em><\/p>\n<p><em><strong>BSc. Eng. in Comp. Sc. &amp; Eng. (BUET)<\/strong><\/em><br \/>\n<em><strong>MSc. in Comp. Sc. (U of Manitoba, Canada)<\/strong><\/em><br \/>\n<em><strong>MSc. in Data Science and Analytics (Ryerson University, Canada)<\/strong><\/em><br \/>\n<em><strong>Linkedin<\/strong>: <a href=\"https:\/\/ca.linkedin.com\/in\/sayedjustetc\">https:\/\/ca.linkedin.com\/in\/sayedjustetc<\/a><br \/>\n<\/em><\/p>\n<p><em><strong>Blog<\/strong>: <a href=\"http:\/\/bangla.salearningschool.com\/\">http:\/\/Bangla.SaLearningSchool.com<\/a>, <a href=\"http:\/\/sitestree.com\">http:\/\/SitesTree.com<\/a> <\/em><br \/>\n<em><strong>Online and Offline Training<\/strong>: <a href=\"http:\/\/training.SitesTree.com\">http:\/\/Training.SitesTree.com<\/a> <\/em><\/p>\n<p><em>Get access to courses on Big Data, Data Science, AI, Cloud, Linux, System Admin, Web Development and Misc. related. Also, create your own course to sell to others to earn a revenue.<\/em><br \/>\n<a href=\"http:\/\/sitestree.com\/training\/\">http:\/\/sitestree.com\/training\/<\/a><\/p>\n<p><em><strong>I<\/strong>f you want to contribute to the operation of this site (Bangla.SaLearn) including occasional free and\/or low cost online training (using Zoom.us): <a href=\"http:\/\/training.sitestree.com\/\">http:\/\/Training.SitesTree.com<\/a> (or charitable\/non-profit work in the education\/health\/social service sector), you can financially contribute to: safoundation at <a href=\"http:\/\/salearningschool.com\">salearningschool.com<\/a> using Paypal or Credit Card (on <\/em><a href=\"http:\/\/sitestree.com\/training\/enrol\/index.php?id=114\">http:\/\/sitestree.com\/training\/enrol\/index.php?id=114<\/a> <em>).<\/em><br \/>\n<strong><\/strong><br \/>\n<strong><em>Affiliate Links: Deals on Amazon :<\/em><\/strong><br \/>\n<em>Hottest Deals on Amazon USA: <a href=\"http:\/\/tiny.cc\/38lddz\">http:\/\/tiny.cc\/38lddz<\/a><br \/>\n<\/em><br \/>\n<em>Hottest Deals on Amazon CA: <a href=\"http:\/\/tiny.cc\/bgnddz\">http:\/\/tiny.cc\/bgnddz<\/a><br \/>\n<\/em><br \/>\n<em>Hottest Deals on Amazon Europe: <a href=\"http:\/\/tiny.cc\/w4nddz\">http:\/\/tiny.cc\/w4nddz<\/a><\/em><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Optimization Concepts: Convex sets: &quot;A convex set is a set of points such that, given any two points A, B in that set, the line AB joining them lies entirely within that set. Intuitively, this means that the set is connected (so that you can pass between any two points without leaving the set) and &hellip; <\/p>\n<p><a class=\"more-link btn\" href=\"http:\/\/bangla.sitestree.com\/?p=16500\">Continue reading<\/a><\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_jetpack_memberships_contains_paid_content":false,"footnotes":""},"categories":[1910,1908,182],"tags":[],"class_list":["post-16500","post","type-post","status-publish","format-standard","hentry","category-ai-ml-ds-rl-dl-nn-nlp-data-mining-optimization","category-math-and-statistics-for-data-science-and-engineering","category---blog","item-wrap"],"jetpack_featured_media_url":"","jetpack_sharing_enabled":true,"jetpack-related-posts":[{"id":16677,"url":"http:\/\/bangla.sitestree.com\/?p=16677","url_meta":{"origin":16500,"position":0},"title":"Part X: Engineering Optimization: Mathematical Optimization","author":"Sayed","date":"January 23, 2020","format":false,"excerpt":"Good intro to: Quadratic Forms and Convexity https:\/\/www.dr-eriksen.no\/teaching\/GRA6035\/2010\/lecture4.pdf Concave Upward and Downward https:\/\/www.mathsisfun.com\/calculus\/concave-up-down-convex.html Convex functions and K-Convexityhttps:\/\/ljk.imag.fr\/membres\/Anatoli.Iouditski\/cours\/convex\/chapitre_3.pdf *** . *** . *** . *** Note: Older short-notes from this site are posted on Medium: https:\/\/medium.com\/@SayedAhmedCanada *** . *** *** . *** . *** . *** Sayed Ahmed BSc. Eng. in\u2026","rel":"","context":"In &quot;AI ML DS RL DL NN NLP Data Mining Optimization&quot;","block_context":{"text":"AI ML DS RL DL NN NLP Data Mining Optimization","link":"http:\/\/bangla.sitestree.com\/?cat=1910"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":16660,"url":"http:\/\/bangla.sitestree.com\/?p=16660","url_meta":{"origin":16500,"position":1},"title":"Optimization and Linear Algebra\/Math from the Internet","author":"Sayed","date":"January 18, 2020","format":false,"excerpt":"Optimization and Linear Algebra\/Math from the Internet First order taylor approximation formula? https:\/\/www.thestudentroom.co.uk\/showthread.php?t=1247928 Hessian Matrix https:\/\/en.wikipedia.org\/wiki\/Hessian_matrix \"In mathematics, the Hessian matrix or Hessian is a square matrix of second-order partial derivatives of a scalar-valued function, or scalar field. It describes the local curvature of a function of many variables.\" Use\u2026","rel":"","context":"In &quot;AI ML DS RL DL NN NLP Data Mining Optimization&quot;","block_context":{"text":"AI ML DS RL DL NN NLP Data Mining Optimization","link":"http:\/\/bangla.sitestree.com\/?cat=1910"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":63958,"url":"http:\/\/bangla.sitestree.com\/?p=63958","url_meta":{"origin":16500,"position":2},"title":"Misc: Optimization: Machine Learning: Data Science Resources","author":"Sayed","date":"June 3, 2021","format":false,"excerpt":"http:\/\/web.mit.edu\/15.053\/www\/AMP-Chapter-09.pdf https:\/\/www.cs.cmu.edu\/~anupamg\/adv-approx\/lecture14.pdf http:\/\/bangla.salearningschool.com\/recent-posts\/misc-optimization\/ https:\/\/www.futurelearn.com\/info\/courses\/maths-linear-quadratic-relations\/0\/steps\/12128 https:\/\/www.mathsisfun.com\/algebra\/systems-linear-equations-matrices.html https:\/\/www.wolframalpha.com\/input\/?i=subspace https:\/\/www.cse.iitk.ac.in\/users\/rmittal\/prev_course\/s14\/notes\/lec3.pdf https:\/\/observablehq.com\/@eliaskal\/point-combinations-linear-conic-affine-convex https:\/\/www.cse.iitk.ac.in\/users\/rmittal\/prev_course\/s14\/course_s14.html http:\/\/bangla.salearningschool.com\/recent-posts\/misc-math-might-relate-to-optimization\/ http:\/\/bangla.salearningschool.com\/recent-posts\/part-x-engineering-optimization-mathematical-optimization\/ https:\/\/www.dr-eriksen.no\/teaching\/GRA6035\/2010\/lecture4.pdf https:\/\/www.mathsisfun.com\/calculus\/concave-up-down-convex.html https:\/\/www-ljk.imag.fr\/membres\/Anatoli.Iouditski\/cours\/convex\/chapitre_3.pdf http:\/\/bangla.salearningschool.com\/recent-posts\/optimization-and-linear-algebra-math-from-the-internet\/ https:\/\/www.thestudentroom.co.uk\/showthread.php?t=1247928 https:\/\/en.wikipedia.org\/wiki\/Hessian_matrix https:\/\/en.wikipedia.org\/wiki\/Newton%27s_method_in_optimization https:\/\/onlinelibrary.wiley.com\/doi\/pdf\/10.1002\/9781118733639.app6 http:\/\/bangla.salearningschool.com\/recent-posts\/optimization-and-linear-algebra-math-from-the-internet\/ https:\/\/en.wikipedia.org\/wiki\/Interior-point_method https:\/\/web.stanford.edu\/~boyd\/papers\/rt_cvx_sig_proc.html Must Read https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2016\/02\/ivantash-optimization_methods_and_their_applications_in_dsp.pdf https:\/\/www.researchgate.net\/publication\/269211254_The_analysis_and_optimization_algorithms_of_the_electronic_circuits_design good one http:\/\/www.bcamath.org\/documentos_public\/courses\/Nogales_2012-13_02_18-22.pdf https:\/\/en.wikipedia.org\/wiki\/Convex_analysis https:\/\/www.khanacademy.org\/search?page_search_query=Optimization%20problems%20(calculus) http:\/\/bangla.salearningschool.com\/recent-posts\/overview-on-optimization-concepts-from-the-internet\/ http:\/\/bangla.salearningschool.com\/recent-posts\/misc-optimization-machine-learning\/ http:\/\/bangla.salearningschool.com\/recent-posts\/misc-math-data-science-machine-learning-optimization-vector-pca-basis-covariance\/ https:\/\/www.mathsisfun.com\/algebra\/matrix-inverse-row-operations-gauss-jordan.html http:\/\/bangla.salearningschool.com\/recent-posts\/misc-math-for-data-science-engineering-and-or-optimization\/","rel":"","context":"In &quot;\u09ac\u09cd\u09b2\u0997 \u0964 Blog&quot;","block_context":{"text":"\u09ac\u09cd\u09b2\u0997 \u0964 Blog","link":"http:\/\/bangla.sitestree.com\/?cat=182"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":16682,"url":"http:\/\/bangla.sitestree.com\/?p=16682","url_meta":{"origin":16500,"position":3},"title":"Misc. Math. Might Relate to Optimization","author":"Sayed","date":"January 26, 2020","format":false,"excerpt":"find the equation for a line http:\/\/www.webmath.com\/_answer.php Parametric forms for lines and vectors https:\/\/www.futurelearn.com\/courses\/maths-linear-quadratic-relations\/0\/steps\/12128 Solving Systems of Linear Equations Using Matrices https:\/\/www.mathsisfun.com\/algebra\/systems-linear-equations-matrices.html Affine Space \" \" Subspace https:\/\/www.wolframalpha.com\/input\/?i=subspace \"What is an affine set? A set is called \u201caffine\u201d iff for any two points in the set, the line through them\u2026","rel":"","context":"In &quot;Math and Statistics for Data Science, and Engineering&quot;","block_context":{"text":"Math and Statistics for Data Science, and Engineering","link":"http:\/\/bangla.sitestree.com\/?cat=1908"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/bangla.salearningschool.com\/wp-content\/uploads\/2020\/01\/image-6.png?resize=350%2C200","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/bangla.salearningschool.com\/wp-content\/uploads\/2020\/01\/image-6.png?resize=350%2C200 1x, https:\/\/i0.wp.com\/bangla.salearningschool.com\/wp-content\/uploads\/2020\/01\/image-6.png?resize=525%2C300 1.5x"},"classes":[]},{"id":16689,"url":"http:\/\/bangla.sitestree.com\/?p=16689","url_meta":{"origin":16500,"position":4},"title":"Misc. Math for Data Science, Engineering, and\/or Optimization","author":"Sayed","date":"January 28, 2020","format":false,"excerpt":"What is the Inverse of a Matrix? https:\/\/www.mathsisfun.com\/algebra\/matrix-inverse.html What is Norm? \"In linear algebra, functional analysis, and related areas of mathematics, a norm is a function that satisfies certain properties pertaining to scalability and additivity, and assigns a strictly positive real number to each vector in a vector space over\u2026","rel":"","context":"In &quot;AI ML DS RL DL NN NLP Data Mining Optimization&quot;","block_context":{"text":"AI ML DS RL DL NN NLP Data Mining Optimization","link":"http:\/\/bangla.sitestree.com\/?cat=1910"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":16742,"url":"http:\/\/bangla.sitestree.com\/?p=16742","url_meta":{"origin":16500,"position":5},"title":"Misc. Optimization:","author":"Sayed","date":"February 4, 2020","format":false,"excerpt":"\"Linear programming (LP, also called linear optimization) is a method to achieve the best outcome (such as maximum profit or lowest cost) in a mathematical model whose requirements are represented by linear relationships. en.wikipedia.org \u203a wiki \u203a Linear_programming Linear programming - Wikipedia\" \"Branch and bound (BB, B&B, or BnB) is\u2026","rel":"","context":"In &quot;AI ML DS RL DL NN NLP Data Mining Optimization&quot;","block_context":{"text":"AI ML DS RL DL NN NLP Data Mining Optimization","link":"http:\/\/bangla.sitestree.com\/?cat=1910"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]}],"_links":{"self":[{"href":"http:\/\/bangla.sitestree.com\/index.php?rest_route=\/wp\/v2\/posts\/16500","targetHints":{"allow":["GET"]}}],"collection":[{"href":"http:\/\/bangla.sitestree.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/bangla.sitestree.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/bangla.sitestree.com\/index.php?rest_route=\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"http:\/\/bangla.sitestree.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=16500"}],"version-history":[{"count":1,"href":"http:\/\/bangla.sitestree.com\/index.php?rest_route=\/wp\/v2\/posts\/16500\/revisions"}],"predecessor-version":[{"id":16717,"href":"http:\/\/bangla.sitestree.com\/index.php?rest_route=\/wp\/v2\/posts\/16500\/revisions\/16717"}],"wp:attachment":[{"href":"http:\/\/bangla.sitestree.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=16500"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/bangla.sitestree.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=16500"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/bangla.sitestree.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=16500"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}