AI ML DS RL DL NN NLP Data Mining Optimization, Math and Statistics for Data Science, and Engineering, ব্লগ । Blog

Part 4: Some Basic Math/Stat Concepts for the wanna be Data Scientists

Part 4: Some Basic Math/Stat Concepts for the wanna be Data Scientists

Also for the Engineers in General

Quadratic form

"In multivariate statistics, if \varepsilon is a vector of n random variables, and \Lambda is an n-dimensional symmetric matrix, then the scalar quantity {\displaystyle \varepsilon ^{T}\Lambda \varepsilon } is known as a quadratic form in \varepsilon.
"

Ref: https://en.wikipedia.org/wiki/Quadratic_form_(statistics)

Please also check matrix related concepts. We will provide some matrix concepts at one point.

"In mathematics, a quadratic form is a polynomial with terms all of degree two. For example, is a quadratic form in the variables x and y. Wikipedia"

"
4x^2 + 2xy - 3y^2is a quadratic form in the variables x and y. The coefficients usually belong to a fixed field K, such as the real or complex numbers, and we speak of a quadratic form over K."

"Quadratic forms are not to be confused with a quadratic equation which has only one variable and includes terms of degree two or less. A quadratic form is one case of the more general concept of homogeneous polynomials."

Ref: https://en.wikipedia.org/wiki/Quadratic_form

Quartic function

"
This article is about the univariate case. For the bivariate case, see Quartic plane curve.

Graph of a polynomial of degree 4, with 3 critical points and four real roots (crossings of the x axis) (and thus no complex roots). If one or the other of the local minima were above the x axis, or if the local maximum were below it, or if there were no local maximum and one minimum below the x axis, there would only be two real roots (and two complex roots). If all three local extrema were above the x axis, or if there were no local maximum and one minimum above the x axis, there would be no real root (and four complex roots). The same reasoning applies in reverse to polynomial with a negative quartic coefficient.

In algebra, a quartic function is a function of the form

f(x)=ax^{4}+bx^{3}+cx^{2}+dx+e,

where a is nonzero, which is defined by a polynomial of degree four, called a quartic polynomial.

Sometimes the term biquadratic is used instead of quartic, but, usually, biquadratic function refers to a quadratic function of a square (or, equivalently, to the function defined by a quartic polynomial without terms of odd degree), having the form

f(x)=ax^{4}+cx^{2}+e.

A quartic equation, or equation of the fourth degree, is an equation that equates a quartic polynomial to zero, of the form

ax^{4}+bx^{3}+cx^{2}+dx+e=0,

where a ≠ 0.

The derivative of a quartic function is a cubic function.

"

Ref: https://en.wikipedia.org/wiki/Quartic_function

Quartic plane curve

Bivariate case

"

A quartic plane curve is a plane algebraic curve of the fourth degree. It can be defined by a bivariate quartic equation:

Ax^4+By^4+Cx^3y+Dx^2y^2+Exy^3+Fx^3+Gy^3+Hx^2y+Ixy^2+Jx^2+Ky^2+Lxy+Mx+Ny+P=0,

with at least one of A, B, C, D, E not equal to zero. This equation has 15 constants. However, it can be multiplied by any non-zero constant without changing the curve; thus by the choice of an appropriate constant of multiplication, any one of the coefficients can be set to 1, leaving only 14 constants. Therefore, the space of quartic curves can be identified with the real projective space {\mathbb {RP}}^{{14}}. It also follows, from Cramer's theorem on algebraic curves, that there is exactly one quartic curve that passes through a set of 14 distinct points in general position, since a quartic has 14 degrees of freedom.

A quartic curve can have a maximum of:

"

Ref: https://en.wikipedia.org/wiki/Quartic_plane_curve

Expected value of Quadratic Forms

"In multivariate statistics, if \varepsilon is a vector of n random variables, and \Lambda is an n-dimensional symmetric matrix, then the scalar quantity {\displaystyle \varepsilon ^{T}\Lambda \varepsilon } is known as a quadratic form in \varepsilon.
"

Expected Value :

"It can be shown that[1]

{\displaystyle \operatorname {E} \left[\varepsilon ^{T}\Lambda \varepsilon \right]=\operatorname {tr} \left[\Lambda \Sigma \right]+\mu ^{T}\Lambda \mu }

where \mu and \Sigma are the expected value and variance-covariance matrix of \varepsilon, respectively, and tr denotes the trace of a matrix. This result only depends on the existence of \mu and \Sigma; in particular, normality of \varepsilon is not required.

"

Note: you might see \varepsilon is replaced with x, and x' is used for transpose(x).

Also,

may be the equation without the second part (sure there will be an explanation)

The equations above hold irrespective of the distribution of x.

Expected value of Quartic form:

Ref: Estimation Books by Yaakov Bar-Shalom, X. Rong Li, Thiagalingam Kirubarajan

Mixture Density

"Mixture distribution. ... In cases where each of the underlying random variables is continuous, the outcome variable will also be continuous and its probability density function is sometimes referred to as a mixture density."

Ref: https://en.wikipedia.org/wiki/Mixture_distribution

Mixture PDF:

"A mixture pdf is a weighted sum of pdfs with the weights summing up to unity"

gaussian mixture pdf consists of weighted sum of gaussian densities

Ref: https://www.slideshare.net/jins0618/clusteringkmeans-expectmaximization-and-gaussian-mixture-model

https://www.mathworks.com/help/stats/gmdistribution.pdf.html

http://digitalcommons.utep.edu/cgi/viewcontent.cgi?article=2110&context=cs_techrep

ML and Mixture Models:

https://www.cs.toronto.edu/~rgrosse/csc321/mixture_models.pdf

https://statweb.stanford.edu/~tibs/stat315a/LECTURES/em.pdf

Definitions: https://www.statisticshowto.datasciencecentral.com/mixture-distribution/

https://www.asc.ohio-state.edu/gan.1/teaching/spring04/Chapter3.pdf

************
Sayed Ahmed

BSc. Eng. in Comp. Sc. & Eng. (BUET)
MSc. in Comp. Sc. (U of Manitoba, Canada)
MSc. in Data Science and Analytics (Ryerson University, Canada)
Linkedin: https://ca.linkedin.com/in/sayedjustetc

Blog: http://Bangla.SaLearningSchool.com, http://SitesTree.com
Online and Offline Training: http://Training.SitesTree.com
FB Group on Learning/Teaching: https://www.facebook.com/banglasalearningschool
Our free or paid events on IT/Data Science/Cloud/Programming/Similar: https://www.facebook.com/justetcsocial

Get access to courses on Big Data, Data Science, AI, Cloud, Linux, System Admin, Web Development and Misc. related. Also, create your own course to sell to others. http://sitestree.com/training/

If you want to contribute to the operation of this site (Bangla.SaLearn) including occasional free and/or low cost online/offline training: http://Training.SitesTree.com (or charitable/non-profit work in the education/health/social service sector), you can financially contribute to: safoundation at salearningschool.com using Paypal or Credit Card (on http://sitestree.com/training/enrol/index.php?id=114 ).