Part 4: Some Basic Math/Stat Concepts for the wanna be Data Scientists

Also for the Engineers in General

Quadratic form

“In multivariate statistics, if is a vector of random variables, and is an -dimensional symmetric matrix, then the scalar quantity is known as a quadratic form in .
”

Ref: https://en.wikipedia.org/wiki/Quadratic_form_(statistics)

Please also check matrix related concepts. We will provide some matrix concepts at one point.

“In mathematics, a quadratic form is a polynomial with terms all of degree two. For example, is a quadratic form in the variables x and y. Wikipedia”

”
is a quadratic form in the variables x and y. The coefficients usually belong to a fixed field K, such as the real or complex numbers, and we speak of a quadratic form over K.”

“Quadratic forms are not to be confused with a quadratic equation which has only one variable and includes terms of degree two or less. A quadratic form is one case of the more general concept of homogeneous polynomials.”

Ref: https://en.wikipedia.org/wiki/Quadratic_form

Quartic function

”
This article is about the univariate case. For the bivariate case, see Quartic plane curve.

Graph of a polynomial of degree 4, with 3 critical points and four real roots (crossings of the x axis) (and thus no complex roots). If one or the other of the local minima were above the x axis, or if the local maximum were below it, or if there were no local maximum and one minimum below the x axis, there would only be two real roots (and two complex roots). If all three local extrema were above the x axis, or if there were no local maximum and one minimum above the x axis, there would be no real root (and four complex roots). The same reasoning applies in reverse to polynomial with a negative quartic coefficient.

In algebra, a quartic function is a function of the form

where a is nonzero, which is defined by a polynomial of degree four, called a quartic polynomial.

Sometimes the term biquadratic is used instead of quartic, but, usually, biquadratic function refers to a quadratic function of a square (or, equivalently, to the function defined by a quartic polynomial without terms of odd degree), having the form

A quartic equation, or equation of the fourth degree, is an equation that equates a quartic polynomial to zero, of the form

where a ≠ 0.

The derivative of a quartic function is a cubic function.

”

Ref: https://en.wikipedia.org/wiki/Quartic_function

Quartic plane curve

Bivariate case

”

A quartic plane curve is a plane algebraic curve of the fourth degree. It can be defined by a bivariate quartic equation:

with at least one of A, B, C, D, E not equal to zero. This equation has 15 constants. However, it can be multiplied by any non-zero constant without changing the curve; thus by the choice of an appropriate constant of multiplication, any one of the coefficients can be set to 1, leaving only 14 constants. Therefore, the space of quartic curves can be identified with the real projective space . It also follows, from Cramer’s theorem on algebraic curves, that there is exactly one quartic curve that passes through a set of 14 distinct points in general position, since a quartic has 14 degrees of freedom.

A quartic curve can have a maximum of:

• Four connected components
• Twenty-eight bi-tangents
• Three ordinary double points.

”

Ref: https://en.wikipedia.org/wiki/Quartic_plane_curve

Expected value of Quadratic Forms

“In multivariate statistics, if is a vector of random variables, and is an -dimensional symmetric matrix, then the scalar quantity is known as a quadratic form in .
”

Expected Value :

“It can be shown that[1]

where and are the expected value and variance-covariance matrix of , respectively, and tr denotes the trace of a matrix. This result only depends on the existence of and ; in particular, normality of is not required.

”

Note: you might see is replaced with x, and x’ is used for transpose(x).

Also,

may be the equation without the second part (sure there will be an explanation)

The equations above hold irrespective of the distribution of x.

Expected value of Quartic form:

Ref: Estimation Books by Yaakov Bar-Shalom, X. Rong Li, Thiagalingam Kirubarajan

Mixture Density

“Mixture distribution. … In cases where each of the underlying random variables is continuous, the outcome variable will also be continuous and its probability density function is sometimes referred to as a mixture density.”

Ref: https://en.wikipedia.org/wiki/Mixture_distribution

Mixture PDF:

“A mixture pdf is a weighted sum of pdfs with the weights summing up to unity“

gaussian mixture pdf consists of weighted sum of gaussian densities

Ref: https://www.slideshare.net/jins0618/clusteringkmeans-expectmaximization-and-gaussian-mixture-model

https://www.mathworks.com/help/stats/gmdistribution.pdf.html

http://digitalcommons.utep.edu/cgi/viewcontent.cgi?article=2110&context=cs_techrep

ML and Mixture Models:

https://www.cs.toronto.edu/~rgrosse/csc321/mixture_models.pdf

https://statweb.stanford.edu/~tibs/stat315a/LECTURES/em.pdf

Definitions: https://www.statisticshowto.datasciencecentral.com/mixture-distribution/

https://www.asc.ohio-state.edu/gan.1/teaching/spring04/Chapter3.pdf

************
Sayed Ahmed

BSc. Eng. in Comp. Sc. & Eng. (BUET)
MSc. in Comp. Sc. (U of Manitoba, Canada)
MSc. in Data Science and Analytics (Ryerson University, Canada)
Linkedin: https://ca.linkedin.com/in/sayedjustetc

Blog: http://Bangla.SaLearningSchool.com, http://SitesTree.com
Online and Offline Training: http://Training.SitesTree.com
FB Group on Learning/Teaching: https://www.facebook.com/banglasalearningschool
Our free or paid events on IT/Data Science/Cloud/Programming/Similar: https://www.facebook.com/justetcsocial

Get access to courses on Big Data, Data Science, AI, Cloud, Linux, System Admin, Web Development and Misc. related. Also, create your own course to sell to others. http://sitestree.com/training/

If you want to contribute to the operation of this site (Bangla.SaLearn) including occasional free and/or low cost online/offline training: http://Training.SitesTree.com (or charitable/non-profit work in the education/health/social service sector), you can financially contribute to: safoundation at salearningschool.com using Paypal or Credit Card (on http://sitestree.com/training/enrol/index.php?id=114 ).

Conditional Probability and PDF

“The conditional probability of an event B is the probability that the event will occur given the knowledge that an event A has already occurred.

This probability is written P(B|A), notation for the probability of B given A. “

“In the case where events A and B are independent (where event A has no effect on the probability of event B), the conditional probability of event B given event A is simply the probability of event B, that is P(B).

If events A and B are not independent, then the probability of the intersection of A and B (the probability that both events occur) is defined by

P(A and B) = P(A)P(B|A).” Multiplication rule

Ref: http://www.stat.yale.edu/Courses/1997-98/101/condprob.htm

Truncated Distribution
“In statistics, a truncated distribution is a conditional distribution that results from restricting the domain of some other probability distribution. Truncated distributions arise in practical statistics in cases where the ability to record, or even to know about, occurrences is limited to values which lie above or below a given threshold or within a specified range.

For example, if the dates of birth of children in a school are examined, these would typically be subject to truncation relative to those of all children in the area given that the school accepts only children in a given age range on a specific date. There would be no information about how many children in the locality had dates of birth before or after the school’s cutoff dates if only a direct approach to the school were used to obtain information.“

“

Probability density function for the truncated normal distribution for different sets of parameters. In all cases, a = −10 and b = 10. For the black: μ = −8, σ = 2; blue: μ = 0, σ = 2; red: μ = 9, σ = 10; orange: μ = 0, σ = 10.
Support
PDF
CDF
Mean
Median

“

https://en.wikipedia.org/wiki/Truncated_distribution

Law of Total Probability

Bayes’s theorem

In probability theory and statistics, Bayes’s theorem describes the probability of an event, based on prior knowledge of conditions that might be related to the event. Wikipedia

Formula

	=	events
	=	probability of A given B is true
	=	probability of B given A is true
	=	the independent probabilities of A and B

Ref: https://en.wikipedia.org/wiki/Bayes’_theorem

Bayes Formula for Random Variables:

http://pwp.gatech.edu/ece-jrom/wp-content/uploads/sites/436/2017/08/16_BayesRVs-su14.pdf

Using the above equation for the bayes rule for discrete random variable

Bayes formula for Continuous Random Variable

Using:

Conditional Expectation : Discrete Case

Conditional Expectation : Continuous Case

Ref: https://www.math.arizona.edu/~tgk/464_07/cond_exp.pdf

Gaussian Random Variables:

The PDF:

Ref: https://www.sciencedirect.com/topics/engineering/gaussian-random-variable

Gaussian Random Vector:

Ref: http://statweb.stanford.edu/~kjross/Lec11_1015.pdf

The text and images are from the Internet. References are provided.