AI ML DS RL DL NN NLP Data Mining Optimization, Math and Statistics for Data Science, and Engineering, ব্লগ । Blog

Misc. Math. Data Science. Machine Learning. Optimization. Vector, PCA, Basis, Covariance

Misc. Math. Data Science. Machine Learning. Optimization. Vector, PCA, Basis, Covariance

Orthonormality: Orthonormal Vectors

"In linear algebra, two vectors in an inner product space are orthonormal if they are orthogonal and unit vectors. A set of vectors form an orthonormal set if all vectors in the set are mutually orthogonal and all of unit length. An orthonormal set which forms a basis is called an orthonormal basis."
https://en.wikipedia.org/wiki/Orthonormality

Basis for a Vector Space
"A vector space's basis is a subset of vectors within the space that are linearly independent and span the space. A basis is linearly independent because the vectors in it cannot be defined as a linear combination of any of the other vectors in the basis."

https://study.com/academy/lesson/finding-the-basis-of-a-vector-space.html

Vector Space
"In linear algebra, you might find yourself working with a set of vectors. When the operations of scalar multiplication and vector addition hold for a set of vectors, we call it a vector space."
https://study.com/academy/lesson/finding-the-basis-of-a-vector-space.html

Explain the concept of covariance matrices based on the shape of data.

Variance:

covariance captures: "The diagonal spread of the data is captured by the covariance."

"The covariance matrix defines the shape of the data. Diagonal spread is captured by the covariance, while axis-aligned spread is captured by the variance."

https://www.visiondummy.com/2014/04/geometric-interpretation-covariance-matrix/

https://www.cs.rutgers.edu/~elgammal/classes/cs536/lectures/i2ml-chap6.pdf

https://pathmind.com/wiki/eigenvector

How to derive variance-covariance matrix of coefficients in linear regression

https://stats.stackexchange.com/questions/68151/how-to-derive-variance-covariance-matrix-of-coefficients-in-linear-regression

"The matrix {\displaystyle \operatorname {K} _{\mathbf {YX} }\operatorname {K} _{\mathbf {XX} }^{-1}} is known as the matrix of regression coefficients, while in linear algebra {\displaystyle \operatorname {K} _{\mathbf {Y|X} }} is the Schur complement of {\displaystyle \operatorname {K} _{\mathbf {XX} }} in {\displaystyle \mathbf {\Sigma } }.
The matrix of regression coefficients may often be given in transpose form, {\displaystyle \operatorname {K} _{\mathbf {XX} }^{-1}\operatorname {K} _{\mathbf {XY} }}, suitable for post-multiplying a row vector of explanatory variables {\displaystyle \mathbf {X} ^{\rm {T}}} rather than pre-multiplying a column vector {\mathbf {X}}. In this form they correspond to the coefficients obtained by inverting the matrix of the normal equations of ordinary least squares (OLS)."
https://en.wikipedia.org/wiki/Covariance_matrix

Statistics 512: Applied Linear Models Topic 3

https://www.stat.purdue.edu/~boli/stat512/lectures/topic3.pdf

*** *** ***
Note: Older short-notes from this site are posted on Medium: https://medium.com/@SayedAhmedCanada

*** . *** *** . *** . *** . ***

Sayed Ahmed

BSc. Eng. in Comp. Sc. & Eng. (BUET)
MSc. in Comp. Sc. (U of Manitoba, Canada)
MSc. in Data Science and Analytics (Ryerson University, Canada)
Linkedin: https://ca.linkedin.com/in/sayedjustetc

Blog: http://Bangla.SaLearningSchool.com, http://SitesTree.com
Online and Offline Training: http://Training.SitesTree.com (Also, can be free and low cost sometimes)

Facebook Group/Form to discuss (Q & A): https://www.facebook.com/banglasalearningschool

Our free or paid training events: https://www.facebook.com/justetcsocial

Get access to courses on Big Data, Data Science, AI, Cloud, Linux, System Admin, Web Development and Misc. related. Also, create your own course to sell to others. http://sitestree.com/training/

If you want to contribute to occasional free and/or low cost online/offline training or charitable/non-profit work in the education/health/social service sector, you can financially contribute to: safoundation at salearningschool.com using Paypal or Credit Card (on http://sitestree.com/training/enrol/index.php?id=114 ).