{"id":16736,"date":"2020-02-02T16:16:13","date_gmt":"2020-02-02T21:16:13","guid":{"rendered":"https:\/\/bangla.salearningschool.com\/recent-posts\/?p=16736"},"modified":"2020-02-08T09:40:13","modified_gmt":"2020-02-08T14:40:13","slug":"misc-basic-statistics-for-data-science","status":"publish","type":"post","link":"http:\/\/bangla.sitestree.com\/?p=16736","title":{"rendered":"Misc Basic Statistics for Data Science"},"content":{"rendered":"

Hypergeometric Distribution<\/strong><\/p>\n

“In probability theory<\/a> and statistics<\/a>, the hypergeometric distribution<\/strong> is a discrete probability distribution<\/a> that describes the probability of \"k\" successes (random draws for which the object drawn has a specified feature) in \"n\" draws, without<\/em> replacement, from a finite population<\/a> of size \"N\" that contains exactly \"K\" objects with that feature, wherein each draw is either a success or a failure. In contrast, the binomial distribution<\/a> describes the probability of \"k\" successes in \"n\" draws with<\/em> replacement.
\nIn statistics<\/a>, the hypergeometric test<\/strong> uses the hypergeometric distribution to calculate the statistical significance of having drawn a specific \"k\" successes (out of \"n\" total draws) from the aforementioned population. The test is often used to identify which sub-populations are over- or under-represented in a sample. This test has a wide range of applications. For example, a marketing group could use the test to understand their customer base by testing a set of known customers for over-representation of various demographic subgroups (e.g., women, people under 30).” https:\/\/en.wikipedia.org\/wiki\/Hypergeometric_distribution<\/a><\/p>\n

Binomial Distribution<\/strong>
\n“In probability theory<\/a> and statistics<\/a>, the binomial distribution<\/strong> with parameters n<\/em> and p<\/em> is the discrete probability distribution<\/a> of the number of successes in a sequence of n<\/em> independent<\/a> experiments<\/a>, each asking a yes\u2013no question<\/a>, and each with its own boolean<\/a>-valued outcome<\/a>: success<\/a>\/yes<\/a>\/true<\/a>\/one<\/a> (with probability<\/a> p<\/em>) or failure<\/a>\/no<\/a>\/false<\/a>\/zero<\/a> (with probability<\/a> q<\/em> = 1 \u2212 p<\/em>). A single success\/failure experiment is also called a Bernoulli trial<\/a> or Bernoulli experiment and a sequence of outcomes is called a Bernoulli process<\/a>; for a single trial, i.e., n<\/em> = 1, the binomial distribution is a Bernoulli distribution<\/a>. The binomial distribution is the basis for the popular binomial test<\/a> of statistical significance<\/a>.<\/p>\n

The binomial distribution is frequently used to model the number of successes in a sample of size n<\/em> drawn with replacement<\/a> from a population of size N<\/em>. If the sampling is carried out without replacement, the draws are not independent and so the resulting distribution is a hypergeometric distribution<\/a>, not a binomial one. However, for N<\/em> much larger than n<\/em>, the binomial distribution remains a good approximation, and is widely used.”
\nhttps:\/\/en.wikipedia.org\/wiki\/Binomial_distribution<\/a><\/p>\n

Negative Binomial Distribution<\/strong>
\n“In probability theory<\/a> and statistics<\/a>, the negative binomial distribution<\/strong> is a discrete probability distribution<\/a> of the number of successes in a sequence of independent and identically distributed Bernoulli trials<\/a> before a specified (non-random) number of failures (denoted r<\/em>) occurs. For example, we can define that when we throw a dice<\/a> and get a 6 it is a failure while rolling any other number is considered a success, and also choose r to be 3. We then throw the dice repeatedly until the third time the number 6 appears. In such a case, the probability distribution of the number of non-6s that appeared will be a negative binomial distribution.<\/p>\n

The Pascal distribution<\/strong> (after Blaise Pascal<\/a>) and Polya distribution<\/strong> (for George P\u00f3lya<\/a>) are special cases of the negative binomial distribution. A convention among engineers, climatologists, and others is to use “negative binomial” or “Pascal” for the case of an integer-valued stopping-time parameter r<\/em>, and use “Polya” for the real-valued case.”
\nhttps:\/\/en.wikipedia.org\/wiki\/Negative_binomial_distribution<\/a><\/p>\n

Probability and Counting<\/p>\n

“To decide “how likely” an event is, we need to count<\/strong> the number of times an event could occur and compare it to the total number of possible events. Such a comparison is called the probability<\/strong> of the particular event occurring. The mathematical theory of counting<\/strong> is known as combinatorial analysis”
\nhttps:\/\/www.intmath.com\/counting-probability\/counting-probability-intro.php<\/a><\/p>\n

Principle of Counting<\/p>\n

“The Fundamental Counting Principle (also called the counting rule) is a way to figure out the number of outcomes in a probability problem. Basically, you multiply the events together to get the total number of outcomes. The formula is:
\nIf you have an event \u201ca\u201d and another event \u201cb\u201d then all the different outcomes for the events is a * b.”
\nhttps:\/\/www.statisticshowto.datasciencecentral.com\/fundamental-counting-principle\/<\/a><\/p>\n

Combinatorics<\/strong>
\nhttps:\/\/mathigon.org\/world\/Combinatorics<\/a><\/p>\n

fundamental principle of counting<\/p>\n

“The Fundamental Counting Principle<\/strong> states that if one event has m possible outcomes and a second independent event has n possible outcomes, then there are m x n total possible outcomes for the two events together.”
\nhttps:\/\/www.mathgoodies.com\/glossary\/term\/Fundamental%20Counting%20Principle<\/a><\/p>\n

Factorial
\n“In mathematics<\/a>, the factorial<\/strong> of a positive integer<\/a> n, denoted by n<\/em>!, is the product<\/a> of all positive integers less than or equal to n:
\n{\\displaystyle n!=n\\times (n-1)\\times (n-2)\\times (n-3)\\times \\cdots \\times 3\\times 2\\times 1\\,.}\"{\\displaystyle”
\nhttps:\/\/en.wikipedia.org\/wiki\/Factorial<\/a><\/p>\n

Factorial with Identical Numbers<\/strong>
\n\"\"<\/a><\/p>\n

“<\/strong><\/em>Bayes’ theorem<\/p>\n

Description<\/h2>\n

In probability theory and statistics, Bayes\u2019s theorem describes the probability of an event, based on prior knowledge of conditions that might be related to the event. Wikipedia<\/a><\/p>\n

Formula
\n\"P(A\\mid<\/p>\n\n\n\n\n\n\n
\"A,<\/td>\n=<\/td>\nevents<\/td>\n<\/tr>\n
\"P(A|B)\"<\/td>\n=<\/td>\nprobability of A given B is true<\/td>\n<\/tr>\n
\"P(B|A)\"<\/td>\n=<\/td>\nprobability of B given A is true<\/td>\n<\/tr>\n
\"P(A),<\/td>\n=<\/td>\nthe independent probabilities of A and B<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

“<\/strong><\/em><\/p>\n

*** . *** . *** . ***<\/strong><\/em>
\nNote: Older short-notes from this site are posted on Medium: <\/strong><\/em>https:\/\/medium.com\/@SayedAhmedCanada<\/a><\/p>\n

*** . *** *** . *** . *** . ***<\/p>\n

Sayed Ahmed<\/strong>
\n<\/em>
\nBSc. Eng. in Comp. Sc. & Eng. (BUET)<\/strong><\/em>
\nMSc. in Comp. Sc. (U of Manitoba, Canada)<\/strong><\/em>
\nMSc. in Data Science and Analytics (Ryerson University, Canada)<\/strong><\/em>
\nLinkedin<\/strong>: https:\/\/ca.linkedin.com\/in\/sayedjustetc<\/a>
\n<\/em><\/p>\n

Blog<\/strong>: http:\/\/Bangla.SaLearningSchool.com<\/a>, http:\/\/SitesTree.com<\/a><\/em>
\nOnline and Offline Training<\/strong>: http:\/\/Training.SitesTree.com<\/a> (Also, can be free and low cost sometimes)<\/em><\/p>\n

Facebook Group\/Form to discuss (Q & A): <\/em>https:\/\/www.facebook.com\/banglasalearningschool<\/a><\/p>\n

Our free or paid training events: https:\/\/www.facebook.com\/justetcsocial<\/a><\/p>\n

Get access to courses on Big Data, Data Science, AI, Cloud, Linux, System Admin, Web Development and Misc. related. Also, create your own course to sell to others. <\/em>http:\/\/sitestree.com\/training\/<\/a><\/p>\n

I<\/strong>f you want to contribute to occasional free and\/or low cost online\/offline training or charitable\/non-profit work in the education\/health\/social service sector, you can financially contribute to: safoundation at salearningschool.com<\/a> using Paypal or Credit Card (on <\/em>http:\/\/sitestree.com\/training\/enrol\/index.php?id=114<\/a> ).
\n<\/em><\/p>\n","protected":false},"excerpt":{"rendered":"

Hypergeometric Distribution “In probability theory and statistics, the hypergeometric distribution is a discrete probability distribution that describes the probability of successes (random draws for which the object drawn has a specified feature) in draws, without replacement, from a finite population of size that contains exactly objects with that feature, wherein each draw is either a … <\/p>\n

Continue reading<\/a><\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_jetpack_memberships_contains_paid_content":false,"footnotes":""},"categories":[1910,182],"tags":[],"class_list":["post-16736","post","type-post","status-publish","format-standard","hentry","category-ai-ml-ds-rl-dl-nn-nlp-data-mining-optimization","category---blog","item-wrap"],"jetpack_featured_media_url":"","jetpack_sharing_enabled":true,"jetpack-related-posts":[{"id":16532,"url":"http:\/\/bangla.sitestree.com\/?p=16532","url_meta":{"origin":16736,"position":0},"title":"Part 1: Some Math\/Stat Background that (true) Data Scientists will know\/use: from the internet","author":"Sayed","date":"December 28, 2019","format":false,"excerpt":"Chebyshev's inequality \"In probability theory, Chebyshev's inequality (also called the Bienaym\u00e9\u2013Chebyshev inequality) guarantees that, for a wide class of probability distributions, no more than a certain fraction of values can be more than a certain distance from the mean. Specifically, no more than 1\/k2 of the distribution's values can be\u2026","rel":"","context":"In "Math and Statistics for Data Science, and Engineering"","block_context":{"text":"Math and Statistics for Data Science, and Engineering","link":"http:\/\/bangla.sitestree.com\/?cat=1908"},"img":{"alt_text":"","src":"https:\/\/upload.wikimedia.org\/wikipedia\/commons\/thumb\/8\/85\/Discrete_probability_distrib.svg\/220px-Discrete_probability_distrib.svg.png","width":350,"height":200},"classes":[]},{"id":16200,"url":"http:\/\/bangla.sitestree.com\/?p=16200","url_meta":{"origin":16736,"position":1},"title":"Important Basic Concepts: Statistics for Big Data","author":"Sayed","date":"September 15, 2019","format":false,"excerpt":"Important Basic Concepts: Statistics for Big Data Graphical : Exploratory Data Analysis (EDA) methods? First of all, EDA is about exploring the data and understanding if the data will be good for the experiment and study. Graphs and plots can easily show the data patterns. The raw data can be\u2026","rel":"","context":"In "Statistics for Big Data"","block_context":{"text":"Statistics for Big Data","link":"http:\/\/bangla.sitestree.com\/?cat=1904"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":16665,"url":"http:\/\/bangla.sitestree.com\/?p=16665","url_meta":{"origin":16736,"position":2},"title":"Bayesian Statistics and Machine Learning","author":"Sayed","date":"January 19, 2020","format":false,"excerpt":"Bayesian Statistics and Machine Learning \"Bayesian inference is a method of statistical inference in which Bayes' theorem is used to update the probability for a hypothesis as more evidence or information becomes available. Bayesian inference is an important technique in statistics, and especially in mathematical statistics.\" en.wikipedia.org \u203a wiki \u203a\u2026","rel":"","context":"In "AI ML DS RL DL NN NLP Data Mining Optimization"","block_context":{"text":"AI ML DS RL DL NN NLP Data Mining Optimization","link":"http:\/\/bangla.sitestree.com\/?cat=1910"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/bangla.sitestree.com\/wp-content\/uploads\/2020\/01\/image-1.png?resize=350%2C200","width":350,"height":200},"classes":[]},{"id":16550,"url":"http:\/\/bangla.sitestree.com\/?p=16550","url_meta":{"origin":16736,"position":3},"title":"Part 3: Some Basic Math\/Stat Concepts for the wanna be Data Scientists","author":"Sayed","date":"December 30, 2019","format":false,"excerpt":"Conditional Probability and PDF \"The conditional probability of an event B is the probability that the event will occur given the knowledge that an event A has already occurred. This probability is written P(B|A), notation for the probability of B given A. \" \"In the case where events A and\u2026","rel":"","context":"In "AI ML DS RL DL NN NLP Data Mining Optimization"","block_context":{"text":"AI ML DS RL DL NN NLP Data Mining Optimization","link":"http:\/\/bangla.sitestree.com\/?cat=1910"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/bangla.sitestree.com\/wp-content\/uploads\/2019\/12\/image-8.png?resize=350%2C200","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/bangla.sitestree.com\/wp-content\/uploads\/2019\/12\/image-8.png?resize=350%2C200 1x, https:\/\/i0.wp.com\/bangla.sitestree.com\/wp-content\/uploads\/2019\/12\/image-8.png?resize=525%2C300 1.5x"},"classes":[]},{"id":26205,"url":"http:\/\/bangla.sitestree.com\/?p=26205","url_meta":{"origin":16736,"position":4},"title":"Important Basic Concepts: Statistics for Big Data #Root","author":"Author-Check- Article-or-Video","date":"April 19, 2021","format":false,"excerpt":"Important Basic Concepts: Statistics for Big Data Graphical : Exploratory Data Analysis (EDA) methods? First of all, EDA is about exploring the data and understanding if the data will be good for the experiment and study. Graphs and plots can easily show the data patterns. The raw data can be\u2026","rel":"","context":"In "FromSitesTree.com"","block_context":{"text":"FromSitesTree.com","link":"http:\/\/bangla.sitestree.com\/?cat=1917"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":16536,"url":"http:\/\/bangla.sitestree.com\/?p=16536","url_meta":{"origin":16736,"position":5},"title":"Part 2: Some basic Math\/Statistics concepts that Data Scientists (the true ones) will usually know\/use","author":"Sayed","date":"December 29, 2019","format":false,"excerpt":"Part 2: Some basic Math\/Statistics concepts that Data Scientists (the true ones) will usually know\/use (came across, studied, learned, used) Covariance and Correlation \"Covariance is a measure of how two variables change together, but its magnitude is unbounded, so it is difficult to interpret. By dividing covariance by the product\u2026","rel":"","context":"In "Math and Statistics for Data Science, and Engineering"","block_context":{"text":"Math and Statistics for Data Science, and Engineering","link":"http:\/\/bangla.sitestree.com\/?cat=1908"},"img":{"alt_text":"[eq5]","src":"https:\/\/i0.wp.com\/www.statlect.com\/images\/covariance-formula__12.png?resize=350%2C200&ssl=1","width":350,"height":200},"classes":[]}],"_links":{"self":[{"href":"http:\/\/bangla.sitestree.com\/index.php?rest_route=\/wp\/v2\/posts\/16736","targetHints":{"allow":["GET"]}}],"collection":[{"href":"http:\/\/bangla.sitestree.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/bangla.sitestree.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/bangla.sitestree.com\/index.php?rest_route=\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"http:\/\/bangla.sitestree.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=16736"}],"version-history":[{"count":2,"href":"http:\/\/bangla.sitestree.com\/index.php?rest_route=\/wp\/v2\/posts\/16736\/revisions"}],"predecessor-version":[{"id":16740,"href":"http:\/\/bangla.sitestree.com\/index.php?rest_route=\/wp\/v2\/posts\/16736\/revisions\/16740"}],"wp:attachment":[{"href":"http:\/\/bangla.sitestree.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=16736"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/bangla.sitestree.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=16736"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/bangla.sitestree.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=16736"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}