Seeing Theory by Delvin, Guo, Kunin, Xiang

• Making predictions about something as seemingly mundane as tomorrow's weather, for example, is actually quite a difficult task.
• In general, it is important in statistics to understand the distinction between theoretical and empirical quantities.
• A probability is always a number between 0 and 1 inclusive.
• The sample space is the set of all possible outcomes in the experiment.
• Collections of outcomes in the sample space are called events.
• The sum of the probabilities of all the outcomes in a sample space must be 1.
• Once we know the probabilities of the outcomes in an experiment, we can compute the probability of any event.
• The probability of an event is the sum of the probabilities of the outcomes it comprises.
• The concept of average value is an important one in statistics.
• The variance of a random variable X is a non-negative number that summarizes on average how much X differs from its mean, or expectation.
• The square root of the variance is called the standard deviation.
• One of the main reasons we do statistics is to make inferences about a population given data from a subset of that population.
• A set is a collection of items, or elements, with no repeats.
• The number of permutations, or orderings, of n distinct objects is given by the factorial expression.
• A function X that maps outcomes in our sample space to real numbers is called a random variable.
• The word "countably" refers to a property of a set. We say a set is countable if we can describe a method to list out all of the elements in the set such that for any particular element in the set, if we wait long enough in our listing process, we will eventually get to that element. In contrast, a set is called uncountable if we cannot provide such a method.
• A random variable X is called discrete if X can only take on finitely many or countably many values.
• We say that X is a continuous random variable if X can take on uncountably many values.
• If X is a continuous random variable, then the probability that X takes on any particular value is 0.
• A continuous random variable is distributed according to a probability density function, usually denoted f, defined on the domain of X.
• There are two fundamental types of errors in hypothesis testing. They are denoted Type I and Type II error.
• A Type I error is made when we reject H0 when it is in fact true.
• A Type II error is made when we accept H0 when it is in fact false.
• The frequentist approach to inference holds that probabilities are intrinsically tied to frequencies.
• Bayesian inference takes a subjective approach and views probabilities as representing degrees of belief.
• The goal of Bayesian inference is to update our prior beliefs by taking into account data that we observe.
• Linear regression is one of the most widely used tools in statistics.
• In reality, most random variables are not actually independent.