Saturday, September 17, 2016

EDX 6.008.1X Computational Probability and Inference Notes (Week 1)

EDX 6.008.1x Computational Probability and Inference Notes (Week 1)

Recently, I started doing the EDX 6.008.1x Computational Probability and Inference online course on EDX. This course is offered as part of the MITx and is similar to the MIT 6.008 Introduction to Inference course offered at MIT by Prof. Polina Golland. As a way to motivate myself keeping on track with the course, I decided to save my notes on the course content and publish them online. I hope you they can be useful for other students who are doing the course now, or in later sessions, and for anyone looking for information about topics related to the course subjects.

Course topics:

The course provides an introduction to probability theory and probabilistic graphical models and how to use to perform inferences. Probabilistic models are used in different application domains including information retrieval, data mining, computer vision, natural language processing, financial analysis, robotics, medical diagnosis, …….. The list goes on and on.

In addition to the theoretical topics covered by the course, it also includes 3 mini projects and one final project in Python programming language.  The programming assignments and project were a strong reason for me to decide doing this course :)

Course Requirements

The course assumes good knowledge of calculus and being comfortable with python programming language. No prior knowledge of probability is required.

Week 1

What is probability?

Probability is the science of representing and computing with uncertainty. It appears in our everyday life, especially when making decisions. For example, In the morning you may decide to take an umbrella because there is high chance that it will rain today.

The goal of the course is to learn how to build computer programs that can perform reasoning under uncertainties by using probabilities.

Simplest example of probability is flipping a fair coin, fair coin is a coin that has equal chance of landing heads or tails when being tossed, the probability of head = ½ and the probability of tail is also ½.  But what does this mean ?

There is two interpretations for probability:

Frequentist interpretation: If you repeat the process of tossing the coin for N times (where N is a very large number, say 10,000 times), Approximately N/2 times the result will be heads and approximately N/2 times the result will be tails.

Bayesian Interpretation: It is hard to use the frequentist approach to explain the meaning of probability in some scenarios where the experiment cannot be repeated. For example, the probability that the a patient will die after being given a certain drug. Of course, this experiment cannot be repeated more than one time. Therefore, the frequentist approach does not help a lot in this scenario. The bayesian interpretation of probability is the that probability value is equal to the state of belief in the experiment outcome.
For example, the probability of heads when tossing a coin = ½ means that if you toss a coin once, then before tossing the coin your belief that the result will be heads is equal to ½.

Luckily, it does not make any difference which interpretation you choose to follow, because all probability laws are the same under the two interpretations.

Two ingredients to model uncertainty

When we represent an uncertain world, we call the process that we observe its result an experiment. To model the uncertainty of the experiment we need to define two things:

  • Sample Space : The set of all possible outcomes of the experiment. The sample space can be either finite or infinite set.
  • Probability of each outcome: Define a probability function that assigns a probability value for each possible outcome .
For example, in the experiment of tossing a fair coin:
  • Sample space:
  • Probability values:

Any subset of the sample Space is called an Event.

The probability of an event   is the sum of probabilities of the possible outcomes that belong to .

Programming Note: representing a probability model

The course recommends using the Python dictionary to represent a probability model, where the keys of the dictionary are the elements of the sample space and the values assigned to them are their probability values.
Example: representing the probability model of a fair coin.
model = {'heads' : 0.5, 'tails' : 0.5}

The following can be used to sample 10 outputs from the preceding model

import numpy as np

items, p_values = zip(*model.items())
samples = np.random.choice(a = items, size = 10, p = p_values)

Three axioms of probability

The probability value assignment has to comply with the following three rules (known as three axioms of probability)
  • If A and B are disjoint events ( ) , then

Random Variables

Random variables, strangely enough despite their name, are functions that map the experiment outcomes to another set of  values.
The course uses the python dictionary to represent a random variable that maps a finite sample space to another finite set of values.

weather_model = {'sunny': 0.7, 'rainy' : 0.25, 'snowy': 0.05}
random_variable_mapping = {'sunny': 1, 'rainy' : 1, 'snowy': 0 }


  1. +1 I hope you keep going. I'm in class but it will take every resource I can find to make it through :)

  2. Get guidance for your dream career | Fair & Lovely Foundation

    Dream of becoming a School teacher, Hair Stylist or Software Programmer? View over 61 Career Options and learn what you need to study.
    Get Free Career Guidance here.

  3. nice Blogs
    Great Information.
    VLCC VANITYCUBE Beauty At Home Beauty & Wellness Services, offers a wide range of beauty services at your doorstep.

  4. Thanks for sharing is so amazing and helpful to us.
    Buy Hydrocodone online


  5. Best Article buy Pain Pills online Excellent post. I appreciate this site. Stick with it! Because the admin of this web page is working, no doubt very quickly it will be well-known, due to its quality contents.This website was how do you say it? Relevant!! Finally, I’ve found something that helped me.
    Best Article buy Roxicodone online Excellent post
    buy Xanax online
    buy Oxycodone online

    Best Article buy Pain Medications online Excellent post. I appreciate this site. Stick with it! Because the admin of this web page is working, no doubt very quickly it will be well-known, due to its quality contents.This website was how do you say it? Relevant!! Finally, I’ve found something that helped me.

    buy Research Chemicals online

    buy Roxicodone online

    buy Cbd Isolate online

  6. Thanks a lot for sharing
    Having good health is what most people out there wants but can not achieve. some people takes buy ibogaine online AND buy weed online to get it.


  7. Welcome to Scottish Fold Cattery and thanks for your interest click here to Adopt A Scottish Fold Kitten online to our scottish cats and kittens For sale! My name is VERA S. Matt. Me and my husband Owns the small scottish fold farm with the best Scottish fold kitten prices and share my biggest hobby/ Scottish Fold Kittens For Sale cat breeding hence scottish fold Breeders. We breed breeds Scottish Folds. All our Scottish Fold cats have very special Grand Champion blood lines from Europe . We love our cats, they live in the Cattery with us.Scottish Folds are known to be very good natured, affectionate cats who are very human-orientated and become extremely attached to their Scottish Fold Cats They are easy going and will appreciate the company and cubby holes of any type. They enjoy attention and are not usually shy with strangers. Affectionate without being “pushy”,they make ideal pets and companions.scottish cats and kittens For saleBrowse through our available scottish fold cats and see the Munchkin Scottish folds,Grey Scottish fold kittens,Black scottish fold kittens Since we offer shipping worldwide.Scottish Fold kittens breederKittens For Sale at the catter.Scottish fold kitten prices We also offer Scottish fold kittens near me,Scottish Fold Breeders,Scottish fold Price.
    Buy passport online We are a group of unique experts who manufacture real and fake passports, IDs, buy driver’s licenses and other documents proving your identity or qualification. We have many years of experience in this field that’s why you can be absolutely sure that no matter what you order from us, you will always receive a high-quality document that meets the necessary…buy IELTS certificate online
    Buy passport online
    buy registered documents online

  8. Despite the fact that Adderall is viewed as a physician recommended tranquilize and most patients
    buy adderall online,Buy Adderall Uk & USA on the web or purchase Adderall 30mg from an online drug store, a few patients will in general get dependent on the medication.can you buy adderal online? When it comes to buying fine research chemicals online, quality is the key. buychemstore research chemicals for sale ,LSD for sale,constantly strives at providing you with the continuing evaluation, innovation . We assure you the highest degree of confidence in all of our products not limited to but including cocaine for sale, LSD for sale, heroin for sale, pain medications for sale etc. Here at buychemstore, we pride ourselves on safety.
    buy research chemicals online
    We are a one-stop-shop for the purchase of anabolic steroids.what is anabolic steroids, We are here to make sure you don’t bother about where to buy anabolic steroids from craigslist, eBay, or any other online forums. We are in contract with a large network of pharmacies in North America, Europe and Asia. With our years of experience in the distribution of anabolic-androgenic steroids, we provide top quality services to our clients that cannot be matched by our competitors.This is te best place to Buy legal steroids online, anabolic steroids for sale
    buy Apetamin vitamin syrup that’s marketed as a weight gain supplement. It was developed by TIL Healthcare PVT, a pharmaceutical company based in India.According to manufacturing labels, 1 teaspoon (5 ml) of Apetamin syrup contains.Apetamin in store,where to buy Apetamin,Is Apetamin syrup Effective in weight gain,buy Apetamin syrup online
    Welcom to our vyvanse online shop where you can buy vyvanse online,Buy vyvanse Uk & USA,learn about vyvanse side effects and have
    vyvanse coupon,vyvanse for sale.
    Buy atorvastatin, sold under the trade name Lipitor among others, is a statin medication used to prevent cardiovascular disease in those at high risk and treat abnormal lipid levels. For the prevention of cardiovascular disease, statins are a first-line treatment. It is taken by mouth.Learn How to use Lipitor, you are free to read about atorvastatin side effects,Buy lipitor UK ,atorvastatin side effects ,
    lipitor for sale. You can also buy juul

  9. dank vapes are the best brand of vape cartridges in the market. These dank vapes cartridges have however suffered a backlash in the recent weeks with multiple reports on how these dank vapes carts cause cancer and other related lung disease.
    dank vapes cart however remain one of the best cartridges. dank vapes are sold in multiple states over the USA. dank vapes cart compost one of the black markets finest vape products. But this backlash has caused a fall in the consumption of the vape by adults.The 710 Kingpen brand is also on the rise in market demands as well as skywalker OG products.
    Recently juul pods also faced a similar backlash. juul starter kits have also fallen in market demand. mango juul pods are refillable juul pods and can be gotten as juul pods bulk. juul skins and juul accessories all come with the juul kits.
    buy juul online at cheap prices buy cocaine online
    buy Keytruda online and other cancer medications such as buy avastin online , buy Herceptin online