Saturday, September 17, 2016

EDX 6.008.1X Computational Probability and Inference Notes (Week 1)

EDX 6.008.1x Computational Probability and Inference Notes (Week 1)

Recently, I started doing the EDX 6.008.1x Computational Probability and Inference online course on EDX. This course is offered as part of the MITx and is similar to the MIT 6.008 Introduction to Inference course offered at MIT by Prof. Polina Golland. As a way to motivate myself keeping on track with the course, I decided to save my notes on the course content and publish them online. I hope you they can be useful for other students who are doing the course now, or in later sessions, and for anyone looking for information about topics related to the course subjects.

Course topics:

The course provides an introduction to probability theory and probabilistic graphical models and how to use to perform inferences. Probabilistic models are used in different application domains including information retrieval, data mining, computer vision, natural language processing, financial analysis, robotics, medical diagnosis, …….. The list goes on and on.

In addition to the theoretical topics covered by the course, it also includes 3 mini projects and one final project in Python programming language.  The programming assignments and project were a strong reason for me to decide doing this course :)

Course Requirements

The course assumes good knowledge of calculus and being comfortable with python programming language. No prior knowledge of probability is required.

Week 1


What is probability?


Probability is the science of representing and computing with uncertainty. It appears in our everyday life, especially when making decisions. For example, In the morning you may decide to take an umbrella because there is high chance that it will rain today.

The goal of the course is to learn how to build computer programs that can perform reasoning under uncertainties by using probabilities.

Simplest example of probability is flipping a fair coin, fair coin is a coin that has equal chance of landing heads or tails when being tossed, the probability of head = ½ and the probability of tail is also ½.  But what does this mean ?

There is two interpretations for probability:

Frequentist interpretation: If you repeat the process of tossing the coin for N times (where N is a very large number, say 10,000 times), Approximately N/2 times the result will be heads and approximately N/2 times the result will be tails.

Bayesian Interpretation: It is hard to use the frequentist approach to explain the meaning of probability in some scenarios where the experiment cannot be repeated. For example, the probability that the a patient will die after being given a certain drug. Of course, this experiment cannot be repeated more than one time. Therefore, the frequentist approach does not help a lot in this scenario. The bayesian interpretation of probability is the that probability value is equal to the state of belief in the experiment outcome.
For example, the probability of heads when tossing a coin = ½ means that if you toss a coin once, then before tossing the coin your belief that the result will be heads is equal to ½.

Luckily, it does not make any difference which interpretation you choose to follow, because all probability laws are the same under the two interpretations.

Two ingredients to model uncertainty

When we represent an uncertain world, we call the process that we observe its result an experiment. To model the uncertainty of the experiment we need to define two things:

  • Sample Space : The set of all possible outcomes of the experiment. The sample space can be either finite or infinite set.
  • Probability of each outcome: Define a probability function that assigns a probability value for each possible outcome .
For example, in the experiment of tossing a fair coin:
  • Sample space:
  • Probability values:


Any subset of the sample Space is called an Event.

The probability of an event   is the sum of probabilities of the possible outcomes that belong to .

Programming Note: representing a probability model


The course recommends using the Python dictionary to represent a probability model, where the keys of the dictionary are the elements of the sample space and the values assigned to them are their probability values.
Example: representing the probability model of a fair coin.
model = {'heads' : 0.5, 'tails' : 0.5}

The following can be used to sample 10 outputs from the preceding model

import numpy as np

items, p_values = zip(*model.items())
samples = np.random.choice(a = items, size = 10, p = p_values)

Three axioms of probability

The probability value assignment has to comply with the following three rules (known as three axioms of probability)
  • If A and B are disjoint events ( ) , then

Random Variables


Random variables, strangely enough despite their name, are functions that map the experiment outcomes to another set of  values.
The course uses the python dictionary to represent a random variable that maps a finite sample space to another finite set of values.

weather_model = {'sunny': 0.7, 'rainy' : 0.25, 'snowy': 0.05}
random_variable_mapping = {'sunny': 1, 'rainy' : 1, 'snowy': 0 }

21 comments:

  1. +1 I hope you keep going. I'm in class but it will take every resource I can find to make it through :)

    ReplyDelete
  2. Get guidance for your dream career | Fair & Lovely Foundation

    Dream of becoming a School teacher, Hair Stylist or Software Programmer? View over 61 Career Options and learn what you need to study.
    Get Free Career Guidance here.

    ReplyDelete
  3. nice Blogs
    Great Information.
    VLCC VANITYCUBE Beauty At Home Beauty & Wellness Services, offers a wide range of beauty services at your doorstep.

    ReplyDelete
  4. Thanks for sharing is so amazing and helpful to us.
    Buy Hydrocodone online

    ReplyDelete

  5. Best Article buy Pain Pills online Excellent post. I appreciate this site. Stick with it! Because the admin of this web page is working, no doubt very quickly it will be well-known, due to its quality contents.This website was how do you say it? Relevant!! Finally, I’ve found something that helped me.
    Best Article buy Roxicodone online Excellent post
    buy Xanax online
    buy Oxycodone online

    Best Article buy Pain Medications online Excellent post. I appreciate this site. Stick with it! Because the admin of this web page is working, no doubt very quickly it will be well-known, due to its quality contents.This website was how do you say it? Relevant!! Finally, I’ve found something that helped me.

    buy Research Chemicals online

    buy Roxicodone online

    buy Cbd Isolate online

    ReplyDelete
  6. Thanks a lot for sharing
    Having good health is what most people out there wants but can not achieve. some people takes buy ibogaine online AND buy weed online to get it.

    ReplyDelete

  7. Welcome to Scottish Fold Cattery and thanks for your interest click here to Adopt A Scottish Fold Kitten online to our scottish cats and kittens For sale! My name is VERA S. Matt. Me and my husband Owns the small scottish fold farm with the best Scottish fold kitten prices and share my biggest hobby/ Scottish Fold Kittens For Sale cat breeding hence scottish fold Breeders. We breed breeds Scottish Folds. All our Scottish Fold cats have very special Grand Champion blood lines from Europe . We love our cats, they live in the Cattery with us.Scottish Folds are known to be very good natured, affectionate cats who are very human-orientated and become extremely attached to their owners.here Scottish Fold Cats They are easy going and will appreciate the company and cubby holes of any type. They enjoy attention and are not usually shy with strangers. Affectionate without being “pushy”,they make ideal pets and companions.scottish cats and kittens For saleBrowse through our available scottish fold cats and see the Munchkin Scottish folds,Grey Scottish fold kittens,Black scottish fold kittens Since we offer shipping worldwide.Scottish Fold kittens breederKittens For Sale at the catter.Scottish fold kitten prices We also offer Scottish fold kittens near me,Scottish Fold Breeders,Scottish fold Price.
    Buy passport online We are a group of unique experts who manufacture real and fake passports, IDs, buy driver’s licenses and other documents proving your identity or qualification. We have many years of experience in this field that’s why you can be absolutely sure that no matter what you order from us, you will always receive a high-quality document that meets the necessary…buy IELTS certificate online
    Buy passport online
    buy registered documents online

    ReplyDelete
  8. Despite the fact that Adderall is viewed as a physician recommended tranquilize and most patients
    buy adderall online,Buy Adderall Uk & USA on the web or purchase Adderall 30mg from an online drug store, a few patients will in general get dependent on the medication.can you buy adderal online? When it comes to buying fine research chemicals online, quality is the key. buychemstore research chemicals for sale ,LSD for sale,constantly strives at providing you with the continuing evaluation, innovation . We assure you the highest degree of confidence in all of our products not limited to but including cocaine for sale, LSD for sale, heroin for sale, pain medications for sale etc. Here at buychemstore, we pride ourselves on safety.
    buy research chemicals online
    We are a one-stop-shop for the purchase of anabolic steroids.what is anabolic steroids, We are here to make sure you don’t bother about where to buy anabolic steroids from craigslist, eBay, or any other online forums. We are in contract with a large network of pharmacies in North America, Europe and Asia. With our years of experience in the distribution of anabolic-androgenic steroids, we provide top quality services to our clients that cannot be matched by our competitors.This is te best place to Buy legal steroids online, anabolic steroids for sale
    buy Apetamin vitamin syrup that’s marketed as a weight gain supplement. It was developed by TIL Healthcare PVT, a pharmaceutical company based in India.According to manufacturing labels, 1 teaspoon (5 ml) of Apetamin syrup contains.Apetamin in store,where to buy Apetamin,Is Apetamin syrup Effective in weight gain,buy Apetamin syrup online
    Welcom to our vyvanse online shop where you can buy vyvanse online,Buy vyvanse Uk & USA,learn about vyvanse side effects and have
    vyvanse coupon,vyvanse for sale.
    Buy atorvastatin, sold under the trade name Lipitor among others, is a statin medication used to prevent cardiovascular disease in those at high risk and treat abnormal lipid levels. For the prevention of cardiovascular disease, statins are a first-line treatment. It is taken by mouth.Learn How to use Lipitor, you are free to read about atorvastatin side effects,Buy lipitor UK ,atorvastatin side effects ,
    lipitor for sale. You can also buy juul

    ReplyDelete
  9. dank vapes are the best brand of vape cartridges in the market. These dank vapes cartridges have however suffered a backlash in the recent weeks with multiple reports on how these dank vapes carts cause cancer and other related lung disease.
    dank vapes cart however remain one of the best cartridges. dank vapes are sold in multiple states over the USA. dank vapes cart compost one of the black markets finest vape products. But this backlash has caused a fall in the consumption of the vape by adults.The 710 Kingpen brand is also on the rise in market demands as well as skywalker OG products.
    Recently juul pods also faced a similar backlash. juul starter kits have also fallen in market demand. mango juul pods are refillable juul pods and can be gotten as juul pods bulk. juul skins and juul accessories all come with the juul kits.
    buy juul online at cheap prices buy cocaine online
    buy Keytruda online and other cancer medications such as buy avastin online , buy Herceptin online

    ReplyDelete
  10. Welcome to Ragdoll Kittens Cattery click here to adopt a ragdoll

    kitten online
    We are a small and loving cattery . We are pleased that you have chosen to visit our Ragdoll

    cats/Ragdoll kittens cattery, and hope you will notice right away from our website how beautiful and loved our

    Ragdoll cats and kittens are. These beauties are easily integrated into our loving family home, first as

    mothers carrying the litters, and then from the time the ragdoll kittens are born until they are adopted so we

    always have Ragdoll kittens for sale|Ragdoll kittens for adoption|Ragdoll kitten price|Ragdolls|Cost of Ragdoll

    kittens|. Our adult cats have tested negative for HCM and PKD1 through University of California Davis. Upon

    request, we have five generations of pedigree documentation on our adults available to anyone who is

    interested. Ragdoll kittens are registered with The International Cat Association (RAGDOLL KITTENS FOR SALE),and are never caged. The cats that are

    in our reputable breeding program can produce mink, sepia and traditional Ragdoll kittens. Ragdolls have a

    laid-back personality and enjoy being physically handled making them one of the best lap cats! We are all

    family here at Ragdoll kittens cattery since we are ragdoll kitten breeders and all the love we bestow by

    holding each cat and kitten daily further Teacup RAGDOLL Kittens

    for sale|Hypoallergenic kittens for sale
    nurtures their loving personalities and temperaments,TICA/CFA REGISTERED RAGDOLL KITTENS FOR SALE thanks for adopting a

    ragdoll ketten from us

    Welcome to our farm where we breed Registered pomeranian puppies

    for sale
    .As a registered pomeranian puppies breeder, we have made it possible for pomeranian puppy lovers

    to
    buy pomeranian puppies online,buy zwergpitz pomeranian from our family run farm. Pomeranian

    dogs are small dogs with a weight of 1.36 to 3.17 kg and a withers height of 15 to 18 cm. They are compact but

    robust dogs with a lush, textured coat and a tall and flat tail. The top coat forms a fur ruff on the neck, for

    which poms are known, and on the hindquarters they have a margin of feathered hair.You can click HERE to view our available pomeranian puppies

    ReplyDelete

  11. Welcome to Scottish Fold Cattery and thanks for your interest click here to Adopt A Scottish Fold Kitten online to our scottish cats and kittens For sale! My name is VERA S. Matt. Me and my husband Owns the small scottish fold farm with the best Scottish fold kitten prices and share my biggest hobby/ Scottish Fold Kittens For Sale cat breeding hence scottish fold Breeders. We breed breeds Scottish Folds. All our Scottish Fold cats have very special Grand Champion blood lines from Europe . We love our cats, they live in the Cattery with us.Scottish Folds are known to be very good natured, affectionate cats who are very human-orientated and become extremely attached to their owners.here Scottish Fold Cats They are easy going and will appreciate the company and cubby holes of any type. They enjoy attention and are not usually shy with strangers. Affectionate without being “pushy”,they make ideal pets and companions.scottish cats and kittens For saleBrowse through our available scottish fold cats and see the Munchkin Scottish folds,Grey Scottish fold kittens,Black scottish fold kittens Since we offer shipping worldwide.Scottish Fold kittens breederKittens For Sale at the catter.Scottish fold kitten prices We also offer Scottish fold kittens near me,Scottish Fold Breeders,Scottish fold Price.
    Buy passport online We are a group of unique experts who manufacture real and fake passports, IDs, buy driver’s licenses and other documents proving your identity or qualification. We have many years of experience in this field that’s why you can be absolutely sure that no matter what you order from us, you will always receive a high-quality document that meets the necessary…buy IELTS certificate online
    Buy passport online
    buy registered documents online

    WELCOME TO pug puppies BREEDERS HOME BUY A PUG ONLINE|PUG PUPPIES FOR SALE|ADOPT A PUG ONLINE|PUG|BLACK PUG PUPPIES FOR SALE
    We are so glad you stopped by to check out our amazing TEACUP Pug puppies. All of our animals come fully up to date on vaccinations, heart-worm tested negative, and de-wormed, as well as neutered, and treated for any ailments found upon veterinary examination Adopt a teacup Pug puppy
    Ownership of a Pug includes a commitment that is wider in scope than providing doggy necessities and allowing this jovial little pet to occupy a portion of your home. Pug ownership is a give-and-take relationship that will continue throughout the little dog’s life. That enjoyable companionship thrives when each member of the pair respects the other and camaraderie rules the union. Your Pug will entertain you, love you, and be obedient and faithful to you if you spend time with your doggy friend.
    Adopt a pug puppies online puppy onlie or pug puppies for sale online does not all depends on the money involve. You have to love
    Black pug puppies for sale at heart before you go in search of pug puppies for sale or adopt a pug puppy.This means that you have to keep the pug puppy price or cost of a pug puppy or a teacup puppy in mind before you harnest the interest of buying a pug online.

    ReplyDelete
  12. medical care is what many people lack in the interior areas of the world. people face a lot of health issues everyday without solution. buy psychedelics online, Buy weed online, Buy fantanyl patches online

    ReplyDelete
  13. Nowadays, getting a doll from a Sex Dolls Shop discreetly has been quite a challenge.
    Big Boobs Sex Dolls for Sale Online is not a common phenomenon in the industry.
    Fantasy and Anime Sex Dolls for Sale are now the official leading trend in the dolls industry. Similar to buying from a
    Discreet Guns Shop

    ReplyDelete