Basics of Bayesian Network

Introduction

There is innumerable text available in the net on Bayesian Network, but most of them are have heavy mathematical formulas and concepts thus quite difficult to understand. Here, I have tried to explain the topic as simple as possible with minimum equations and a real-world example.

Basic Concept

Bayesian Network is a very important tool in understanding the dependency among events and assigning probabilities to them thus ascertaining how probable or what is the change of occurrence of one event given the other. For example, suppose you are getting scolded at school by your teacher for being late and there could be many reasons for being late like waking up late, traffic jams, etc. So here, scolding is dependent on the events like waking up late or traffic jam i.e., these reasons have a direct influence on you being scolded. This can be efficiently represented using Bayesian Network which we will see soon.

Fig. 1

Let {X1, X2,…….Xn} be some events just like being scolded, waking up late etc. In Bayesian Network, they can be represented as nodes. Now if a node has some dependency on another node then an arrow/arc is drawn from one node to another as shown in fig.1. It is interpreted as the child node’s occurrence is influenced by the occurrence of its parent node. So Bayesian Network represents a directed acyclic graph(DAG). It is not allowed to form cyclical patterns within nodes so it is acyclic.

Fig. 2: The Formula for Joint Probability Distribution

Now using Conditional probability and chain rule, we can easily get the full joint distribution i.e., the probability of the final event given all other dependent events. (Note: If you are unsure of conditional probability and chain rules, follow these lectures. They are the extensive resources to learn probability and Bayesian Statistics.) From the equation, we can see the probability of child nodes depends on its immediate parents. Note: π denotes product sign.

Using the above formulas we can derive the joint probability distribution formula of the below models.

Fig. 3

So the results of the joint probability estimation of the above-given networks can be easily found using the formula given in Fig.2.

Real-World Example of Bayesian Network Model

Now let us wrap up the tutorial with a real-world example for better understanding.

Fig. 4

Let, P(A) denotes the event that you wake up late in the morning.

P(B) denotes the event you are late for school.

P(C) denotes the event you are scolded by your teacher.

P(D) denotes the event there is a traffic jam on the road

P(E) denotes the event your father boards the public bus every morning.

P(F) denotes the event your father goes to office.

So what is the probability that you are scolded at school? Now, you can easily find that out using the formula. So the final result becomes P(C|B).P(B|A,D).P(D).P(A)

Hey, but wait!!! What about P(E) and P(F) ??? They are also present there in the model. Remember, two events which are not connected with each other with arcs directly or indirectly given any other node are called independent events and occurrence of one has no effect on the occurrence of another. We can safely predict from the model the value of P(C|F) i.e., the probability you being scolded because of the event that your father is going to office is 0.

The probability value for each node given their parent node is represented in the form of a table called Conditional Probability Table(CPT). The below example is a representation of CPT.

Fig. 5: Conditional Probability Table (Creator: Muhammad Nazim Razali)

Bayesian Network has a huge application in the real world. Given a symptom, a Bayesian Network can predict the probability of a particular disease causing the symptoms. Also by taking the evidence collected from two crime scenes, Bayesian Network can give the investigation team valuable insights on whether the criminals at two places are related or not.

Some MOOCs to explore more on Bayesian Networks:

1. Probabilistic Graphical Model by Stanford University

2. Bayesian Methods for Machine Learning by National Research University Higher School of Economics

3. Bayesian Statistics: From Concept to Data Analysis by University of California, Santa Cruz