Even though it has an ornate name, the underlying concept is actually quite simple and intuitive. The concept of Empirical Risk Minimization becomes relevant in the world of supervised learning. The actual goal of supervised learning is to find a model that solves a problem as opposed to finding a model that best fits the … Continue reading What is Empirical Risk Minimization
Tag: Probability
What Is Monte Carlo Simulation
There are many phenomena in everyday life where it's very difficult to model the problem. There are so many variables and so many dependencies that any approximation or assumption would lead to a huge errors in outputs. This is usually a combination of uncertainty and variability. Even though we have access to all the historical information, … Continue reading What Is Monte Carlo Simulation
How To Compute Confidence Measure For SVM Classifiers
Support Vector Machines are machine learning models that are used to classify data. Let's say you want to build a system that can automatically identify if the input image contains a given object. For ease of understanding, let's limit the discussion to three different types of objects i.e. chair, laptop, and refrigerator. To build this, … Continue reading How To Compute Confidence Measure For SVM Classifiers
What Is Relative Entropy?
In this blog post, we will be using a bit of background from my previous blog post. If you are familiar with the basics of entropy coding, you should be fine. If not, you may want to quickly read through my previous blog post. So coming to the topic at hand, let's continue our discussion … Continue reading What Is Relative Entropy?
What Is Entropy Coding?
Entropy Coding appears everywhere in modern digital systems. It is a fundamental building block of data compression, and data compression is pretty much needed everywhere, especially for internet, video, audio, communication, etc. Let's consider the following scenario. You have a stream of English alphabets coming in and you want to store them in the best … Continue reading What Is Entropy Coding?
What Is A Markov Chain?
If you have studied probability theory, then you must have heard Markov's name. When we study probability and statistics, we tend to deal with independent trials. What this means is that if you conduct an experiment a lot of times, we assume that the outcome of one trial doesn't influence the outcome of the next … Continue reading What Is A Markov Chain?
What Is Maximum Likelihood Estimation?
Let's say you are trying to estimate the height of a group of people somewhere. If the group is small enough, you can just measure all of them and be done with it. But in real life, the groups are pretty large and you cannot measure each and every person. So we end up having … Continue reading What Is Maximum Likelihood Estimation?
What Are Confidence Intervals?
Confidence interval is a concept in statistics that is used extensively in many diverse areas like physics, chemistry, computer vision, machine learning, genetics, etc. This concept is so fundamental that any modern science would eventually end up using it. Let's say you have collected some data and you want to understand the behavior of that … Continue reading What Are Confidence Intervals?
What Are P-Values?
Let's say you are a part of the sub-atomic physics team and you are working on discovering an important effect. The thing about sub-atomic physics is that nothing is certain and you cannot say something has happened with 100% certainty. The best we can do is to say that we are x-percent sure that something interesting happened. … Continue reading What Are P-Values?
What Is Random Walk?
Consider the following situation. We have a drunkard who is clinging to a lamppost, and now he decides to start walking. He is in the middle of the street and the road runs from east to west. In his inebriated state, he is as likely to take a step towards the east as he is … Continue reading What Is Random Walk?
Perpetual Enigma 