Even though it has an ornate name, the underlying concept is actually quite simple and intuitive. The concept of Empirical Risk Minimization becomes relevant in the world of supervised learning. The actual goal of supervised learning is to find a model that solves a problem as opposed to finding a model that best fits the … Continue reading What is Empirical Risk Minimization
Tag: Distribution
What Is Pareto Optimality
Let's consider a business deal where there are multiple parties negotiating the terms. In such a situation, it's usually not possible for every single party to get everything it wants. They need to optimize their demands so that everyone comes out with something positive. Similar situations arise across many areas of engineering where we have to … Continue reading What Is Pareto Optimality
What Is Relative Entropy?
In this blog post, we will be using a bit of background from my previous blog post. If you are familiar with the basics of entropy coding, you should be fine. If not, you may want to quickly read through my previous blog post. So coming to the topic at hand, let's continue our discussion … Continue reading What Is Relative Entropy?
What Is Entropy Coding?
Entropy Coding appears everywhere in modern digital systems. It is a fundamental building block of data compression, and data compression is pretty much needed everywhere, especially for internet, video, audio, communication, etc. Let's consider the following scenario. You have a stream of English alphabets coming in and you want to store them in the best … Continue reading What Is Entropy Coding?
What Are Confidence Intervals?
Confidence interval is a concept in statistics that is used extensively in many diverse areas like physics, chemistry, computer vision, machine learning, genetics, etc. This concept is so fundamental that any modern science would eventually end up using it. Let's say you have collected some data and you want to understand the behavior of that … Continue reading What Are Confidence Intervals?
What Are P-Values?
Let's say you are a part of the sub-atomic physics team and you are working on discovering an important effect. The thing about sub-atomic physics is that nothing is certain and you cannot say something has happened with 100% certainty. The best we can do is to say that we are x-percent sure that something interesting happened. … Continue reading What Are P-Values?
Gaussian Mixture Models
Let's say you have a lot of data and you want to estimate the underlying statistical model. Wait a minute, why on earth would I care about that? Well, if you estimate the model, then you can analyze unknown data that is not under our control. Some of the common examples would be weather estimation, … Continue reading Gaussian Mixture Models
Expectation Maximization
Probabilistic models are commonly used to model various forms of data, including physical, biological, seismic, etc. Much of their popularity can be attributed to the existence of efficient and robust procedures for learning parameters from observations. Often, however, the only data available for training a probabilistic model are incomplete. Missing values can occur which will … Continue reading Expectation Maximization
Reimann Hypothesis And Its Connection To Cryptography
Over the centuries, mathematicians have been involved in solving some of most complex problems. But what is the motivation behind that? The pursuit of truth! But The Clay Mathematics Institute thought that there should be a little more than that. So to celebrate mathematics in the new millennium, they established seven Millennium Prize Problems. The … Continue reading Reimann Hypothesis And Its Connection To Cryptography
The Butterfly Effect
This blog post is a continuation of my previous post on Chaos Theory. Although it is not required for you to read that post to understand this post, it would be better if you glance through it once. All of us have heard about the Butterfly Effect. It is one of the very famous examples … Continue reading The Butterfly Effect
Perpetual Enigma 