blank

a post with table of contents on a sidebar

2023-04-25T14:14:00+00:00

This post shows how to add a table of contents as a sidebar.

Adding a Table of Contents

To add a table of contents to a post as a sidebar, simply add

toc:
  sidebar: left

to the front matter of the post. The table of contents will be automatically generated from the headings in the post. If you wish to display the sidebar to the right, simply change left to right.

Example of Sub-Heading 1

Jean shorts raw denim Vice normcore, art party High Life PBR skateboard stumptown vinyl kitsch. Four loko meh 8-bit, tousled banh mi tilde forage Schlitz dreamcatcher twee 3 wolf moon. Chambray asymmetrical paleo salvia, sartorial umami four loko master cleanse drinking vinegar brunch. Pinterest DIY authentic Schlitz, hoodie Intelligentsia butcher trust fund brunch shabby chic Kickstarter forage flexitarian. Direct trade cold-pressed meggings stumptown plaid, pop-up taxidermy. Hoodie XOXO fingerstache scenester Echo Park. Plaid ugh Wes Anderson, freegan pug selvage fanny pack leggings pickled food truck DIY irony Banksy.

Example of another Sub-Heading 1

Customizing Your Table of Contents

If you want to learn more about how to customize the table of contents of your sidebar, you can check the bootstrap-toc documentation. Notice that you can even customize the text of the heading that will be displayed on the sidebar.

Example of Sub-Heading 2

Example of another Sub-Heading 2

displaying beautiful tables with Bootstrap Tables

2023-03-20T18:37:00+00:00

Using markdown to display tables is easy. Just use the following syntax:

| Left aligned | Center aligned | Right aligned |
| :----------- | :------------: | ------------: |
| Left 1       | center 1       | right 1       |
| Left 2       | center 2       | right 2       |
| Left 3       | center 3       | right 3       |

That will generate:

Left aligned	Center aligned	Right aligned
Left 1	center 1	right 1
Left 2	center 2	right 2
Left 3	center 3	right 3

It is also possible to use HTML to display tables. For example, the following HTML code will display a table with Bootstrap Table, loaded from a JSON file:

  id="table"
  data-toggle="table"
  data-url="{{ '/assets/json/table_data.json' | relative_url }}">
  
    
       data-field="id">ID
       data-field="name">Item Name
       data-field="price">Item Price
    
  

ID	Item Name	Item Price

By using Bootstrap Table it is possible to create pretty complex tables, with pagination, search, and more. For example, the following HTML code will display a table, loaded from a JSON file, with pagination, search, checkboxes, and header/content alignment. For more information, check the documentation.

  data-click-to-select="true"
  data-height="460"
  data-pagination="true"
  data-search="true"
  data-toggle="table"
  data-url="{{ '/assets/json/table_data.json' | relative_url }}">
  
    
       data-checkbox="true">
       data-field="id" data-halign="left" data-align="center" data-sortable="true">ID
       data-field="name" data-halign="center" data-align="right" data-sortable="true">Item Name
       data-field="price" data-halign="right" data-align="left" data-sortable="true">Item Price
    
  

	ID	Item Name	Item Price

a post with table of contents

2023-03-20T15:59:00+00:00

This post shows how to add a table of contents in the beginning of the post.

Adding a Table of Contents

To add a table of contents to a post, simply add

toc:
  beginning: true

to the front matter of the post. The table of contents will be automatically generated from the headings in the post.

Example of Sub-Heading 1

Example of another Sub-Heading 1

Table of Contents Options

If you want to learn more about how to customize the table of contents, you can check the jekyll-toc repository.

Example of Sub-Heading 2

Example of another Sub-Heading 2

a post with giscus comments

2022-12-10T15:59:00+00:00

This post shows how to add GISCUS comments.

Displaying External Posts on Your al-folio Blog

2022-04-23T23:20:09+00:00

a post with redirect

2022-02-01T17:39:00+00:00

you can also redirect to assets like pdf

Cross Entropy Demistified

2021-07-15T15:09:00+00:00

Understanding “Entropy”, “Cross-Entropy Loss” and “KL Divergence”

Classification is one of the preliminary steps in most Machine Learning and Deep learning projects and Cross-Entropy Loss is the most commonly used loss function, but have you ever tried to explore what is cross-entropy or entropy exactly.

Ever imagined why cross-entropy works for classification?

This series of articles consisting of 2 parts is designed to explain in detail the intuition behind what is cross-entropy and why cross-entropy has been used as the most popular cost function for classification.

Before diving into cross-entropy loss and its application to classification, the concept of entropy and cross-entropy must be clear, so this article is dedicated to exploring what these 2 terms mean.

The content and images in this post are inspired by the amazing tutorial A Short Introduction to Entropy, Cross-Entropy and KL-Divergence by Aurélien Géron

Entropy

The concept of entropy and cross-entropy comes from Claude Shannon’s Information theory which he introduces through his classical paper “A Mathematical Theory of Communication”

According to Shannon, Entropy is the minimum no of useful bits required to transfer information from a sender to a receiver.

Let’s understand the two terms by looking into an example.

Suppose that we need to share the weather information of a place with another friend who stays in a different city, and the weather has a 50–50 chance of being sunny or rainy every day.

This information can be transmitted using just a single bit (0 or 1) and the uncertainty associated with this event is 2 as there are 2 possibilities, either weather is sunny or rainy.

If the probability of occurrence of an event is $p$, then the uncertainty raised due to that event is given as $\frac{1}{p}$

In our example, the probability of occurrence of both events is 0.5 so the uncertainty for each event is $\frac{1}{0.5} = 2$

Even if the information is transferred as a string “RAINY” having 5 characters each of 1 byte, the total information transferred is 40 bits but only 1 bit of useful information is transferred.

Given the uncertainty due to an event is $N$, the minimum number of bits required to transfer the information about that event can be calculated as $log(N)$

Here, as uncertainty for weather being rainy or sunny is 2, the minimum no. of useful bits required to transfer information about being sunny or rainy is $log(2) = 1$

Note : In the article, $log(x)$ means logarithm with base $2$ and $ln(x)$ means natural logarithm with base $e$

Now suppose that the event “Weather” had 8 possibilities, all equally likely with $12.5\%$ probability of occurrence of each.

So now as the no. of uncertainties is 8, the minimum no. of useful bits required to transfer information about each event can be calculated as $log(8) = 3$

Let us consider a case which is similar to the 1st case that we saw with 2 possibilities, sunny or rainy, but now both are not equally likely. One occurs with a probability of $75\%$ and the other with a probability of $25\%$.

Now the events are not occuring with equal probabilities, so the uncertainties for the events will be different. the uncertainty of the weather being rainy is $\frac{1}{0.25} = 4$ and for the weather being sunny is $\frac{1}{0.75} = 1.33$

The minimum number of useful bits required the information is rainy is $log(4) = 2$ and for the weather being sunny is $log(1.33) = 0.4$

This also be derived from the probability directly as, given the probability of a given event is $p$ , then the uncertainty associated with the occurrence of that event is $1/p$ and hence the minimum number of useful bits required to transfer information about it is,

\[log\left(\frac{1}{p}\right) \text{ or} -log(p), \text{ since } [log\left(\frac{1}{p}\right) = - log(p)]\]

2 bits are required to say whether the weather is rainy and 0.4 bits are required to say if the weather is sunny, so the average no. of useful bits required to transmit the information can be calculated as,

\[0.75 \times log\left(\frac{1}{0.75}\right) + 0.25 \times log\left(\frac{1}{0.25}\right) = 0.81\]

So on average, we would receive 0.81 bits of information and this is the minimum number of bits required to transfer the weather information, following the above-mentioned probability distribution.This is known as Entropy.

\[\boxed{Entropy : H(p) = - \sum_{n=1}^{n}{p_i \times log(p_i)}}\]

Entropy (expressed in ‘bits’) is a measure of how unpredictable the probability distribution is. So more the individual events vary, the more is its entropy.

Cross-Entropy

Cross entropy is the average message length that is used to transmit the message.

In this example, there are 8 variations all equally likely. So the entropy of this system is 3, but suppose that the probability distribution changes with probabilities something like this :

Though the probability distribution has changed, we still use 3 bits to transfer this information.

Now the entropy of this distribution will be,

E = -{ 0.35 x log(0.35) + 0.35 x log(0.35)+ 0.1 x log(0.1) + 0.1 x log(0.1) + 0.04 x log(0.04) + 0.04 x log(0.04) + 0.01 x log(0.01) + 0.01 x log(0.01)} = 2.23 bits
which is the minimum number of useful bits transmitted, and entropy of the system.

So though we are sending 3 bits of information, the user gets 2.23 useful bits. This can be improved by changing the no. of bits used to address each kind of information. Suppose we use a following distribution :

The average no. of bits transmitted using the following bit pattern is,

$CE = 0.35 \times 2 + 0.35 \times 2 + 0.1 \times 3 + 0.1 \times 3 + 0.04 \times 4 + 0.04 \times 4 + 0.01 \times 5 + 0.01 \times 5 = 2.42 \text{ bits}$ which is close to the entropy. This is the Cross Entropy

But suppose the same bit pattern is used for a different probability distribution :

$CE = 0.01 \times 2 + 0.01 \times 2 + 0.04 \times 3 + 0.04 \times 3 + 0.1 \times 4 + 0.1 \times 4 + 0.35 \times 5 + 0.35 \times 5 = 4.58 \text{ bits}$ which is significantly grater than the entropy.

This happens because the bit code we are using is making some implicit estimation of the probability distribution of the weather as,

\[\boxed{p = \left(\frac{1}{2^{\text{no. of bits}}}\right)}\]

So we can express cross-entropy as a function of both the true distribution and predicted distribution as,

\[\boxed{\text{Cross Entropy }: H(p,q) = -\sum_{n=1}^{n}{p_i \times log(q_i)}}\]

Here instead of taking the $log$ of the true probability, we are taking the $log$ of the predicted probability distribution $q$.

Basically, when we know the probability of occurrence of the events, but we don’t know the bit distribution of the events, so a random distribution can be taken and given the probabilities and assumed bit distribution, cross-entropy of the events can be calculated and cross-checked with the original entropy to see if the assumed distribution gives the minimum uncertainty for the given probabilities or not. Hence it is termed as “Cross” entropy.

Usually Cross entropy is larger than the entropy of a distribution. When the predicted distribution is equal to true distribution, the cross-entropy is equal to entropy.

Kullback–Leibler Divergence

The amount by which the cross-entropy exceeds the entropy is called Relative Entropy or commonly known as Kullback-Leibler Divergence or KL Divergence.

\[\boxed{d_{KL}(p\text{ }||\text{ }q) = H(p,q) - H(p)}\]

So the key take away from this article is,

given a probability distribution, the minimum average no. of useful bits required to transfer the information about the distribution is its Entropy which can also be said as the minimum possible randomness that can be associated with a probability distribution.

In the last example, we took an assumed bit distribution for each event and found the cross-entropy of that distribution with the original probabilities of the events. This cross-entropy resulted to be higher than the original entropy. So we tried to change the assumed bit distribution so that we can reduce the cross-entropy and make it as close as possible to the entropy.

But wait for a second !!

Isn’t that exactly what we try to do in a classification?

We start with a randomly initialized model that outputs an assumed bit distribution for the different classes that we want to classify, and in the process of training, we try to achieve an optimal distribution that can get us close enough to the lowest possible Entropy for the probability distribution.

So does that mean cross-entropy can be used to quantify how bad is the model performing in assuming the distribution?

Can we use KL Divergence as a metric to measure how bad the model is performing?

In the subsequent article, we shall explore the answer to all these questions and understand the intuition behind why Cross-Entropy is an appropriate loss function for our requirement.

Got some doubts/suggestions?

Please feel free to share your suggestions, questions, queries, and doubts through comments below — I will be happy to talk/discuss them all.

a post with diagrams

2021-07-04T17:39:00+00:00

This theme supports generating various diagrams from a text description using jekyll-diagrams plugin. Below, we generate a few examples of such diagrams using languages such as mermaid, plantuml, vega-lite, etc.

Note: different diagram-generation packages require external dependencies to be installed on your machine. Also, be mindful of that because of diagram generation the fist time you build your Jekyll website after adding new diagrams will be SLOW. For any other details, please refer to jekyll-diagrams README.

Mermaid

Install mermaid using node.js package manager npm by running the following command:

npm install -g mermaid.cli

The diagram below was generated by the following code:

{% mermaid %}
sequenceDiagram
    participant John
    participant Alice
    Alice->>John: Hello John, how are you?
    John-->>Alice: Great!
{% endmermaid %}

a distill-style blog post

2021-05-22T00:00:00+00:00

Equations

This theme supports rendering beautiful math in inline and display modes using MathJax 3 engine. You just need to surround your math expression with $$, like $$ E = mc^2 $$. If you leave it inside a paragraph, it will produce an inline expression, just like $E = mc^2$.

To use display mode, again surround your expression with $$ and place it as a separate paragraph. Here is an example:

\[\left( \sum_{k=1}^n a_k b_k \right)^2 \leq \left( \sum_{k=1}^n a_k^2 \right) \left( \sum_{k=1}^n b_k^2 \right)\]

Note that MathJax 3 is a major re-write of MathJax that brought a significant improvement to the loading and rendering speed, which is now on par with KaTeX.

Citations

Citations are then used in the article body with the tag. The key attribute is a reference to the id provided in the bibliography. The key attribute can take multiple ids, separated by commas.

The citation is presented inline like this: (a number that displays more information on hover). If you have an appendix, a bibliography is automatically created and populated in it.

Distill chose a numerical inline citation style to improve readability of citation dense articles and because many of the benefits of longer citations are obviated by displaying more information on hover. However, we consider it good style to mention author last names if you discuss something at length and it fits into the flow well — the authors are human and it’s nice for them to have the community associate them with their work.

Footnotes

Just wrap the text you would like to show up in a footnote in a tag. The number of the footnote will be automatically generated.This will become a hoverable footnote.

Code Blocks

Syntax highlighting is provided within tags. An example of inline code snippets: let x = 10;. For larger blocks of code, add a block attribute:

var x = 25; function(x) { return x * x; }

Note: blocks do not look good in the dark mode. You can always use the default code-highlight using the highlight liquid tag:

var x = 25;
function(x) {
  return x * x;
}

Interactive Plots

You can add interative plots using plotly + iframes :framed_picture:

The plot must be generated separately and saved into an HTML file. To generate the plot that you see above, you can use the following code snippet:

import pandas as pd
import plotly.express as px
df = pd.read_csv(
  'https://raw.githubusercontent.com/plotly/datasets/master/earthquakes-23k.csv'
)
fig = px.density_mapbox(
  df,
  lat='Latitude',
  lon='Longitude',
  z='Magnitude',
  radius=10,
  center=dict(lat=0, lon=180),
  zoom=0,
  mapbox_style="stamen-terrain",
)
fig.show()
fig.write_html('assets/plotly/demo.html')

Details boxes

Details boxes are collapsible boxes which hide additional information from the user. They can be added with the details liquid tag:

Click here to know more

a post with github metadata

2020-09-28T21:01:00+00:00

A sample blog page that demonstrates the accessing of github meta data.

What does Github-MetaData do?

Propagates the site.github namespace with repository metadata
Setting site variables :
- site.title
- site.description
- site.url
- site.baseurl
Accessing the metadata - duh.
Generating edittable links.

Additional Reading

If you’re recieving incorrect/missing data, you may need to perform a Github API authentication.
Go through this README for more details on the topic.
This page highlights all the feilds you can access with github-metadata.

Example MetaData

Host Name :
URL :
BaseURL :
Archived :
Contributors :