Search results for “site:eli.thegreenplace.net”

Eli Bendersky's website

A function that satisfies for its entire domain is called an even function (also sometimes referred to as symmetric). Its graph is symmetric w.r.t. the y axis. Some examples of even functions are and : called an odd function (also sometimes referred to as anti-symmetric). Its graph is flipped to...

In short: Discover the properties of even and odd functions, learn how to sum and multiply them, and understand integrating over symmetric intervals. Master essential math concepts with Eli Bendersky. (AI summary)

Notes on even and odd functions - Eli Bendersky's website

A function that satisfies for its entire domain is called an even function (also sometimes referred to as symmetric). Its graph is symmetric w.r.t. the y axis. Some examples of even functions are and : called an odd function (also sometimes referred to as anti-symmetric). Its graph is flipped to...

Bloom filters - Eli Bendersky's website

The original motivation for the creation of Bloom filters is efficient set membership, using a probabilistic approach to significantly reduce the time and space required to reject items that are not members in a certain set. The data structure was proposed by Burton Bloom in a 1970 paper titled...

Cross-entropy and KL divergence - Eli Bendersky's website

Cross-entropy is widely used in modern ML to compute the loss for classification tasks. This post is a brief overview of the math behind it and a related concept called Kullback-Leibler (KL) divergence. Information content of a single random event We'll start with a single event (E) that has...

Notes on implementing Attention - Eli Bendersky's website

Some notes on implementing attention blocks in pure Python + Numpy. The focus here is on the exact implementation in code, explaining all the shapes throughout the process. The motivation for why attention works is not covered here too deeply - there are plenty of excellent online resources...

In short: Learn to implement Attention blocks in Python + Numpy from this detailed guide, focusing on code and shapes. Gain insights into Basic Scaled Self-Attention, ... (AI summary)

Summary of reading: April - June 2025 - Eli Bendersky's website

Christina Soontornvat - a very nice account of the unusual event that kept the world enthralled in 2018. "The Frontiersmen: A Narrative" by Allan W. Eckert - tales from the US frontier in the period right before the revolutionary war and until the war of 1812 (back when the frontier was Kentucky...

In short: Discover thrilling tales from the past and deep learning insights! Dive into 'All Thirteen', a gripping account of the Thai soccer team rescue. Learn about f... (AI summary)

Sparsely-gated Mixture Of Experts (MoE) - Eli Bendersky's website

is typically followed by a feed forward layer (FF), which is a simple fully-connected NN with a hidden layer and nonlinearity. Here's the code for such a block that uses ReLU: def feed_forward_relu(x, W1, W2): """Feed-forward layer with...

In short: Learn about Sparsely-gated Mixture Of Experts (MoE) in transformer models, improving efficiency by splitting large feed-forward layers into smaller 'experts'... (AI summary)

Teaching coding with JavaScript and p5.js - Eli Bendersky's website

When asked which programming language to learn first - especially for kids - my usual answer is JavaScript [1]. Nothing beats the direct feedback you get from code that's able to paint things on the screen, without having to install anything. One library that makes it a particularly pleasant...

Articles in tag "Math"

2025.07.04: Notes on even and odd functions 2025.06.14: Notes on Cramer's rule 2025.05.20: Convolutions, Polynomials and Flipped Kernels 2025.05.01: Bloom filters 2025.04.18: Sparsely-gated Mixture Of Experts (MoE) 2025.04.12: Cross-entropy and KL divergence 2025.04.05: Reproducing word2vec with...

- Eli Bendersky's website

Hello, and welcome to my modest corner of the web! This blog began in 2003 as a personal online journal; in the past few years it became mostly an outlet for technical, programming-related posts. It's my way to document things I find...