Stochastic Gradient Descent

For curve fitting using linear regression, there exists a minor variant of Batch Gradient Descent algorithm, called Stochastic Gradient Descent. In the Batch Gradient Descent, the parameter vector  is updated as, . (loop over all elements of training set in one iteration) For Stochastic Gradient Descent, the vector gets updated as, at each iteration the…

Read More

Sigma delta modulation

In an earlier post, it was mentioned that delta modulator without the quantizer is identical to convolving an input sequence with . Let us first try to validate that thought using a small MATLAB example and using the delta modulator circuit shown in Figure 9.13a of DSP-Proakis [1]. % delta modulation xn = sin(2*pi*1/64*[0:63]); xhatn…

Read More

Chi Square Random Variable

While trying to derive the theoretical bit error rate (BER) for BPSK modulation in a Rayleigh fading channel, I realized that I need to discuss chi square random variable prior. What is chi-square random variable? Let there be independent and identically distributed Gaussian random variables with mean and variance and we form a new random…

Read More