The post covers various neural network based word embedding models. Starting from the Neural Probabilistic Language Model from Bengio et al 2003, then reduction of complexity using Hierarchical softmax and Noise Contrastive Estimation. Further works like CBoW, GlOVe, Skip Gram and Negative Sampling which helped to train on much higher data.
I noticed that the scripts provided from http://wolverinex02.googlepages.com/emoticonsforblogger2 for writing equations on blogger stopped working. Reason: The address of the public mimeTEX server is changed to /cgi-bin/mimetex.cgi? from Thursday, March15 2007.
In a previous post, we discussed about a probable first order digital PLL for tracking constant phase offset. The assumption was that as the phase offset is small and the bits gets decoded correctly, the phase difference between the ideal and actual constellation gives the initial value of phase. However, in typical scenarios it may…
Long back in time we discussed the BER (bit error rate) for BPSK modulation in a simple AWGN channel (time stamps states August 2007). Almost an year back! It high time we discuss the BER for BPSK in a Rayleigh multipath channel. In a brief discussion on Rayleigh channel, wherein we stated that a circularly…
Let us try to understand the formula for Channel Capacity with an Average Power Limitation, described in Section 25 of the landmark paper A Mathematical Theory for Communication, by Mr. Claude Shannon. Further, the following writeup is based on Section 12.5.1 from Fundamentals of Communication Systems by John G. Proakis, Masoud Salehi
In this post, the objective is to figure out the minimum separation between two sinusoidals having frequencies , of duration each to be orthogonal. Let the phase difference between the sinusoidals is where can take any value from to (Refer Example 4.3 [DIG-COMM-SKLAR]). For the two sinuosidals to be orthogonal,
Question 6 on digital circuit from GATE (Graduate Aptitude Test in Engineering) 2012 Electronics and Communication Engineering paper. Q6. Consider the given circuit In this circuit, the race around (A) does not occur (B) occurs when CLK=0 (C) occurs when CLK=1 and A=B=1 (D) occurs when CLK=1 and A=B=0
Some among you will be aware that in a wireless link having multiple antenna’s at the receiver (aka receive diversity) improves the bit error rate (BER) performance. In this post, let us try to understand the BER improvement with receive diversity. And, since we are just getting started, let us limit ourselves to additive white…
Question 3 on Communication from GATE (Graduate Aptitude Test in Engineering) 2012 Electronics and Communication Engineering paper. Q3. In a baseband communications link, frequencies upto 3500Hz are used for signalling. Using a raised cosine pulse with 75% excess bandwidth and for no inter-symbol interference, the maximum possible signaling rate in symbols per second is, (A)…
The IEEE 802.11a specifications are used by many to understand a wireless communication link built using OFDM. In this post, I have put together a set of 10 multiple choice questions based on 802.11a specifications. The questions are on the building blocks in 802.11a specifications, preamble structure and so on. Upon completion of the quiz,…
In typical digital signal processing applications, there arises need to increase the sampling frequency of a signal sequence, where the higher sampling frequency is an integer multiple of the original sampling frequency i.e for a signal sequence with a sampling frequency , change the sampling frequency to , where is an integer.
In a classification problem, the output (also called the label or class) takes a small number of discrete values rather than continuous values. For a simple binary classification problem, where output takes only two discrete values : 0 or 1, the sigmoid function can be used to transform the output of a linear regression model…
In the previous post on Batch Gradient Descent and Stochastic Gradient Descent, we looked at two iterative methods for finding the parameter vector which minimizes the square of the error between the predicted value and the actual output for all values in the training set. A closed form solution for finding the parameter vector is possible, and in this post…
Are you bothered by speed of the speed of the simulations which you develop in Matlab/Octave? I was not bothered much, till I ran into the Viterbi decoder. If you recall, the Matlab/Octave simulation script for BER computation with hard soft decision Viterbi algorithm provided in post Viterbi with finite survivor state memory took around…