Speaker authentication has become increasingly important. It goes with the other forms of security checks such as user login and personal identification number and has a say in the final decision about the authenticity. One of the issues with the authentication algorithms is that the automated devices that take the call have to work with a limited data set. In this chapter, a new class of intelligent element called differentially fed artificial neural network has been introduced to predict the data and use it effectively. It keeps the model simple and helps in taking online and crisp decisions with the available limited data.