Artificial Neural Networks (ANNs) have become popular among researchers and practitioners for modeling complex real-world problems. One of the latest research areas in this field is evolving ANNs. In this chapter, we investigate the simultaneous evolution of network architectures and connection weights in ANNs. In simultaneous evolution, we use the well-known concept of multiobjective optimization and subsequently evolutionary multiobjective algorithms to evolve ANNs. The results are promising when compared with the traditional ANN algorithms. It is expected that this methodology would provide better solutions to many applications of ANNs.