A Comprehensive Review of State-of-the-Art Generative AI Models in Natural Language Processing: Architectures, Innovations, Applications, and Future Directions

Abstract

Over the last few years, generative AI models make NLP work new and unprecedented potential in terms of language understanding and generation. This paper aims to systematically review recently published generative AI models in premier venues and includes most popular BERT, GPT, RoBERTa, XLNet, ALBERT, ERNIE, DistilBERT, T5, ELECTRA, and DeBERTa. These new models have incorporated significant novelties to the field such as bidirectional context comprehension, permutation training techniques, parameters’ sharing and multi-task learning resulting in a significant boost in performance standard for numerous typing NLP operations. We review these models, focusing on the key aspects, improvements and usages in several tasks including text generation, machine translation, summarization, and question and answering. This review demonstrates the added value of each model by also comparing their strengths and weaknesses and how it contributes to the progress of NLP. In addition, we describe the practical applications of these models with examples of implementation of such models in the healthcare, finance, and entertainment industries. The purpose of this paper is three-fold: to offer a systematic literature review and analysis of generative AI approaches for language representation to establish the current state of knowledge and emerging trends. It also points to further investigation by indicating areas for model improvement relating to efficiency, interpretability and ethics. It is thus our intention, through this systematic review, to provide useful knowledge to researchers and practitioners, which shall help the growth of generative AI and its further incorporation in natural language processing.